Analysis: bound the freshness query, let window sums skip chunks, share the Overview's catalog
ci / build-test (push) Successful in 2m34s
ci / build-test (push) Successful in 2m34s
Three things the 1,000-meter x 10-year measurement found, each proved by EXPLAIN or a statement count before and after. No tally moves. Freshness had two jobs in one unbounded query. The mark -- when a meter last delivered -- is now stored on meter_rollup_state and maintained by every recompute, with a one-pass backfill in the migration, so an import-only meter keeps its years-old last activity without reading a single raw row. The rhythm that decides stale versus live is sampled inside a 90-day window and only for meters that actually have a live source; a source silent for longer than that is re-read unbounded, so it is still called stale by its own rhythm rather than by a default. The portfolio query went from 13.8 ms planning plus 36.1 ms execution across all 123 reading chunks to 0.58 plus 0.44 ms across four. Window sums took their time bounds only from the unnest join, so the planner could not exclude chunks: a 1,960-window case scanned 1.39 M rows in parallel and spilled a 45 MB sort. Repeating the overall min and max as constants makes it five chunks and nested-loop index scans, 121.5 ms to 8.7 ms. The Overview read the catalog three times, once for the quantities and once for each of its two bills. One context and one catalog snapshot now feed all three: 31 statements per load to 23. The final timings on an idle machine are in docs/ANALYSIS_REPORT.md: the brief's target request (100 meters, ten years, monthly) is 286 ms against two seconds, and a startup rebuild of 1,000 meters is 279 s.
This commit is contained in:
@@ -24,15 +24,18 @@ Back up the database (`pg_dump`) first.
|
||||
state of each meter.
|
||||
- This runs **before the web server listens**. The app is unreachable meanwhile, and Compose may report the
|
||||
container unhealthy. Let it finish.
|
||||
- The time grows with the number of raw readings. One meter took about 0.1 s with monthly readings, ~0.6 s with ten
|
||||
years of daily readings (3,700 readings), and ~1.4 s with a year of hourly readings (9,300 readings).
|
||||
- A synthetic 1,000-meter × 10-year instance (1.34 M readings) took 5.6–6.5 minutes (338–392 s) on a Ryzen 9
|
||||
9950X3D. That run was preliminary, on a busy machine. It is about 20 % slower per meter than the 0.3.0 rebuild,
|
||||
because rollups and coverage are written too.
|
||||
- The time grows with the number of raw readings. One meter took about 0.05 s with monthly readings, ~0.5 s with ten
|
||||
years of daily readings (3,700 readings), and ~1.2 s with a year of hourly readings (9,300 readings).
|
||||
- A synthetic 1,000-meter × 10-year instance (1.34 M readings) took 4.7 minutes (279 s) on an idle Ryzen 9 9950X3D —
|
||||
about 3.6 meters a second. It writes rollups and coverage on top of consumption, so it does more per meter than
|
||||
the 0.3.0 rebuild did.
|
||||
- Progress is logged every 100 meters. A meter that fails is logged, kept pending and retried at the next start.
|
||||
A meter whose stored consumption is older than its oldest remaining reading is skipped and logged, so its
|
||||
history is not truncated.
|
||||
- Until a meter is rebuilt, its pages say "analysis being prepared", never "no data".
|
||||
- **The migration records each meter's last reading time** in the analysis state table and fills it in for the meters
|
||||
you already have, in one pass over the readings. It is what the pages show as "last activity" and what decides
|
||||
whether a live source is late, so no page has to search the raw readings for it any more.
|
||||
- **Virtual meters without a formula** (such as a seeded *Summe Solar* from an earlier version) are converted at
|
||||
startup. When their incoming links name meters of one unit and kind, the implied sum is stored as an explicit
|
||||
formula (D-28). The log lists the converted meters, and the meters that still need configuration because their
|
||||
@@ -214,7 +217,7 @@ engine, as listed above: actuals stop at now, virtual meters are evaluated, and
|
||||
and offers the monthly view.
|
||||
- **Bonus, Discount and Tax tariffs are stored but not applied** (D-57).
|
||||
- **Every live reading recomputes its meter in full** (D-57). That is fine for monthly and daily meters, but costs
|
||||
about 1.4 s per reading for a meter with a year of hourly data, and grows with history.
|
||||
about 1.2 s per reading for a meter with a year of hourly data, and grows with history.
|
||||
- **Months cannot switch billing basis:** the billing basis (grid meter or household use) is chosen per energy type
|
||||
for all time (A-17).
|
||||
- **Batteries are not modelled.** Without a grid-export meter, Solar's feed-in is calculated, and labelled as such.
|
||||
|
||||
Reference in New Issue
Block a user