Files
MeterVault/docs/RELEASE_NOTES.md
T
Florian Schmidt 5a6f34a467
ci / build-test (push) Successful in 2m34s
Analysis: bound the freshness query, let window sums skip chunks, share the Overview's catalog
Three things the 1,000-meter x 10-year measurement found, each proved by
EXPLAIN or a statement count before and after. No tally moves.

Freshness had two jobs in one unbounded query. The mark -- when a meter
last delivered -- is now stored on meter_rollup_state and maintained by
every recompute, with a one-pass backfill in the migration, so an
import-only meter keeps its years-old last activity without reading a
single raw row. The rhythm that decides stale versus live is sampled
inside a 90-day window and only for meters that actually have a live
source; a source silent for longer than that is re-read unbounded, so it
is still called stale by its own rhythm rather than by a default. The
portfolio query went from 13.8 ms planning plus 36.1 ms execution across
all 123 reading chunks to 0.58 plus 0.44 ms across four.

Window sums took their time bounds only from the unnest join, so the
planner could not exclude chunks: a 1,960-window case scanned 1.39 M rows
in parallel and spilled a 45 MB sort. Repeating the overall min and max as
constants makes it five chunks and nested-loop index scans, 121.5 ms to
8.7 ms.

The Overview read the catalog three times, once for the quantities and
once for each of its two bills. One context and one catalog snapshot now
feed all three: 31 statements per load to 23.

The final timings on an idle machine are in docs/ANALYSIS_REPORT.md: the
brief's target request (100 meters, ten years, monthly) is 286 ms against
two seconds, and a startup rebuild of 1,000 meters is 279 s.
2026-09-20 11:16:48 +02:00

18 KiB
Raw Blame History

Release notes

User-visible changes per release. Earlier releases are described in the git history and in the README's upgrade sections. Decision ids (D-nn, A-nn) refer to ANALYSIS_IMPLEMENTATION_NOTE.md.

0.4.0 — Dashboard, navigation and historical analysis (unreleased)

This release rebuilds how MeterVault analyses and prices data, and the pages that show it. Every page now works on the same selected period and the same numbers. A virtual (calculated) meter can be analysed like a physical one. A true zero, missing data, data that exists only per month, and a missing price are always told apart.

Some figures change on purpose. Most visibly, an energy type's bill now counts its grid meter, not every meter that exists. Changed figures lists every deliberate change. The seeded demo's yearly costs now match the spreadsheet: 2025 comes to 7,907.65 € against the sheet's 7,907.64 €. The seeded "this year" cost for 2026 was 4,402.19 € in 0.3.0, which priced Haus + Netz + Auto and water at 5 €/m³. It is now 2,940.19 €, the sheet's figure.

Upgrading

Back up the database (pg_dump) first.

  • The first start rebuilds all analysis data (normalization revision 3):
    • For every meter it rebuilds consumption and the new day and month rollups plus their coverage. It records the state of each meter.
    • This runs before the web server listens. The app is unreachable meanwhile, and Compose may report the container unhealthy. Let it finish.
    • The time grows with the number of raw readings. One meter took about 0.05 s with monthly readings, ~0.5 s with ten years of daily readings (3,700 readings), and ~1.2 s with a year of hourly readings (9,300 readings).
    • A synthetic 1,000-meter × 10-year instance (1.34 M readings) took 4.7 minutes (279 s) on an idle Ryzen 9 9950X3D — about 3.6 meters a second. It writes rollups and coverage on top of consumption, so it does more per meter than the 0.3.0 rebuild did.
    • Progress is logged every 100 meters. A meter that fails is logged, kept pending and retried at the next start. A meter whose stored consumption is older than its oldest remaining reading is skipped and logged, so its history is not truncated.
    • Until a meter is rebuilt, its pages say "analysis being prepared", never "no data".
  • The migration records each meter's last reading time in the analysis state table and fills it in for the meters you already have, in one pass over the readings. It is what the pages show as "last activity" and what decides whether a live source is late, so no page has to search the raw readings for it any more.
  • Virtual meters without a formula (such as a seeded Summe Solar from an earlier version) are converted at startup. When their incoming links name meters of one unit and kind, the implied sum is stored as an explicit formula (D-28). The log lists the converted meters, and the meters that still need configuration because their links are ambiguous, in mixed units, or loop. The conversion runs once; a rerun changes nothing. From then on, links are topology only and never change a calculation.
  • The migration drops the unused continuous aggregates and their hourly refresh jobs. It also deletes consumption rows stored for virtual meters, which nothing read (D-17).
  • Check the attention items on the Overview after the first start. They name missing prices with a direct "Add tariff" link, invalid calculations, stale live sources, rows dated after now, and meters that may be counted twice.
  • Rolling back to 0.3.0 works. The old version ignores the new tables and never read the dropped aggregates. Consumption stays as 0.4.0 booked it until each meter next ingests a reading. Stored virtual formulas remain, and 0.3.0 goes back to summing links.

What's new

Navigation.

  • The sidebar has a fixed structure: Overview, Analysis, Meters, an Energy types group, Specialized views (Solar, Tanks & consumables; always listed, with a setup hint when not configured), Data import, and Configuration.
  • "Configuration → Energy types" edits the definitions; the Energy types group is for analysis.
  • Expanded groups are remembered, and the current page's group always opens. If the energy types cannot be loaded, the menu shows an error with Retry instead of silently dropping them.
  • Breadcrumbs (Overview → energy type → meter) keep the selected period, and so does Back.
  • "Find a meter" opens the meter's analysis with the current period and still offers quick entry. It is a text button on desktop and an icon on phones.

One period everywhere.

  • Every analysis page has the same toolbar: this month to date, last month, year to date, previous year, last 12 or 24 months, all history, or custom dates with one Apply.
  • It also offers a bucket size (automatic, day, week, month, year) and a comparison: previous period, same period last year (the default), or any calendar year.
  • The effective dates are shown next to the choice, with the time zone. The selection lives in the address, so reload, Back and shared links reproduce the page.
  • Rapid clicks can no longer leave one page showing another selection's data.

Overview.

  • It shows one selected period (default month to date):
    • the cost, split into metered use, standing charges, manual costs and feed-in credit;
    • a card per energy type with its quantities in their own units, its cost and billing basis, the change and freshness;
    • a history chart with a table view;
    • "What changed", by category or by meter;
    • the cost composition;
    • attention items, each with one targeted action.
  • Changes are compared only over the part both periods cover, and the page states both date ranges.
  • When the period has no data, the page names the dates that do have data and offers "Go to latest data". It never silently switches to an older month.

Analysis page (/trends, formerly the cost trend).

  • Explore everything, one energy type, a cost category, one meter, or up to six meters side by side, by quantity or by cost.
  • Compare calendar years with an overlay and a comparison table. Click a bar or a row to drill into a finer period.
  • Export exactly what is shown as CSV.

Energy type pages are titled with the type's own name and have four tabs:

  • Overview: measures such as total use, grid import, generation and runtime, each in its own unit and never added across units; the cost with its billing basis; coverage; the largest changes.
  • History: the total, or up to six individual meters with an explanation of how each one counts.
  • Flow: the Sankey, now using the same values as every other page. Calculated and estimated connections are marked. It comes with a table version and Manage connections.
  • Meters: each meter's value for the period and its data quality.

Meter page.

  • The tabs sit directly under the header: Analysis, Readings, Normalized data, Events, Tariffs, Sources. A calculated meter has Calculation instead of Sources.
  • The Analysis tab shows:
    • the period total with its unit and status, and the cost with its rule, or the reason it has none;
    • the change against the comparison period;
    • a labelled projection, where there is enough data for one;
    • a full chart with the previous-year overlay, and a table;
    • a "Data quality and coverage" section;
    • events and tariff changes in the range.
  • The record tabs page through the whole history (100 rows at a time, filtered by the selected dates), no longer just the latest 200. Rows dated after now are marked.
  • Existing links such as ?tab=consumption, ?tab=readings&action=reading and ?tab=events&action=swap still open the intended tab and dialog once.

Virtual (calculated) meters.

  • Create them with Sum, Difference or a Formula over other meters, picked by name. A live preview of the selected period shows every source's values and flags incomplete months.
  • The result kind (consumption, generation, net or indicator), unit and cost rule are stored with the formula.
  • A virtual meter gets the same analysis as a physical one, plus "Source meters" showing each source's contribution.
  • The rules:
    • A missing source month makes the result "no data" for that month, never a silent zero.
    • An observed zero is a real zero.
    • A division by zero or a loop is reported, and names the meters involved.
    • Differences stay negative.
  • New calculated meters are analysis only: they never add to a type's totals or the bill. The "Always count" option lets one replace its sources instead.

Solar and Tanks & consumables use the same toolbar, cards and charts.

  • Solar works out self-consumption, feed-in, site use, savings and autarky from the meters' roles. For a missing role it shows a setup card with candidate meters instead of raw role tags.
  • Tanks keep "Last dipstick (date)" apart from "Estimated now". A past period shows the contents at its end, not today's. The forecast is a labelled projection, hidden when the dipstick is older than 60 days.

Tariffs and configuration.

  • An "Add tariff" link from a missing price opens the tariff editor once, prefilled with the scope, component and first uncovered month.
  • The unit is checked against what it prices: a wrong unit or currency blocks the save.
  • A new tariff needs a value; a typed 0 is a deliberate free period.
  • The editor notes that Bonus, Discount and Tax are stored but not applied yet.
  • Meter roles have friendly names and one-line meanings. A role is unique per energy type, and saving it names the meter it moves from.
  • The Settings page shows the analysis data state and that raw retention is not enforced.

Also:

  • CSV export of any analysis view (/export/analysis.csv): statuses, provenance, costs and comparison values; unknown values are empty cells, never 0.
  • The light/dark choice persists across reloads and language switches, and charts follow it immediately.
  • Visible keyboard focus, and tables for every chart.
  • MudBlazor's own labels are in German too.
  • Pages work at phone width.
  • Amounts use the configured currency (MeterVault__Currency) instead of a hard-coded €.
  • Deleting a meter or an energy type also deletes the tariffs scoped to it.
  • JSON export/import now carries meter connections, re-links virtual formulas to the new meter ids, and no longer restores the tariffs of deleted meters onto other meters.

Changed figures

Every change below is deliberate. The golden spreadsheet reconciliation (consumption of all four sheets, Netz Einsparung) is unchanged. The seeded yearly bill matches the sheet's Jahreskosten within ±0.02 € for 2022, 2025 and 2026. It differs by 3.78 € (2023) and 0.46 € (2024) only because the sheet multiplies by unrounded prices.

Area 0.3.0 0.4.0
What an energy type's bill counts Every meter's cost was summed: Haus + Netz + Auto for the seeded Strom The type's grid import meter when it has one, otherwise its household use. Submeters are breakdowns; generation is never billed. Seeded Strom is Zähler Netz × price, like the sheet (2025: 4,742.64 €) (D-22, D-34)
Feed-in credit Credited on all generation Only on a meter with the grid-export role, at the feed-in price (D-34)
A subsection with its own meter price Added on top Billed at its own price and taken out of the meter above it; quantities unchanged (D-35, A-19)
Missing tariff Cost 0 "Not priced (no tariff)" with an Add tariff action; a hole in a price history is a price gap (unavailable). The quantities stay visible. An explicit 0 tariff is still a valid zero (D-38)
Standing charges Per meter, per month that had readings, and copied onto every meter of the type Per local day over the scope's service period (including reading gaps), once per scope. Type and global charges are their own rows; meter fees stay on their meter (D-40, A-18)
Price of a longer bucket Month buckets used the price of the 15th, year buckets the price of 1 July Every bucket is priced month by month at the price of the 15th, so a year equals the sum of its months and changing the bucket never changes a total (D-36)
Months in which the billed grid meter was not yet (or no longer) in service — (0.3.0 summed every meter) Unavailable rather than free while use was measured, with an attention item naming the grid meter and the months (A-17)
Tank, runtime, direct-delta or instant-rate readings more than a month apart (a tank dipped every few months, quarterly burner hours) Booked whole in the month of the later reading, with zeros in between The months in between read "only coarser data" (not zero). A year or longer bucket is priced when all its months share one price; otherwise it is unavailable with an attention item (A-16)
Manual costs Counted in the Overview but not in the trend; a cost dated later this month counted at once Counted once, on their start day, when that day has come, everywhere: Overview, Analysis, categories, export (D-41)
Cost categories Sum of their member meters' costs The priced non-overlapping cover of their members plus their manual costs. Seeded Strom = Netz × price. A category overlapping another is shown as a view, apart from the composition. A category whose members price nothing says so (D-42, A-22)
Virtual meters No analysis; the flow summed incoming links, ignoring any formula Full analysis from the stored formula; the Sankey uses the same values (D-27, D-30)
Summe Solar and other generation sums Analysed as generation (seeded 2025: 4,750 kWh = Solar 1 3,123 + Solar 2 1,627) and not costed, because generation is never billed (A-15)
Readings at exactly midnight Booked in the following day Booked in the day they close (D-11)
"Last 12 months" 1314 buckets, including a partial future month 12 calendar buckets ending with the current month; actuals stop at now (D-02)
Rows dated after now Counted in "to date" totals Excluded and shown as "recorded after now", e.g. a sheet row labelled the current month or a future-stamped reading. A day that holds such a row reads partial (D-04, A-14, A-20)
Overview "this month / this year" Current month and year against the complete previous ones The selected period against the same elapsed part of the comparison period, measured over what both cover (D-07)
"Latest month with data" An amount without its month; consumption only The month and its basis (meter data, manual costs or both) (D-19)
Percentages A negative baseline was divided by its absolute value "Not applicable" for a zero or negative baseline; the absolute difference is always shown (D-08)
Meter dates Outside its install and retire dates a meter counts as a known zero; retired meters keep their history (D-24)
Currency € hard-coded in most views MeterVault__Currency everywhere; a tariff in another currency is reported as not fitting, never converted (D-43)
Continuous aggregates Refreshed hourly, read by nothing Dropped; rollup tables are written with each recompute (D-12, D-17)
Seed Adds the water price of 7.00 €/m³ from 2026-01-01, and stores Summe Solar's formula (m4 + m5, generation, not costed). This affects new seeds; existing seeded instances get the formula through the startup conversion (D-44)

REST API

Every existing field keeps its name and type. What changed is only added as new fields. The numbers follow the new engine, as listed above: actuals stop at now, virtual meters are evaluated, and costs are the bill's.

  • GET /api/v1/consumption:
    • New fields: status, issue, kind, unit.
    • A month without data is left out instead of reported as 0.
    • from/to with a UTC offset are accepted; they used to fail with a server error.
    • Virtual meters return evaluated values.
  • GET /api/v1/cost:
    • New fields: costStatus (Priced, Partial, NotPriced, PriceGap, UnitMismatch), costAvailability (the state of the quantities behind the cost, e.g. Unresolved, Invalid), costRule, notCosted, missingPrices[] (component, reason, scope, first and last month, tariff, unit issue, credit), status, issue, kind, unit.
    • cost stays numeric and is 0 when nothing could be priced; check costStatus and costAvailability before trusting a 0.
    • Behaviour change: generation and runtime meters, indicators and calculations that cannot be evaluated now report costStatus: NotPriced, with notCosted giving the reason. They used to report Priced, and a generation meter could carry a negative feed-in cost (A-21).
  • GET /api/v1/dashboard/summary:
    • New fields: deltaPercentApplicable per KPI, and latestMonth {period, basis}.
    • The month and year windows are unchanged (the calendar month and year to now, against the whole previous ones), but the values are the new bill. deltaPercent is 0 when not applicable.

Known limitations

  • Raw retention is not enforced (D-57). MeterVault__RawRetentionDays is shown but nothing deletes readings, because every recompute rebuilds a meter from its readings.
  • Monthly data is never interpolated to days (D-57). A day or week view of monthly data says "only coarser data" and offers the monthly view.
  • Bonus, Discount and Tax tariffs are stored but not applied (D-57).
  • Every live reading recomputes its meter in full (D-57). That is fine for monthly and daily meters, but costs about 1.2 s per reading for a meter with a year of hourly data, and grows with history.
  • Months cannot switch billing basis: the billing basis (grid meter or household use) is chosen per energy type for all time (A-17).
  • Batteries are not modelled. Without a grid-export meter, Solar's feed-in is calculated, and labelled as such.
  • No CSV export on the Solar page, because the export has no derived measures.