Three things the 1,000-meter x 10-year measurement found, each proved by EXPLAIN or a statement count before and after. No tally moves. Freshness had two jobs in one unbounded query. The mark -- when a meter last delivered -- is now stored on meter_rollup_state and maintained by every recompute, with a one-pass backfill in the migration, so an import-only meter keeps its years-old last activity without reading a single raw row. The rhythm that decides stale versus live is sampled inside a 90-day window and only for meters that actually have a live source; a source silent for longer than that is re-read unbounded, so it is still called stale by its own rhythm rather than by a default. The portfolio query went from 13.8 ms planning plus 36.1 ms execution across all 123 reading chunks to 0.58 plus 0.44 ms across four. Window sums took their time bounds only from the unnest join, so the planner could not exclude chunks: a 1,960-window case scanned 1.39 M rows in parallel and spilled a 45 MB sort. Repeating the overall min and max as constants makes it five chunks and nested-loop index scans, 121.5 ms to 8.7 ms. The Overview read the catalog three times, once for the quantities and once for each of its two bills. One context and one catalog snapshot now feed all three: 31 statements per load to 23. The final timings on an idle machine are in docs/ANALYSIS_REPORT.md: the brief's target request (100 meters, ten years, monthly) is 286 ms against two seconds, and a startup rebuild of 1,000 meters is 279 s.
53 KiB
Analysis rework: implementation note
Companion to DASHBOARD_ANALYSIS_CHANGE_BRIEF.md. This is the Phase 1
deliverable that resolves the brief's open choices. Every decision has an ID (D-nn) so code, tests and the final
report can refer to it. Written against c0f52db; revised after an adversarial design review.
The note was kept current through Phase 5:
- §11: amendments from the Phase 1 module review.
- §12: amendments from the acceptance review.
- §13: decisions recorded with the final documentation.
- §14: amendments from the performance measurement.
- §9 and §10: extended with what the implementation measured and changed.
The outcome is in ANALYSIS_REPORT.md, the user-facing changes in RELEASE_NOTES.md.
1. What the code review established
The brief's findings A01–A14 are confirmed by the source, with these refinements:
- Nothing evaluates a virtual meter.
VirtualNormalizerandExpressionEvaluatorrun only in tests.NormalizationServiceskips virtual meters. The only value in production isFlowService's sum of incoming links, which ignores any stored formula. SDD §14.1 is not implemented in either direction. - The expression evaluator is unsafe for user input. It has no AST, unknown identifiers evaluate to 0, and recursion is unbounded.
- Consumption rows store only the interval end. There is no unit column, and
Estimatedcovers four different cases. Coverage cannot be recovered from sums; it has to be captured during normalization. - Month division only exists for cumulative and generation counters. Tanks, runtime, direct-delta and instant-rate modes book a whole interval at its end. The same is true of swaps, resets, decreases and first readings.
- The continuous aggregates cannot be used: they are Berlin-only, materialized-only on TimescaleDB ≥ 2.13, and never backfilled. No reader uses them, yet they are refreshed hourly.
- Raw retention is not implemented. Turning it on would destroy history, because every recompute rebuilds a meter from the readings that remain.
- The seeded costs are wrong in a way the spreadsheet proves. The sheet bills
Kosten = Netz × price, but the seed prices Haus + Netz + Auto. The seed is also missing the water price rise to 7.00 €/m³ in 2026. - The tooling gaps are real. There is no clock abstraction, no bUnit or Playwright, and the API responses have no contract tests.
2. Periods and comparisons
- D-01 Clock.
TimeProvideris registered. Pages and API endpoints read "now" once per request and resolve a period with the pure resolver, then pass the resolved period down. Services never read the clock. Where one genuinely needs "now" (freshness, forecast), it takes a trailing optionalTimeProvider? time = null. Tests use a smallFixedTimeProvider. - D-02 Presets and URL tokens.
period=mtd|last-month|ytd|prev-year|12m|24m|all|custom, withfrom/to(yyyy-MM-dd) used only forcustom.- Overview default:
mtd. History pages default:12m, which is 12 calendar buckets ending with the current partial month. allspans the availability metadata (D-19), not a fixed century.- A page default applies only when no period key is present. An invalid token falls back to the default and shows a notice.
- D-03 Bounds.
- A period resolves once, in the instance zone, into two forms: a local inclusive date range for display, and
a half-open UTC range
[from, to)for queries. tois the local midnight after the end date, or the captured "now" for to-date periods.- Quantities, costs, comparisons and exports all use the same bounds.
- A period resolves once, in the instance zone, into two forms: a local inclusive date range for display, and
a half-open UTC range
- D-04 Now and the future.
- Actual figures stop at "now". A row counts as recorded after now when its source interval ends after now. Examples: a current-month label row, a future-stamped Tasmota row.
- Such rows are excluded from actuals and reported in a separately labelled "recorded after now" block (rows, amount, dates), with an attention item.
- A range entirely in the future reports "not yet occurred".
- D-05 Buckets.
- Buckets are
day|week|month|year|auto. Weeks start on Monday in local time. autopicks one bucket for the whole chart: the coarsest resolution any plotted series needs, at most 400 points.- An explicit bucket above 400 points is refused with a coarser suggestion.
- Bucket bounds are local midnights, clipped to the period.
- Buckets are
- D-06 Comparisons.
compare=none|prev-period|prev-year|year:YYYY.year:YYYYneeds a year-aligned range.- Shifting uses local calendar units, never durations:
- Whole years shift by years, and whole months by months. Anything else shifts by days.
12m/24mcompare with the N months before.allhas no comparison.
- The cut-off maps as local date plus wall time:
- A nonexistent time takes the first valid instant after the gap.
- An ambiguous time takes the occurrence with "now"'s offset if one matches, otherwise the first occurrence.
- A day that does not exist in the target month (31st, 29 Feb) cuts at that month's end.
- D-07 Matched coverage.
- A change figure is "confident" only over the range both periods actually cover. The current period's covered range is shifted, intersected with the comparison's coverage, and trimmed to whole buckets where resolution is coarser than the cut.
- Both requested ranges and the matched range are shown.
- An empty match means "not comparable": absolute values only, no percentage.
- D-08 Change figures.
- The absolute difference is always shown.
- The percentage is "not applicable" when the baseline is ≤ 0 or unavailable.
- Colours depend on the metric: more consumption is not "good", more generation is.
- D-09 Projections are separate and labelled "Projection (straight-line from N days)".
- Method: the covered rate × the remaining days. Standing charges are added exactly per day.
- A projection is suppressed when:
- coverage ends more than 2× the meter's typical interval before now;
- covered elapsed time is under 7 days (month) or 30 days (year);
- the resolution is coarser than the period.
- A change chip never compares a projection with an actual.
3. Data layer
-
D-10 Engine intervals. Every
Consumptionrow carries its source interval:IntervalStart,IntervalEndandDivided, as EF-ignored properties, so the schema does not change. Each mode sets them:Mode Interval Counters previous effective reading → this one, with GapSegmentbounds for divided sharesRuntime previous effective time Tank previous TankLevel event Instant rate previous sample Direct delta previous reading, or the labelled month for a label First reading the labelled month for a label; [InstalledAt, t]when set; otherwise an unknown start (D-14)Coalescekeeps the minimum start and the maximum end. -
D-11 Midnight stamps. A non-label row whose interval end falls exactly on a local midnight is stamped 1 second earlier, inside the day it describes, mirroring
InsideSegment. Label rows keepStampTime.[from, to)stays everywhere. -
D-12 Tables. These are plain tables, not hypertables. Each has an FK to
meterwithON DELETE CASCADE. They are written byRecomputeMeterAsyncin the caller's transaction, by diff, so only changed rows are touched.Table Key Columns consumption_rollup(meter_id, day, kind)amount,measured,manual,imported,estimated,rows,flags(baseline-delta, divided)consumption_rollup_month(meter_id, month, kind)same columns; month and year reads use it meter_coverage(meter_id, span_from)span_to,resolution_class,divided_at_months,gap_reasonmeter_rollup_state(meter_id)revision,zone,normalized_unit,kind,built_atAll local dates use the configured zone.
-
D-13 Coverage runs.
- Consecutive intervals of the same resolution class merge into one run. The classes are ≤ 1 h, ≤ 1 day, ≤ 7 days, ≤ 1 local month, and coarser.
- An interval longer than a month is its own run.
- These open a gap instead of coverage:
- an unexplained decrease;
- a reset without PrevValue;
- an instant-rate gap longer than max(1 h, 10 × the median sample interval);
- deliveries before a tank's first level.
- Coverage is capped at now.
-
D-14 Bucket status.
- missing: no run overlaps the bucket.
- partial: runs cover only part of it.
- unresolved: an undivided interval crosses the bucket edge by more than 5 % of the bucket length. The only exception is an edge at a local month boundary the normalizer divided at.
- available: everything else. An available bucket with no rows is a true zero.
- Opening balance: a first reading with unknown start marks its bucket "partial (opening balance, start unknown)". It is excluded from comparisons and projections, and the UI offers to set an install date.
- Provenance is a separate dimension, derived from the per-quality amounts plus
derivedfor virtual meters.
-
D-15 Reading a period.
- Rollups (month table for month/year buckets, day table otherwise) cover complete local days.
- For at most two partial edge days per range, one direct
consumptionquery covers[edge-day midnight, cutoff)(meter_id = ANY(@ids)). - Each request makes one query per table and one tariff load. Virtual dependencies are expanded in memory first. The 400-point and 6-series limits are enforced before any SQL runs.
-
D-16 Rebuild.
CurrentRevisionbecomes 3, because the engine books differently (D-11, intervals).- The startup upgrade rebuilds consumption, rollups and coverage, and records
meter_rollup_state. - A meter whose state is missing or outdated (revision, or zone ≠ the reader's zone) reads as "analysis being prepared", never as "no data".
- The migration purges derived rows of virtual meters.
RecomputeMeterAsyncpurges them if a meter becomes virtual. - The upgrade skips and logs any meter whose oldest consumption predates its oldest reading or event, instead of truncating history (D-44).
-
D-17 Continuous aggregates. The new migration removes their policies and drops the three views.
Monthly_continuous_aggregate_refreshes_and_matches_baseis replaced by a rollup-equals-consumption test (water Dec 2022 = 14 m³). -
D-18 Freshness.
- The last reading or event time is the freshness mark.
- A live source is stale when that time is older than the larger of 3 × the median of its last 20 intervals and 3 × its poll interval.
- Import-only meters are "historical", never "stale".
-
D-19 Availability.
- A quantity scope's availability is its coverage.
- A cost scope's availability is its billed meters' coverage plus its manual costs'
PeriodStartdays. - Both are capped at now.
- "Latest period with data" is the latest local month ≤ now in that union. It is returned with its month and basis (meters / manual / both).
4. Quantities, units and totals
- D-20 Normalized quantity. A Core function
NormalizedQuantity(meter, tank, definition)returns(kind, unit).-
Kinds: consumption, generation, export, runtime, and for virtual meters also net or indicator.
-
Units by mode:
Mode Unit RuntimeCounter h, or the tank unit with a Fixed rate (kind stays runtime; provenance estimated)InstantRate the rate unit without /h(W→Wh, kW→kWh)ConsumableBalance the tank unit Virtual its declared result unit Others Meter.Unit -
Aliases are normalized (m3 = m³).
-
The normalized unit is stored in
meter_rollup_state. Raw units appear only on the Readings tab.
-
- D-21 Roles.
total_load,grid_importandgrid_exportare unique per energy type; saving a role moves it and says who held it. The editor shows localized names and one-line meanings, and offers each role only for compatible modes. - D-22 Per-type totals algorithm. Pure and ordered:
- Supply meters are grid_import or grid_export meters, GenerationCounter meters, and generation-kind virtual meters. A link out of a supply meter is a supply edge and never makes its target a submeter.
- Containment. A link from a physical, consumption-kind, non-supply meter makes the target a breakdown of its parent.
- Measures per type:
- Use is the total_load meter if there is one, otherwise the consumption roots. Consumption roots are physical consumption-kind meters that are not supply meters, have no containment parent and are not runtime meters. Tanks count.
- Grid import is the grid_import meters.
- Export is the grid_export meters, which are never consumption.
- Generation is the GenerationCounter roots.
- Runtime is the runtime meters.
- Measures are never added across units.
- Virtual meters are analysis views. Retired meters keep their history.
- Seeded result: use = {Haus}, breakdown = {Auto}, grid import = {Netz}, generation = {Solar 1, Solar 2}, analysis-only = {Summe Solar}, runtime = {Brenner}, water use = {Wasser}, oil use = {Öltank}.
- D-23 Override.
Meter.Meta.totalsisauto|always|never.alwayson a virtual meter replaces its expanded dependencies in that measure and in the bill.alwayson a meter whose ancestor or dependent is already counted is refused on save, naming the other meter.neverremoves a meter from the measures it would join.- The resulting cover is shared by the quantity totals and the bill.
- D-24 Lifecycle. Outside
[InstalledAt, RetiredAt], when set, a meter contributes a known zero to totals and to virtual evaluation.
5. Virtual meters
- D-25 Definition.
Meter.Metaholdsexpression,referencedMeterIds(always derived from the expression and rewritten on save),resultKind(consumption|generation|net|indicator),resultUnitandcostRule(none|sourceCosts|ownQuantity).- Topology links never define a calculation.
- D-26 Formula.
- The grammar is the existing one (
+ - * /, parentheses, numbers). It is parsed to an AST, with limits of 2,000 characters and nesting depth 64. - References are
m<id>; any other identifier is invalid. - Validation on save and on read covers: syntax, unknown or self references, cycles through nested virtual meters (with the path), and kind/unit.
- Kind/unit rules for operands:
+/−need the same unit and kind, or a declarednet.- Meter × or ÷ meter needs a declared
resultUnitand kindindicator. - Indicators are non-additive, never totalled and never costed.
- The grammar is the existing one (
- D-27 Evaluation.
- Evaluation runs on read, per bucket, from the sources' rollups.
- A virtual meter's coverage is the intersection of its sources' coverage, and its resolution is the coarsest among them.
- A missing source makes the bucket missing (strict); an observed zero is a valid input. A non-finite result makes it invalid, with the reason.
- A period total is the formula applied to the sources' totals over the joint coverage. It is partial when that coverage is smaller than the period. For a linear formula without a constant this equals the sum of its buckets; otherwise the series is marked non-additive.
- The result carries every source's series, status and dependency path.
- D-28 Legacy definitions.
- At startup, an expression-less virtual meter whose same-type incoming links name sources of one unit and kind gets the equivalent explicit sum. Meters are processed in dependency order, the run is idempotent, and the counts are logged.
- Anything else is flagged "needs configuration".
- Until converted, the reader evaluates the implied sum with status "legacy — confirm".
ReferenceDataImporterwrites Summe Solar's definition directly:m(Solar 1) + m(Solar 2), generation, kWh.
- D-29 One evaluator.
VirtualNormalizeris removed fromNormalizationEngine.CreateDefault. Its tests and the golden Netz Einsparung reconciliation move to the new evaluator over month buckets (≥ 20 matches, ±1 kWh). - D-30 Flow.
- A pure-sum virtual meter's incoming edges are its calculation dependencies, drawn at each source's own value and marked "calculated".
- Other virtual meters appear only in the table view.
- Links are capped at the parent's value, and proportional splits are marked estimated.
- The Flow tab gets "Manage connections".
- D-31 Editor.
- Sum, Difference and Advanced modes, with source pickers by name (showing unit, kind and install/retire dates) and a live preview for the selected period.
- On a virtual meter's page, a Calculation tab replaces Sources. Register details and Readings are removed. Events keeps Note.
- Saving a Sum offers to sync the incoming links.
- D-32 Export/import carries
meter_linkand remaps meter ids inside definitions. - D-33 Deleting a meter lists the virtual meters that depend on it and requires confirmation.
6. Costs
- D-34 Billing.
- Per energy type, the grid_import meters are billed if the type has one, otherwise the use meters (D-22).
- Generation meters are never billed.
- The feed-in credit is the FeedIn price × the export of grid_export meters.
- Runtime and virtual meters are not billed unless D-39 applies.
- D-35 Separately billed submeter. A containment child with an applicable meter-scoped UnitPrice is billed at its own price. Its monthly quantity is subtracted from its billed ancestor's for pricing. Quantity totals do not change.
- D-36 Prices.
- The monthly convention is kept: the price valid on the 15th of each local month.
- Every bucket size is priced month by month, so week and year buckets are split by local month. Changing the bucket never changes a total.
- D-37 Tariff applicability.
- A UnitPrice or FeedIn tariff applies only when the unit denominator matches the meter's normalized unit. Known scales are converted (ct, per 100 L, per MWh).
- A parsed mismatch makes the cost "unavailable (unit)". An unparseable unit applies, with a warning.
- BasePrice units are per day, per month (the default) or per year.
- The tariff editor validates units on save and states that Bonus, Discount and Tax are not applied yet.
- D-38 Coverage.
- A billed scope with no UnitPrice tariff at any date is "not priced (no tariff)". That is an attention item, not "unavailable".
- A gap inside a priced scope's tariff history makes the cost "unavailable" for those months.
- An explicit zero tariff is a valid zero.
- A missing FeedIn price is reported only where a grid_export meter exists.
- D-39 Virtual costs.
sourceCostsadds the sources' metered costs. It is allowed only for pure sums and excludes scope-level standing charges.ownQuantityprices the virtual quantity with normal precedence. It is allowed only for linear formulas without a constant.- The default is
sourceCostsfor pure sums andnoneotherwise. The rule is named next to every virtual cost. - A virtual meter is part of the bill only through D-23.
- D-40 Standing charges.
- A standing charge accrues per local day, at value ÷ days in that local month, over the scope's service period. The service period runs from the earliest InstalledAt or first data to the latest RetiredAt or now, regardless of reading gaps.
- Meter-scoped charges stay on their meter.
- Type- and global-scoped charges are their own rows ("Standing charge — " / "— global"), never split across meters.
- D-41 Manual costs are booked in full on their
PeriodStartlocal day, when that day is in[from, to)and ≤ today.PeriodEndis informational. A cost withMeterIdset goes to that meter's categories. - D-42 Categories.
- A category's cost is the priced cost of the non-overlapping cover of its members, using the bill algorithm restricted to them, plus its manual costs. For example, Strom {Haus, Netz, Auto, Solar 1, Solar 2} gives Netz × price, and a category {Auto} gives Auto × price.
- A type- or global-scoped standing-charge row joins a category only if the whole type (or, for global, every billed meter) is a member.
- The composition is the disjoint categories, plus Uncategorized, plus standing-charge rows, and it reconciles to the bill.
- A category that overlaps another, or covers meters outside the bill, is an "overlapping view" and stays outside the composition.
- The donut is drawn only when every slice is ≥ 0; otherwise signed bars are used.
- D-43 Currency.
MeterVault__Currencyis used everywhere through oneFormat.Money. - D-44 Seed.
- The water tariff 7.00 €/m³ from 2026-01-01 is added.
- Summe Solar gets an explicit definition.
- The seed tariffs are otherwise unchanged.
- Golden: the seeded yearly bill equals the sheet's
Jahreskostenwithin ±0.02 € for 2022 (421.52), 2025 (7,907.64) and 2026 (2,940.19). This is computed on a frozen clock after 2026-05-31, with the tank unpriced. - 2023 and 2024 differ by 3.78 € and 0.46 €, because the sheet rounds its displayed prices. That is documented and not tuned away.
7. API
- D-45 Compatibility.
- Contract tests for
/consumption,/costand/dashboard/summaryare written before anything is rerouted. - Every existing field and type is kept.
/consumptionand/cost:- They keep exact-instant bounds, now converted with
ToUniversalTime(). coststays numeric (0 when nothing is priced), andcostStatusandmissingPrices[]are added.- Virtual meters return evaluated values with a status.
- They keep exact-instant bounds, now converted with
/dashboard/summary:- It keeps its calendar month and year windows (legacy semantics, documented) but uses the new billing set.
- It adds
deltaPercentApplicableandlatestMonth.
- The numeric change is listed in the release notes.
- Contract tests for
8. Pages, navigation, state
- D-46 URL state.
- The query parses into an immutable
AnalysisQueryvalue. Analysis reloads only when that value changes, with a generation counter and cancellation. - An action drop or tab change never reloads it.
- Toolbar and tab changes replace the history entry; drill-downs push. Defaults are never written on load, so a deep-linked dialog is not dismissed.
- Initial loads stay in
OnInitialized/OnParametersSet, because render tests read prerendered data.
- The query parses into an immutable
- D-47 Keys.
- Meter tabs are
analysis|readings|normalized|events|tariffs|sources|calculation. Legacyconsumptionmaps tonormalized. On virtual meters,sourcesmaps tocalculationandreadingstoanalysis. - Energy-type tabs are
overview|history|flow|meters. - Analysis page scope is
scope=portfolio|type|category|meter|meterswithid/ids(at most 6), plusmetric=consumption|generation|export|runtime|net|cost|balance. - Link helpers append the new keys after the existing ones.
- Meter tabs are
- D-48 Navigation.
- Sidebar entries: Overview, Analysis, Meters, Energy types (with a retry item on error), Specialized views (Solar, Tanks & consumables — always listed, with a setup state when unsupported), Data import, Configuration.
- Expanded groups persist in a cookie.
NavStategainsMetersChanged. - Breadcrumbs (Overview → type → meter) carry the period, and Back returns to the parent.
- Search shows a text label from the md breakpoint up, and its results link to Analysis (with the period) plus quick entry.
- D-49 Theme.
- A scoped
ThemeStateis backed by a cookie that App reads, so prerender and the language switch keep the mode. - Charts use a transparent background and the theme's mode, and are re-keyed on theme or result change.
- Units and currency go into JS formatter strings. Points are nullable; there is no smoothing and no joining across gaps.
- A scoped
- D-50 Tables.
- Readings, Normalized data and Events are paged server-side (100 rows), keyset-ordered, with
from/tofilters. - The manual-entry dialog runs its own queries (latest reading, reading at T with flags, boundaries), so its verdicts never depend on a page of rows.
- Readings, Normalized data and Events are paged server-side (100 rows), keyset-ordered, with
- D-51 Drill-down. Clicking a chart bucket keeps the scope, sets the bucket's range and the next finer supported bucket. If there is none, it opens Normalized data filtered to that bucket.
- D-52 Deep links.
- Tariffs:
/admin/tariffs?scope=&id=&component=&from=&action=newopens a pre-filled new-tariff dialog. Missing-cost explanations link there with the first uncovered month.
- Tariffs:
- D-53 Attention items. Missing required price (scope and first month), stale live source, invalid or unconverted virtual definition, recorded-after-now rows, possible overlap (a total_load and a grid_import root that are not linked).
- D-54 Solar and consumables.
- Both adopt the shared toolbar, cards and charts. Units come from D-20.
- Solar shows a setup card for each missing role.
- Tanks show "Last dipstick: on " separately from "Estimated now (incl. deliveries since)". For a historical range they show the balance at the range end. Deliveries are filtered to the range.
- The forecast is suppressed when the dipstick is older than 60 days.
- D-55 CSV export. The analysis table as CSV: one row per bucket and series, with local ISO bucket bounds, timezone, invariant numbers, empty cells for unavailable values, and status, provenance, cost, cost status, currency and the comparison value. Served by an App endpoint that takes the same URL keys.
9. Evidence, limitations, deviations
- D-56 Evidence.
- Frozen-clock tests cover New Year, Berlin DST in spring and autumn, 29 Feb, 31 Jan → Feb, and New York.
- Seeded goldens: D-44, the D-22 classification, and water Dec 2022 = 70 € / 14 m³.
- Worked virtual examples: A+B, A−B, missing vs zero, nested, cycle, division by zero.
- A synthetic generator (test trait) for 1,000 meters × 10 years with recorded timings.
- Screenshots and a manual checklist (EN/DE × light/dark × 360/768/desktop) from the seeded instance. No bUnit or Playwright is added.
- As implemented:
- The frozen-clock, seeded-golden and worked-virtual suites exist as planned.
docs/ANALYSIS_REPORT.mdlists them with counts. At the end: Core 1,733 tests, Integration 746. - The synthetic generator and timings are
tests/Integration.Tests/Performance(traitCategory=Performance, skipped unlessMETERVAULT_PERF=1). - The manual checklist ran as Chrome DevTools Protocol scripts against seeded instances: four acceptance
reviewers plus the page agents, in EN/DE, light/dark, at 1440/390/360 px. Those scripts and the screenshots are
outside the repository. Server-rendered pages are covered by
HtmlRenderer-based render tests in EN and DE.
- The frozen-clock, seeded-golden and worked-virtual suites exist as planned.
- D-57 Limitations.
- Raw retention is not implemented;
/admin/settingslabels it "not enforced", and the Readings tab explains it. The brief's "Retained history" scenario is a documented blocker. - Monthly imports are never interpolated to days (SDD §8.7 / §14.2 unchanged).
- A full recompute still runs per live reading.
- Bonus, Discount and Tax tariffs are not applied.
- Measured and found later (see
docs/ANALYSIS_REPORT.md):- The per-reading recompute is linear in a meter's reading count: ~0.1 s for a monthly meter, ~1.4 s for a year of hourly data. The startup rebuild is ~20 % slower per meter than in 0.3.0.
- The freshness query (
AnalysisQueries.RecentReadingsAsync) has no time bound, so it plans across every raw chunk. - The window-sum query (
AnalysisQueries.WindowSumsAsync) gets no plan-time chunk exclusion. - Both grow with history length, and neither is fixed.
- The billing basis cannot change month by month (A-17).
- Batteries are not modelled in Solar's calculated feed-in.
- The Solar page has no CSV export, because the export has no derived measures.
- Raw retention is not implemented;
- D-58 SDD deviations.
- §14.1: virtual meters are computed on read, and nothing is materialized.
- §5.4 / §10: the configured zone is used, and rollups replace the continuous aggregates.
- §8.1: the Overview shows one selected period.
- §7.4: prices inside expressions are not supported; the
ownQuantitycost rule covers savings. - Also marked in the SDD at the end of the rework:
- §3 (FR-9, FR-11, FR-12, FR-16), §4.1 / §4.2 (no aggregates, the two-reader pipeline) and §5.1 (the new tables).
- §5.5 (raw retention not enforced, D-57).
- §7.1 (revision 3), §7.3 (tank "now" vs period), §7.5 (the bill, D-34 – D-43).
- §8.0 (the shared contract), §8.2 – §8.7 (the pages), and the monthly-history note (no interpolation).
- §9 (additive API fields, D-45, A-21), §10 (half-open bounds,
TimeProvider), §11 – §13 (layout, milestones, tests) and §14.1 – §14.4 plus the new §14.9 – §14.14. - Appendix B (Ersparnis is not an expression).
10. Deliberate behaviour changes
| Area | Old | New |
|---|---|---|
| Seeded Strom bill | Haus + Netz + Auto priced | Netz (grid import) billed; matches the sheet |
| Feed-in | credited on all generation | credited on grid_export only |
| Missing tariff | cost 0 | not priced / unavailable (D-38) |
| Standing charge | per meter per month with data | once per scope, per day of service |
| Midnight readings | booked in the next day | booked in the day they close (D-11) |
| "Last 12 months" | 13–14 buckets, including a future month | 12 buckets, actuals up to now |
| Virtual meters | no analysis; flow sums links | full analysis from the formula |
| Overview "this year" | full year vs complete previous year | selected period vs matched coverage |
| Currency | hard-coded € | configured currency |
| Continuous aggregates | refreshed hourly, unused | dropped |
| API | — | additive fields only; the summary's values follow the new bill |
Added as the amendments and pages landed (the release notes, docs/RELEASE_NOTES.md, list them for users):
| Area | Old | New |
|---|---|---|
| Year and week buckets | a year priced at the 1 July price, a type/global base price per meter and bucket | every bucket priced month by month at the price of the 15th (D-36) |
| Separately priced subsection | added on top of its parent | billed at its own price, out of its parent (D-35, A-19) |
| Manual costs | in the summary but not the trend; a cost later this month counted at once | once, on its start day, once that day has come, everywhere (D-41) |
| Categories | sum of member meters' costs | priced non-overlapping cover; overlapping categories are views (D-42) |
| Intervals longer than a month (tank, runtime, direct delta) | booked whole in the later month, zeros between | months "only coarser data"; longer buckets priced when the months share one price (A-16) |
| Months without a grid meter in service | — | cost unavailable, with an attention item (A-17) |
| Meter fee on a meter no line prices | per meter and bucket | its own standing-charge row (A-18) |
| Rows recorded after now | counted in to-date totals | reported apart; a day holding one reads partial (D-04, A-14, A-20) |
| Summe Solar / generation sums | not costed (no analysis) | analysed; cost rule none (A-15) |
| Percentage against a negative baseline | divided by its absolute value | not applicable (D-08) |
/api/v1/cost of generation, runtime, invalid meters |
Priced (a generation meter could carry a negative feed-in cost) |
NotPriced with costRule/notCosted (A-21) |
/api/v1/consumption months without data |
0 | left out (only rows holding quantity data) |
| Deleting a meter or energy type | its scoped tariffs stayed behind and could be restored onto another meter | deleted with it (EntityDeletion); export/import skips such orphans (A-37) |
Tests rewritten on purpose (none weakened; each rewrite states the new rule):
MeterPeriodServiceTests: deleted withMeterPeriodService. Its cases moved toMeterAnalysisLoaderTests, with the virtual-null case replaced by positive and error-state cases (the worked A+B example, missing vs zero, an invalid calculation).FlowServiceTests.Virtual_sum_meter_aggregates_its_upstreamsbecameVirtual_sum_meter_is_its_formula, plus legacy, non-sum, capped-link, missing-sub-meter, other-unit and after-now cases.- The CAgg test became
CostReconciliationTests.Monthly_rollup_equals_the_consumption_it_sums.SchemaTestsnow pins that the aggregates and their jobs are gone. FormatCultureTests: the currency case becameMoney_is_in_the_configured_currency_written_the_readers_way, plus formatter cases.LocalTimeEntryTests.Tab_keys_map_to_panel_indexesbecameTab_keys_resolve_by_key_and_mode.VirtualMeterTestsandElectricityReconciliationTests.Netz_einsparung_virtual_matches_the_sheetnow run throughVirtualEvaluator(D-29).ExpressionEvaluatorTests: deleted with the evaluator.FormulaParserTestspins that unknown identifiers are errors.DashboardRenderTests: the literal labels of the old pages, replaced by assertions on the new ones in EN and DE.DashboardServicesTests: four Solar and tank tests moved toSpecialized/with the new services' assertions.AnalysisChartModelTests: a missing bucket is marked "–", not "*" (A-28).MeterAnalysisLoaderTestsreads the cost change's new type (A-23).AttentionItemsTestshas the specific duplicate-role text.
11. Amendments after the Phase 1 module review
These refine the decisions above where the independently built Core modules met.
- A-01 Opening balance. An opening balance is never a coverage run. The rollup day it is booked in
carries a baseline-delta flag, which the coverage evaluator takes as an input. Matched coverage gets the
booked stamps of opening-balance rows.
CoverageGapReason.OpeningBalanceis removed. - A-02 DividedAtMonths. This is true when no interval of a run straddles a local month boundary undivided. An interval is either divided at month boundaries, or lies inside one local month. A zero increase across a month boundary counts as divided, because a register that did not move is exactly 0 in every month.
- A-03 Divided intervals are classified at most
Month: they are never coarser for month and year buckets. Runs are rejoined by source interval identity, not by adjacency. - A-04 Capping at now.
- Stored runs are uncapped and carry
LastIntervalStart. A reader caps a run at now, but when now falls inside the run's final interval, the run ends atLastIntervalStart. That final interval's row is recorded after now (D-04). - A to-date bucket counts as fully covered when its coverage reaches within one interval of the run's class of its end. The consumption since the last reading is not yet known, and the bucket is not reported Partial for that reason.
- One module owns capping:
CoverageRuns.CapAt.
- Stored runs are uncapped and carry
- A-05 Recorded after now (Phase 2). Each rollup day stores the latest interval end among its rows. A day whose rows end after now is reported as recorded after now, not as an actual.
- A-06 Auto bucket is chosen from the period's nominal range (ytd → month, mtd → day), so one URL renders the same way all year. The point limit is checked on the elapsed range.
- A-07 Roles.
- Analysis reads roles only through
MeterRoleRules.Effective: case-insensitive, and only for modes that may hold them. - Virtual meters never hold a role.
- A role is unique among meters that are not retired. A retired meter keeps its role for its history, and each meter counts only within its own service period (D-24).
- Analysis reads roles only through
- A-08 Virtual result kinds are consumption, generation, net or indicator, nothing else. Save writes the effective (inferred) kind, unit and cost rule, so readers never re-infer them.
- A-09 One classifier, one unit table.
- The resolution classifier lives once, in
Coverage. Units(Quantities) is the only unit normalizer, and every module uses its comparer.
- The resolution classifier lives once, in
- A-10 Comparison mapping details (D-06).
- A range starting on a day the target month lacks (30 March → February) starts at the end of that month, so a range starting 30 March matches from 1 March.
- A "now" in the second pass of an autumn fold maps to the end of the fold when the target date has no fold, which keeps the mapping monotonic.
- Comparison buckets are paired with the current buckets by index (
ComparisonResolver.PairBuckets), never planned separately.
- A-11 A separately connected heat pump (its own supply, not below the main meter) is modelled as its own energy type with its own grid_import meter. Containment children with their own price remain D-35.
- A-12 Joint coverage. The reader derives each virtual source's per-day coverage and bucket states from
CoverageEvaluator, and feeds them toVirtualEvaluator. The evaluator does not re-derive coverage rules. - A-13 Default comparison. D-06 lists the comparisons without fixing a default. Every page compares with the
previous year when
compareis absent (prev-year): the Overview's month to date with the same elapsed days a year earlier, a history page's last 12 months with the 12 months a year before. Compared with the period just before, a seasonal utility would show the season as a trend. Like every default it applies only to an absent key and is never written into a URL;compare=noneandcompare=prev-periodremain one click away. - A-14 Capping inside an earlier interval (A-04). When now falls inside an interval that is not the run's last (two readings stamped ahead, a reading stamped weeks ahead whose month shares are several intervals, a sheet row that carries the current month's register into a later month), the stored run does not say where that interval starts. The run then ends at the earliest instant it can start: the later of the local month start of now and now minus one interval of the run's class; a coarse run ends where it starts. Such a run is always divided at months, because an undivided interval across a month edge is its own run. So coverage never claims time whose row is recorded after now: a label run gives up exactly the current month, finer data at most one interval. The shares of such an interval that closed before now stay actuals, as A-05 reads interval ends per share.
- A-15 Virtual source costs (D-39).
sourceCostsadds, for each physical source, what that source's own scope costs: a consumption source at its unit price (or its bill line), an export source as its feed-in credit, a generation or runtime source nothing. A sum whose sources price nothing is not costed, and the reason is named (generation, runtime).- The sources come from the formula's weights, through nested pure sums, each once:
m1 + m1 - m1 + m2is m1 and m2. - A generation sum defaults to
none, because generation is never billed (D-34). The seed and the legacy derivation write the default, so Summe Solar is stored withnone. A storedsourceCostson a generation sum costs nothing. - A sum over a nested calculation that is not a pure sum is not a sum of metered costs. Its default is
none. A storedsourceCostsstays valid for the quantity but is taken asnoneon read (not costed: "a source calculation is not a plain sum"), and is reported asCostRuleNeedsPureSumfor the editor to refuse on save (VirtualValidation.CostRuleProblem,IsSavable).
- A-16 Intervals longer than a month (D-36). Pricing month by month left a meter whose reading intervals span
several months (a tank dipped every few months, a quarterly delta, burner hours read quarterly) without a cost at
every bucket size. When a bucket of several months holds an unresolved month, the cost engine also reads the bucket
whole. If every month the bucket has data in has the same price (the same tariff outcome and converted unit price,
D-37), the bucket costs its quantity at that price. A price change inside it leaves it unavailable, with an attention
item naming the meter and the months. Month buckets stay unknown; year buckets and period totals are priced, and the
bucket size still never changes a total. The legacy adapters no longer turn such an unknown cost into a priced 0:
ConsumableSummary.CostKnown,MeterPeriodView.YearToDateCostKnown, andcostAvailabilityon/api/v1/cost(additive, D-45). The dashboard summary keeps D-45's numeric legacy windows. - A-17 Months without a grid meter (D-34 with D-24). The billing basis is chosen per energy type for all time. In a month where no billed grid_import meter is in service on every day (before its install date, after it retired without a successor) while a use meter in service measured something, the grid meter's known zero would bill that use as free. Such months are unavailable instead, with an attention item naming the grid meter and the months. Switching the basis month by month is deferred: the category composition (D-42) would need the same per-month basis to stay reconciled with the bill.
- A-18 Meter fees without a line (D-40). A meter-scoped standing charge of a physical meter that no line of the figure prices (a PV or house meter behind the billed grid meter) accrues as its own standing-charge row on that meter, over its service period: in the type's bill, the portfolio and the meter's own scope. In the composition it is a row like the type's: it joins the one disjoint category that holds its meter, and is a slice of its own otherwise.
- A-19 Kaskade (D-35 with D-22). A consumer with its own meter-scoped unit price, linked directly below a billed grid_import meter with no house meter in between, is billed at its own price and taken out of the grid meters that link to it. D-22 still reads that link as a supply edge, so the measures do not change. A priced meter nothing links is still reported as an unused meter price.
- A-20 Withheld days are not complete (A-05, D-14). A-05 withholds a whole rollup day or month once a row in it closes after now, and that can take rows recorded before now with it (a current-month label beside live readings takes the day's live share). Coverage cannot see this, so such a bucket, and a total holding it, reads partial with the issue "recorded after now", never available: an empty day is not a true zero. The rows stay in the "recorded after now" block.
12. Amendments after the acceptance review
These refine decisions where the acceptance review found a gap. No golden bill or reconciliation figure changes.
- A-21 Not-costed meters on
/api/v1/cost(A-16, D-45). A meter without a cost rule (generation, runtime, an indicator, a calculation that cannot be evaluated) returnscostStatus: NotPricedand, ascostAvailability, the status of its quantity (Invalidfor a loop or a division by zero) — never "Priced, Available" beside the numeric 0 D-45 keeps. Two additive fields say why:costRule(MeterCostRule) andnotCosted(MeterNotCostedReason). A costed meter's month whose quantity is invalid or pending never reports an available cost either. Release note: physical generation and runtime meters changed fromPricedtoNotPriced. - A-22 A category whose members price nothing (D-39, D-42). The cost math is unchanged: a calculated view, a
generation or runtime meter adds nothing to a category. The cost reader now reports it
(
CostAttentionKind.CategoryPricesNothing, with the category and the members), for a category scope and for every category of a portfolio read. The Analysis page shows the explanation instead of "No data yet", the Overview lists the category in its composition as "No cost – members not billed", and the meter editor says under a virtual meter's cost categories that membership adds no cost. - A-23 One cost-change rule on every page (D-07). The Overview's rule (
OverviewComparison.Between: the totals when both periods are complete, else the paired buckets complete on both sides, else not comparable) is used by the energy type page, the Analysis page (cards and the table's total row) and the meter page (which now also states it from the totals when both are complete), with the same "over the part both periods cover" caption. The Analysis page's one-meter quantity view reads the meter's comparison cost for its cost card. - A-24 A measure's resolution (D-51). A per-type measure carries the coarsest resolution of the meters it counts (a virtual member's evaluated resolution), so a type or Solar view over monthly data never drills a month into days; the Overview's own fallback is gone. Solar bounds drilling by every series it charts. Pages offer a click, a drill column and a drill hint only where some bucket leads somewhere.
- A-25 A virtual meter's bucket with nothing finer (D-51). A virtual meter has no records, so where a physical meter opens its Normalized data, a virtual meter's bucket opens the meter's own analysis over that bucket; its source contributions link on to each source's records for it. The bucket that already is the whole view leads nowhere.
- A-26 Nothing booked (D-19, D-41). A cost bucket with no line, no charge, no manual cost and nothing missing stays
unknown in the engine (SeededBillTests) and now reads "No data" everywhere: never "Priced" beside "—", never complete,
and
Missing(notAvailable) in the CSV export. - A-27 A tariff's value (D-38, D-52). The tariff editor starts a new tariff without a value and refuses to save without one, so the missing-price deep link cannot turn a gap into a free period by one click. A typed 0 for a unit price, base price or feed-in is saved as the valid zero D-38 defines, with the note "A price of 0 makes this period free of charge".
- A-28 Chart marks (D-49, brief §4.3). Bars are outlined in their colour, so a true zero is a line on the baseline and a gap draws nothing; a bucket without a value is marked "–" (its own note), a qualified value keeps "*". A chart with nothing to draw says why: data only coarser than the buckets (naming the resolution, with the interval that shows it) or a cost without a price — "no data" only when there is none. A unit mismatch in an attention item says what does not fit: the currency, a base price's period, or the meter's unit.
- A-29 Record tabs and "now" (D-04, D-50). The record tabs list the whole named range, so their toolbar shows those dates (the end of the month for month to date), and every row dated after now carries an "After now" mark.
- A-30 Preview period (D-31). The calculation preview opens on the period of the page the editor was opened from and offers every preset, custom dates and all available history (the sources' own dates, however old), through the shared toolbar; a range too long for months previews in years.
13. Amendments recorded with the final documentation
The page agents and the integration made these decisions while building. They are implemented and tested, but were not written down above. They are recorded here so the note stays the complete list. None of them changes a golden figure.
- A-31 Page-specific URL keys (D-46, D-47).
- The energy History tab uses
view=total|meters. - The Overview uses
chart=for its chart selection. - The record tabs reuse the page's
period/from/to, andallthere means no date bound. - None of these keys belongs to
AnalysisUrlKeys. They are written with replace and never reload the analysis.
- The energy History tab uses
- A-32 Overview projection (D-09).
- It is offered only for month or year to date, only from a complete figure, and only after 7 or 30 days.
- Metered use (net of feed-in credit) and standing charges are extended at their observed rate per elapsed day. D-09 said standing charges are added exactly per day; that is not done.
- Manual costs are kept as booked, not projected.
- A-33 Series on one chart (D-15, brief §7.4).
- The Analysis page draws comparison overlays for at most three series. With more, the comparison stays in the table, with a note.
- The energy History "individual meters" view charts at most six meters and links to the Analysis page for the rest.
- A category is analysed by quantity only when all its meters share one kind and unit. The meters are then shown side by side and never added, because members can overlap. Otherwise the page explains why and offers the alternatives.
- A-34 Context and counts on the meter page (brief §7.2, D-50).
- Events and tariff changes in the range are listed under the chart, not drawn on it: the shared chart has no annotation support.
- Record tab labels carry no counts, because that meant counting every row on each load. Each table states its own count, capped at 10,000.
- A-35 Solar figures (D-54).
- Self-consumption is total load − grid import, else generation − grid export.
- Feed-in is the grid export meter, else generation − self-consumption. The calculated form is labelled, and batteries are not modelled.
- Site use is the total load meter, else self-consumption + grid import.
- Savings are self-consumption × the grid unit price, month by month through
CostCalculator. The feed-in credit is the cost reader's own line. - Mixed units make a figure invalid, naming the units.
- A-36 Tariff deep link (D-52, A-27).
- The prefilled dialog opens once.
action,componentandfromare then dropped from the address. scope/idstay and filter the list to the tariffs that can price that meter or type, with "Show all tariffs".- The suggested unit follows the scope until the user types one.
- A stored tariff whose unit no longer fits shows an issue icon and cannot be saved again until the unit is fixed.
- Both admin and meter tariff lists show the effective end: the day before the next tariff of the same kind starts
(
TariffValidity).
- The prefilled dialog opens once.
- A-37 Deleting a meter or an energy type (D-32, D-33).
tariff.scope_idhas no foreign key, soEntityDeletiondeletes the tariffs scoped to the meter or type together with it, in one transaction. For a meter, its readings and consumption go too. Export/import no longer restores a tariff whose meter or type is gone onto whichever id replaces it. - A-38 Where the toolbar sits (brief §7.2, §7.3).
- The energy page has one toolbar above its four tabs:
- Interval and comparison show on Overview and History; metric and export only on History.
- A bucket refused for too many points is read again on auto, so every tab still shows figures while the toolbar offers the coarser size.
- The meter page keeps its toolbar inside the Analysis tab, so its tab bar sits directly under the header.
- The energy page has one toolbar above its four tabs:
- A-39 Shell after the acceptance review.
- Buttons, icon buttons, links, chips, tabs and nav links get a 2 px focus ring in the theme's text colour.
MeterVaultMudLocalizergives MudBlazor's own labels German text. The English values are MudBlazor's own.- Meter → Sources links each source's connector to its editor, and the source dialog has "Edit connector". Both keep the existing detour and its draft.
- The theme defaults to dark when no
mv-themecookie is set. - The Calculation tab words a calculation problem as the attention list does, one wording per
VirtualProblemKind.
14. Amendments from the performance measurement
The measurement of brief §9.10 / D-56 on a synthetic 1,000-meter × 10-year instance named three costs that grow with history rather than with what a page asks for. They are fixed here. No golden bill, reconciliation figure or displayed value changes; only what a request reads does.
- A-40 What a request reads (D-15, D-18, D-56).
- The freshness mark is stored, not searched.
meter_rollup_stategainslast_reading_at: the stamp of the meter's latest raw reading, written by the recompute that every write path already runs. D-18 is unchanged — the last reading or event time is still the mark — but a request no longer queriesreadingto find it, so an import-only meter's mark stays exactly as old as its data without planning across a decade of raw chunks. The migration backfills the column in one pass, so nothing waits for a rebuild. - Only a live source's rhythm is sampled, and only from the recent past. The median of D-18 is consulted for a
meter with a live source, so the reading times are read for those meters alone, bounded by
FreshnessRules.RecentWindow(90 days) — the bound is what lets PostgreSQL exclude the older chunks at plan time. A live meter that delivered nothing inside the window has no rhythm there; those few meters are read again over their whole history, so a long-silent source is still called stale by its own rhythm. An instance without any live source readsreadingnot at all. - The window-sum statement carries its overall bounds. Its windows arrive through an
unnestjoin, so their bounds are columns and exclude no chunk. The minimum start and maximum end of the window set are repeated as constants in theWHEREclause. No row outside them can match any window, so no tally changes; with enough windows it is the difference between an indexed probe and a parallel scan of the whole hypertable with a sort spilling to disk. - One Overview load holds one catalog. The page's quantities, its bill and its comparison's bill are three
figures of one period.
DashboardService.GetOverviewAsyncloads the meters, tanks, links and rollup states once and passes that snapshot to all three (CostReader.ReadAsync(db, catalog, …)), so every figure answers from the same snapshot and the load is not repeated. The cards, the change table and the composition are derived from those results and are never priced again.OverviewReadBudgetTestsasserts the statement count of one load.
- The freshness mark is stored, not searched.