Docs: the analysis rework's implementation report and screenshots
ci / build-test (push) Successful in 2m23s
ci / build-test (push) Successful in 2m23s
The brief (§12) asks for a report: which of its findings A01-A14 were
fixed and how, how physical and virtual results now agree, which cost and
aggregation decisions were applied to existing data, what was tested, and
what is still missing. docs/ANALYSIS_REPORT.md is that, with a file path
and a pinning test for every row, the acceptance scenarios with their
evidence, the measured performance, and the limitations -- including the
three the note already records (raw retention not enforced, monthly
imports never interpolated, a full recompute per ingested reading) and the
three hotspots the measurement found.
The screenshots it references show the Overview, a physical and a virtual
meter, and an energy type's history at desktop and phone widths, plus two
"before" shots at c0f52db for comparison: the old Overview, and the old
virtual meter page, which offered a flow link instead of any analysis.
This commit is contained in:
@@ -0,0 +1,425 @@
|
||||
# Dashboard, navigation and historical analysis: implementation report
|
||||
|
||||
Final report for [DASHBOARD_ANALYSIS_CHANGE_BRIEF.md](DASHBOARD_ANALYSIS_CHANGE_BRIEF.md) §12. It states which of
|
||||
the brief's findings were fixed and how, how physical and virtual results now agree, which cost and aggregation
|
||||
decisions were applied to existing data, what was tested, and what is still missing.
|
||||
|
||||
- **Base revision:** `c0f52db` (the revision the brief reviewed). **Release:** 0.4.0.
|
||||
- **Decisions:** D-01 – D-58 and amendments A-01 – A-39 in
|
||||
[ANALYSIS_IMPLEMENTATION_NOTE.md](ANALYSIS_IMPLEMENTATION_NOTE.md). Nothing here decides anything new; where a
|
||||
behaviour is deliberate, the decision id says so.
|
||||
- **User-facing changes and the upgrade path:** [RELEASE_NOTES.md](RELEASE_NOTES.md). This report does not repeat
|
||||
them.
|
||||
- **Build state at the end:** `dotnet build` 0 errors; Core.Tests 1,733 of 1,733 and Integration.Tests 746 of 746
|
||||
passing; the golden spreadsheet reconciliation and the seeded bill goldens unmoved by the last fix round.
|
||||
|
||||
## 1. Findings A01 – A14
|
||||
|
||||
One row per finding of the brief's §2. "Pinned by" names the test that would fail if the behaviour returned; a
|
||||
class name without a method means the whole suite covers that area.
|
||||
|
||||
| Finding | What was wrong | What it is now | Where the code lives | Pinned by |
|
||||
|---|---|---|---|---|
|
||||
| **A01** / P0 — virtual meters excluded from meter analysis | `MeterPeriodService.GetAsync` returned `null` for `MeterMode.Virtual`; the meter page showed a notice and sent the user to the flow page. A test pinned the refusal. | A virtual meter is a full analysis subject: the reader expands its dependencies and evaluates it per bucket, returning period total, history, comparison, status, provenance, source contributions and a cost rule — the same shape a physical meter returns. `MeterPeriodService` and its test are deleted. | `src/Core/Analysis/Virtual/VirtualEvaluator.cs`, `src/Infrastructure/Analysis/AnalysisReader.cs`, `src/App/MeterDetails/MeterAnalysisLoader.cs`, `src/App/Components/Pages/MeterPage/MeterAnalysisTab.razor` | `AnalysisReaderTests.A_two_source_generation_sum_reads_like_a_physical_meter`; `MeterAnalysisLoaderTests.A_virtual_sum_of_two_generation_meters_reads_250_and_200_by_month_and_450_in_total` |
|
||||
| **A02** / P0 — "virtual" meant different things per layer | `FlowService` summed every incoming link, `VirtualNormalizer` ran expressions (in tests only), and the editor offered upstream selection with no formula editor. A subtraction could disagree with the flow diagram. | One canonical definition in `Meter.Meta` (`expression`, `referencedMeterIds`, `resultKind`, `resultUnit`, `costRule`), one evaluator, one validator. Links are topology only and never change a calculation (D-25). The Sankey draws a pure sum from its calculation inputs, over the same reader result as the page. `VirtualNormalizer` and `ExpressionEvaluator` are deleted. | `src/Core/Analysis/Virtual/` (`VirtualDefinition`, `VirtualDefinitionJson`, `FormulaParser`, `VirtualValidator`, `DependencyGraph`), `src/App/MeterEditing/`, `src/App/Components/Shared/MeterEditing/`, `src/Infrastructure/Dashboard/FlowService.cs` | `FlowServiceTests.Virtual_sum_meter_is_its_formula`; `VirtualValidatorTests`; `VirtualDefinitionJsonTests`; `MeterEditorLogicTests` |
|
||||
| **A03** / P0 — the virtual read/materialize pipeline was incomplete | Normalization skipped virtual meters and costing read stored consumption, so a virtual meter had neither stored nor computed values; assigning a cost category was not a fix. | Virtual meters are evaluated on read and store nothing (D-27; the migration purges the rows they used to have). Costing resolves them through the same evaluator and prices them by their named rule (D-39, A-15), independently of category membership. | `src/Core/Analysis/Virtual/VirtualEvaluator.cs`, `src/Infrastructure/Analysis/AnalysisRun.cs`, `src/Infrastructure/Costing/BillRun.cs`, `src/Infrastructure/Normalization/NormalizationService.cs` | `VirtualEvaluatorTests`; `CostReaderTests.Virtual_meters_are_costed_by_their_named_rule`; `SchemaTests.The_analysis_tables_are_plain_tables_that_go_with_their_meter` |
|
||||
| **A04** / P0 — costs could count overlapping meters twice | The Overview summed every meter, the energy page summed every meter of the type beside a topology-root throughput, and categories deduplicated ids only inside one category. | One pure classification decides what counts: supply edges, containment, the measures Use / GridImport / Export / Generation / Runtime, and a per-meter `totals` override. The bill prices the grid import meters where a type has them, otherwise its use meters; a separately priced subsection is taken out of its parent. A category prices the non-overlapping cover of its members; an overlapping category is a view and stays out of the composition. | `src/Core/Analysis/Totals/TotalsPolicy.cs`, `TotalsGraph.cs`, `CategoryCover.cs`, `src/Core/Analysis/Costing/CostCalculator.cs`, `src/Infrastructure/Costing/CostReader.cs` | `TotalsPolicyTests` (incl. `The_seeded_topology_classifies_every_meter_as_the_note_pins`); `CategoryCoverTests`; `EnergyTypePageTests.An_overlapping_topology_totals_the_parent_with_the_child_as_a_breakdown`; `SeededBillTests.Standing_charges_and_categories_compose_the_portfolio_bill` |
|
||||
| **A05** / P0 — zero, missing, unpriced and invalid were conflated | History filled absent months with 0 and hid all-zero history; a missing tariff resolved to 0; virtual evaluation substituted 0 for absent sources and for non-finite results. | Every bucket carries a `BucketStatus` (`Available`, `Partial`, `Missing`, `Unresolved`, `Invalid`, `Pending`) derived from coverage runs and their resolution class, never from the amount; provenance, value issue and freshness are separate dimensions. A missing price is `NotPriced` or `PriceGap`, an explicit 0 tariff is a valid zero, a non-finite virtual result is `Invalid` with its reason. The chart draws a true zero as an outlined bar on the baseline and an unknown bucket as a gap marked "–". | `src/Core/Analysis/Coverage/` (`CoverageBuilder`, `CoverageEvaluator`, `CoverageRuns`, `ResolutionClassifier`), `src/Core/Analysis/Costing/CostAmount.cs`, `src/App/Analysis/FigureText.cs`, `src/App/Analysis/AnalysisChartModel.cs` | `CoverageEvaluatorTests`; `CostAmountTests`; `AnalysisChartModelTests.A_true_zero_bar_is_drawn_on_the_baseline_and_a_bucket_without_value_is_marked_as_such`; `AnalysisReaderTests.An_all_zero_year_is_visible_and_a_net_balance_stays_signed` |
|
||||
| **A06** / P1 — inconsistent time ranges | No selector on the Overview, a fixed 12-month strip on the meter page, 24 months with Apply on Trends, 60 months elsewhere, and "all time" meaning 1,200 months. | One URL-borne period contract with one toolbar on every analysis page: `period=mtd|last-month|ytd|prev-year|12m|24m|all|custom` with `from`/`to`, `bucket` and `compare`. The Overview defaults to `mtd`, history pages to `12m` (12 calendar buckets ending with the current partial month); `all` spans the availability metadata. Defaults are never written into the address. | `src/App/Analysis/AnalysisQuery.cs`, `AnalysisDefaults.cs`, `AnalysisPeriods.cs`, `src/Core/Analysis/Time/PeriodResolver.cs`, `src/App/Components/Shared/Analysis/PeriodToolbar.razor` | `AnalysisQueryTests`; `PeriodResolverTests.The_last_12_months_are_twelve_calendar_months_ending_with_the_current_partial_one`; `PeriodResolverTests.All_history_spans_the_available_data_and_stops_at_its_last_day_when_that_is_in_the_past` |
|
||||
| **A07** / P0 — comparison and cut-off semantics differed | The summary asked for full calendar years, the breakdown ended at `asOf.AddMonths(1)`, the difference truncated to whole months, several pages derived "today" from UTC, and the meter quantity SQL had no upper bound. | A period resolves once per request, in the instance zone, into a local inclusive display range and a half-open UTC range `[from, to)` used by quantities, costs, comparisons and the export alike. "Now" comes from the registered `TimeProvider`, read once per page. Comparisons shift in calendar units and are measured only over the coverage both periods share; rows closing after now are excluded from actuals and reported apart. | `src/Core/Analysis/Time/PeriodResolver.cs`, `ComparisonResolver.cs`, `src/Core/Analysis/Coverage/MatchedCoverage.cs`, `src/App/InstanceClock.cs`, `src/App/Analysis/CostChanges.cs` | `PeriodResolverTests`; `ComparisonResolverTests`; `MatchedCoverageTests`; `AnalysisReaderTests.A_comparison_is_confident_only_over_the_coverage_both_periods_share`; `AnalysisReaderTests.Rows_that_close_after_now_are_left_out_and_reported`; `CostConsistencyTests.A_partial_period_states_the_same_matched_cost_change_on_every_page` |
|
||||
| **A08** / P1 — too little history to investigate a change | Trends was one monthly total-cost chart, energy pages had no series at all, and the record tabs showed the latest 200 rows. | `/trends` is a scope × metric exploration page (portfolio, type, category, one meter, or up to six meters); energy types have a History tab with a total or per-meter view; the meter page has a full Analysis tab with chart, table, comparison overlay, projection and coverage. Record tabs page through the whole history, 100 rows at a time, keyset-ordered and filtered by the selected dates. Everything shown exports as CSV. | `src/App/AnalysisPage/`, `src/App/Energy/`, `src/App/Components/Pages/Energy/EnergyHistoryTab.razor`, `src/App/MeterDetails/RecordPager.cs`, `src/Infrastructure/Dashboard/MeterDetailService.cs`, `src/App/Analysis/AnalysisExportEndpoints.cs` | `AnalysisPageSelectionTests`; `AnalysisPageLoaderTests`; `MeterDetailServiceTests.Readings_page_newest_first_by_keyset_and_filter_by_a_half_open_range`; `MeterPageLogicTests.The_pager_walks_keyset_pages_and_back`; `AnalysisExportEndpointTests` |
|
||||
| **A09** / P1 — Overview and Trends disagreed | The summary counted manual costs, the monthly trend did not; "latest month with data" gave an amount without its month and judged recency from consumption alone. | Both read the one bill. Manual costs are booked once, in full, on their `PeriodStart` day, everywhere. "Latest period with data" returns its month and its basis (meter data, manual costs or both), from the union of billed-meter coverage and manual-cost days, capped at now. One cost-change rule (`OverviewComparison.Between`) serves the Overview, the energy page, the Analysis page and the meter page. | `src/Core/Analysis/Costing/CostCalculator.cs`, `src/Infrastructure/Dashboard/OverviewModels.cs`, `src/Infrastructure/Costing/CostReader.cs`, `src/App/Energy/EnergyAnalysisLoader.cs` | `CostReaderTests.Manual_costs_are_booked_once_on_their_start_day_wherever_they_belong`; `CostReaderTests.The_latest_period_with_data_includes_manual_costs`; `OverviewDataTests.The_seeded_previous_year_is_the_sheet_s_bill_and_every_panel_adds_up_to_it`; `CostConsistencyTests` |
|
||||
| **A10** / P1 — navigation mixed analysis with specialized pages | Type links opened a page headed "flow", settings were spread across editor, Sources tab and admin, and a nav database error silently removed the energy-type links. | A fixed sidebar (Overview, Analysis, Meters, Energy types, Specialized views, Data import, Configuration) with persisted groups and a Retry item when the types cannot be loaded. Energy pages are titled with the type's own name and have Overview / History / Flow / Meters tabs. Breadcrumbs Overview → type → meter carry the period; link helpers are the only way links are built. | `src/App/Components/Layout/NavMenu.razor`, `src/App/NavGroups.cs`, `src/App/MeterLinks.cs`, `AnalysisLinks.cs`, `TariffLinks.cs`, `src/App/Analysis/AnalysisNavigation.cs`, `src/App/Components/Shared/Analysis/AnalysisBreadcrumbs.razor` | `AppLinkTests`; `AnalysisNavigationTests.Breadcrumbs_carry_the_period_up_and_end_at_the_current_page`; `ShellPreferenceTests`; `DashboardRenderTests` |
|
||||
| **A11** / P1 — visual semantics varied | The meter history was a hand-built 110 px HTML bar chart using absolute values, and the chart components hard-coded dark mode. | One `AnalysisChart` (ApexCharts) with one axis per unit, nullable points, no smoothing or joining across gaps, a real zero line for signed data, outlined bars and a per-bucket mark, plus an accessible `AnalysisTable` for every chart. Light/dark lives in a scoped `ThemeState` backed by a cookie the app reads at prerender; charts re-key on a theme change. `SeriesChart`, `TrendChart`, `CategoryDonut` and `DeltaChip` are deleted. | `src/App/Components/Shared/Analysis/AnalysisChart.razor`, `AnalysisTable.razor`, `src/App/Analysis/AnalysisChartModel.cs`, `AnalysisChartOptions.cs`, `src/App/Theme/ThemeState.cs` | `AnalysisChartModelTests`; `AnalysisTableModelTests`; `AnalysisComponentRenderTests`; `ShellPreferenceTests.The_theme_defaults_to_dark_and_tokens_round_trip` |
|
||||
| **A12** / P1 — misleading labels and units | Most cost views called `Format.Euro` while the meter page used the configured currency, and `Meter.Unit` was used for normalized period results. | `NormalizedQuantity(meter, tank, definition)` gives every meter its analysis kind and unit (runtime in `h` or the tank unit at a fixed rate, instant rate integrated, the tank unit, a virtual meter's declared unit); `Units` is the only normalizer, and raw units appear only on the Readings tab. Amounts go through `Format.Money` over `InstanceCurrency`; `Format.Euro` is gone and no `€` is written in code or resources. | `src/Core/Analysis/Quantities/NormalizedQuantity.cs`, `Units.cs`, `TariffUnit.cs`, `src/App/Format.cs`, `src/App/InstanceCurrency.cs` | `NormalizedQuantityTests`; `UnitsTests`; `TariffUnitTests`; `FormatCultureTests.Money_is_in_the_configured_currency_written_the_readers_way` |
|
||||
| **A13** / P1 — rapid navigation discarded, no retry | An `_loading` early return dropped later requests, panels had no retry, and nav errors removed links. | Every page commits one value through `LoadSequencer.RunAsync` into a `LoadState<T>`: superseded loads are cancelled and never committed, and a failure keeps the previous value visible while reporting itself. `LoadPanel` renders the initial, refreshing and `PanelError`-with-Retry states; the nav keeps its group with a Retry item on error. Subscriptions are disposed with the circuit. | `src/App/Analysis/LoadSequencer.cs`, `src/App/Components/Shared/Analysis/LoadPanel.razor`, `PanelError.razor`, `RefreshIndicator.razor` | `LoadSequencerTests` (incl. `A_delayed_first_load_cannot_overwrite_the_later_one` and `A_failure_keeps_the_previous_value_visible_and_is_reported`) |
|
||||
| **A14** / P1 — long-history reads scanned `consumption` | Several services aggregated `consumption` directly; the continuous aggregates were Berlin-only, amount-only, never backfilled and read by nothing; flow summed all meters before filtering by type. | Normalization writes per-meter day and month rollups, coverage runs and a rollup state by diff, in the same transaction as consumption. `AnalysisReader` reads the month table for month and year buckets, the day table otherwise, plus at most two partial edge days from `consumption`; limits are checked before any SQL runs. The three continuous aggregates and their jobs are dropped. Measured: 8–16 statements per request whatever the meter count. | `src/Infrastructure/Normalization/AnalysisDataWriter.cs`, `src/Core/Analysis/Rollups/RollupBuilder.cs`, `src/Infrastructure/Analysis/AnalysisQueries.cs`, `AnalysisReader.cs`, `src/Infrastructure/Persistence/Migrations/20260919090259_AnalysisRollups.cs` | `SchemaTests.The_continuous_aggregates_and_their_refresh_jobs_are_gone`; `CostReconciliationTests.Monthly_rollup_equals_the_consumption_it_sums`; `AnalysisReaderTests`; `Performance/ReaderTimingTests` (opt-in) |
|
||||
|
||||
Two of the brief's starting points were kept rather than removed. `Core/Costing/TariffResolver` survives as the
|
||||
legacy resolver behind `CostService`, which is now only the adapter under `/api/v1/consumption|cost`; and
|
||||
`DashboardService.GetMonthlyTrendAsync` / `GetCategoryBreakdownAsync` / `GetCategoryDifferenceAsync` survive as
|
||||
legacy entry points that only tests call. Nothing a page renders goes through either.
|
||||
|
||||
## 2. How physical and virtual results agree
|
||||
|
||||
### 2.1 One read path
|
||||
|
||||
Pages, `/api/v1`, the CSV export, Solar, Consumables, Flow and the Overview read quantities through
|
||||
`AnalysisReader` and money through `CostReader`. There is no second path: a figure that queried `consumption` or
|
||||
`reading` directly would be the only one able to disagree, and none is left. `AnalysisReader` loads its catalog
|
||||
once, expands virtual dependencies **in memory**, then reads each table once for all physical meters involved.
|
||||
A virtual series and a physical series therefore come out of the same rollup rows, the same coverage runs, the
|
||||
same bucket plan and the same comparison.
|
||||
|
||||
### 2.2 The worked example (brief §5.4)
|
||||
|
||||
Generation meters A and B, complete monthly data, evaluated as `m(A) + m(B)`:
|
||||
|
||||
| Month | A | B | Virtual sum A+B |
|
||||
|---|---:|---:|---:|
|
||||
| January | 100 kWh | 150 kWh | 250 kWh |
|
||||
| February | 80 kWh | 120 kWh | 200 kWh |
|
||||
| **Period total** | **180 kWh** | **270 kWh** | **450 kWh** |
|
||||
|
||||
The sum reads 450 kWh for the two months, `Available`, kind `Generation`, unit kWh, with a year bucket of 450, the
|
||||
contributions A +1 × 180 kWh and B +1 × 270 kWh, and the two physical series unchanged on their own pages. The
|
||||
type's Generation measure over the same period is the same 250 / 200 / 450, and the sum is classed `AnalysisOnly`,
|
||||
so it is never added on top of its sources. A separate `m(A) − m(B)` meter reads −50 and −40 kWh, total −90 kWh,
|
||||
kind `Net`, plotted below a real zero line and exported signed: it does not inherit the old flow service's sum.
|
||||
Pinned by `VirtualEvaluatorTests.A_plus_B_is_250_and_200_by_month_and_450_in_total`,
|
||||
`A_minus_B_is_minus_50_and_minus_40_and_stays_signed` and
|
||||
`AnalysisReaderTests.A_two_source_generation_sum_reads_like_a_physical_meter`; reproduced in a running instance
|
||||
during the acceptance review (scenarios V1 and V3).
|
||||
|
||||
### 2.3 Seeded Summe Solar
|
||||
|
||||
The seed now stores the definition directly: `expression = m4 + m5`, `resultKind = generation`,
|
||||
`resultUnit = kWh`, `costRule = none`, `referencedMeterIds = [4, 5]`, with the links 4→9 and 5→9 kept as topology.
|
||||
For 2025 it reads 4,750 kWh, which is Zähler Solar 1 (3,123) + Zähler Solar 2 (1,627), the same figure the Strom
|
||||
type's Generation measure shows month by month. It is not costed, because generation is never billed (D-34, A-15),
|
||||
and the page says so rather than showing a blank. An installation seeded by an earlier version gets the same
|
||||
definition from `VirtualDefinitionUpgrade` at startup: the implied sum of the incoming links is stored when they
|
||||
name sources of one unit and kind; the run is idempotent (meta and `updated_at` of every virtual meter were
|
||||
byte-identical after a second start); an existing explicit expression always wins; and anything ambiguous, mixed
|
||||
or looping is flagged "needs configuration" and logged. Pinned by
|
||||
`NormalizedQuantityTests.Summe_Solar_is_the_generation_it_declares`, `LegacyVirtualDerivationTests`,
|
||||
`VirtualManagementTests` and `TotalsPolicyTests`.
|
||||
|
||||
### 2.4 Joint coverage, missing versus observed zero
|
||||
|
||||
The reader derives each source's per-day coverage and bucket states from `CoverageEvaluator` and hands them to
|
||||
`VirtualEvaluator`; the evaluator never re-derives coverage rules (A-12). From that:
|
||||
|
||||
- **A missing source is unknown, not zero.** With B absent in February, February is `Missing` with
|
||||
`MissingSource` and the dependency path naming B, and the period total is `Partial` 250 — not a confident 80.
|
||||
- **An observed zero is a value.** With B observed as 0 in February, February is a complete 80 and the two-month
|
||||
total is a complete 330.
|
||||
- **Partial and coarser coverage propagate.** A source covering part of a bucket makes it partial with the jointly
|
||||
covered value; a monthly source cut inside its month is `Unresolved`, not a plausible-looking partial; a source
|
||||
still being rebuilt makes every bucket `Pending`.
|
||||
- **A meter outside its install and retire dates contributes a known zero**, while a gap inside its lifetime stays
|
||||
missing (D-24).
|
||||
- **Non-finite arithmetic and loops are explained, never numbers.** A division by zero is `Invalid` with the
|
||||
reason; a cycle is `Invalid` with the loop path, and both ends are named by meter name, never by `#id`.
|
||||
- **A period total is the formula over the joint coverage.** For a linear formula without a constant that equals
|
||||
the sum of its buckets; otherwise the series is marked non-additive and the total is the ratio of totals. An
|
||||
indicator (meter × or ÷ meter) is never additive, never totalled into a measure and never costed.
|
||||
|
||||
Pinned by `VirtualEvaluatorTests` and by
|
||||
`AnalysisReaderTests.A_missing_source_is_unknown_and_an_observed_zero_is_a_value`,
|
||||
`A_difference_stays_negative_and_nested_meters_resolve_once`, `A_dependency_loop_is_named_and_never_a_number` and
|
||||
`A_division_by_zero_is_invalid_and_a_ratio_is_not_additive`.
|
||||
|
||||
### 2.5 Contributions, and what a virtual meter does not get
|
||||
|
||||
Every virtual result carries each source's own series, status and dependency path, which the page renders as
|
||||
"Source meters" (`SeriesContributions`), with the weight the formula gave it and a link on to that source's
|
||||
records. Nested sums appear nested. A virtual meter has no Readings and no Normalized data tab, and the page never
|
||||
offers "add a reading" to fix missing history; it has a Calculation tab instead, showing the formula with meter
|
||||
names beside the `m<id>` tokens and any problem worded exactly as the attention list words it. Where a physical
|
||||
meter's bucket drills into its normalized records, a virtual meter's bucket opens its own analysis over that
|
||||
bucket, from which each source links on (A-25). A virtual meter never holds a role, and it joins a type's totals or
|
||||
the bill only through an explicit `totals = always` override, which is refused — naming the other meter — when an
|
||||
ancestor or a dependent already counts.
|
||||
|
||||
## 3. Cost and aggregation decisions applied to existing data
|
||||
|
||||
The bill is computed by `BillRun` → `CostCalculator` over a `TariffBook`, from the same reader passes as the
|
||||
quantities. What that changed for data that already exists:
|
||||
|
||||
| Decision | Applied as | Pinned by |
|
||||
|---|---|---|
|
||||
| **D-34** Billing set | Per energy type, the grid import meters where it has any, otherwise the use meters. Generation, runtime and virtual views are never billed. | `TotalsPolicyTests.The_seeded_bill_is_the_grid_import_for_electricity_and_household_use_elsewhere`; `SeededBillTests` |
|
||||
| **D-34** Feed-in | Credited only on meters holding the `grid_export` role, at the FeedIn price. A missing feed-in price is an optional credit, reported only where an export meter exists. | `CostReaderTests.Export_is_credited_at_the_feed_in_price_and_a_missing_one_is_an_optional_credit` |
|
||||
| **D-35 / A-19** Separately priced subsection | A containment child with its own meter-scoped unit price is billed at that price, and its monthly quantity is deducted from the ancestor that bills it. Quantities do not change. | `CostReaderTests.A_separately_billed_subsection_is_priced_at_its_own_price_out_of_its_parent`; `SeparatelyBilledSubmeterTests` |
|
||||
| **D-36** Prices per local month | Every bucket is cut into local months and priced at the price valid on the 15th, so a year equals the sum of its months and the bucket size never changes a total. | `CostReaderTests.The_bucket_size_never_changes_a_total_and_every_part_is_priced_in_its_month`; `CostCalculatorPricingTests` |
|
||||
| **A-16** Intervals longer than a month | A multi-month bucket holding an unresolved month is priced whole when every month it has data in shares one price; a price change inside it leaves it unavailable, with an attention item naming the meter and the months. | `CostReaderTests.A_monthly_import_prices_its_month_although_its_days_are_unresolved`; `CostCalculatorCoverageTests` |
|
||||
| **D-40 / A-18** Standing charges | Accrued per local day over the scope's service period, **once per scope**, regardless of reading gaps. Type and global charges are their own rows; meter fees stay on their meter, including on a meter no bill line prices. | `CostReaderTests.A_standing_charge_accrues_once_per_scope_and_a_meter_fee_on_its_meter`; `CostCalculatorStandingChargeTests`; `CostConsistencyTests.The_energy_card_counts_the_meter_fees_on_bill_lines_like_every_other_card` |
|
||||
| **D-41** Manual costs | Booked once, in full, on their `PeriodStart` local day, once that day has come, in every figure that covers it. | `CostReaderTests.Manual_costs_are_booked_once_on_their_start_day_wherever_they_belong`; `CostCalculatorManualCostTests` |
|
||||
| **D-42 / A-22** Categories | A category prices the non-overlapping cover of its members, plus its manual costs. The disjoint categories, Uncategorized and the standing-charge rows form the composition and reconcile to the bill; an overlapping category is a view. A category whose members price nothing says so instead of reading "No data yet". | `CategoryCoverTests`; `SeededBillTests.Standing_charges_and_categories_compose_the_portfolio_bill`; `CostReviewFixTests` |
|
||||
| **D-37 / D-38** Price coverage | A tariff applies only when its unit fits the meter's normalized unit and the instance currency; otherwise `UnitMismatch`. No tariff at any date is `NotPriced` (an attention item, never a silent partial), a hole in a priced history is `PriceGap`, and an explicit 0 is a valid zero. | `CostReaderTests.A_missing_tariff_is_not_priced_and_a_zero_tariff_is_a_valid_zero`, `A_gap_in_a_price_history_makes_those_months_unavailable`, `A_price_in_another_unit_is_a_unit_mismatch_and_a_currency_follows_the_options` |
|
||||
| **D-39 / A-15** Virtual cost rules | `sourceCosts` for pure sums (each physical source once, at what its own scope costs), `ownQuantity` for linear formulas, `none` otherwise and for generation sums. The rule is named next to every virtual cost. | `CostReaderTests.Virtual_meters_are_costed_by_their_named_rule`, `A_virtual_meter_counted_by_an_override_is_billed_by_its_cost_rule` |
|
||||
| **A-17** Months without a grid meter | A month in which no billed grid meter was in service on every day, while a use meter measured something, is unavailable rather than free, with an attention item naming the meter and the months. | `CostCalculatorBillTests`; `CostReaderTests.A_retired_meter_is_a_known_zero_on_the_bill_and_missing_on_its_own_page` |
|
||||
| **A-21 / A-26** Honest cost status | A meter with no cost rule reports `NotPriced` with `costRule` and `notCosted`, never a priced zero; a bucket with nothing booked reads "No data" on pages and `Missing` in the export, never "Priced" beside "—". | `ApiContractTests.A_meter_that_is_not_costed_or_cannot_be_evaluated_never_reads_as_a_priced_zero`; `AnalysisExportEndpointTests.A_month_with_nothing_booked_is_exported_as_no_data_not_as_an_available_priced_blank` |
|
||||
| **D-43** Currency | One `Format.Money` over `MeterVault__Currency`; a tariff in another currency is reported as not fitting, never converted. | `FormatCultureTests`; acceptance scenario C8 (a USD instance shows no € or EUR on any amount) |
|
||||
|
||||
### 3.1 The seeded bill against the spreadsheet
|
||||
|
||||
`SeededBillTests.The_seeded_yearly_bill_equals_the_sheet_s_Jahreskosten` runs on a frozen clock after 2026-05-31,
|
||||
with the oil tank unpriced:
|
||||
|
||||
| Year | Sheet `Jahreskosten` | 0.4.0 | Note |
|
||||
|---|---:|---:|---|
|
||||
| 2022 | 421.52 € | 421.52 € | within ±0.02 € |
|
||||
| 2023 | — | — | differs by 3.78 €: the sheet multiplies by its rounded displayed price |
|
||||
| 2024 | — | — | differs by 0.46 €, same cause |
|
||||
| 2025 | 7,907.64 € | 7,907.65 € | Strom alone is 4,742.64 € = Zähler Netz × price, as the sheet bills it |
|
||||
| 2026 | 2,940.19 € | 2,940.19 € | was 4,402.19 € in 0.3.0, which priced Haus + Netz + Auto and water at 5 €/m³ |
|
||||
|
||||
The 2026 figure is the most visible deliberate change in the release: the old sum of every meter is replaced by the
|
||||
grid import meter, and the seed gained the water price rise to 7.00 €/m³ from 2026-01-01 (D-44). The two
|
||||
differences are documented and not tuned away. The golden consumption reconciliation of all four fixture CSVs, and
|
||||
the Netz Einsparung relation, are unchanged — the latter now computed through `VirtualEvaluator` instead of the
|
||||
deleted normalizer (D-29).
|
||||
|
||||
### 3.2 What an existing installation sees change
|
||||
|
||||
The complete list, with old and new values, is the **Changed figures** table in
|
||||
[RELEASE_NOTES.md](RELEASE_NOTES.md#changed-figures), together with the upgrade steps (the first start rebuilds all
|
||||
analysis data at normalization revision 3, before the web server listens) and the additive REST API changes. It is
|
||||
not repeated here.
|
||||
|
||||
## 4. What was tested
|
||||
|
||||
### 4.1 Suites and counts
|
||||
|
||||
| Suite | Count | Needs | Covers |
|
||||
|---|---:|---|---|
|
||||
| `tests/Core.Tests` | 1,733 | nothing | parsers, normalizers, swap→12, and `Analysis/`: periods, DST, comparisons, buckets, coverage, rollups, totals, category cover, virtual formulas and evaluation, the cost calculator |
|
||||
| `tests/Integration.Tests` | 746 | Docker (TimescaleDB) for the database facts | reconciliation against the four golden CSVs, import commit and revert, ingestion and events, rollups, the reader, the cost engine, the seeded bill, API contracts, the export, rendered pages, plus the pure UI-model tests under `Analysis/`, `MeterPage/`, `Overview/`, `Editor/` and `Specialized/` |
|
||||
| `tests/Integration.Tests/Performance` | 2 | `METERVAULT_PERF=1` | the synthetic 1,000-meter × 10-year dataset, reader and cost timings, statement counts, query plans; skipped by default |
|
||||
|
||||
All green at the end of the last fix round, where Integration rose from 727 to 746.
|
||||
|
||||
### 4.2 Kinds of tests
|
||||
|
||||
- **Frozen-clock period and DST tests.** `PeriodResolverTests`, `ComparisonResolverTests` and `BucketPlannerTests`
|
||||
cover half an hour into New Year, the 23 h spring day and the month containing it, the repeated autumn hour, 29
|
||||
February, 31 March compared with all of February, and New York as a zone behind UTC. `AnalysisReaderTests`
|
||||
asserts the same against real rollups ([24, 24], [23], [25], and [24, 24] in New York).
|
||||
- **Golden fixture reconciliation.** `Reconciliation/` reconciles electricity, water, oil and costs against the
|
||||
four `sampledata/` CSVs, including `GapSplittingIsInertOnFixturesTests` (no fixture interval is divided, in UTC
|
||||
or in Berlin) and `CoverageOfFixturesTests`.
|
||||
- **Seeded bill goldens.** `SeededBillTests` pins the yearly bill against the sheet, the composition, and the
|
||||
manual-cost-only instance; `CostReconciliationTests` pins water Dec 2022 = 14 m³ / 70 € and rollup = consumption.
|
||||
- **Reader and rollup tests.** `AnalysisReaderTests`, `AnalysisDataTests`, `AnalysisCatalogTests`,
|
||||
`RollupBuilderTests`, `CoverageBuilderTests`, `CoverageEvaluatorTests`, `SchemaTests`.
|
||||
- **Cost engine.** `CostCalculator*Tests` (bill, pricing, coverage, standing charges, manual costs),
|
||||
`CostingTariffBookTests`, `CategoryCoverTests`, `CostReaderTests`, `CostConsistencyTests`.
|
||||
- **Virtual evaluator and definitions.** `VirtualEvaluatorTests`, `VirtualValidatorTests`, `FormulaParserTests`
|
||||
(an unknown identifier is an error, never 0), `DependencyGraphTests`, `LegacyVirtualDerivationTests`,
|
||||
`VirtualManagementTests`, `MeterDraftPreviewTests`.
|
||||
- **URL, link, chart and table models.** `AnalysisQueryTests`, `AppLinkTests`, `AnalysisNavigationTests`,
|
||||
`LoadSequencerTests`, `AnalysisChartModelTests`, `AnalysisTableModelTests`, `AttentionItemsTests`,
|
||||
`AnalysisCsvWriterTests`.
|
||||
- **Rendered HTML** through the framework `HtmlRenderer`, in English and German: `AnalysisComponentRenderTests`,
|
||||
`DashboardRenderTests`, `OverviewPageTests`, `AdminPagesRenderTests`, `MeterSourcesRenderTests`.
|
||||
- **API contracts.** `ApiContractTests` pins every existing field and type of `/consumption`, `/cost` and
|
||||
`/dashboard/summary`, the offset-bounds acceptance, and the not-costed cases.
|
||||
- **CSV export.** `AnalysisExportEndpointTests` and `AnalysisCsvWriterTests`: one row per bucket and series, local
|
||||
ISO bounds with offset, invariant numbers, empty cells for unknown values, and a 400 with a reason for anything
|
||||
the endpoint cannot answer.
|
||||
- **Localization.** `StringResourceTests` fails the build on a missing or blank translation, a placeholder
|
||||
mismatch, an orphan or an unreferenced key; `EnumDisplayNameTests` fails on a localized enum value with no
|
||||
wording.
|
||||
|
||||
### 4.3 Acceptance scenarios
|
||||
|
||||
Four independent reviewers walked the brief's §11 scenarios against the built application — each in its own
|
||||
worktree and its own seeded database, driving the browser with Chrome DevTools Protocol scripts and reading tests —
|
||||
and grouped them as costs (C), time and history (T), virtual meters (V), and UI and navigation (U). Status is as
|
||||
recorded at review time; every confirmed finding behind a "partial" was fixed afterwards (§4.4).
|
||||
|
||||
| Id | Scenario | Status | Evidence |
|
||||
|---|---|---|---|
|
||||
| C1 | Overlapping topology: parent 300, child 100 | pass | `EnergyTypePageTests.An_overlapping_topology_totals_the_parent_with_the_child_as_a_breakdown`; live, a linked parent/child pair reads 300 kWh on the type page, the Overview card, `/trends`, the portfolio measure and the CSV, never 400, with the child shown as a breakdown |
|
||||
| C2 | Missing versus free tariff, and the Add-tariff deep link | partial | `CostReaderTests` (not priced / price gap / valid zero) pass; live, the oil tank reads "Not priced (no tariff)" while its 2,100 L stays visible, and the link opens the tariff dialog prefilled with scope, component and first uncovered month. Finding F1: the dialog could be saved with no value, turning a gap into a free period |
|
||||
| C3 | Cost stability: manual costs once, standing charges once, bucket-invariant totals | pass | Five cost tests plus `SeededBillTests`; live, seeded 2025 totals 7,907.65 € (metered 5,807.64 + manual 2,100.01) at every bucket size, on the Overview, on `/trends` and in the CSV |
|
||||
| C4 | Historical-only instance | pass | With `period=mtd` every page says "No data for this period", names the available dates and offers "Go to latest data"; the Overview names "Latest month with data: May 2026 (Meter data and manual costs)"; `prev-year` shows physical and virtual quantities and costs |
|
||||
| C5 | Manual-cost-only instance | pass | `SeededBillTests` and `OverviewDataTests` agree; live, on an empty database with three manual costs, the Overview, Analysis, the category scope and the latest month all read the same figures without a single meter |
|
||||
| C6 | Phase 4 exit: scopes reconcile across cards, charts, tables and exports | partial | Overview 7,907.65 €, Strom 4,742.64 €, the energy card, `/trends` (portfolio, type, category), the chart series read through CDP and the CSV all match row by row; quantities stay visible where cost is unavailable. Findings F2 (the energy card understated standing charges), F5 and F6 (chart and CSV wording) |
|
||||
| C7 | Discoverability: Overview → type or breakdown row → scoped analysis; missing cost → tariff editor | partial | Links carry period, bucket and comparison; category rows land on the matching figure; the Add-tariff action opens once and is then dropped from the address. Findings F1 and F3 (a currency mismatch worded as a unit problem) |
|
||||
| C8 | Currency | pass | With `MeterVault__Currency=USD` no page shows € or EUR on an amount in either language; the only EUR text is user-entered tariff units |
|
||||
| T1 | Local-time boundaries, half-open bounds | pass | 232 Core and 90 Integration tests in the reviewer's worktree; frozen-clock coverage of New Year, both DST changes, 29 February, short months and New York; the SQL is `>= from AND < to` throughout `AnalysisQueries` |
|
||||
| T2 | Future rows and partial periods | pass | Readings pushed at 23:00 today and six days ahead are excluded from a 363 kWh month to date, reported as "values dated after now … are not counted yet" with a link to those records, and comparisons state the matched dates |
|
||||
| T3 | Zero and negative history, percentage rules | pass | A register that never moved gives 12 `Available` zeros and a 0 total; a net series stays signed; a zero or negative baseline gives "percentage not applicable" with the absolute difference still shown |
|
||||
| T4 | Monthly legacy resolution | partial | Day buckets over monthly data are 31 `Unresolved` rows with the month total intact — no spike, no zeros — and `auto` never plans finer than the data. Findings F1 (type measures drilled a month into empty days), F3 and F4 |
|
||||
| T5 | Retained history | documented blocker | Raw retention is not enforced (D-57). `/admin/settings` and the Readings tab say so, asserted in English and German by `AdminPagesRenderTests` |
|
||||
| T6 | Import, correction and freshness without duplicates | pass | A throwaway test drove swap, manual-reading deletion, tariff edit, definition edit and an import commit through the reader: each shows in the next read, for the physical meter and for the virtual meter above it, with no duplicates and no cache |
|
||||
| T7 | Compare two historical years; explain a spike; paged records | partial | Virtual 2024 against 2023 reads 4,778 versus 4,583 kWh with an overlay, comparison table and source contributions; the physical path drills a March bar into that month's records. Findings F2 (a virtual bucket was a dead end) and F5 |
|
||||
| T8 | Phase 3 exit: Overview → type → meter → month and back | partial | The whole path was driven with CDP: breadcrumbs and Back return through each step with the dates intact. Findings F1 and F2 |
|
||||
| V1 | Two-source virtual generation sum | pass | 250 / 200 / 450 kWh, generation, kWh, contributions, type history agreeing; seeded Summe Solar 2025 = 4,750 kWh = 3,123 + 1,627 |
|
||||
| V2 | Missing versus zero source | pass | Missing B makes February `Missing` with the source named and the total `Partial` 250; an observed zero gives a complete 80 and a complete 330 total; the API and the CSV agree |
|
||||
| V3 | Difference and nesting; named cycle error | partial | −50 / −40 below a real zero line, total −90 kWh; a nested sum resolves its leaves once; a cycle is `Invalid` with the loop path everywhere. Finding F1: two views named the meter as `#id` |
|
||||
| V4 | Invalid arithmetic and non-additive rollup | partial | Division by zero gives "—" with "The formula has no finite result", the ratio total is the ratio of totals with a non-additive note, and the CSV writes an empty value with status `Invalid` — never 0 or infinity. Finding F4 (no reason column in the CSV) was refuted as outside D-55 |
|
||||
| V5 | Legacy virtual configuration | pass | The seed stores the definition; an expression-less meter is converted at startup and logged; a rerun changes nothing (byte-identical meta); an existing formula wins over links; adding links afterwards changes no calculation |
|
||||
| V6 | Virtual overlap | pass | Sources 100 + 150 and the virtual 250 stay 250 in the portfolio; with "always" the sum replaces its sources; the seeded bill and the Strom cost do not move |
|
||||
| V7 | Inspect and configure a combined meter (EN and DE) | pass | Search → the meter's Analysis tab with the period carried and source contributions shown; the editor offers Sum by default, sources by name with unit, kind and dates, and a live preview |
|
||||
| V8 | Phase 2 exit | pass | Every worked meter — sum, difference, nested, missing, zero, ratio, cycle — has the full Analysis tab without raw readings or categories; new calculated meters default to analysis-only; 145 filtered Integration and 342 Core tests |
|
||||
| U1 | Rapid filter and navigation changes, URL state | pass | `LoadSequencerTests` plus CDP runs switching type, meter and period within 30 ms: the last request wins every time; toolbar and tab changes replace, drill-downs push |
|
||||
| U2 | Compatibility actions and legacy tab keys | pass | Every legacy `?tab=`/`?action=` URL opened the intended tab and dialog once and then dropped `action` from the address, including `tab=consumption` → Normalized data and a virtual meter's `tab=sources` → Calculation |
|
||||
| U3 | Theme, EN/DE, narrow screens, keyboard, tables | partial | The in-circuit theme toggle re-keys every chart; 23 resource tests pass; no page-wide overflow on 33 URLs at 360 px in German. Finding F3: the keyboard focus indicator was near-invisible |
|
||||
| U4 | Sidebar, persisted groups, search, breadcrumbs | pass | The fixed structure renders as specified, the cookie survives a reload, the current route's group opens, the active item is visible, and search carries the period |
|
||||
| U5 | Discoverability paths | partial | Quick entry from the header, the list and search; Flow → Manage connections names both ends; calendar-year comparison from the toolbar. Findings F5 (no way to edit the connector in use) and F6 (refuted) |
|
||||
| U6 | Definition of done in the running app | partial | Checked at 1440 and 390 px, English and German, light and dark: the virtual meter has totals, comparison, table, export, contributions and its Calculation tab; a true zero reads "0 kWh Complete" against "— No data". Findings F1, F2 and F4 |
|
||||
|
||||
### 4.4 The review-and-fix round
|
||||
|
||||
The four reviewers raised 24 findings. Each was re-checked by an independent verifier before anything was changed:
|
||||
20 were confirmed, three were refuted as deliberate design (C-F4, the "What changed" table omitting rows with no
|
||||
figure on either side; V-F4, the CSV column set, which D-55 fixes; U-F6, sidebar entries opening at their own
|
||||
default, which D-48 and D-02 intend), and one (T-F4, `cost_status` in the export) was left uncertain and is covered
|
||||
by the fix for C-F6. The fix agent reports all 19 findings it took on as fixed — T-F2 and U-F4 were the same
|
||||
problem and took one fix — with the build green and the golden figures unmoved. The decisions those fixes needed
|
||||
are recorded as A-21 – A-30 in §12 of the implementation note; A-31 – A-39 in §13 record what the page agents and
|
||||
the integration decided while building.
|
||||
|
||||
Tests added in that round: `ApiContractTests` (not-costed and unevaluable meters), `CostReviewFixTests`,
|
||||
`CostConsistencyTests` (the same cost change on four pages, and matching standing charges) and
|
||||
`MeterSourcesRenderTests`, plus cases in `AnalysisPageLoaderTests`, `MeterAnalysisLoaderTests`,
|
||||
`MeterDraftPreviewTests`, `AttentionItemsTests`, `AnalysisChartModelTests`, `AnalysisNavigationTests`,
|
||||
`AnalysisTableModelTests`, `AnalysisExportEndpointTests`, `TariffEditingTests` and `MeterPageLogicTests`.
|
||||
|
||||
Tests rewritten on purpose across the whole rework — none weakened, each rewrite stating the new rule — are listed
|
||||
at the end of §10 of the implementation note.
|
||||
|
||||
## 5. Performance
|
||||
|
||||
Measured with the opt-in Performance trait (`METERVAULT_PERF=1`,
|
||||
`dotnet test tests/Integration.Tests -c Release --filter "FullyQualifiedName~Performance.ReaderTimingTests"`, about
|
||||
10 minutes) against a deterministic synthetic instance: 1,000 meters over 10 years, ≈1.34 M readings, 7 energy
|
||||
types, 21 links, 109 tariff rows, 64 manual costs and 20 virtual meters nested up to three levels.
|
||||
|
||||
Hardware: Ryzen 9 9950X3D (16 cores, 32 threads), 64 GB, Windows 11, Docker Desktop, PostgreSQL 16.6 with
|
||||
TimescaleDB 2.17.2 (`jit=off`), .NET 10 Release. **These numbers are preliminary**: they were taken while other
|
||||
agents were building and testing on the same machine (host 20–40 % CPU). They are re-runnable as above.
|
||||
|
||||
### 5.1 Reader and cost requests
|
||||
|
||||
Median and p95 of 10 runs after 2 warm-ups; SQL is the number of commands one extra, untimed run sent.
|
||||
|
||||
| Request | Median ms | p95 ms | SQL |
|
||||
|---|---:|---:|---:|
|
||||
| 100 selected meters, 10 years by month, catalog cached — **the brief's 2 s target** | 374 | 509 | 8 |
|
||||
| Portfolio, last 12 months by month (measures only, as the Overview) | 268 | 284 | 12 |
|
||||
| … with one series per meter (1,000 series) | 379 | 403 | 12 |
|
||||
| … with the previous-year comparison | 414 | 458 | 14 |
|
||||
| One energy type (402 meters), 10 years by month | 313 | 347 | 12 |
|
||||
| … with one series per meter (the type page's table) | 917 | 1,016 | 12 |
|
||||
| One meter: 10 years by week, or 365 days by day | 29–36 | 32–58 | 11 |
|
||||
| Portfolio bill, 12 months by month, with categories | 320 | 357 | 16 |
|
||||
| Portfolio bill, 10 years by month | 838 | 875 | 15 |
|
||||
| Virtual meter nested three levels, 10 years by month | 245 | 272 | 11 |
|
||||
| Virtual difference of a 40-meter and an 8-meter sum, 10 years by month | 377 | 442 | 11 |
|
||||
| Catalog load: 1,000 meters, tanks, links, states, validation, classification | 7 | 8 | 4 |
|
||||
| 1,000-meter selection with the limit raised, 12 months by month | 353 | 394 | 12 |
|
||||
| Refused: 1,000 meters against the 6-series limit; portfolio or bill by day over 10 years | 0 | 0 | **0** |
|
||||
|
||||
The brief's target is met with margin: 374 ms median and 509 ms p95 against 2,000 ms. The statement count per
|
||||
request is constant at 8–16 whether the request covers 1, 100 or 1,000 meters — there is no per-meter query storm.
|
||||
Refused requests send no SQL at all, because the 400-point and 6-series limits are checked before any query is
|
||||
built. Rollup reads use the primary-key index: 4 ms for 125 meters over 10 years of month rollups, 7 ms for 980
|
||||
meters over 11 months.
|
||||
|
||||
### 5.2 Rebuild and per-reading recompute
|
||||
|
||||
Rebuilding all 1,000 meters at startup through `NormalizationUpgrade` took 344–392 s: about 0.1 s per monthly
|
||||
meter, ~0.6 s for a daily meter (3,670 readings) and ~1.4 s for a meter with a year of hourly data (9,314
|
||||
readings). That is also the cost of one ingested reading, because every write path recomputes its meter in full
|
||||
(D-57).
|
||||
|
||||
### 5.3 Page loads, old against new
|
||||
|
||||
The same synthetic raw data was loaded into a database migrated by the old app (`c0f52db`) and one migrated by the
|
||||
new app; each rebuilt its own derived data, then prerendered GETs were timed (median of 10), with the statements
|
||||
per GET taken from a separate logging pass.
|
||||
|
||||
| Page | 0.3.0 median ms | new median ms | SQL per GET |
|
||||
|---|---:|---:|---|
|
||||
| `/` Overview | 28,175 | 2,384 | 48,244 → 205 |
|
||||
| `/trends` | 5,101 | 1,290 | 6,004 → 30 |
|
||||
| `/energy/1` | 3,244 | 1,136 | 2,426 → 42 |
|
||||
| `/meters` | 222 | 163 | 4 → 8 |
|
||||
| `/solar` | 92 | 18 | 32 → 20 |
|
||||
|
||||
The old Overview ran a meter load, a tariff load and a `time_bucket` query 8,021 times per request. The individual
|
||||
meter pages and `/consumables` stayed in the same range (tens to low hundreds of milliseconds) and are not listed.
|
||||
The new-side page numbers were taken on the foundation-era pages, before the page rework landed, so they are an
|
||||
order of magnitude rather than a final figure.
|
||||
|
||||
### 5.4 Hotspots found and not fixed
|
||||
|
||||
1. **The freshness query has no time bound.** `AnalysisQueries.RecentReadingsAsync` (the latest 20 reading times
|
||||
per meter, D-18) plans across every raw chunk: 14–24 ms of planning over 123 `reading` chunks, 189 ms on a cold
|
||||
connection, and 38–46 ms of execution for a portfolio walking back through compressed chunks. With a constant
|
||||
90-day lower bound the same single-meter statement plans in 0.9 ms. It grows with history length.
|
||||
2. **Window sums get no plan-time chunk exclusion.** `AnalysisQueries.WindowSumsAsync` takes its bounds only from
|
||||
the `unnest` join. Real requests send few windows and cost 4.6 ms, but a stress case of ~2,000 windows flips to
|
||||
a parallel sequential scan of 1.39 M rows with a sort spilling to disk, 138–152 ms. Latent, and it grows with
|
||||
total history rather than with the number of windows.
|
||||
3. **The reworked Overview computes several portfolio bills per request** — nine in the measured build — which
|
||||
shows as repeated catalog, tariff and manual-cost loads (526 ms of SQL inside a 2.4 s page).
|
||||
|
||||
In-process work dominates the rest and scales with series × buckets (roughly 10–16 µs per series bucket). That is
|
||||
the budget that matters for very large exports and tables, not for a normal page.
|
||||
|
||||
## 6. Remaining limitations and known gaps
|
||||
|
||||
**Documented in D-57:**
|
||||
|
||||
- **Raw retention is not enforced.** `MeterVault__RawRetentionDays` is displayed but nothing deletes readings,
|
||||
because every recompute rebuilds a meter from the readings that remain. `/admin/settings` and the Readings tab
|
||||
say so. This is the brief's "Retained history" acceptance scenario, and it stands as a documented blocker.
|
||||
- **Monthly imports are never interpolated to days.** A day or week view of monthly data says "only coarser data"
|
||||
and offers the monthly interval; it never invents measured daily detail.
|
||||
- **A full recompute runs per ingested reading.** Fine for monthly and daily meters (0.1–0.6 s); about 1.4 s for a
|
||||
meter with a year of hourly data, and it grows with history.
|
||||
- **Bonus, Discount and Tax tariffs are stored but not applied.** The tariff editor says so.
|
||||
|
||||
**Further limitations, measured or decided during the work:**
|
||||
|
||||
- The three performance hotspots in §5.4 are known and unfixed.
|
||||
- The billing basis (grid meter or household use) is chosen per energy type for all time and cannot switch month by
|
||||
month, because the category composition would need the same per-month basis to stay reconciled with the bill
|
||||
(A-17).
|
||||
- Batteries are not modelled: without a grid-export meter, Solar's feed-in is calculated, and labelled as such.
|
||||
- The Solar page has no CSV export, because the export has no derived measures.
|
||||
- An old explicit difference formula over two generation meters stores `meter_rollup_state.kind` as `Consumption`.
|
||||
Nothing reads that column, and the meter's declared kind governs every figure.
|
||||
- `Dashboard.razor` and `Trends.razor` subscribe to `LocationChanged` without checking that the location is still
|
||||
their own page, so leaving them can start a load on a disposing component. No error from it appeared in the
|
||||
application log.
|
||||
- The navigation's retry-on-error path for energy types is code-reviewed only: no automated test covers it, and it
|
||||
could not be forced in a running instance.
|
||||
|
||||
**Testing gaps:**
|
||||
|
||||
- **No bUnit and no Playwright** (a committed stack decision). Browser behaviour — interactive ApexCharts updates,
|
||||
browser history, responsive layout, keyboard focus — was checked with Chrome DevTools Protocol scripts against
|
||||
seeded instances by the page agents and the four acceptance reviewers, in English and German, light and dark, at
|
||||
1440 / 390 / 360 px. Those scripts and their raw screenshots live outside the repository. Inside it,
|
||||
server-rendered pages are covered by `HtmlRenderer`-based render tests in both languages.
|
||||
- For three of the last-round fixes (naming a culprit meter, and the virtual-bucket dead end) the tests were
|
||||
written after the fix, so they were never watched to fail; the reviewers' reproductions are the "before".
|
||||
- The "After now" mark on record rows is covered only by a test: the seeded demo holds no rows dated after now, so
|
||||
it could not be seen in a browser.
|
||||
- The performance figures are preliminary (§5); re-run them on a quiet machine before quoting them as a property.
|
||||
|
||||
**Deviations from the SDD** are listed in D-58 and marked in place in [`SDD.md`](SDD.md). The largest is §14.1:
|
||||
virtual meters are computed on read and nothing is materialized.
|
||||
|
||||
## 7. Screenshots
|
||||
|
||||
Taken from a seeded instance. "Before" is `c0f52db`; the rest is 0.4.0.
|
||||
|
||||
| Image | What it shows |
|
||||
|---|---|
|
||||
| [`screenshots/analysis/before-overview-desktop.png`](screenshots/analysis/before-overview-desktop.png) | The 0.3.0 Overview: no period selector, totals that summed every meter, and a cost card that needed cost categories configured before it said anything |
|
||||
| [`screenshots/analysis/before-meter-virtual-desktop.png`](screenshots/analysis/before-meter-virtual-desktop.png) | The 0.3.0 meter page for a virtual meter: the "virtual" notice and a link to the flow page instead of any history |
|
||||
| [`screenshots/analysis/overview-desktop.png`](screenshots/analysis/overview-desktop.png) | The new Overview at desktop width: the period toolbar, the cost split into metered use, standing charges, manual costs and feed-in credit, one card per energy type in its own unit, the history chart, "What changed", the cost composition and the attention items |
|
||||
| [`screenshots/analysis/overview-mobile.png`](screenshots/analysis/overview-mobile.png) | The same page at phone width: the toolbar and cards wrap, and wide tables scroll in their own region |
|
||||
| [`screenshots/analysis/meter-physical-desktop.png`](screenshots/analysis/meter-physical-desktop.png) | A physical meter's Analysis tab: tabs directly under the header, the period total with its status, the cost with its rule, the change against the comparison period, the chart with the previous-year overlay, the table, and "Data quality and coverage" |
|
||||
| [`screenshots/analysis/meter-virtual-desktop.png`](screenshots/analysis/meter-virtual-desktop.png) | Summe Solar in 0.4.0: the same analysis as a physical meter, "Not costed — generation is never billed", the "Source meters" contributions, and a Calculation tab in place of Sources |
|
||||
| [`screenshots/analysis/meter-virtual-mobile.png`](screenshots/analysis/meter-virtual-mobile.png) | The virtual meter page at phone width, in German |
|
||||
| [`screenshots/analysis/energy-history-desktop.png`](screenshots/analysis/energy-history-desktop.png) | An energy type's History tab: one toolbar above the four tabs, the total or per-meter view with an explanation of how each meter counts, chart plus table, and Export CSV |
|
||||
| [`screenshots/analysis/energy-history-mobile.png`](screenshots/analysis/energy-history-mobile.png) | The same tab at phone width |
|
||||
|
||||
## 8. Related documents
|
||||
|
||||
| Document | What it holds |
|
||||
|---|---|
|
||||
| [`DASHBOARD_ANALYSIS_CHANGE_BRIEF.md`](DASHBOARD_ANALYSIS_CHANGE_BRIEF.md) | The work order: findings A01 – A14, the target journeys, the acceptance scenarios, the definition of done |
|
||||
| [`ANALYSIS_IMPLEMENTATION_NOTE.md`](ANALYSIS_IMPLEMENTATION_NOTE.md) | Every decision (D-01 – D-58) and amendment (A-01 – A-39), the deliberate behaviour changes, the evidence and the limitations |
|
||||
| [`RELEASE_NOTES.md`](RELEASE_NOTES.md) | What a user sees change in 0.4.0: the upgrade path, the changed figures, the API changes |
|
||||
| [`SDD.md`](SDD.md) | The design reference, with each deviation marked in place |
|
||||
| [`../CLAUDE.md`](../CLAUDE.md) | The architecture as it now is |
|
||||
Reference in New Issue
Block a user