FinalFactory/MeterVault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Analysis: one selected period, one set of numbers, on every page
ci / build-test (push) Successful in 2m31s
Details

Browse Source 
The dashboards told several stories at once. Overview asked for full
calendar years, meter detail for a fixed 12-month window that was really
13, Trends for 24 months with an Apply button, and the energy pages for
60. Each page derived "today" from UTC, so the first hours of a local day
belonged to yesterday. A missing tariff, a month nobody measured and a
genuine zero all rendered as 0. And a virtual meter -- the one thing the
spreadsheet leans on hardest -- was excluded from analysis outright:
MeterPeriodService returned null for it and the page offered a flow
diagram instead.

docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it
left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58
plus amendments A-01..A-30; code, tests and release notes cite those ids.

The analysis layer

Core/Analysis holds the pure rules: period presets resolved once in the
instance zone into a local date range and a half-open UTC range, bucket
plans, calendar-unit comparisons, coverage runs with a resolution class,
normalized quantities and units, the totals policy, the virtual formula
parser/validator/evaluator, and the cost calculator. "Now" comes from
TimeProvider; services never read the clock.

Normalization now writes, in the same transaction as consumption and by
diff, per-meter rollups by local day and month plus coverage runs and a
rollup state (AnalysisDataWriter). AnalysisReader answers a request from
those tables -- month rollups for month and year buckets, day rollups
otherwise, at most two partial edge days from consumption -- and
CostReader prices the result month by month. Pages, /api/v1 and the CSV
export read nothing else. The unused continuous aggregates are dropped.

The reader's statement count per request is constant whether it covers one
meter or a thousand. On a synthetic 1,000-meter, ten-year instance the
brief's target request (100 meters, ten years, monthly) takes 374 ms
against a two-second target, and the Overview went from 48,244 SQL
statements per load to 205.

Missing is not zero

Every bucket carries a status -- available, partial, missing, unresolved,
invalid, pending -- derived from coverage, never from the amount, with
provenance and a reason code beside it. A true zero is a number and a bar
on the baseline; an unknown bucket is a gap that says why; a month whose
data only exists monthly says so instead of inventing daily detail; a
scope with no tariff says "not priced" instead of 0. Rows whose interval
closes after now are reported separately rather than counted.

Virtual meters are analysis subjects

A virtual meter stores a canonical definition -- expression over m<id>
references, result kind, unit and cost rule -- validated on save and on
read for syntax, unknown or self references, loops and unit/kind rules.
It is evaluated on read from its sources' rollups over their joint
coverage: a missing source makes the bucket missing, an observed zero is
a valid input, a non-finite result is invalid with its dependency path,
and the page lists each source's contribution. Topology links are
topology only and never rewrite a saved calculation; expression-less
meters from older installs are converted once at startup. The editor has
Sum, Difference and Advanced modes with a live preview.

Totals and the bill

Per energy type the totals policy separates use, grid import, export,
generation and runtime, marks breakdown meters as breakdowns and virtual
meters as views, and never adds across units. The bill follows it: grid
import where there is one, separately priced subsections at their own
price, feed-in only on export meters, standing charges once per scope per
local day, manual costs once on their start day, categories as
non-overlapping covers whose composition reconciles to the bill. The
seeded demo's yearly totals now match the spreadsheet.

Pages and navigation

The period lives in the URL and every page reads the same contract, so a
link, a reload and the browser's Back button keep it. Shared components
carry it: page header with breadcrumbs, period toolbar, theme-aware chart
with an accessible table beside it, metric cards, comparison and
availability states, attention items that each link to the one action
that fixes them. Meter detail leads with an Analysis tab and resolves its
tabs by key; the energy page has Overview, History, Flow and Meters; the
old cost-only Trends page is a general Analysis page over portfolio, type,
category, meter or a meter comparison. Records tabs are paged server-side
instead of showing the latest 200. Everything is English and German,
light and dark, down to 360px.

Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and
what the first start after the update does (it rebuilds all analysis data
before the web server listens). docs/SDD.md and CLAUDE.md describe the
system as it now is.

Tests: 1,733 Core and 746 integration, all green, plus an opt-in
performance suite with a synthetic 1,000-meter generator.
This commit is contained in:
Florian Schmidt
2026-09-20 10:29:13 +02:00
parent c0f52dbb6f
commit 8940ef25c3
384 changed files with 82753 additions and 4518 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SDD.md
+569 -5
View File
@@ -5,6 +5,13 @@
> **What this is:** a self-hosted, local-first energy & utility metering platform that pulls meter data from Home Assistant, Tasmota and MQTT on a schedule, stores every reading with a timestamp, and turns it into cost dashboards. Not limited to electricity/water/oil — energy types are user-defined.
>
> **Status:** design spec, pre-code. This document doubles as the build brief for Claude Code.
>
> **Implementation status (0.4.0):** M0–M7 are implemented, and the dashboard/analysis rework
> ([`DASHBOARD_ANALYSIS_CHANGE_BRIEF.md`](DASHBOARD_ANALYSIS_CHANGE_BRIEF.md)) replaced the aggregation, virtual-meter,
> cost and page model. Its decisions are numbered D-01 – D-58 and A-01 – A-39 in
> [`ANALYSIS_IMPLEMENTATION_NOTE.md`](ANALYSIS_IMPLEMENTATION_NOTE.md). The original design text below is kept.
> Wherever the system now works differently, a **Deviation** or **Current behaviour** block says so in place (list:
> D-58). The results are in [`ANALYSIS_REPORT.md`](ANALYSIS_REPORT.md).
---
@@ -104,6 +111,14 @@ Early rows (1997–2004) only carry **deliveries** (no burner hours — tracking
| FR-18 | **Deploy** via Docker Compose (app + TimescaleDB); Unraid template; healthcheck endpoint; backup guidance. |
| FR-19 | **Auth**: optional local accounts *and* reverse-proxy trust (honour `X-Forwarded-User` behind Authelia/Traefik). |
> **Where the implementation differs (D-58):**
> - **FR-9:** prices are monthly (the 15th of each local month); there is no day-accurate proration mode. Bonus,
> discount and tax are stored but not applied (§7.5, D-57).
> - **FR-11:** rollups are by local day and month. Weeks and years are built from them, and there is no hourly level
> (§5.4).
> - **FR-12:** the Overview shows one selected period instead of fixed today/month/year cards (§8.1).
> - **FR-16:** raw retention is shown but not enforced (§5.5).
---
## 4. Architecture & tech stack
@@ -125,6 +140,10 @@ Early rows (1997–2004) only carry **deliveries** (no burner hours — tracking
> The **domain model and DB schema are UI-agnostic.** If a future maintainer swaps Blazor for an SPA, everything from §5–§7 and the REST API in §9 is reusable.
> **Deviation (D-17, D-58):** continuous aggregates are not used. They were dropped in favour of rollup tables that the
> recompute writes (§5.4). Dapper serves the hot-path reads of those tables (`AnalysisQueries`). No hosted service
> refreshes aggregates.
### 4.2 Components & data flow
```
@@ -146,6 +165,13 @@ Early rows (1997–2004) only carry **deliveries** (no burner hours — tracking
Ingestion writes **raw `reading`** rows. A normalization step derives **`consumption`** (append-only, base unit). Continuous aggregates roll consumption up to hourly/daily/monthly/yearly. The cost engine joins aggregates with time-ranged tariffs. The dashboard and API read aggregates + cost views (never scan raw for charts).
> **Current pipeline (D-12 – D-17, D-27, D-34):** the recompute derives `consumption`. In the same transaction it
> writes per-meter rollups by local day and local month, plus coverage runs, all in the configured zone (§5.4). One
> shared **analysis reader** (`AnalysisReader`) reads those rollups. It evaluates virtual meters on read (§7.4) and
> classifies per-type totals. One **cost engine** (`CostReader`) prices the bill from those quantities and the
> time-ranged tariffs (§7.5). The pages, the REST API and the CSV export read only these two readers. None of them
> scans `reading`.
---
## 5. Data model & database
@@ -168,6 +194,18 @@ Design principles: raw readings are immutable audit truth; everything derived (c
- `import_batch` — provenance + revert for CSV/manual bulk loads.
- `app_setting` — currency, locale, timezone, retention, fallbacks.
Added since this sketch (see §5.4 and the note, D-12):
- `meter_link` — directed flow topology (`from → to`: the downstream meter is a subsection of the upstream one;
several parents are allowed). It is topology only: it never defines a virtual meter's calculation (D-25).
- `consumption_rollup` / `consumption_rollup_month` — per-meter sums of `consumption` by local day and local month,
with per-provenance amounts, row count, flags and the latest interval end.
- `meter_coverage` — per-meter coverage runs with their resolution class.
- `meter_rollup_state` — the revision, zone, normalized unit and kind each meter's analysis rows were built with.
> **Deviation (D-57):** the principle "long-horizon retention is served by aggregates, not by keeping every raw row
> forever" is not realised. Raw readings are kept indefinitely (§5.5).
### 5.2 Measurement modes (`meter.mode`)
| Mode | Meaning | Consumption derived by |
@@ -183,6 +221,10 @@ Design principles: raw readings are immutable audit truth; everything derived (c
### 5.3 Schema sketch (PostgreSQL + TimescaleDB)
> Illustrative DDL; EF Core migrations own the relational tables, and **raw-SQL migrations** own the Timescale-specific DDL (`create_hypertable`, compression, continuous aggregates, retention). Timescale objects are *not* expressible through EF's model builder.
>
> *Current:* the Timescale DDL in use is the two hypertables (`reading`, `consumption`) and raw compression. The
> continuous aggregates were dropped by the `AnalysisRollups` migration (D-17), and no retention policy exists (D-57).
> The rollup, coverage and state tables are ordinary EF-owned tables with `ON DELETE CASCADE` to `meter` (D-12).
```sql
CREATE TABLE energy_type (
@@ -353,6 +395,48 @@ CREATE TABLE app_setting (
### 5.4 Continuous aggregates & cost view
> **Deviation (analysis rework, `docs/ANALYSIS_IMPLEMENTATION_NOTE.md` D-10 – D-17, D-36, D-58):** the continuous
> aggregates below were Berlin-only and materialized-only. They were never backfilled after historical imports, and
> no reader used them. The `AnalysisRollups` migration dropped them together with their refresh policies. The SQL
> below is kept as the original design. What replaced them:
>
> - **Rollup tables** (plain tables, FK to `meter` with `ON DELETE CASCADE`), all keyed by local dates of the
> **configured** zone:
>
> | Table | Key | Holds |
> |---|---|---|
> | `consumption_rollup` | `(meter_id, day, kind)` | amount; measured/manual/imported/estimated shares; row count; flags (baseline delta, divided); latest interval end |
> | `consumption_rollup_month` | `(meter_id, month, kind)` | the same per local month (read for month and year buckets) |
> | `meter_coverage` | `(meter_id, span_from)` | coverage runs: span, resolution class (≤ 1 h, ≤ 1 day, ≤ 7 days, ≤ 1 month, coarser), whether divided at months, gap reason, last interval start |
> | `meter_rollup_state` | `(meter_id)` | normalization revision, zone, normalized unit and kind, build time |
>
> - **Written by the recompute, by diff, in its transaction.** `NormalizationService.RecomputeMeterAsync` stages them
> through `AnalysisDataWriter` in the same transaction as the meter's `consumption`. Only changed rows are touched.
> Import, revert, events, manual readings, live ingestion and meter edits all reach the next read without a refresh
> job. No cache sits in between.
> - **Intervals.** Every normalized row carries its source interval (EF-ignored, D-10), so coverage is captured while
> normalizing instead of being guessed from sums. A non-label row that ends exactly on a local midnight is stamped one
> second earlier, inside the day it closes (D-11).
> - **Bucket status** (D-14) comes from coverage, never from the amount:
> - `Available`: covered. A zero is a true zero.
> - `Partial`: only partly covered.
> - `Missing`: nothing covers the bucket.
> - `Unresolved`: covered only by an undivided interval that crosses the bucket edge, e.g. monthly data asked by day.
> - `Invalid`: a calculation failed (virtual meters, §7.4).
> - `Pending`: the rollups are being rebuilt.
> A to-date read caps coverage at "now". Rows whose interval closes after now are reported, never counted (D-04,
> A-04, A-05, A-14, A-20).
> - **Reads.** Month and year buckets read the month table. Days and weeks read the day table. At most two partial
> edge days per range come straight from `consumption`. Every request makes one query per table, and the point and
> series limits are checked before any SQL runs (D-15).
> - **Rebuild.** Normalization revision 3 (D-16) rebuilds consumption, rollups and coverage for every meter at the
> next start (`NormalizationUpgrade`). It rebuilds again whenever the revision or the configured zone differs from
> `meter_rollup_state`. Until then a meter reads as "analysis being prepared" (`Pending`), never as "no data".
>
> **Cost is not a view either.** The cost engine (`CostReader`, §7.5) prices each bucket month by month, at the price
> valid on the 15th of each local month (D-36), so the bucket size never changes a total. There is no day-accurate
> proration mode (§14.3). A price change inside a reading interval longer than a month is reported, not guessed (A-16).
```sql
-- Daily normalized consumption per meter (local-tz buckets)
CREATE MATERIALIZED VIEW consumption_daily
@@ -390,6 +474,20 @@ Normalized/rolled-up volumes are tiny regardless: daily consumption = 1000 × 36
> Design conclusion: 1000 meters × 50 years is comfortably within TimescaleDB on modest hardware **provided** dashboards read aggregates and raw retention is bounded. Don't let the UI scan `reading` for charts.
> **Deviation (D-57, a documented limitation):** raw retention is **not enforced**. `MeterVault__RawRetentionDays`
> (default 1095) is shown, but nothing deletes readings. Every recompute rebuilds a meter's consumption from the
> readings that remain, so dropping old readings would destroy analytical history. Bounded raw retention first needs a
> recompute that can start from stored consumption. `/admin/settings` shows "Not enforced" with that reason, and the
> meter page's Readings tab explains it.
>
> **History reads the rollups** (§5.4 deviation): `consumption_rollup_month` for month/year buckets and
> `consumption_rollup` for day/week buckets. `consumption` itself is read only for at most two partial edge days per
> range and by the meter page's Normalized data tab. No chart reads `reading`. It is read by the paged Readings tab
> (100 rows per page, keyset-ordered, date-filtered, D-50), by the manual-entry checks, and for freshness: the latest 20
> reading times per meter (D-18). That freshness query has no time bound yet (§13, report). `consumption` and the
> rollups are the analytical history. Performance was measured against a synthetic 1,000-meter × 10-year dataset
> (`docs/ANALYSIS_REPORT.md`, §13).
---
## 6. Ingestion
@@ -435,46 +533,385 @@ Imported monthly tables keep the golden fixtures reconciling (§13). A row label
Every reader that buckets consumption by month or day (cost, trends, solar, consumables, flow, meter detail) buckets in the configured instance timezone — the same zone the division uses — never a hard-coded one, and starts and ends requested periods at local midnight. The zone id is normalised to its IANA form, and one unknown to .NET or PostgreSQL is reported at startup. Because consumption is derived, a change to these rules is applied to stored data at startup: `app_setting.normalization_revision` and `normalization_zone` record the rule revision and zone the stored series was built with, and every non-virtual meter is recomputed when either differs (the first run also flags month rows of earlier monthly imports, identified from each batch's stored mapping; a batch whose dates were auto-detected counts as monthly when all its rows sit on the 1st across at least two months, which is logged; if the flagging fails, nothing is rebuilt and the upgrade is retried at the next start). A meter whose rebuild fails is logged and listed in `normalization_pending`, retried at the next start, and never stops the application from starting.
> **Current (normalization revision 3, D-10, D-11, D-16):**
> - Every normalized row carries its source interval, so coverage and resolution can be recorded (§5.4).
> - A reading that closes exactly at a local midnight is booked in the day it closes, not the next one.
> - The startup rebuild also writes the rollup and coverage tables, and records each meter's `meter_rollup_state`.
> Virtual meters store no consumption: the migration purged their rows, and a meter that becomes virtual is purged
> on its next recompute.
> - A meter whose oldest consumption predates its oldest reading or event is skipped and logged, so its history is not
> truncated.
> - Month division still applies only to cumulative and generation counters. Tank, runtime, direct-delta and
> instant-rate intervals are booked whole. Where such an interval crosses a month edge, the months on either side are
> `Unresolved` and only coarser buckets are resolved (D-14, A-16).
### 7.2 Runtime → consumption (burner)
For `runtime_counter`: `amount = Δhours × rate`. `rate` comes from the linked `tank`: `fixed` (nozzle spec, L/h) or `empirical` (`Δlevel ÷ Δhours` measured between deliveries/level reads — reproduce the spreadsheet's 1.87/1.94/2.92 … behaviour). Expose both; default empirical when level data exists, else fixed.
### 7.3 Consumable/tank balance & forecast
`balance(t) = Σ deliveries(≤t) − Σ consumption(≤t)`, reconciled to physical `tank_level` events when present (cm → litres via calibration). Forecast to empty from a trailing consumption rate (e.g. last-30-day L/day) → `Vorraussichtliches Ende`. Surface low/reorder thresholds.
> **Current (D-54, `TankLevels`):** the last dipstick ("Last dipstick: value on date") is kept apart from "Estimated
> now (incl. deliveries since)", which does not deduct use since the dipstick. For a period that ended before now,
> the Consumables page shows the contents at the period's end, never today's balance. The forecast is a labelled
> projection. It is hidden when the dipstick is older than 60 days, when too little time has passed, or when no use
> was measured. Deliveries before a tank's first level open a coverage gap (D-13).
### 7.4 Virtual meters
For `virtual`: evaluate `config.expression` (whitelisted, sandboxed — a small safe expression evaluator over referenced meters' consumption/generation series), e.g. `self_consumption = generation − grid_feed_in`, `savings = self_consumption * unit_price`. Persist results to `consumption` (or compute on read — decide per §14). This is how PV self-consumption/savings and net figures are modelled without hardcoding.
> **Deviation (D-25 – D-33, D-39, A-08, A-12, A-15, D-58):**
>
> - **Canonical definition in `meter.meta`:** `expression`, `referencedMeterIds` (always derived from the expression
> and rewritten on save), `resultKind`, `resultUnit` and `costRule`. The result kind is one of `consumption`,
> `generation`, `net` or `indicator`. Save writes the effective (inferred) kind, unit and cost rule, so readers never
> re-infer them (A-08). Topology links (`meter_link`) never define or change a calculation. `meter_source` rows of
> type `virtual` are not used for this.
> - **Formula:**
> - Grammar: `+ − * /`, parentheses and numbers, parsed to an AST (`FormulaParser`), at most 2,000 characters and
> nesting depth 64.
> - References are `m<id>`. Any other identifier is an error; it is never read as 0.
> - The old string evaluator and `VirtualNormalizer` were removed.
> - **Validation** (`VirtualValidator`), on save and again on read for legacy data, covers:
> - syntax;
> - unknown or self references;
> - loops through nested virtual meters, reported with their path;
> - kinds and units: `+`/`−` need the same unit and kind or a declared `net`; meter × or ÷ meter needs a declared
> unit and kind `indicator`. Indicators are non-additive, never totalled and never costed.
> - **Evaluated on read, never materialized** (`VirtualEvaluator`, through `AnalysisReader`):
> - Evaluation runs per bucket from the sources' rollups, in dependency order. Each physical source is read once,
> however deeply it is nested.
> - Coverage is the intersection of the sources' coverage, and the resolution is the coarsest among them.
> - Strict: a missing source bucket makes the result missing and names the source. An observed zero is a valid input.
> - A non-finite result (division by zero) is `Invalid` with the reason. So is a loop, which is reported with its
> dependency path.
> - A period total is the formula over the sources' totals across their joint coverage. For a linear formula without
> a constant this equals the sum of its buckets; any other formula (for example a ratio) is marked non-additive,
> and its total is the ratio of totals.
> - The result carries every source's series (the "source contributions" on the meter page).
> - Nothing is written to `consumption` for a virtual meter. This decides §14.1.
> - **Totals:** a virtual meter is an *analysis view* by default and is never added on top of the meters it reads.
> The meter's `totals` override (`auto|always|never`) can make it replace its sources in its type's totals and in the
> bill (D-23).
> - **Legacy definitions (D-28):** at startup (`VirtualDefinitionUpgrade`), an expression-less virtual meter whose
> same-type incoming links name sources of one unit and kind gets the equivalent explicit sum stored. The run is
> idempotent and logged. Anything ambiguous is flagged "needs configuration" and is never guessed. The seed writes
> Summe Solar as `m4 + m5`, generation, kWh, cost rule `none`.
> - **Prices are not part of expressions** (`savings = self_consumption * unit_price` is not supported). A virtual
> meter's cost follows its cost rule (§7.5): `sourceCosts` (pure sums: the sources' own metered costs), `ownQuantity`
> (linear formulas: the evaluated quantity at its unit price) or `none`. PV savings are computed by the Solar page
> (self-consumption × the grid unit price, month by month) or through `ownQuantity`.
> - Export/import carries `meter_link` and remaps meter ids inside definitions (D-32). Deleting a meter names the
> virtual meters that read it and asks for confirmation (D-33).
### 7.5 Cost
`cost(bucket) = Σ(consumption_amount × active_unit_price) + base_price(prorated) − feed_in_credit − bonus`. Prices resolved by date from `tariff` (time-ranged). Currency from `app_setting`. Provide monthly-price and day-accurate-proration modes (§5.4). Categories roll costs up per `cost_category`; add `manual_cost` for meter-less categories (pool).
> **Deviation: the bill (D-22, D-34 – D-43, A-15 – A-19, A-21 – A-22, A-26 – A-27).** One cost engine (`CostReader` →
> `BillRun` → the Core `CostCalculator`) prices the portfolio, an energy type, a meter or a category for one resolved
> period. Every page, the REST API and the CSV export use it. Before the rework, costs summed every meter; this is
> what the engine does now:
>
> - **What is billed (D-22, D-34):**
> - Per energy type, the `grid_import` meters are billed if the type has one. Otherwise its *use* meters are billed:
> the `total_load` meter, or else the consumption roots of the topology.
> - Generation meters are never billed.
> - The feed-in credit is the FeedIn price × the export of `grid_export` meters.
> - Runtime meters and virtual views are not billed.
> - Submeters (topology children) are breakdowns, never added. So the seeded Strom bill is Zähler Netz × price, as
> the sheet's `Kosten` is.
> - **Separately billed subsections (D-35, A-19):**
> - A containment child with its own meter-scoped unit price is billed at that price, and its quantity is taken out
> of its billed ancestor.
> - The same applies to a consumer linked directly below a billed grid meter.
> - Quantity totals do not change.
> - **Prices (D-36):**
> - The spreadsheet's monthly convention is kept: the price valid on the **15th of each local month**.
> - Every bucket is cut into its local months and each part priced at its month's price, so week, month and year
> buckets, and the period total, agree.
> - There is no day-accurate proration (§14.3).
> - A reading interval longer than a month (a tank dipped every few months, quarterly burner hours) leaves its months
> unknown. A longer bucket over such months is priced as a whole when every month in it has the same price;
> otherwise it is unavailable, with an attention item (A-16).
> - **Tariff applicability (D-37):**
> - A UnitPrice or FeedIn tariff applies only when its unit's denominator matches the meter's normalized unit.
> Known scales are converted (ct, per 100 L, per MWh).
> - A parsed unit or currency mismatch makes the cost "unavailable (unit)", and the explanation names what does not
> fit (A-28). An unparseable unit applies, with a warning.
> - BasePrice units are per day, per month (the default) or per year.
> - The tariff editor checks units on save. It states that **Bonus, Discount and Tax are stored but not applied**
> (D-57).
> - **Not priced vs price gap vs zero (D-38, A-26, A-27):**
> - A billed scope with no UnitPrice tariff at any date is **not priced (no tariff)**. That is an attention item, not
> a partial total.
> - A hole in a priced scope's tariff history makes those months a **price gap** (cost unavailable).
> - An explicit zero tariff is a **valid zero**. The tariff editor refuses to save a new tariff without a value, so a
> deep link cannot create a free period by accident.
> - A missing FeedIn price is reported only where a `grid_export` meter exists.
> - Months with no grid meter in service, while use was measured, are unavailable rather than free (A-17).
> - A bucket with nothing booked reads "No data", never "Priced" (A-26).
> - The quantity analysis stays visible whenever the cost is unavailable.
> - **Standing charges (D-40, A-18):**
> - A standing charge accrues per local day, at value ÷ days in that local month, over the scope's service period
> (install or first data to retirement or now, regardless of reading gaps).
> - Type- and global-scoped charges are their own rows, **once per scope**, never copied onto each meter.
> - Meter-scoped fees stay on their meter. A fee on a meter that no bill line prices is its own row.
> - **Manual costs (D-41):**
> - A manual cost is booked in full, **once**, on its `PeriodStart` local day when that day is in the period and not
> after today. `PeriodEnd` is informational.
> - They count in the Overview, the Analysis page, categories and the export alike.
> - **Categories (D-42, A-22):**
> - A category's cost is the priced **non-overlapping cover** of its members, plus its manual costs.
> - The disjoint categories, *Uncategorized* and the standing-charge rows form the **composition**, which reconciles
> to the bill.
> - A category that overlaps another, or covers meters outside the bill, is an **overlapping view**. It is listed
> apart, never summed, and never drawn in the donut. The donut is drawn only when every slice is ≥ 0; otherwise
> signed bars are used.
> - A category whose members price nothing (calculated views, generation, runtime) says so rather than "no data".
> - **Virtual meters (D-39, A-15):** they are costed by their named rule:
> - `sourceCosts` (pure sums only): each physical source's own metered cost, once, with no scope-level standing
> charges.
> - `ownQuantity` (linear formulas without a constant): the evaluated quantity at its unit price.
> - `none`: every other formula, and generation sums (generation is never billed), which is why Summe Solar is not
> costed.
> - The rule is named next to every virtual cost. A virtual meter enters the bill only through the `always`
> override, and then replaces its sources.
> - The REST API reports a meter without a cost rule as `NotPriced` with the reason (A-21).
> - **Currency (D-43):** the configured `MeterVault__Currency` everywhere, through `Format.Money` / `InstanceCurrency`.
> A tariff in another currency is a unit mismatch; no conversion is made.
> - **Changes (D-07, A-23):** a cost change is stated only between complete figures, by one rule on every page
> (`OverviewComparison.Between`). If both totals are complete, it compares the totals. Otherwise it compares only the
> paired buckets that are complete on both sides, with the caption "over the part both periods cover".
---
## 8. Dashboard & UX
> **Deviation (analysis rework, brief §3–§8, D-46 – D-55, D-58):** the panels below were rebuilt as one set of pages
> that share one period contract, one reader pair and one set of components. §8.0 describes what they share; the
> notes under §8.1 – §8.7 say what each page now is.
### 8.0 Shared analysis contract (current)
**Pages and navigation (D-47, D-48):**
| Sidebar entry | Route | What it is |
|---|---|---|
| Overview | `/` | One selected period (default month to date): the cost with its composition, a card per energy type (quantities in their own units, cost with billing basis, change, freshness), the history chart, "What changed", attention items |
| Analysis | `/trends` (route kept) | Explore any scope: `portfolio`, energy `type`, cost `category`, one `meter`, or up to six `meters` side by side, by quantity or cost |
| Meters | `/meters` | All meters with period values, status, resolution, "data up to", how each counts, quick entry, filter by type |
| Energy types → *type* | `/energy/{id}` | Tabs `overview`, `history` (total or `view=meters`), `flow` (Sankey + table + Manage connections), `meters` |
| Specialized views | `/solar`, `/consumables` | Always listed; each shows a setup state when unsupported (no generation meter, no tank) |
| Data import | `/import`, `/import/wizard` | Import batches with revert; the CSV mapping wizard |
| Configuration | `/admin/*` | Energy type *definitions*, tariffs, cost categories, connectors, settings |
- The meter page `/meters/{id}` is the per-meter hub. Its tabs are `analysis`, `readings`, `normalized`, `events`,
`tariffs`, `sources`. A virtual meter shows `analysis`, `events`, `tariffs`, `calculation`: no Readings or
Normalized data, and Calculation instead of Sources.
- Old links still work: `tab=consumption` opens `normalized`, and on a virtual meter `sources` opens `calculation` and
`readings` opens `analysis`. Tabs are resolved by key and meter mode, never by index.
- One-shot `action=` links (reading, swap, reset, delivery, tank level, note, edit, source) open their dialog once and
are then dropped from the address.
- Breadcrumbs (Overview → type → meter) carry the period. Expanded sidebar groups persist in a cookie (`mv-nav`), and
the group of the current page is always open. If the energy types cannot be loaded, the nav shows an error with
Retry instead of dropping the group.
- The app-bar "Find a meter" (a text button from the md breakpoint, an icon below) opens a meter's Analysis tab with
the current period, next to the quick-entry action.
**Period and URL state (D-01 – D-08, D-46, A-13, `AnalysisQuery`):**
| Key | Values |
|---|---|
| `scope`, `id`, `ids` | `portfolio`, `type`, `category`, `meter`, `meters` (`ids`, at most 6) |
| `metric` | `consumption`, `generation`, `export`, `runtime`, `net`, `cost`, `balance` |
| `period` | `mtd`, `last-month`, `ytd`, `prev-year`, `12m`, `24m`, `all`, `custom` (with `from`/`to`, local, inclusive) |
| `bucket` | `auto`, `day`, `week` (Monday), `month`, `year` |
| `compare` | `none`, `prev-period`, `prev-year`, `year:YYYY` |
- The URL is the state: reload, share and Back reproduce the page. Defaults are never written: the Overview uses
`mtd`, history pages `12m` (12 calendar buckets ending with the current partial month), and every page compares with
the previous year (A-13).
- Toolbar and tab changes replace the history entry; drill-downs push. An invalid token falls back to the default
with a notice.
- `all` spans the scope's available data, never a fixed century.
- `auto` picks one bucket, at most 400 points per series. A finer explicit bucket is refused with a coarser
suggestion.
- A period resolves once per request against a captured "now" in the configured zone (§10). Quantities, costs,
comparisons and the export use the same half-open bounds.
- Comparisons shift in calendar units. The change figure is measured only over the range both periods cover, and both
exact ranges are shown (D-07). The percentage is "not applicable" for a zero or negative baseline; the absolute
difference is always shown (D-08). Colours depend on the metric: more generation is good, more consumption is not.
- Projections are separate, labelled with their method, and suppressed where coverage is insufficient (D-09).
- Pages load through a `LoadSequencer`, so a superseded load never overwrites a later one. Panels show a
loading/refresh/error-with-Retry state.
- Other page-specific keys: the energy page's `tab` and `view`, the Overview's `chart`, the record tabs' `from`/`to`.
**Missing vs zero vs not priced (brief §4.3, D-14, D-38, A-24 – A-28):**
- Every figure carries its bucket status (§5.4) and provenance (measured, manual, imported, estimated, derived,
opening balance), worded next to it (`FigureText`).
- A true zero is a number and a bar on the baseline.
- An unknown bucket is a gap in the chart, marked "–", and "—" with its reason in the table: no data, only coarser
data, cannot be calculated, being prepared.
- Qualified values (partial, estimated) are marked "*".
- A period without data says "No data for this period", names the available dates and offers "Go to latest data". A
future range says it has not started yet.
- A chart never says "no data" when the reason is a coarser resolution (it names the resolution and offers the
interval that shows it) or a missing price (it names the cost status).
- Drill-downs only go where finer data exists. A monthly bucket never opens 31 unknown days. A physical meter's finest
bucket opens its Normalized data. A virtual meter's opens its own analysis over that bucket, whose source list links
on to the sources' records.
- Missing prices, stale sources, invalid calculations, rows dated after now and possible overlaps become attention
items, each with one targeted action (D-53). For example, "Add tariff" opens
`/admin/tariffs?scope=&id=&component=&from=&action=new` prefilled with the first uncovered month (D-52).
**Shared components (`Components/Shared/Analysis`):** `PageHeader`, `AnalysisBreadcrumbs`, `PeriodToolbar`,
`AnalysisChart` (ApexCharts; one axis per unit; signed values around a real zero line; follows the light/dark theme
in the current circuit), `AnalysisTable` (the accessible equivalent of every chart), `MetricCard`, `ChangeChip`,
`ValueStatus`, `EmptyPeriodState`, `PendingState`, `PanelError`, `LoadPanel`, `ProjectionNote`, `ComparisonSummary`,
`AttentionList`, `SeriesContributions`.
**CSV export (D-55):** `GET /export/analysis.csv` takes the same URL keys as the pages; every toolbar has "Export CSV".
It writes one row per bucket and series, with these columns:
- `series_id`, `series_name`, `kind`, `unit`;
- `bucket_start`, `bucket_end` (local ISO with offset; the end is exclusive, and a to-date bucket ends at now),
`timezone`;
- `value` (invariant, full precision; empty when unknown), `status`, `provenance`;
- `cost`, `cost_status`, `currency`, `comparison_value`.
A bucket with nothing booked is `Missing`, never `Available`. Invalid requests (a notice, too many buckets, an unknown
scope) get a 400 with a plain-text reason. The endpoint is a UI endpoint, like the pages, and needs no API key.
### 8.1 Overview
- KPI cards: **Today**, **This month**, **This year** cost — each with Δ (absolute + %) vs the previous comparable period and an ↑/↓ indicator.
- "Cost now" total across all categories.
> **Deviation (D-58, brief §7.1):** the Overview shows **one selected period** instead of Today/This month/This year
> cards. It has a period toolbar and defaults to month to date. The page shows:
>
> - the cost of the period, split into metered use, standing charges, manual costs and feed-in credit;
> - one card per energy type, with its measures in their own units (unlike quantities are never added), the cost with
> its billing basis, the change, freshness, and a link to that type with the same period;
> - the history chart with a metric selector (`chart=`), the previous-year overlay, a table toggle and drill-down;
> - "What changed", by category or by meter, with rows linking to the scoped Analysis page with the same dates;
> - the cost composition (§7.5): a donut only for non-negative disjoint slices, otherwise signed bars. Overlapping
> views are listed apart;
> - attention items, and "Latest month with data" with its month and basis (meters, manual costs or both, D-19).
>
> Changes are measured over the coverage both periods share (D-07). A period without data offers "Go to latest data";
> the page never switches to history on its own. Missing categories or tariffs are small setup notes, never a
> prerequisite for seeing quantities. The REST summary keeps its legacy month/year windows (D-45).
### 8.2 Cost breakdown / "what costs most"
- Stacked bar or donut by `cost_category` for a selectable period; ranked list (most → least).
- **Difference view** (explicitly requested): a table answering *"what cost more, what cost less this time"* — per category **and** per meter, **this month vs last month** and **this year vs last year**, columns `now | previous | Δ | Δ% | ↑/↓`, sorted by absolute impact.
> **Current:** both live on the Overview for the selected period and its comparison, not for fixed month/year windows.
> The composition is §7.5's. "What changed" lists categories or meters with current, previous, change and percentage
> where applicable, sorted by impact, plus a bill total row. A change is shown only between complete figures (A-23).
### 8.3 Trends
- Consumption and cost over time; **granularity toggle** day/week/month/year; per-meter or per-category; **previous-year overlay**.
> **Current:** this is the **Analysis** page (`/trends`, brief §7.4, nav "Analysis"). It offers:
>
> - Scope: all energy types, one energy type, a cost category, one meter, or a comparison of up to six meters. A
> seventh is refused with an explanation.
> - Metric: only what the scope supports. A category is always available by cost, and by quantity only when all its
> meters share one kind and unit; otherwise the page explains why and offers the alternatives.
> - The shared toolbar, with a calendar-year select (compare with any of the five years before), the chart (overlays
> up to three series; above that the comparison stays in the table), the table, drill-down and CSV export.
> - For per-type measures, total use and grid import side by side, never added. Portfolio cost is the same bill the
> Overview shows, with manual costs once.
### 8.4 PV / Solar panel
- Generation, self-consumption, grid feed/draw, **savings (Ersparnis)**, **autarky %**, **self-consumption %**. Time-filtered.
> **Current (D-54, `SolarService`):** one section per energy type that has generation. Nothing is inferred from names:
> meters are found by mode and by the effective roles `total_load`, `grid_import`, `grid_export` (A-07).
>
> - **Generation** is the type's generation measure, so a virtual sum such as Summe Solar is listed as a view and never
> added twice.
> - **Self-consumption** is total load − grid import, or else generation − grid export.
> - **Feed-in** is the grid export meter, or else generation − self-consumption (labelled as calculated; batteries are
> not modelled).
> - **Site use** is the total load meter, or else self-consumption + grid import.
> - **Savings** are self-consumption × the grid unit price, month by month through the cost calculator.
> - Autarky % and self-consumption % are shown when the roles allow.
>
> Every figure shows its status and how it was obtained. Units come from the meters; mixed units give "cannot be
> calculated". A missing role gets a setup card with the candidate meters, which lead into the meter editor (no raw
> role tags). The page has no CSV export, because the export has no derived measures.
### 8.5 Oil / consumable panel
- Tank level (cm + L), balance vs capacity gauge, deliveries log, burner runtime, effective **L/h** (fixed/empirical), **forecast to empty**, monthly cost.
> **Current (D-54, `ConsumableService`):** "Now" and "Selected period" are separate parts.
>
> - **Now:** the last dipstick with its date (and cm reading), an estimate that includes the deliveries since then,
> the fill bar (only when the level is known), and the forecast as a labelled projection (§7.3).
> - **Selected period:** use from the analysis reader, the deliveries of the period only, burner runtime of the type's
> runtime meters, and the burn rate (fixed, or empirical when runtime is in hours).
> - A period that ended shows the contents at its end.
> - The cost stays unknown, never 0 €, when the tank has no tariff or its months cannot be placed (A-16).
### 8.6 Meter detail
- Raw readings, normalized consumption, source status (last-seen, last value), tariff timeline, events (swaps/deliveries/corrections), measured-vs-estimated markers.
> **Current (brief §7.2, D-47, D-50):** the header carries identity, the energy type, mode and retirement chips, and
> the actions: primary entry by mode (Add reading / Record tank level), the "Record event" menu and Edit. The tab bar
> sits directly below the header.
>
> - **Analysis** tab:
> - The quantity card in the normalized unit (D-20), with any projection shown separately inside it.
> - The cost card, with its rule named or "Not costed" plus the reason, and the cost change (A-23).
> - The previous-year overlay, a full chart, the table, drill-down and CSV export.
> - Events and tariff changes inside the range, listed as context under the chart.
> - A "Data quality and coverage" section: resolution, data range, freshness, opening balance with "Set install
> date", and rows recorded after now.
> - For virtual meters, the source contributions.
> - **Readings**, **Normalized data** and **Events** tabs:
> - Server-side paging, 100 rows per page, keyset-ordered, filtered by the page's `period`/`from`/`to`.
> - Their toolbar shows the whole range listed, and rows dated after now carry an "After now" mark (A-29).
> - The Readings tab explains that raw readings are the audit record and that raw retention is not enforced.
> - **Tariffs** lists meter, type and global tariffs with their effective end and the one that applies now, plus "Add
> tariff for this meter".
> - **Sources** links each source's connector to its editor (the connector detour keeps the typed draft).
> - A virtual meter's **Calculation** tab shows the status, the formula with meter names beside each `m<id>`, result
> kind and unit, cost rule, the meters read (also through nested calculations), and any problem with its dependency
> path.
> - The manual-entry dialog runs its own queries for the entered time, so its verdict never depends on a page of rows.
### 8.7 Admin / config
- CRUD for energy types, meters, sources, tariffs, cost categories, connectors; retention & locale/currency settings; import wizard; API keys.
> **Current (D-21, D-23, D-31, D-37, D-52, A-27, A-30):**
>
> - **Meter editor** (shared `MeterEditor`):
> - Roles use friendly names and one-line meanings, and are offered only for compatible modes. Saving a role moves it
> and names the meter that held it.
> - The totals override (Automatic / Always / Never) states its meaning and "In the totals now: …". A conflicting
> "Always" is refused, naming the other meter.
> - A virtual meter gets the calculation editor: Sum, Difference or Formula mode; sources picked by name, with unit,
> kind and dates; only valid cost rules offered, each with its reason.
> - A live preview uses the page's period (every preset, custom dates, all history) and shows per-source values and
> the incomplete months.
> - Saving a Sum can bring the incoming links in line with its sources, but links never change a calculation.
> - **Tariffs:** the deep link opens a prefilled dialog once, scoped to what can price that meter or type. Units are
> checked live, a new tariff needs a value, and Bonus/Discount/Tax are marked "not applied".
> - **Energy types** under Configuration edit the definitions ("Energy type definitions"); analysis lives under the
> Energy types nav group.
> - **Settings** is read-only. It shows the zone, the currency, raw retention ("Not enforced"), the normalization
> revision and zone, how many meters have current analysis data or are waiting for a rebuild, and calculated meters
> by status.
> - Flow topology is edited from the energy type's Flow tab ("Manage connections"): cycle-, type- and
> duplicate-checked, and it never touches a stored formula. Physical meters can also set their upstream meters in
> the editor.
> **Legacy monthly history:** imported data is monthly-granular. Offer per-import choice: keep native monthly buckets, or **linearly interpolate to daily** (energietracker-style) so old and new data render on the same axes. Interpolated points are marked `quality = interpolated`.
>
> **Current behaviour (D-14, D-57):** monthly data stays monthly and is never interpolated. A day or week bucket over
> it reads "only coarser data", the chart names the data's resolution and offers the interval that shows it, and a
> drill-down never opens days a monthly import cannot resolve (D-51).
---
@@ -496,17 +933,54 @@ OpenAPI/Swagger published. API-key auth for automation endpoints; UI uses the se
| `GET /api/v1/sources/status` | Connector/source health. |
| `GET /healthz` | Liveness/readiness (for Gatus). |
> **Current contract (D-45, A-16, A-21; pinned by `ApiContractTests`):** `/consumption`, `/cost` and
> `/dashboard/summary` keep every existing field, name and type. Their numbers now come from the analysis reader and
> the cost engine, so they match the pages. What changed is only added as new fields:
>
> | Endpoint | Behaviour | Added fields |
> |---|---|---|
> | `GET /api/v1/consumption?meter=&from=&to=` | Monthly rows as before. Instants with any offset are accepted (converted to UTC; an offset used to be a 500). Actuals stop at now. A month without data is absent, not a 0. A virtual meter is evaluated from its formula. | `status` (BucketStatus), `issue` (why a value is not plain), `kind`, `unit` (normalized) |
> | `GET /api/v1/cost?meter=&from=&to=` | `cost` stays numeric: 0 when nothing could be priced, with the reason beside it. The meter is priced by its rule (the bill line, a subsection at its unit price, a virtual meter by its cost rule). Generation and runtime meters, and meters that cannot be evaluated, are not costed. | `costStatus` (`Priced`, `Partial`, `NotPriced`, `PriceGap`, `UnitMismatch`), `costAvailability` (the status of the quantities behind the cost, e.g. `Unresolved`, `Invalid`), `costRule`, `notCosted` (the reason), `missingPrices[]` (component, reason, scope, first/last month, tariff, unit issue, credit), `status`, `issue`, `kind`, `unit` |
> | `GET /api/v1/dashboard/summary` | Keeps its legacy windows (calendar month and year to now against the whole previous ones) but prices the bill (§7.5). | `deltaPercentApplicable` per KPI (the percentage is 0 and not applicable for a zero or negative baseline), `latestMonth` `{period, basis}` |
>
> `/consumption` and `/cost` take one `meter` and answer by calendar month, as before the rework; the table's `scope`
> and `bucket` parameters are not implemented. The UI's analysis CSV (`GET /export/analysis.csv`, §8.0) is not part of
> `/api/v1` and needs no API key.
---
## 10. Non-functional
- **Time & DST:** store UTC; bucket and display in the instance timezone (default `Europe/Berlin`). "Daily cost" boundaries are local-midnight — use `time_bucket(..., 'Europe/Berlin')`.
*Deviation (D-01 – D-06, D-58):*
- **Zone:** the **configured** zone (`MeterVault__TimeZone`, normalized to its IANA id) is used everywhere:
normalization, rollups, readers, periods, the export. Berlin is never hard-coded, and a zone change rebuilds the
rollups.
- **Bounds:** a period resolves into a local inclusive date range for display and a **half-open** UTC range
`[from, to)` for queries. `to` is the local midnight after the end date, or the captured "now" for to-date
periods.
- **Calendar:** days start at local midnight, and a DST day really has 23 or 25 hours. Weeks start on Monday. Months
and years are local.
- **Clock:** "now" is read once per request from the registered `TimeProvider` (`InstanceClock` for pages).
Services never read the clock; tests freeze it.
- **After now:** rows whose interval ends after now are never counted as actuals; they are reported as "recorded
after now" (D-04).
- **Auth:** optional built-in local accounts; **reverse-proxy trust** mode honouring `X-Forwarded-User`/`Remote-User` behind Authelia/Traefik; API keys for machine access. Default: single admin user + one ingest API key.
- **Observability:** `/healthz`, structured logs (Serilog), optional Prometheus `/metrics`.
- **Config:** environment variables + a settings UI; secrets via env/Docker secrets or encrypted at rest (never in DB plaintext) — see §6.4.
- **i18n:** `en` (default for OSS) + `de`; locale-aware number/currency/date. Ship a German locale that matches the source data conventions.
- **Backup:** document `pg_dump`/Timescale backup; provide a full **JSON export/import** for portability.
- **Performance:** dashboards read aggregates only; raw reads paginated and time-bounded.
*Current:* dashboards read the rollup tables (§5.4). A request sends a constant handful of statements, whatever the
meter count. The point limit (400 per series) and the series limit (6 on a chart) are enforced before any SQL runs.
The brief's target is a 10-year monthly request for 100 meters in under 2 s; it was measured against a synthetic
1,000-meter × 10-year dataset (§13, `docs/ANALYSIS_REPORT.md`). Raw record tabs are paged (100 rows, keyset) and
date-filtered.
- **Accessibility & layout (brief §8):** every chart has a table equivalent; colour is never the only cue (words,
arrows, signed values, dashed overlays); visible focus rings; keyboard-reachable drill links; no page-wide overflow
at 360 px; charts follow the light/dark theme (cookie `mv-theme`) within the circuit. MudBlazor's own labels are
localized (`MeterVaultMudLocalizer`).
- **Currency:** `MeterVault__Currency` (default `EUR`) for every amount (D-43).
---
@@ -528,6 +1002,18 @@ OpenAPI/Swagger published. API-key auth for automation endpoints; UI uses the se
/docs architecture, setup, HA/Tasmota wiring, API, screenshots
```
> *Current layout:* the analysis rework added these folders.
> - `src/Core/Analysis`: pure rules for time and periods, coverage, rollups, quantities and units, totals and category
> cover, virtual formulas, and the cost calculator.
> - `src/Infrastructure/Analysis`: the reader, the catalog and the virtual upgrade.
> - `src/Infrastructure/Costing`: `CostReader` and `BillRun`.
> - `src/App/Analysis`: the URL contract and the chart, table and attention models.
> - `src/App/Components/Shared/Analysis`: the shared components.
> - Page folders under `src/App/Components/Pages`.
>
> The fixtures live in `sampledata/`. CI is Gitea Actions (`.gitea/workflows/`), publishing to the Gitea container
> registry. `CLAUDE.md` holds the maintained layout.
- **CI (GitHub Actions):** build → test (spin Timescale) → publish Docker image to **GHCR** (amd64; add arm64 if desired) on tag.
- **License:** pick before release — **MIT** (max adoption; matches energietracker/your prior assets) or **AGPL-3.0** (keeps hosted forks open). Default suggestion: **MIT**, unless keeping SaaS forks open-source matters to you.
- **Docs:** a "wire up HA/Tasmota" guide is the highest-leverage doc for adoption.
@@ -545,6 +1031,16 @@ OpenAPI/Swagger published. API-key auth for automation endpoints; UI uses the se
- **M6 — API & auth.** REST + OpenAPI; API keys; reverse-proxy trust. *Exit:* HA can push via `POST /readings`; Swagger published.
- **M7 — Release polish.** i18n (de/en); retention settings; JSON export/import; Unraid template; CI → GHCR; README + wiring guide. *Exit:* `docker compose up` from a clean host yields a working, documented instance.
> *After M7:* the dashboard/analysis rework was built in five phases:
> 1. shared semantics and fixtures;
> 2. virtual evaluation and migration;
> 3. history and navigation;
> 4. Overview and specialized pages;
> 5. integration, performance and documentation.
>
> It replaced M4's continuous aggregates and "monthly + prorated" cost view with rollup tables and the month-by-month
> bill (§5.4, §7.5), and M5's panels with the pages of §8.0. M7's "retention settings" remain display-only (D-57).
---
## 13. Testing strategy
@@ -554,21 +1050,89 @@ OpenAPI/Swagger published. API-key auth for automation endpoints; UI uses the se
- **Golden fixtures:** the four CSVs with expected monthly consumption/cost tables. A regression test asserts computed ≈ spreadsheet (define tolerance for rounding; the sheet rounds to cents / whole kWh).
- **Load smoke (optional):** synthetic 1000-meter × N-year generator to validate aggregate query latency and compression ratio.
> **Current (analysis rework, D-56):** `Core.Tests` has 1,733 tests and `Integration.Tests` 746, all passing at the end
> of the rework. The continuous-aggregate refresh test was replaced by
> `CostReconciliationTests.Monthly_rollup_equals_the_consumption_it_sums` (water Dec 2022 = 14 m³). `SchemaTests`
> pins that the aggregates and their jobs are gone and that the analysis tables cascade with their meter.
>
> - **Frozen clock, pure (Core):**
> - `PeriodResolverTests`, `ComparisonResolverTests`, `BucketPlannerTests`: local New Year, Berlin DST in spring
> and autumn (23 h and 25 h days, the repeated hour), 29 Feb, 31 March against February, shorter months, New York
> (a zone behind UTC).
> - Coverage, rollups, provenance and freshness: `Coverage*Tests`, `RollupBuilderTests`, `MatchedCoverageTests`,
> `ProvenanceRulesTests`, `FreshnessRulesTests`.
> - Totals and categories: `TotalsPolicyTests` (the seeded classification, D-22), `CategoryCoverTests`,
> `SeparatelyBilledSubmeterTests`.
> - Virtual formulas: `FormulaParserTests`, `VirtualValidatorTests`, `VirtualEvaluatorTests`, `DependencyGraphTests`,
> `LegacyVirtualDerivationTests`: A+B, A−B, missing vs zero, nested, loop, division by zero.
> - Costing: `CostCalculator*Tests`, `CostingTariffBookTests`, `TariffUnitTests`.
> - Units and changes: `UnitsTests`, `ChangeTests`.
> - **Reader and costs (Testcontainers):**
> - `AnalysisReaderTests`: the worked examples, local days across DST and in New York, rows after now, new readings
> and corrections visible on the next read.
> - `AnalysisDataTests`: rollups written by diff, rows removed behind the tracker's back.
> - `CostReaderTests`: missing vs zero tariff, price gaps, bucket-independent totals, standing charges once per
> scope, manual costs once, virtual cost rules. `CostReviewFixTests` adds a category whose members price nothing.
> - `SeededBillTests`: the D-44 goldens. The yearly bill equals the sheet's `Jahreskosten` within ±0.02 € for 2022,
> 2025 and 2026; 2023 and 2024 are pinned to the documented 3.78 € / 0.46 € differences. It also pins the
> composition and a manual-cost-only instance.
> - `CostConsistencyTests`: the same cost change on four pages, and matching standing charges.
> - `CoverageOfFixturesTests`, and the unchanged reconciliation suites (electricity, water, oil, costs; Netz
> Einsparung through the new evaluator, D-29).
> - **Contracts and export:** `ApiContractTests` (fields and types of `/consumption`, `/cost`, `/dashboard/summary`,
> offsets, not-costed meters), `AnalysisExportEndpointTests`, `AnalysisCsvWriterTests`, and `ExportRoundTripTests`
> (links and virtual definitions remapped, a deleted meter's tariffs not restored).
> - **Pages without a browser:**
> - Loaders against a database: `MeterAnalysisLoaderTests`, `AnalysisPageLoaderTests`, `EnergyTypePageTests`,
> `OverviewDataTests`, `SolarServiceTests`, `ConsumableServiceTests`, `MeterDraftPreviewTests`,
> `VirtualManagementTests`, `MeterDetailServiceTests` (keyset paging, half-open filters).
> - Pure UI models: `AnalysisQueryTests`, `AnalysisNavigationTests`, `AnalysisChartModelTests`,
> `AnalysisTableModelTests`, `AttentionItemsTests`, `LoadSequencerTests`, `MeterPageLogicTests`,
> `EnergyPageTests`, `OverviewLogicTests`, `MeterEditorLogicTests`, `TariffEditingTests`.
> - Server-rendered HTML in EN and DE: `DashboardRenderTests`, `OverviewPageTests`, `AnalysisComponentRenderTests`,
> `AdminPagesRenderTests`, `MeterSourcesRenderTests`.
> - `StringResourceTests`, `EnumDisplayNameTests` and `FormatCultureTests` pin both languages and the currency.
> - **Browser checks:** interactive ApexCharts, browser history, theme switching and responsive layout were checked
> against the seeded instance. CDP scripts drove Chrome in EN/DE, light/dark, 1440/390/360 px. That acceptance walk
> is recorded in `docs/ANALYSIS_REPORT.md`. No bUnit or Playwright suite is in the repository.
> - **Performance** (`tests/Integration.Tests/Performance`, trait `Category=Performance`): skipped unless
> `METERVAULT_PERF=1`. `SyntheticLoadTests` loads a deterministic 1,000-meter × 10-year dataset (≈1.34 M readings;
> monthly, daily and hourly meters, tanks, roles, links, 20 virtual meters nested up to three levels) through the
> real pipeline. `ReaderTimingTests` times the reader and cost scenarios (the brief's target: 100 meters, 10 years
> monthly, under 2 s), counts SQL statements, records query plans, the startup rebuild and per-meter recompute
> cost. Results are in `docs/ANALYSIS_REPORT.md`.
---
## 14. Open questions & defaults
Pick the **default** and flag it if unsure; only ask when a question isn't listed here.
1. **Virtual meters: compute-on-write or compute-on-read?** *Default:* compute-on-read for dashboards, materialize to `consumption` only if a virtual meter is referenced by cost. (Avoids recompute storms; revisit if slow.)
2. **Legacy monthly import → daily interpolation or native monthly?** *Default:* offer both per import; interpolation off by default, points marked `interpolated`.
3. **Cost proration when price changes mid-month.** *Default:* day-accurate proration available, but the **displayed** monthly figure uses the month's dominant price to match the spreadsheet unless the user opts into proration.
4. **Instant-rate (power/flow) integration in v1?** *Default:* schema-supported, worker deferred to post-v1 (Tasmota already gives cumulative `ENERGY.Total`, so it's rarely needed).
1. **Virtual meters: compute-on-write or compute-on-read?** *Default:* compute-on-read for dashboards, materialize to `consumption` only if a virtual meter is referenced by cost. (Avoids recompute storms; revisit if slow.) *Decided (D-58):* computed on read everywhere, costs included; nothing is materialized.
2. **Legacy monthly import → daily interpolation or native monthly?** *Default:* offer both per import; interpolation off by default, points marked `interpolated`. *Current (D-57):* native monthly only; interpolation is not offered.
3. **Cost proration when price changes mid-month.** *Default:* day-accurate proration available, but the **displayed** monthly figure uses the month's dominant price to match the spreadsheet unless the user opts into proration. *Decided (D-36, A-16):* one price per local month, the one valid on the 15th (the sheet's convention); no proration mode. Every bucket is priced month by month, so bucket size never changes a total. A reading interval longer than a month is priced as a whole only when its months share one price; otherwise it is unavailable, with an attention item. Standing charges accrue per day (D-40).
4. **Instant-rate (power/flow) integration in v1?** *Default:* schema-supported, worker deferred to post-v1 (Tasmota already gives cumulative `ENERGY.Total`, so it's rarely needed). *Current:* the `instant_rate` normalizer integrates the rate over time (trapezoidal), and a gap longer than max(1 h, 10 × the median sample interval) opens a coverage gap (D-13). There is no rate-specific ingestion worker: rate values arrive through the ordinary sources like any other reading.
5. **Multi-user?** *Default:* single admin + reverse-proxy trust; full accounts post-v1.
6. **.NET version pin.** *Default:* current LTS at implementation time; keep `TargetFramework` in one place.
7. **License.** *Default:* MIT unless you want AGPL's copyleft on hosted forks.
8. **Name.** `MeterVault` is a placeholder — decide before first public tag.
The analysis rework settled these further questions. Each is recorded in `ANALYSIS_IMPLEMENTATION_NOTE.md`.
9. **Raw retention (§5.5).** *Decided (D-57):* not enforced until recompute can start from stored consumption. The
setting is shown as "Not enforced".
10. **What a type's total and bill count.** *Decided (D-22, D-34):* the non-overlapping topology roots, with
consumption and generation apart. The bill counts grid import where there is one, otherwise use. Submeters and
virtual views are never added; the `always` override lets a virtual meter replace its sources (D-23).
11. **Virtual costs.** *Decided (D-39, A-15):* a named cost rule. `sourceCosts` for pure sums, `ownQuantity` for
linear formulas, `none` otherwise and for generation sums. The default for new virtual meters is "analysis only"
in the totals.
12. **Default comparison.** *Decided (A-13):* the previous year, at the same elapsed point, measured over what both
periods cover (D-07).
13. **Default periods.** *Decided (D-02):* the Overview uses month to date. History pages use the last 12 months: 12
calendar buckets, the current one partial.
14. **Missing tariff.** *Decided (D-38):* "not priced", never 0. An explicit zero tariff is a valid zero.
---
## Appendix A — CSV import dialect (from the reference files)
@@ -589,7 +1153,7 @@ Pick the **default** and flag it if unsure; only ask when a question isn't liste
|------------|---------------|
| Zähler (Haus/Netz/Auto/Solar) | `meter` (`cumulative_`/`generation_counter`) |
| Verbrauch / Erzeugung | `consumption.amount` (kind 0/1) |
| Solar Erzeugung / Eigenverbrauch / Ersparnis / Netz Einsparung | `virtual` meters via expressions |
| Solar Erzeugung / Eigenverbrauch / Ersparnis / Netz Einsparung | `virtual` meters via expressions (quantities only: Solar Erzeugung = `m(Solar 1) + m(Solar 2)`, Netz Einsparung = `m(Haus) − m(Netz)`). Ersparnis is a price × quantity, which expressions do not support (§7.4). It is the Solar page's savings, or a virtual meter's `ownQuantity` cost. |
| €/kWh, €/m³, €/100l | `tariff.unit_price` (time-ranged) |
| Grundpreis / Abschlag | `tariff.base_price` |
| Betriebststunden | `runtime_counter` meter |