MeterVault/docs/SDD.md

# MeterVault — Software Development Document

> **Working codename:** `MeterVault` (rename freely before publishing). Your existing spreadsheet system is called *Energiebilanz*; for a public GitHub repo an English name reaches more people, but `Energiebilanz` is also fine.
>
> **What this is:** a self-hosted, local-first energy & utility metering platform that pulls meter data from Home Assistant, Tasmota and MQTT on a schedule, stores every reading with a timestamp, and turns it into cost dashboards. Not limited to electricity/water/oil — energy types are user-defined.
>
> **Status:** design spec, pre-code. This document doubles as the build brief for Claude Code.
>
> **Implementation status (0.4.0):** M0–M7 are implemented, and the dashboard/analysis rework
> ([`DASHBOARD_ANALYSIS_CHANGE_BRIEF.md`](DASHBOARD_ANALYSIS_CHANGE_BRIEF.md)) replaced the aggregation, virtual-meter,
> cost and page model. Its decisions are numbered D-01 – D-58 and A-01 – A-39 in
> [`ANALYSIS_IMPLEMENTATION_NOTE.md`](ANALYSIS_IMPLEMENTATION_NOTE.md). The original design text below is kept.
> Wherever the system now works differently, a **Deviation** or **Current behaviour** block says so in place (list:
> D-58). The results are in [`ANALYSIS_REPORT.md`](ANALYSIS_REPORT.md).

---

## 0. How to build this with Claude Code

This document is the source of truth. Suggested working protocol:

1. Drop this file in the repo root as `SDD.md`, and create a `CLAUDE.md` that references it (`See SDD.md for the full spec; work milestone by milestone; do not skip tests`).
2. Build strictly in the **milestone order** in §12. Each milestone is independently runnable and testable — do not start Mn+1 until Mn's tests pass.
3. The **four reference CSVs** (`Energiebilanz_-_*.csv`) are golden fixtures. Every parsing/consumption/cost rule in this doc must be validated against them (§13). If a computed number disagrees with the spreadsheet, the spreadsheet wins unless the discrepancy is documented as a deliberate correctness fix.
4. Prefer small, reviewable PRs per milestone. Keep the domain layer free of infrastructure concerns.
5. When a design decision is ambiguous, check §14 (open questions) — if it's listed, pick the stated default and flag it; if it isn't, ask before guessing.

---

## 1. Vision & scope

### 1.1 Problem

Utility/energy bookkeeping today lives in a hand-maintained Google Sheet (the *Energiebilanz*). It works but: readings are entered manually and monthly, there's no live pull from the sensors that already exist (Tasmota plugs, HA entities), the cost logic is buried in cell formulas, and it doesn't scale to fine-grained data or many meters.

### 1.2 Goals

- **Automatic ingestion** from HA, Tasmota and raw MQTT on a schedule, plus manual entry and CSV import.
- **Every reading timestamped** and preserved (auditable), with a normalized consumption/cost layer on top.
- **Versatile by design:** energy types (electricity, water, heating oil, gas, district heat, pool operation, …) and meters are user-defined, not hardcoded. Multiple meters of the same type are first-class (the reference data has **five** electricity meters).
- **Cost engine** with time-ranged tariffs (price history), per-type and per-category rollups.
- **Neat dashboard:** daily / monthly / yearly cost; period-over-period difference ("what cost more, what cost less"); PV savings; oil/tank forecasting; cost ranking.
- **Scale target:** up to ~1,000 meters, data retained up to 50 years.
- **Self-hosted, zero cloud dependency.** Runs on the existing homelab (Docker on Unraid, or Proxmox LXC).
- **Releasable OSS quality:** clean repo, Docker image, docs, CI, license.

### 1.3 Non-goals (v1)

- Not a smart-meter *reading* device (no P1/SML hardware decoding — that stays in HA/ESPHome/Tasmota upstream; we ingest the resulting values).
- Not a billing/invoicing system for third parties.
- Not multi-tenant SaaS. Single household/instance; optional lightweight multi-user, but not tenant isolation.
- No mobile native app (responsive web is enough).

---

## 2. What the reference data establishes

The four CSVs are the minimum feature bar. Summary of what each proves the app must support:

### 2.1 `Kosten` (cost overview) — monthly
Columns: `Datum, Jahreskosten, Kosten, Heizung, Strom, Wasser, Pool Betrieb`.
→ A **cost-category** rollup: monthly total plus a breakdown per category (Heizung/Strom/Wasser/Pool), and a yearly total (on December rows). "Heizung" is a *category* that may be fed by oil today but gas/heat-pump tomorrow — categories are decoupled from energy types. "Pool Betrieb" can be a **flat manual monthly cost** with no meter.

### 2.2 `Strom_Verbrauch` (electricity) — monthly, rich
Meters (cumulative registers): `Zähler Haus`, `Zähler Netz`, `Zähler Auto` (wallbox/EV), `Zähler Solar 1`, `Zähler Solar 2` → **5 electricity meters**.
Derived quantities: `Solar Erzeugung` (generation), `Netz Einsparung` (grid-balance), `Anlage Eigenverbrauch` (self-consumption), `Ersparnis` (savings), `Verbrauchskosten`, `Kosten` (= `Verbrauchskosten − Ersparnis`), plus yearly rollups.
Price `€/kWh` with **history** (0.16 → 0.44 → 0.31 → 0.27).
→ Must support: multiple registers per type; **generation** meters; **virtual/derived** meters computed from others via a formula; PV self-consumption & savings; register→consumption deltas; tariff-time-ranged cost. Verified relation: `Netz Einsparung = Haus − Netz`, `Ersparnis = Netz Einsparung × €/kWh`, `Kosten = Verbrauchskosten − Ersparnis`. The exact formula is user-domain — the app must let users **define** such derived metrics, not hardcode these.

### 2.3 `Wasser` (water) — monthly
`Zähler Wasser, Wasserverbrauch, €/m³, Kosten, Jahreskosten`. The register **swaps mid-series** (…861 → 2 → 15 → …).
→ Must support **meter swaps / counter resets** with consumption continuity across the boundary.

### 2.4 `Heizöl_Verbrauch` (heating oil) — the versatility stress test
Header KPIs: total delivered `64042`, tank size `7000`, consumption per month/day/year.
Table columns: `Betriebststunden` (cumulative burner hours), `Differenz Betrieb` (Δ hours), `Betrieb / Tag`, `Füllstand cm` (manual dipstick), `Tankfüllstand`/`Tank Aktuell` (litres), `Vorhersage`, `Lieferungmenge` (delivery litres), `Differenz Tank` (Δ level = monthly consumption), `Verbrauch / Tag`, `Vorraussichtliches Ende` (predicted empty), `Verbrauch / Betrieb Stunde` (L/h — **empirically derived**, e.g. 1.87, 1.94, 2.92 …), `€/100l`, `Monatskosten`.
Early rows (1997–2004) only carry **deliveries** (no burner hours — tracking started later).
→ Must support a **consumable/tank** model: deliveries add to a balance; consumption derivable **two ways** — (a) tank-level Δ, (b) burner **runtime × rate** — where the rate can be **fixed** (nozzle spec) or **empirical** (tank Δ ÷ hours Δ); physical level readings (cm) via a tank calibration curve; and a **forecast to empty**. This is the "not limited to oil" generalization: any consumable drawn from a store and/or consumed proportionally to a runtime signal.

### 2.5 Cross-cutting data facts (drive the CSV importer, §Appendix A)
- Decimal separator is **comma**; thousands separator **dot**; currency like `2.940,19 €`.
- Values carry **unit suffixes**: `411kWh`, `49` cm, `2287` L.
- Two date formats: `"September 2022"` (month tables) and `10.06.1997` / `13.07.2026` (`DD.MM.YYYY`, event rows).
- **Summary rows** exist inline (`Total`, `Heute`, `Seitbeginn Tage`, `Seit 2023`) and must be skipped, not ingested.
- **Placeholder future rows** (Dec 2026 all-zero) must be treated as no-data.
- Negative values are valid (savings, grid balance).

---

## 3. Functional requirements

| ID | Requirement |
|----|-------------|
| FR-1 | Define arbitrary **energy types** (key, display name, base unit, icon, colour). Ships with sensible defaults but nothing is hardcoded. |
| FR-2 | Define **meters** (≤ ~1000), each bound to an energy type and a **measurement mode** (§5.2). Multiple meters per type. Serial/model/install/retire metadata. |
| FR-3 | Attach one or more **data sources** to a meter: MQTT topic, Tasmota field, HA entity, manual, import, or virtual (formula). Per-source scale/offset, priority, enable flag, last-seen status. |
| FR-4 | **Ingest on schedule / on message** from MQTT (incl. Tasmota) and HA (WebSocket push or REST poll). Idempotent; guard cumulative registers against spurious decreases; sample/debounce high-frequency sources. |
| FR-5 | **Manual entry**: add a reading, delivery, tank level, swap, or correction from the UI. |
| FR-6 | **CSV import** with column mapping, saved mapping profiles, German dialect handling (Appendix A), dry-run preview, and revertible import batches. The four reference CSVs must import correctly. |
| FR-7 | **Consumption normalization**: convert raw readings (register/level/runtime/rate) into normalized consumption/generation in the base unit, handling deltas, swaps, resets, deliveries, runtime×rate. |
| FR-8 | **Virtual meters**: values computed from other meters via a user-defined expression (self-consumption, savings, net). |
| FR-9 | **Tariffs** with time ranges (price history): unit price, base/standing price, feed-in tariff, bonus, discount, tax. Scope: global, per energy type, or per meter. Support **monthly** pricing *and* **day-accurate proration** when price changes mid-period. |
| FR-10 | **Cost categories** (Heizung/Strom/Wasser/Pool …) mapping one or more meters/types → a category; plus **manual flat costs** (e.g. pool) with no meter. |
| FR-11 | **Aggregation**: hourly/daily/monthly/yearly consumption, generation and cost, per meter and per category. |
| FR-12 | **Dashboard** (§8): today/month/year cost KPIs with Δ vs previous period; cost breakdown & "what costs most"; period-over-period difference view; trends with granularity toggle and previous-year overlay; PV panel; oil/consumable panel; meter detail. |
| FR-13 | **PV analytics**: generation, self-consumption, autarky %, self-consumption %, savings. |
| FR-14 | **Consumable/tank analytics**: balance, deliveries log, burner runtime, effective L/h (fixed or empirical), forecast to empty. |
| FR-15 | **REST API + OpenAPI** for ingest (push), query, and automation; API-key auth. Lets HA *push* as an alternative to us *pulling*. |
| FR-16 | **Retention & storage** configurable to satisfy up to 50 years of data (§5.5). |
| FR-17 | **i18n**: English + German UI; locale-aware number/currency/date formatting. |
| FR-18 | **Deploy** via Docker Compose (app + TimescaleDB); Unraid template; healthcheck endpoint; backup guidance. |
| FR-19 | **Auth**: optional local accounts *and* reverse-proxy trust (honour `X-Forwarded-User` behind Authelia/Traefik). |

> **Where the implementation differs (D-58):**
> - **FR-9:** prices are monthly (the 15th of each local month); there is no day-accurate proration mode. Bonus,
>   discount and tax are stored but not applied (§7.5, D-57).
> - **FR-11:** rollups are by local day and month. Weeks and years are built from them, and there is no hourly level
>   (§5.4).
> - **FR-12:** the Overview shows one selected period instead of fixed today/month/year cards (§8.1).
> - **FR-16:** raw retention is shown but not enforced (§5.5).

---

## 4. Architecture & tech stack

### 4.1 Stack (recommended, committed)

| Layer | Choice | Rationale |
|-------|--------|-----------|
| Runtime | **.NET (current LTS — .NET 10; .NET 8 acceptable)**, C# | Matches your toolchain (Rider, C#-first) and the MQTTower precedent (.NET Blazor + MQTT). |
| Web/UI | **ASP.NET Core + Blazor Server** | Same model as MQTTower; server-side keeps DB/time-series logic close, good for a homelab dashboard. |
| Component kit | **MudBlazor** | Mature, clean, good tables/cards/dialogs. |
| Charts | **ApexCharts (Blazor-ApexCharts)** | Solid for time-series, stacked bars, mixed cost/consumption. |
| MQTT | **MQTTnet** | The standard .NET MQTT lib; used for Tasmota + HA-published topics. |
| DB | **PostgreSQL + TimescaleDB** | Postgres you already know, with hypertables, native compression, and continuous aggregates — the right tool for 1000 meters × 50 years. |
| ORM / data | **EF Core (Npgsql)** for schema/migrations/CRUD; **Dapper** for hot-path time-series reads | EF for productivity; Dapper + raw SQL where Timescale features (hypertables, CAggs, `time_bucket`) need it. |
| Background work | **`BackgroundService` / hosted services** | MQTT subscriber, HA poller, aggregation refresh, forecast recompute. |
| Tests | **xUnit + Testcontainers (Timescale image)** | Real DB in integration tests; the 4 CSVs as fixtures. |
| Container | **Docker + docker-compose**; **GHCR** multi-arch (amd64 primary) via GitHub Actions | Homelab-native distribution. |

> The **domain model and DB schema are UI-agnostic.** If a future maintainer swaps Blazor for an SPA, everything from §5–§7 and the REST API in §9 is reusable.

> **Deviation (D-17, D-58):** continuous aggregates are not used. They were dropped in favour of rollup tables that the
> recompute writes (§5.4). Dapper serves the hot-path reads of those tables (`AnalysisQueries`). No hosted service
> refreshes aggregates.

### 4.2 Components & data flow

```
 ┌──────────── sources ────────────┐
 │  Tasmota  ──tele/<t>/SENSOR──┐   │
 │  HA (WS/REST)  ──────────────┤   │        ┌─────────────────────────────┐
 │  raw MQTT  ──────────────────┼──▶│ Ingestion │  normalize  │  TimescaleDB │
 │  Manual / CSV import ────────┘   │  workers  │  pipeline   │  (hypertables│
 └──────────────────────────────────┘          │  (deltas,   │   + CAggs)   │
                                                │  swaps,     └──────┬───────┘
                                                │  runtime×r) │       │
                                                └─────────────┘       │
                                          Cost engine (tariff join)   │
                                                     │                │
                                    ┌────────────────┴────────────────┴───┐
                                    │  Blazor dashboard   +   REST API     │
                                    └──────────────────────────────────────┘
```

Ingestion writes **raw `reading`** rows. A normalization step derives **`consumption`** (append-only, base unit). Continuous aggregates roll consumption up to hourly/daily/monthly/yearly. The cost engine joins aggregates with time-ranged tariffs. The dashboard and API read aggregates + cost views (never scan raw for charts).

> **Current pipeline (D-12 – D-17, D-27, D-34):** the recompute derives `consumption`. In the same transaction it
> writes per-meter rollups by local day and local month, plus coverage runs, all in the configured zone (§5.4). One
> shared **analysis reader** (`AnalysisReader`) reads those rollups. It evaluates virtual meters on read (§7.4) and
> classifies per-type totals. One **cost engine** (`CostReader`) prices the bill from those quantities and the
> time-ranged tariffs (§7.5). The pages, the REST API and the CSV export read only these two readers. None of them
> scans `reading`.

---

## 5. Data model & database

Design principles: raw readings are immutable audit truth; everything derived (consumption, cost, balances, forecasts) is computed on top and reproducible; the big table is a Timescale hypertable; long-horizon retention is served by aggregates, not by keeping every raw row forever.

### 5.1 Entity overview

- `energy_type` — user-defined categories of measurement.
- `meter` — the "device" (≤1000). Has a **measurement mode**.
- `meter_source` — 0..n ingest bindings per meter.
- `reading` *(hypertable)* — raw timestamped values.
- `consumption` *(hypertable)* — normalized, append-only deltas in base unit.
- `meter_event` — discrete events: swap, reset, delivery, tank level, correction, note.
- `tank` — consumable store: capacity, cm→litre calibration, thresholds, cached balance.
- `tariff` — price components with validity ranges (history).
- `cost_category` + `cost_category_member` — reporting groups (Heizung/Strom/Wasser/Pool).
- `manual_cost` — flat costs with no meter (e.g. pool).
- `ingestion_endpoint` — broker / HA connection configs (secrets by reference).
- `import_batch` — provenance + revert for CSV/manual bulk loads.
- `app_setting` — currency, locale, timezone, retention, fallbacks.

Added since this sketch (see §5.4 and the note, D-12):

- `meter_link` — directed flow topology (`from → to`: the downstream meter is a subsection of the upstream one;
  several parents are allowed). It is topology only: it never defines a virtual meter's calculation (D-25).
- `consumption_rollup` / `consumption_rollup_month` — per-meter sums of `consumption` by local day and local month,
  with per-provenance amounts, row count, flags and the latest interval end.
- `meter_coverage` — per-meter coverage runs with their resolution class.
- `meter_rollup_state` — the revision, zone, normalized unit and kind each meter's analysis rows were built with.

> **Deviation (D-57):** the principle "long-horizon retention is served by aggregates, not by keeping every raw row
> forever" is not realised. Raw readings are kept indefinitely (§5.5).

### 5.2 Measurement modes (`meter.mode`)

| Mode | Meaning | Consumption derived by |
|------|---------|------------------------|
| `cumulative_counter` | Monotonic register (Zähler Haus/Netz/Auto, water) | Δ register between readings; handle swaps/resets. |
| `generation_counter` | Monotonic generation register (Solar 1/2) | Δ register → generation. |
| `runtime_counter` | Cumulative operating hours (burner) | Δ hours × rate (fixed or empirical) → consumption. |
| `consumable_balance` | Tank/bottle with deliveries + level | Deliveries add; usage from level-Δ and/or runtime×rate; forecast to empty. |
| `direct_delta` | Source already reports increments | Value *is* the increment. |
| `instant_rate` | Power/flow sensor (optional v1) | Integrate rate over time. |
| `virtual` | Computed from other meters | Evaluate `expression` over referenced meters. |

### 5.3 Schema sketch (PostgreSQL + TimescaleDB)

> Illustrative DDL; EF Core migrations own the relational tables, and **raw-SQL migrations** own the Timescale-specific DDL (`create_hypertable`, compression, continuous aggregates, retention). Timescale objects are *not* expressible through EF's model builder.
>
> *Current:* the Timescale DDL in use is the two hypertables (`reading`, `consumption`) and raw compression. The
> continuous aggregates were dropped by the `AnalysisRollups` migration (D-17), and no retention policy exists (D-57).
> The rollup, coverage and state tables are ordinary EF-owned tables with `ON DELETE CASCADE` to `meter` (D-12).

```sql
CREATE TABLE energy_type (
    id            SMALLSERIAL PRIMARY KEY,
    key           TEXT NOT NULL UNIQUE,          -- 'electricity','water','heating_oil','gas','heat','pool'
    display_name  TEXT NOT NULL,
    base_unit     TEXT NOT NULL,                 -- 'kWh','m3','L','h'
    default_mode  TEXT NOT NULL,                 -- see 5.2
    icon          TEXT,
    color_hex     TEXT,
    created_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE meter (
    id             INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    name           TEXT NOT NULL,
    energy_type_id SMALLINT NOT NULL REFERENCES energy_type(id),
    mode           TEXT NOT NULL,
    unit           TEXT NOT NULL,                -- defaults from energy_type.base_unit
    location       TEXT,
    serial_number  TEXT,
    model          TEXT,
    manufacturer   TEXT,
    installed_at   DATE,
    retired_at     DATE,
    is_active      BOOLEAN NOT NULL DEFAULT true,
    meta           JSONB NOT NULL DEFAULT '{}',  -- rate config, formula, tank ref, etc.
    created_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at     TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX ix_meter_type_active ON meter(energy_type_id, is_active);

CREATE TABLE meter_source (
    id           INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    meter_id     INT NOT NULL REFERENCES meter(id) ON DELETE CASCADE,
    source_type  TEXT NOT NULL,                  -- 'mqtt','tasmota','homeassistant','manual','import','virtual'
    endpoint_id  INT REFERENCES ingestion_endpoint(id),
    config       JSONB NOT NULL DEFAULT '{}',    -- topic / field path / entity_id / expression / poll interval
    value_kind   TEXT NOT NULL,                  -- 'register','delta','rate','level','runtime'
    scale        DOUBLE PRECISION NOT NULL DEFAULT 1,
    "offset"     DOUBLE PRECISION NOT NULL DEFAULT 0,
    priority     INT NOT NULL DEFAULT 0,
    is_enabled   BOOLEAN NOT NULL DEFAULT true,
    last_seen_at TIMESTAMPTZ,
    last_value   DOUBLE PRECISION,
    last_status  TEXT
);

-- BIG TABLE: raw readings
CREATE TABLE reading (
    time      TIMESTAMPTZ      NOT NULL,
    meter_id  INT              NOT NULL REFERENCES meter(id),
    value     DOUBLE PRECISION NOT NULL,        -- register value / level / hours / rate, in meter.unit
    source_id INT REFERENCES meter_source(id),
    quality   SMALLINT         NOT NULL DEFAULT 0, -- 0 measured,1 estimated,2 manual,3 imported,4 interpolated
    flags     INT              NOT NULL DEFAULT 0, -- bitmask: reset, anomaly, ...
    PRIMARY KEY (meter_id, time)
);
SELECT create_hypertable('reading', 'time',
       chunk_time_interval => INTERVAL '30 days');
ALTER TABLE reading SET (timescaledb.compress,
       timescaledb.compress_segmentby = 'meter_id',
       timescaledb.compress_orderby   = 'time DESC');
SELECT add_compression_policy('reading', INTERVAL '30 days');

-- Normalized consumption (append-only deltas in base unit)
CREATE TABLE consumption (
    time      TIMESTAMPTZ      NOT NULL,         -- end of the interval this delta covers
    meter_id  INT              NOT NULL REFERENCES meter(id),
    amount    DOUBLE PRECISION NOT NULL,         -- consumption(+) or generation(+) in base unit
    kind      SMALLINT         NOT NULL DEFAULT 0, -- 0 consumption, 1 generation
    quality   SMALLINT         NOT NULL DEFAULT 0,
    PRIMARY KEY (meter_id, time, kind)
);
SELECT create_hypertable('consumption', 'time',
       chunk_time_interval => INTERVAL '90 days');

CREATE TABLE meter_event (
    id          INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    meter_id    INT NOT NULL REFERENCES meter(id) ON DELETE CASCADE,
    time        TIMESTAMPTZ NOT NULL,
    event_type  TEXT NOT NULL,                   -- 'meter_swap','counter_reset','delivery','tank_level','correction','note'
    amount      DOUBLE PRECISION,                -- delivery litres / correction value
    prev_value  DOUBLE PRECISION,                -- swap: old register final
    new_value   DOUBLE PRECISION,                -- swap: new register initial
    unit        TEXT,
    notes       TEXT,
    meta        JSONB NOT NULL DEFAULT '{}'
);
CREATE INDEX ix_event_meter_time ON meter_event(meter_id, time);

CREATE TABLE tank (
    id              INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    meter_id        INT NOT NULL REFERENCES meter(id) ON DELETE CASCADE,
    capacity        DOUBLE PRECISION NOT NULL,   -- e.g. 7000
    unit            TEXT NOT NULL DEFAULT 'L',
    calibration     JSONB,                       -- cm→litre curve or geometry for level readings
    rate_mode       TEXT NOT NULL DEFAULT 'empirical', -- 'fixed' | 'empirical'
    fixed_rate      DOUBLE PRECISION,            -- L per runtime-hour when rate_mode='fixed'
    low_threshold   DOUBLE PRECISION,
    reorder_threshold DOUBLE PRECISION,
    cached_balance  DOUBLE PRECISION,            -- last computed Tank Aktuell
    cached_at       TIMESTAMPTZ
);

CREATE TABLE tariff (
    id          INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    scope_type  TEXT NOT NULL,                   -- 'global','energy_type','meter'
    scope_id    INT,                             -- energy_type.id or meter.id (null for global)
    component   TEXT NOT NULL,                   -- 'unit_price','base_price','feed_in','bonus','discount','tax'
    value       DOUBLE PRECISION NOT NULL,
    unit        TEXT NOT NULL,                   -- 'EUR/kWh','EUR/m3','EUR/100L','EUR/month'
    currency    TEXT NOT NULL DEFAULT 'EUR',
    valid_from  DATE NOT NULL,
    valid_to    DATE,                            -- null = open-ended
    notes       TEXT
);
CREATE INDEX ix_tariff_scope ON tariff(scope_type, scope_id, component, valid_from);

CREATE TABLE cost_category (
    id        INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    name      TEXT NOT NULL,
    color_hex TEXT,
    sort      INT NOT NULL DEFAULT 0
);
CREATE TABLE cost_category_member (
    category_id    INT NOT NULL REFERENCES cost_category(id) ON DELETE CASCADE,
    meter_id       INT REFERENCES meter(id) ON DELETE CASCADE,
    energy_type_id SMALLINT REFERENCES energy_type(id) ON DELETE CASCADE,
    CHECK (meter_id IS NOT NULL OR energy_type_id IS NOT NULL)
);

CREATE TABLE manual_cost (
    id          INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    category_id INT REFERENCES cost_category(id),
    meter_id    INT REFERENCES meter(id),
    period_start DATE NOT NULL,
    period_end   DATE NOT NULL,
    amount      DOUBLE PRECISION NOT NULL,
    currency    TEXT NOT NULL DEFAULT 'EUR',
    notes       TEXT
);

CREATE TABLE ingestion_endpoint (
    id         INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    type       TEXT NOT NULL,                    -- 'mqtt_broker','homeassistant'
    name       TEXT NOT NULL,
    config     JSONB NOT NULL DEFAULT '{}',      -- host/port/tls/base_url; secrets by *reference* only
    is_enabled BOOLEAN NOT NULL DEFAULT true,
    last_status TEXT,
    last_seen_at TIMESTAMPTZ
);

CREATE TABLE import_batch (
    id          INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    source_name TEXT,
    mapping     JSONB,
    row_count   INT,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    reverted_at TIMESTAMPTZ
);

CREATE TABLE app_setting (
    key   TEXT PRIMARY KEY,
    value JSONB NOT NULL
);
```

### 5.4 Continuous aggregates & cost view

> **Deviation (analysis rework, `docs/ANALYSIS_IMPLEMENTATION_NOTE.md` D-10 – D-17, D-36, D-58):** the continuous
> aggregates below were Berlin-only and materialized-only. They were never backfilled after historical imports, and
> no reader used them. The `AnalysisRollups` migration dropped them together with their refresh policies. The SQL
> below is kept as the original design. What replaced them:
>
> - **Rollup tables** (plain tables, FK to `meter` with `ON DELETE CASCADE`), all keyed by local dates of the
>   **configured** zone:
>
>   | Table | Key | Holds |
>   |---|---|---|
>   | `consumption_rollup` | `(meter_id, day, kind)` | amount; measured/manual/imported/estimated shares; row count; flags (baseline delta, divided); latest interval end |
>   | `consumption_rollup_month` | `(meter_id, month, kind)` | the same per local month (read for month and year buckets) |
>   | `meter_coverage` | `(meter_id, span_from)` | coverage runs: span, resolution class (≤ 1 h, ≤ 1 day, ≤ 7 days, ≤ 1 month, coarser), whether divided at months, gap reason, last interval start |
>   | `meter_rollup_state` | `(meter_id)` | normalization revision, zone, normalized unit and kind, build time |
>
> - **Written by the recompute, by diff, in its transaction.** `NormalizationService.RecomputeMeterAsync` stages them
>   through `AnalysisDataWriter` in the same transaction as the meter's `consumption`. Only changed rows are touched.
>   Import, revert, events, manual readings, live ingestion and meter edits all reach the next read without a refresh
>   job. No cache sits in between.
> - **Intervals.** Every normalized row carries its source interval (EF-ignored, D-10), so coverage is captured while
>   normalizing instead of being guessed from sums. A non-label row that ends exactly on a local midnight is stamped one
>   second earlier, inside the day it closes (D-11).
> - **Bucket status** (D-14) comes from coverage, never from the amount:
>   - `Available`: covered. A zero is a true zero.
>   - `Partial`: only partly covered.
>   - `Missing`: nothing covers the bucket.
>   - `Unresolved`: covered only by an undivided interval that crosses the bucket edge, e.g. monthly data asked by day.
>   - `Invalid`: a calculation failed (virtual meters, §7.4).
>   - `Pending`: the rollups are being rebuilt.
>   A to-date read caps coverage at "now". Rows whose interval closes after now are reported, never counted (D-04,
>   A-04, A-05, A-14, A-20).
> - **Reads.** Month and year buckets read the month table. Days and weeks read the day table. At most two partial
>   edge days per range come straight from `consumption`. Every request makes one query per table, and the point and
>   series limits are checked before any SQL runs (D-15).
> - **Rebuild.** Normalization revision 3 (D-16) rebuilds consumption, rollups and coverage for every meter at the
>   next start (`NormalizationUpgrade`). It rebuilds again whenever the revision or the configured zone differs from
>   `meter_rollup_state`. Until then a meter reads as "analysis being prepared" (`Pending`), never as "no data".
>
> **Cost is not a view either.** The cost engine (`CostReader`, §7.5) prices each bucket month by month, at the price
> valid on the 15th of each local month (D-36), so the bucket size never changes a total. There is no day-accurate
> proration mode (§14.3). A price change inside a reading interval longer than a month is reported, not guessed (A-16).

```sql
-- Daily normalized consumption per meter (local-tz buckets)
CREATE MATERIALIZED VIEW consumption_daily
WITH (timescaledb.continuous) AS
SELECT time_bucket('1 day', time, 'Europe/Berlin') AS day,
       meter_id, kind,
       sum(amount) AS amount
FROM consumption
GROUP BY day, meter_id, kind;
-- + monthly and yearly CAggs the same way (bucket '1 month' / '1 year').
SELECT add_continuous_aggregate_policy('consumption_daily',
       start_offset => INTERVAL '3 days', end_offset => INTERVAL '1 hour',
       schedule_interval => INTERVAL '1 hour');
```

Cost is **not** baked into a continuous aggregate (tariffs are a slowly-changing dimension; a CAgg can't join them cleanly). Instead, compute cost in a regular SQL view / function on top of the consumption CAggs, resolving the active `unit_price`/`base_price`/`feed_in` for each bucket by date. Support two modes: **monthly price** (one price per calendar month, matching the spreadsheet) and **day-accurate proration** (split a bucket if a price change falls inside it).

### 5.5 Capacity & retention (the 1000×50y requirement)

Worst-case raw volume, 1000 meters:

| Ingest granularity | Rows/day | Rows/year | Rows/50y |
|--------------------|----------|-----------|----------|
| 1/min | 1.44 M | 525 M | **26.3 B** |
| 5/min (typical Tasmota TelePeriod 300) | 288 k | 105 M | **5.25 B** |
| daily (manual/legacy) | 1 k | 365 k | 18 M |

Normalized/rolled-up volumes are tiny regardless: daily consumption = 1000 × 365 × 50 = **18.25 M**, monthly **600 k**, yearly **50 k**.

**Strategy:**
- **Raw `reading`:** Timescale hypertable + columnar compression (`segmentby meter_id`), typically 10–20× on monotonic sensor data. Raw is kept for a **configurable window** (default: 3 years) — long enough for full-resolution drill-down.
- **`consumption` + continuous aggregates:** the long-term source of truth. Kept effectively **forever** (they're small), which is what actually satisfies "50 years of data" for dashboards and cost.
- If the user *insists* on 50 years of raw high-frequency data, it's still feasible on a homelab NAS (single-digit TB compressed) — expose raw retention as a setting, don't hardcode.
- Recommend `space` partitioning by `meter_id` only if meter count and query patterns justify it; start with time-only chunks.

> Design conclusion: 1000 meters × 50 years is comfortably within TimescaleDB on modest hardware **provided** dashboards read aggregates and raw retention is bounded. Don't let the UI scan `reading` for charts.

> **Deviation (D-57, a documented limitation):** raw retention is **not enforced**. `MeterVault__RawRetentionDays`
> (default 1095) is shown, but nothing deletes readings. Every recompute rebuilds a meter's consumption from the
> readings that remain, so dropping old readings would destroy analytical history. Bounded raw retention first needs a
> recompute that can start from stored consumption. `/admin/settings` shows "Not enforced" with that reason, and the
> meter page's Readings tab explains it.
>
> **History reads the rollups** (§5.4 deviation): `consumption_rollup_month` for month/year buckets and
> `consumption_rollup` for day/week buckets. `consumption` itself is read only for at most two partial edge days per
> range and by the meter page's Normalized data tab. No chart reads `reading`. It is read by the paged Readings tab
> (100 rows per page, keyset-ordered, date-filtered, D-50), by the manual-entry checks, and for freshness: the latest 20
> reading times per meter (D-18). That freshness query has no time bound yet (§13, report). `consumption` and the
> rollups are the analytical history. Performance was measured against a synthetic 1,000-meter × 10-year dataset
> (`docs/ANALYSIS_REPORT.md`, §13).

---

## 6. Ingestion

### 6.1 MQTT / Tasmota worker
- A hosted `BackgroundService` maintains a persistent MQTTnet connection per enabled `ingestion_endpoint` of type `mqtt_broker`.
- Subscribes to the union of topics from enabled `meter_source`s (and Tasmota patterns like `tele/+/SENSOR`).
- On message: resolve topic → source(s); extract value via JSON path/template from `config` (Tasmota energy under `ENERGY.Total`, `ENERGY.Today`, `ENERGY.Power`; generic sensors by path); apply `scale`/`offset`; write `reading` with a timestamp (prefer the payload's own time field, else Tasmota `Time`, else receive time).
- **Sampling/debounce:** per-source policy — store on-change and/or at most 1/min for chatty sources, to keep raw volume in check.
- **Idempotent** upsert on `(meter_id, time)`. For `cumulative_counter`/`generation_counter`, reject decreases unless an active `counter_reset`/`meter_swap` event explains it.

### 6.2 Home Assistant connector
- Prefer **WebSocket API** (`auth` with long-lived token → `subscribe_events` / `state_changed`) for push; fall back to **REST poll** `/api/states/<entity_id>` on a per-source interval.
- Extract `state` or a named `attribute`; parse units; write `reading`.
- Optional **backfill** on first connect / after downtime via `/api/history/period`.
- HA can also simply publish to MQTT — in which case it's an `mqtt`/`tasmota` source and no HA connector is needed.

### 6.3 Manual & CSV import
- UI quick-add for readings, deliveries, tank levels, swaps, corrections.
- CSV wizard: upload → detect dialect (Appendix A) → map columns to meters + row semantics → **dry-run preview** (computed consumption/cost shown) → commit as an `import_batch` (revertible).
- Ship the four reference CSVs as built-in example imports and as test fixtures.

### 6.4 Secrets
Tokens/passwords are **never** stored in plaintext in the DB. Two storage forms satisfy this, chosen per connector in the admin UI:

- **By reference** — `ingestion_endpoint.config` names an env var / Docker secret path (`token_env`, `password_env`); the app resolves it at runtime.
- **Encrypted at rest** — the operator types the secret into the connector dialog and it is stored encrypted (`token_enc`, `password_enc`) under the ASP.NET Core data-protection key ring.

Exactly one form survives a save; switching clears the other. The encrypted form exists because reference-only forced a file edit plus a service restart to add a connector, which in practice led to tokens being pasted into the env-var *name* field. It keeps the guarantee that matters — a `pg_dump` or JSON export carries nothing usable — but note the trust boundary: the key ring is on disk, so it protects against leaked database content, not against an attacker who already has the host. That is the same boundary as an env var, which is equally readable from `/proc`.

The key ring must be persisted outside the app directory (`MeterVault__DataProtectionKeyPath`, default `/var/lib/metervault/keys`), or a redeploy that replaces the content root will orphan every stored secret. MQTT *usernames* are not secrets and are stored as-is. JSON exports drop `*_enc` values: they are bound to the originating key ring and so are useless where an export would be restored — expect to re-enter secrets after a restore.

---

## 7. Consumption normalization & cost engine

### 7.1 Register → consumption
For `cumulative_counter`/`generation_counter`: for each new reading, `amount = value − previous_value`. Persist to `consumption`. Cross a `meter_swap` as `(old_final − prev) + (curr − new_initial)`; a `counter_reset` starts a fresh baseline. Ignore/annotate negative deltas that lack an explaining event (flag as anomaly).

**Month attribution.** A reading is an instant, and the consumption between two readings accrued over the time between them. Booking the whole delta at the closing reading misfiles it whenever the interval crosses a month boundary: readings on 1 August and 16 September would show six weeks of use in September and none in August. So a plain increase whose interval crosses one or more **local** month boundaries (instance timezone, §10 — the months the charts bucket by) is divided at those boundaries in proportion to elapsed time. Each share is stamped inside its month — the closing reading keeps its own timestamp for the month it falls in, other shares take the last second of their month — and a divided interval's rows are marked `quality = estimated`: the meter recorded a total, not a shape. The parts always sum to the original.

Imported monthly tables keep the golden fixtures reconciling (§13). A row labelled "Mai 2026" carries the register at the *end* of May and May's consumption, but is stamped 00:00 UTC on 1 May so it files under its month. The importer flags such a row `reading.flags & month_label` — only it still knows whether the date cell named a month or a day, and a day-dated "01.08.2026" at the same midnight is an ordinary instant. A month label is read as the end of its month: readings are walked in that effective order by every register normalizer and by the checks that judge a new reading against its predecessor (so a sheet imported after live readings of the same month does not count the month twice, and a mid-month reading below the month's end value is not a decrease), consecutive rows span exactly their closing month and book unchanged, a skipped month is shared between the months the gap covers, and a live reading after the last imported row counts from the end of that row's month rather than claiming it a second time. A label is stamped inside the month it names — at its own timestamp where that lies in the local month, otherwise (zones behind UTC) at the month's local start. Swap and reset amounts are never divided: they are explicit corrections booked at the event. A swap the importer detected at a month row applies from the start of that local month, i.e. to the first reading in it, and a new register's recorded start value never counts above that reading. Should two rows still land on the same instant, the engine adds them into one estimated row rather than producing a duplicate key.

Every reader that buckets consumption by month or day (cost, trends, solar, consumables, flow, meter detail) buckets in the configured instance timezone — the same zone the division uses — never a hard-coded one, and starts and ends requested periods at local midnight. The zone id is normalised to its IANA form, and one unknown to .NET or PostgreSQL is reported at startup. Because consumption is derived, a change to these rules is applied to stored data at startup: `app_setting.normalization_revision` and `normalization_zone` record the rule revision and zone the stored series was built with, and every non-virtual meter is recomputed when either differs (the first run also flags month rows of earlier monthly imports, identified from each batch's stored mapping; a batch whose dates were auto-detected counts as monthly when all its rows sit on the 1st across at least two months, which is logged; if the flagging fails, nothing is rebuilt and the upgrade is retried at the next start). A meter whose rebuild fails is logged and listed in `normalization_pending`, retried at the next start, and never stops the application from starting.

> **Current (normalization revision 3, D-10, D-11, D-16):**
> - Every normalized row carries its source interval, so coverage and resolution can be recorded (§5.4).
> - A reading that closes exactly at a local midnight is booked in the day it closes, not the next one.
> - The startup rebuild also writes the rollup and coverage tables, and records each meter's `meter_rollup_state`.
>   Virtual meters store no consumption: the migration purged their rows, and a meter that becomes virtual is purged
>   on its next recompute.
> - A meter whose oldest consumption predates its oldest reading or event is skipped and logged, so its history is not
>   truncated.
> - Month division still applies only to cumulative and generation counters. Tank, runtime, direct-delta and
>   instant-rate intervals are booked whole. Where such an interval crosses a month edge, the months on either side are
>   `Unresolved` and only coarser buckets are resolved (D-14, A-16).

### 7.2 Runtime → consumption (burner)
For `runtime_counter`: `amount = Δhours × rate`. `rate` comes from the linked `tank`: `fixed` (nozzle spec, L/h) or `empirical` (`Δlevel ÷ Δhours` measured between deliveries/level reads — reproduce the spreadsheet's 1.87/1.94/2.92 … behaviour). Expose both; default empirical when level data exists, else fixed.

### 7.3 Consumable/tank balance & forecast
`balance(t) = Σ deliveries(≤t) − Σ consumption(≤t)`, reconciled to physical `tank_level` events when present (cm → litres via calibration). Forecast to empty from a trailing consumption rate (e.g. last-30-day L/day) → `Vorraussichtliches Ende`. Surface low/reorder thresholds.

> **Current (D-54, `TankLevels`):** the last dipstick ("Last dipstick: value on date") is kept apart from "Estimated
> now (incl. deliveries since)", which does not deduct use since the dipstick. For a period that ended before now,
> the Consumables page shows the contents at the period's end, never today's balance. The forecast is a labelled
> projection. It is hidden when the dipstick is older than 60 days, when too little time has passed, or when no use
> was measured. Deliveries before a tank's first level open a coverage gap (D-13).

### 7.4 Virtual meters
For `virtual`: evaluate `config.expression` (whitelisted, sandboxed — a small safe expression evaluator over referenced meters' consumption/generation series), e.g. `self_consumption = generation − grid_feed_in`, `savings = self_consumption * unit_price`. Persist results to `consumption` (or compute on read — decide per §14). This is how PV self-consumption/savings and net figures are modelled without hardcoding.

> **Deviation (D-25 – D-33, D-39, A-08, A-12, A-15, D-58):**
>
> - **Canonical definition in `meter.meta`:** `expression`, `referencedMeterIds` (always derived from the expression
>   and rewritten on save), `resultKind`, `resultUnit` and `costRule`. The result kind is one of `consumption`,
>   `generation`, `net` or `indicator`. Save writes the effective (inferred) kind, unit and cost rule, so readers never
>   re-infer them (A-08). Topology links (`meter_link`) never define or change a calculation. `meter_source` rows of
>   type `virtual` are not used for this.
> - **Formula:**
>   - Grammar: `+ − * /`, parentheses and numbers, parsed to an AST (`FormulaParser`), at most 2,000 characters and
>     nesting depth 64.
>   - References are `m<id>`. Any other identifier is an error; it is never read as 0.
>   - The old string evaluator and `VirtualNormalizer` were removed.
> - **Validation** (`VirtualValidator`), on save and again on read for legacy data, covers:
>   - syntax;
>   - unknown or self references;
>   - loops through nested virtual meters, reported with their path;
>   - kinds and units: `+`/`−` need the same unit and kind or a declared `net`; meter × or ÷ meter needs a declared
>     unit and kind `indicator`. Indicators are non-additive, never totalled and never costed.
> - **Evaluated on read, never materialized** (`VirtualEvaluator`, through `AnalysisReader`):
>   - Evaluation runs per bucket from the sources' rollups, in dependency order. Each physical source is read once,
>     however deeply it is nested.
>   - Coverage is the intersection of the sources' coverage, and the resolution is the coarsest among them.
>   - Strict: a missing source bucket makes the result missing and names the source. An observed zero is a valid input.
>   - A non-finite result (division by zero) is `Invalid` with the reason. So is a loop, which is reported with its
>     dependency path.
>   - A period total is the formula over the sources' totals across their joint coverage. For a linear formula without
>     a constant this equals the sum of its buckets; any other formula (for example a ratio) is marked non-additive,
>     and its total is the ratio of totals.
>   - The result carries every source's series (the "source contributions" on the meter page).
>   - Nothing is written to `consumption` for a virtual meter. This decides §14.1.
> - **Totals:** a virtual meter is an *analysis view* by default and is never added on top of the meters it reads.
>   The meter's `totals` override (`auto|always|never`) can make it replace its sources in its type's totals and in the
>   bill (D-23).
> - **Legacy definitions (D-28):** at startup (`VirtualDefinitionUpgrade`), an expression-less virtual meter whose
>   same-type incoming links name sources of one unit and kind gets the equivalent explicit sum stored. The run is
>   idempotent and logged. Anything ambiguous is flagged "needs configuration" and is never guessed. The seed writes
>   Summe Solar as `m4 + m5`, generation, kWh, cost rule `none`.
> - **Prices are not part of expressions** (`savings = self_consumption * unit_price` is not supported). A virtual
>   meter's cost follows its cost rule (§7.5): `sourceCosts` (pure sums: the sources' own metered costs), `ownQuantity`
>   (linear formulas: the evaluated quantity at its unit price) or `none`. PV savings are computed by the Solar page
>   (self-consumption × the grid unit price, month by month) or through `ownQuantity`.
> - Export/import carries `meter_link` and remaps meter ids inside definitions (D-32). Deleting a meter names the
>   virtual meters that read it and asks for confirmation (D-33).

### 7.5 Cost
`cost(bucket) = Σ(consumption_amount × active_unit_price) + base_price(prorated) − feed_in_credit − bonus`. Prices resolved by date from `tariff` (time-ranged). Currency from `app_setting`. Provide monthly-price and day-accurate-proration modes (§5.4). Categories roll costs up per `cost_category`; add `manual_cost` for meter-less categories (pool).

> **Deviation: the bill (D-22, D-34 – D-43, A-15 – A-19, A-21 – A-22, A-26 – A-27).** One cost engine (`CostReader` →
> `BillRun` → the Core `CostCalculator`) prices the portfolio, an energy type, a meter or a category for one resolved
> period. Every page, the REST API and the CSV export use it. Before the rework, costs summed every meter; this is
> what the engine does now:
>
> - **What is billed (D-22, D-34):**
>   - Per energy type, the `grid_import` meters are billed if the type has one. Otherwise its *use* meters are billed:
>     the `total_load` meter, or else the consumption roots of the topology.
>   - Generation meters are never billed.
>   - The feed-in credit is the FeedIn price × the export of `grid_export` meters.
>   - Runtime meters and virtual views are not billed.
>   - Submeters (topology children) are breakdowns, never added. So the seeded Strom bill is Zähler Netz × price, as
>     the sheet's `Kosten` is.
> - **Separately billed subsections (D-35, A-19):**
>   - A containment child with its own meter-scoped unit price is billed at that price, and its quantity is taken out
>     of its billed ancestor.
>   - The same applies to a consumer linked directly below a billed grid meter.
>   - Quantity totals do not change.
> - **Prices (D-36):**
>   - The spreadsheet's monthly convention is kept: the price valid on the **15th of each local month**.
>   - Every bucket is cut into its local months and each part priced at its month's price, so week, month and year
>     buckets, and the period total, agree.
>   - There is no day-accurate proration (§14.3).
>   - A reading interval longer than a month (a tank dipped every few months, quarterly burner hours) leaves its months
>     unknown. A longer bucket over such months is priced as a whole when every month in it has the same price;
>     otherwise it is unavailable, with an attention item (A-16).
> - **Tariff applicability (D-37):**
>   - A UnitPrice or FeedIn tariff applies only when its unit's denominator matches the meter's normalized unit.
>     Known scales are converted (ct, per 100 L, per MWh).
>   - A parsed unit or currency mismatch makes the cost "unavailable (unit)", and the explanation names what does not
>     fit (A-28). An unparseable unit applies, with a warning.
>   - BasePrice units are per day, per month (the default) or per year.
>   - The tariff editor checks units on save. It states that **Bonus, Discount and Tax are stored but not applied**
>     (D-57).
> - **Not priced vs price gap vs zero (D-38, A-26, A-27):**
>   - A billed scope with no UnitPrice tariff at any date is **not priced (no tariff)**. That is an attention item, not
>     a partial total.
>   - A hole in a priced scope's tariff history makes those months a **price gap** (cost unavailable).
>   - An explicit zero tariff is a **valid zero**. The tariff editor refuses to save a new tariff without a value, so a
>     deep link cannot create a free period by accident.
>   - A missing FeedIn price is reported only where a `grid_export` meter exists.
>   - Months with no grid meter in service, while use was measured, are unavailable rather than free (A-17).
>   - A bucket with nothing booked reads "No data", never "Priced" (A-26).
>   - The quantity analysis stays visible whenever the cost is unavailable.
> - **Standing charges (D-40, A-18):**
>   - A standing charge accrues per local day, at value ÷ days in that local month, over the scope's service period
>     (install or first data to retirement or now, regardless of reading gaps).
>   - Type- and global-scoped charges are their own rows, **once per scope**, never copied onto each meter.
>   - Meter-scoped fees stay on their meter. A fee on a meter that no bill line prices is its own row.
> - **Manual costs (D-41):**
>   - A manual cost is booked in full, **once**, on its `PeriodStart` local day when that day is in the period and not
>     after today. `PeriodEnd` is informational.
>   - They count in the Overview, the Analysis page, categories and the export alike.
> - **Categories (D-42, A-22):**
>   - A category's cost is the priced **non-overlapping cover** of its members, plus its manual costs.
>   - The disjoint categories, *Uncategorized* and the standing-charge rows form the **composition**, which reconciles
>     to the bill.
>   - A category that overlaps another, or covers meters outside the bill, is an **overlapping view**. It is listed
>     apart, never summed, and never drawn in the donut. The donut is drawn only when every slice is ≥ 0; otherwise
>     signed bars are used.
>   - A category whose members price nothing (calculated views, generation, runtime) says so rather than "no data".
> - **Virtual meters (D-39, A-15):** they are costed by their named rule:
>   - `sourceCosts` (pure sums only): each physical source's own metered cost, once, with no scope-level standing
>     charges.
>   - `ownQuantity` (linear formulas without a constant): the evaluated quantity at its unit price.
>   - `none`: every other formula, and generation sums (generation is never billed), which is why Summe Solar is not
>     costed.
>   - The rule is named next to every virtual cost. A virtual meter enters the bill only through the `always`
>     override, and then replaces its sources.
>   - The REST API reports a meter without a cost rule as `NotPriced` with the reason (A-21).
> - **Currency (D-43):** the configured `MeterVault__Currency` everywhere, through `Format.Money` / `InstanceCurrency`.
>   A tariff in another currency is a unit mismatch; no conversion is made.
> - **Changes (D-07, A-23):** a cost change is stated only between complete figures, by one rule on every page
>   (`OverviewComparison.Between`). If both totals are complete, it compares the totals. Otherwise it compares only the
>   paired buckets that are complete on both sides, with the caption "over the part both periods cover".

---

## 8. Dashboard & UX

> **Deviation (analysis rework, brief §3–§8, D-46 – D-55, D-58):** the panels below were rebuilt as one set of pages
> that share one period contract, one reader pair and one set of components. §8.0 describes what they share; the
> notes under §8.1 – §8.7 say what each page now is.

### 8.0 Shared analysis contract (current)

**Pages and navigation (D-47, D-48):**

| Sidebar entry | Route | What it is |
|---|---|---|
| Overview | `/` | One selected period (default month to date): the cost with its composition, a card per energy type (quantities in their own units, cost with billing basis, change, freshness), the history chart, "What changed", attention items |
| Analysis | `/trends` (route kept) | Explore any scope: `portfolio`, energy `type`, cost `category`, one `meter`, or up to six `meters` side by side, by quantity or cost |
| Meters | `/meters` | All meters with period values, status, resolution, "data up to", how each counts, quick entry, filter by type |
| Energy types → *type* | `/energy/{id}` | Tabs `overview`, `history` (total or `view=meters`), `flow` (Sankey + table + Manage connections), `meters` |
| Specialized views | `/solar`, `/consumables` | Always listed; each shows a setup state when unsupported (no generation meter, no tank) |
| Data import | `/import`, `/import/wizard` | Import batches with revert; the CSV mapping wizard |
| Configuration | `/admin/*` | Energy type *definitions*, tariffs, cost categories, connectors, settings |

- The meter page `/meters/{id}` is the per-meter hub. Its tabs are `analysis`, `readings`, `normalized`, `events`,
  `tariffs`, `sources`. A virtual meter shows `analysis`, `events`, `tariffs`, `calculation`: no Readings or
  Normalized data, and Calculation instead of Sources.
- Old links still work: `tab=consumption` opens `normalized`, and on a virtual meter `sources` opens `calculation` and
  `readings` opens `analysis`. Tabs are resolved by key and meter mode, never by index.
- One-shot `action=` links (reading, swap, reset, delivery, tank level, note, edit, source) open their dialog once and
  are then dropped from the address.
- Breadcrumbs (Overview → type → meter) carry the period. Expanded sidebar groups persist in a cookie (`mv-nav`), and
  the group of the current page is always open. If the energy types cannot be loaded, the nav shows an error with
  Retry instead of dropping the group.
- The app-bar "Find a meter" (a text button from the md breakpoint, an icon below) opens a meter's Analysis tab with
  the current period, next to the quick-entry action.

**Period and URL state (D-01 – D-08, D-46, A-13, `AnalysisQuery`):**

| Key | Values |
|---|---|
| `scope`, `id`, `ids` | `portfolio`, `type`, `category`, `meter`, `meters` (`ids`, at most 6) |
| `metric` | `consumption`, `generation`, `export`, `runtime`, `net`, `cost`, `balance` |
| `period` | `mtd`, `last-month`, `ytd`, `prev-year`, `12m`, `24m`, `all`, `custom` (with `from`/`to`, local, inclusive) |
| `bucket` | `auto`, `day`, `week` (Monday), `month`, `year` |
| `compare` | `none`, `prev-period`, `prev-year`, `year:YYYY` |

- The URL is the state: reload, share and Back reproduce the page. Defaults are never written: the Overview uses
  `mtd`, history pages `12m` (12 calendar buckets ending with the current partial month), and every page compares with
  the previous year (A-13).
- Toolbar and tab changes replace the history entry; drill-downs push. An invalid token falls back to the default
  with a notice.
- `all` spans the scope's available data, never a fixed century.
- `auto` picks one bucket, at most 400 points per series. A finer explicit bucket is refused with a coarser
  suggestion.
- A period resolves once per request against a captured "now" in the configured zone (§10). Quantities, costs,
  comparisons and the export use the same half-open bounds.
- Comparisons shift in calendar units. The change figure is measured only over the range both periods cover, and both
  exact ranges are shown (D-07). The percentage is "not applicable" for a zero or negative baseline; the absolute
  difference is always shown (D-08). Colours depend on the metric: more generation is good, more consumption is not.
- Projections are separate, labelled with their method, and suppressed where coverage is insufficient (D-09).
- Pages load through a `LoadSequencer`, so a superseded load never overwrites a later one. Panels show a
  loading/refresh/error-with-Retry state.
- Other page-specific keys: the energy page's `tab` and `view`, the Overview's `chart`, the record tabs' `from`/`to`.

**Missing vs zero vs not priced (brief §4.3, D-14, D-38, A-24 – A-28):**
- Every figure carries its bucket status (§5.4) and provenance (measured, manual, imported, estimated, derived,
  opening balance), worded next to it (`FigureText`).
- A true zero is a number and a bar on the baseline.
- An unknown bucket is a gap in the chart, marked "–", and "—" with its reason in the table: no data, only coarser
  data, cannot be calculated, being prepared.
- Qualified values (partial, estimated) are marked "*".
- A period without data says "No data for this period", names the available dates and offers "Go to latest data". A
  future range says it has not started yet.
- A chart never says "no data" when the reason is a coarser resolution (it names the resolution and offers the
  interval that shows it) or a missing price (it names the cost status).
- Drill-downs only go where finer data exists. A monthly bucket never opens 31 unknown days. A physical meter's finest
  bucket opens its Normalized data. A virtual meter's opens its own analysis over that bucket, whose source list links
  on to the sources' records.
- Missing prices, stale sources, invalid calculations, rows dated after now and possible overlaps become attention
  items, each with one targeted action (D-53). For example, "Add tariff" opens
  `/admin/tariffs?scope=&id=&component=&from=&action=new` prefilled with the first uncovered month (D-52).

**Shared components (`Components/Shared/Analysis`):** `PageHeader`, `AnalysisBreadcrumbs`, `PeriodToolbar`,
`AnalysisChart` (ApexCharts; one axis per unit; signed values around a real zero line; follows the light/dark theme
in the current circuit), `AnalysisTable` (the accessible equivalent of every chart), `MetricCard`, `ChangeChip`,
`ValueStatus`, `EmptyPeriodState`, `PendingState`, `PanelError`, `LoadPanel`, `ProjectionNote`, `ComparisonSummary`,
`AttentionList`, `SeriesContributions`.

**CSV export (D-55):** `GET /export/analysis.csv` takes the same URL keys as the pages; every toolbar has "Export CSV".
It writes one row per bucket and series, with these columns:
- `series_id`, `series_name`, `kind`, `unit`;
- `bucket_start`, `bucket_end` (local ISO with offset; the end is exclusive, and a to-date bucket ends at now),
  `timezone`;
- `value` (invariant, full precision; empty when unknown), `status`, `provenance`;
- `cost`, `cost_status`, `currency`, `comparison_value`.

A bucket with nothing booked is `Missing`, never `Available`. Invalid requests (a notice, too many buckets, an unknown
scope) get a 400 with a plain-text reason. The endpoint is a UI endpoint, like the pages, and needs no API key.

### 8.1 Overview
- KPI cards: **Today**, **This month**, **This year** cost — each with Δ (absolute + %) vs the previous comparable period and an ↑/↓ indicator.
- "Cost now" total across all categories.

> **Deviation (D-58, brief §7.1):** the Overview shows **one selected period** instead of Today/This month/This year
> cards. It has a period toolbar and defaults to month to date. The page shows:
>
> - the cost of the period, split into metered use, standing charges, manual costs and feed-in credit;
> - one card per energy type, with its measures in their own units (unlike quantities are never added), the cost with
>   its billing basis, the change, freshness, and a link to that type with the same period;
> - the history chart with a metric selector (`chart=`), the previous-year overlay, a table toggle and drill-down;
> - "What changed", by category or by meter, with rows linking to the scoped Analysis page with the same dates;
> - the cost composition (§7.5): a donut only for non-negative disjoint slices, otherwise signed bars. Overlapping
>   views are listed apart;
> - attention items, and "Latest month with data" with its month and basis (meters, manual costs or both, D-19).
>
> Changes are measured over the coverage both periods share (D-07). A period without data offers "Go to latest data";
> the page never switches to history on its own. Missing categories or tariffs are small setup notes, never a
> prerequisite for seeing quantities. The REST summary keeps its legacy month/year windows (D-45).

### 8.2 Cost breakdown / "what costs most"
- Stacked bar or donut by `cost_category` for a selectable period; ranked list (most → least).
- **Difference view** (explicitly requested): a table answering *"what cost more, what cost less this time"* — per category **and** per meter, **this month vs last month** and **this year vs last year**, columns `now | previous | Δ | Δ% | ↑/↓`, sorted by absolute impact.

> **Current:** both live on the Overview for the selected period and its comparison, not for fixed month/year windows.
> The composition is §7.5's. "What changed" lists categories or meters with current, previous, change and percentage
> where applicable, sorted by impact, plus a bill total row. A change is shown only between complete figures (A-23).

### 8.3 Trends
- Consumption and cost over time; **granularity toggle** day/week/month/year; per-meter or per-category; **previous-year overlay**.

> **Current:** this is the **Analysis** page (`/trends`, brief §7.4, nav "Analysis"). It offers:
>
> - Scope: all energy types, one energy type, a cost category, one meter, or a comparison of up to six meters. A
>   seventh is refused with an explanation.
> - Metric: only what the scope supports. A category is always available by cost, and by quantity only when all its
>   meters share one kind and unit; otherwise the page explains why and offers the alternatives.
> - The shared toolbar, with a calendar-year select (compare with any of the five years before), the chart (overlays
>   up to three series; above that the comparison stays in the table), the table, drill-down and CSV export.
> - For per-type measures, total use and grid import side by side, never added. Portfolio cost is the same bill the
>   Overview shows, with manual costs once.

### 8.4 PV / Solar panel
- Generation, self-consumption, grid feed/draw, **savings (Ersparnis)**, **autarky %**, **self-consumption %**. Time-filtered.

> **Current (D-54, `SolarService`):** one section per energy type that has generation. Nothing is inferred from names:
> meters are found by mode and by the effective roles `total_load`, `grid_import`, `grid_export` (A-07).
>
> - **Generation** is the type's generation measure, so a virtual sum such as Summe Solar is listed as a view and never
>   added twice.
> - **Self-consumption** is total load − grid import, or else generation − grid export.
> - **Feed-in** is the grid export meter, or else generation − self-consumption (labelled as calculated; batteries are
>   not modelled).
> - **Site use** is the total load meter, or else self-consumption + grid import.
> - **Savings** are self-consumption × the grid unit price, month by month through the cost calculator.
> - Autarky % and self-consumption % are shown when the roles allow.
>
> Every figure shows its status and how it was obtained. Units come from the meters; mixed units give "cannot be
> calculated". A missing role gets a setup card with the candidate meters, which lead into the meter editor (no raw
> role tags). The page has no CSV export, because the export has no derived measures.

### 8.5 Oil / consumable panel
- Tank level (cm + L), balance vs capacity gauge, deliveries log, burner runtime, effective **L/h** (fixed/empirical), **forecast to empty**, monthly cost.

> **Current (D-54, `ConsumableService`):** "Now" and "Selected period" are separate parts.
>
> - **Now:** the last dipstick with its date (and cm reading), an estimate that includes the deliveries since then,
>   the fill bar (only when the level is known), and the forecast as a labelled projection (§7.3).
> - **Selected period:** use from the analysis reader, the deliveries of the period only, burner runtime of the type's
>   runtime meters, and the burn rate (fixed, or empirical when runtime is in hours).
> - A period that ended shows the contents at its end.
> - The cost stays unknown, never 0 €, when the tank has no tariff or its months cannot be placed (A-16).

### 8.6 Meter detail
- Raw readings, normalized consumption, source status (last-seen, last value), tariff timeline, events (swaps/deliveries/corrections), measured-vs-estimated markers.

> **Current (brief §7.2, D-47, D-50):** the header carries identity, the energy type, mode and retirement chips, and
> the actions: primary entry by mode (Add reading / Record tank level), the "Record event" menu and Edit. The tab bar
> sits directly below the header.
>
> - **Analysis** tab:
>   - The quantity card in the normalized unit (D-20), with any projection shown separately inside it.
>   - The cost card, with its rule named or "Not costed" plus the reason, and the cost change (A-23).
>   - The previous-year overlay, a full chart, the table, drill-down and CSV export.
>   - Events and tariff changes inside the range, listed as context under the chart.
>   - A "Data quality and coverage" section: resolution, data range, freshness, opening balance with "Set install
>     date", and rows recorded after now.
>   - For virtual meters, the source contributions.
> - **Readings**, **Normalized data** and **Events** tabs:
>   - Server-side paging, 100 rows per page, keyset-ordered, filtered by the page's `period`/`from`/`to`.
>   - Their toolbar shows the whole range listed, and rows dated after now carry an "After now" mark (A-29).
>   - The Readings tab explains that raw readings are the audit record and that raw retention is not enforced.
> - **Tariffs** lists meter, type and global tariffs with their effective end and the one that applies now, plus "Add
>   tariff for this meter".
> - **Sources** links each source's connector to its editor (the connector detour keeps the typed draft).
> - A virtual meter's **Calculation** tab shows the status, the formula with meter names beside each `m<id>`, result
>   kind and unit, cost rule, the meters read (also through nested calculations), and any problem with its dependency
>   path.
> - The manual-entry dialog runs its own queries for the entered time, so its verdict never depends on a page of rows.

### 8.7 Admin / config
- CRUD for energy types, meters, sources, tariffs, cost categories, connectors; retention & locale/currency settings; import wizard; API keys.

> **Current (D-21, D-23, D-31, D-37, D-52, A-27, A-30):**
>
> - **Meter editor** (shared `MeterEditor`):
>   - Roles use friendly names and one-line meanings, and are offered only for compatible modes. Saving a role moves it
>     and names the meter that held it.
>   - The totals override (Automatic / Always / Never) states its meaning and "In the totals now: …". A conflicting
>     "Always" is refused, naming the other meter.
>   - A virtual meter gets the calculation editor: Sum, Difference or Formula mode; sources picked by name, with unit,
>     kind and dates; only valid cost rules offered, each with its reason.
>   - A live preview uses the page's period (every preset, custom dates, all history) and shows per-source values and
>     the incomplete months.
>   - Saving a Sum can bring the incoming links in line with its sources, but links never change a calculation.
> - **Tariffs:** the deep link opens a prefilled dialog once, scoped to what can price that meter or type. Units are
>   checked live, a new tariff needs a value, and Bonus/Discount/Tax are marked "not applied".
> - **Energy types** under Configuration edit the definitions ("Energy type definitions"); analysis lives under the
>   Energy types nav group.
> - **Settings** is read-only. It shows the zone, the currency, raw retention ("Not enforced"), the normalization
>   revision and zone, how many meters have current analysis data or are waiting for a rebuild, and calculated meters
>   by status.
> - Flow topology is edited from the energy type's Flow tab ("Manage connections"): cycle-, type- and
>   duplicate-checked, and it never touches a stored formula. Physical meters can also set their upstream meters in
>   the editor.

> **Legacy monthly history:** imported data is monthly-granular. Offer per-import choice: keep native monthly buckets, or **linearly interpolate to daily** (energietracker-style) so old and new data render on the same axes. Interpolated points are marked `quality = interpolated`.
>
> **Current behaviour (D-14, D-57):** monthly data stays monthly and is never interpolated. A day or week bucket over
> it reads "only coarser data", the chart names the data's resolution and offers the interval that shows it, and a
> drill-down never opens days a monthly import cannot resolve (D-51).

---

## 9. REST API (v1)

OpenAPI/Swagger published. API-key auth for automation endpoints; UI uses the session.

| Method & path | Purpose |
|---------------|---------|
| `POST /api/v1/readings` | Ingest one/many readings (idempotent). Lets HA **push** instead of us pulling. |
| `GET  /api/v1/meters` · `POST/PUT/DELETE` | Meter CRUD. |
| `GET  /api/v1/energy-types` · CRUD | Energy-type CRUD. |
| `GET  /api/v1/consumption?meter=&from=&to=&bucket=` | Normalized consumption/generation. |
| `GET  /api/v1/cost?scope=&id=&from=&to=&bucket=` | Cost by meter/category/type. |
| `GET  /api/v1/dashboard/summary` | KPI cards + Δ payload. |
| `POST /api/v1/events` | Delivery, swap, tank level, correction. |
| `GET/POST /api/v1/tariffs` | Tariff CRUD (time-ranged). |
| `POST /api/v1/import` (multipart) | CSV import with a mapping profile. |
| `GET  /api/v1/sources/status` | Connector/source health. |
| `GET  /healthz` | Liveness/readiness (for Gatus). |

> **Current contract (D-45, A-16, A-21; pinned by `ApiContractTests`):** `/consumption`, `/cost` and
> `/dashboard/summary` keep every existing field, name and type. Their numbers now come from the analysis reader and
> the cost engine, so they match the pages. What changed is only added as new fields:
>
> | Endpoint | Behaviour | Added fields |
> |---|---|---|
> | `GET /api/v1/consumption?meter=&from=&to=` | Monthly rows as before. Instants with any offset are accepted (converted to UTC; an offset used to be a 500). Actuals stop at now. A month without data is absent, not a 0. A virtual meter is evaluated from its formula. | `status` (BucketStatus), `issue` (why a value is not plain), `kind`, `unit` (normalized) |
> | `GET /api/v1/cost?meter=&from=&to=` | `cost` stays numeric: 0 when nothing could be priced, with the reason beside it. The meter is priced by its rule (the bill line, a subsection at its unit price, a virtual meter by its cost rule). Generation and runtime meters, and meters that cannot be evaluated, are not costed. | `costStatus` (`Priced`, `Partial`, `NotPriced`, `PriceGap`, `UnitMismatch`), `costAvailability` (the status of the quantities behind the cost, e.g. `Unresolved`, `Invalid`), `costRule`, `notCosted` (the reason), `missingPrices[]` (component, reason, scope, first/last month, tariff, unit issue, credit), `status`, `issue`, `kind`, `unit` |
> | `GET /api/v1/dashboard/summary` | Keeps its legacy windows (calendar month and year to now against the whole previous ones) but prices the bill (§7.5). | `deltaPercentApplicable` per KPI (the percentage is 0 and not applicable for a zero or negative baseline), `latestMonth` `{period, basis}` |
>
> `/consumption` and `/cost` take one `meter` and answer by calendar month, as before the rework; the table's `scope`
> and `bucket` parameters are not implemented. The UI's analysis CSV (`GET /export/analysis.csv`, §8.0) is not part of
> `/api/v1` and needs no API key.

---

## 10. Non-functional

- **Time & DST:** store UTC; bucket and display in the instance timezone (default `Europe/Berlin`). "Daily cost" boundaries are local-midnight — use `time_bucket(..., 'Europe/Berlin')`.
  *Deviation (D-01 – D-06, D-58):*
  - **Zone:** the **configured** zone (`MeterVault__TimeZone`, normalized to its IANA id) is used everywhere:
    normalization, rollups, readers, periods, the export. Berlin is never hard-coded, and a zone change rebuilds the
    rollups.
  - **Bounds:** a period resolves into a local inclusive date range for display and a **half-open** UTC range
    `[from, to)` for queries. `to` is the local midnight after the end date, or the captured "now" for to-date
    periods.
  - **Calendar:** days start at local midnight, and a DST day really has 23 or 25 hours. Weeks start on Monday. Months
    and years are local.
  - **Clock:** "now" is read once per request from the registered `TimeProvider` (`InstanceClock` for pages).
    Services never read the clock; tests freeze it.
  - **After now:** rows whose interval ends after now are never counted as actuals; they are reported as "recorded
    after now" (D-04).
- **Auth:** optional built-in local accounts; **reverse-proxy trust** mode honouring `X-Forwarded-User`/`Remote-User` behind Authelia/Traefik; API keys for machine access. Default: single admin user + one ingest API key.
- **Observability:** `/healthz`, structured logs (Serilog), optional Prometheus `/metrics`.
- **Config:** environment variables + a settings UI; secrets via env/Docker secrets or encrypted at rest (never in DB plaintext) — see §6.4.
- **i18n:** `en` (default for OSS) + `de`; locale-aware number/currency/date. Ship a German locale that matches the source data conventions.
- **Backup:** document `pg_dump`/Timescale backup; provide a full **JSON export/import** for portability.
- **Performance:** dashboards read aggregates only; raw reads paginated and time-bounded.
  *Current:* dashboards read the rollup tables (§5.4). A request sends a constant handful of statements, whatever the
  meter count. The point limit (400 per series) and the series limit (6 on a chart) are enforced before any SQL runs.
  The brief's target is a 10-year monthly request for 100 meters in under 2 s; it was measured against a synthetic
  1,000-meter × 10-year dataset (§13, `docs/ANALYSIS_REPORT.md`). Raw record tabs are paged (100 rows, keyset) and
  date-filtered.
- **Accessibility & layout (brief §8):** every chart has a table equivalent; colour is never the only cue (words,
  arrows, signed values, dashed overlays); visible focus rings; keyboard-reachable drill links; no page-wide overflow
  at 360 px; charts follow the light/dark theme (cookie `mv-theme`) within the circuit. MudBlazor's own labels are
  localized (`MeterVaultMudLocalizer`).
- **Currency:** `MeterVault__Currency` (default `EUR`) for every amount (D-43).

---

## 11. Repository, CI, licensing

```
/                       CLAUDE.md, README.md, LICENSE, docker-compose.yml
/src
  /Core                 domain entities, enums, interfaces, expression eval
  /Infrastructure       EF Core + Npgsql, Dapper repos, Timescale SQL migrations,
                        MQTT client, HA client, CSV importer
  /App                  ASP.NET Core host: Blazor Server UI + REST API + hosted workers
/tests
  /Core.Tests           unit: deltas, swaps, tariff resolution, oil rate, CSV parsing
  /Integration.Tests    Testcontainers (Timescale): ingest→aggregate→cost e2e
  /fixtures             the 4 reference CSVs + expected outputs
/deploy
  Dockerfile            docker-compose.yml (app + timescaledb), unraid-template.xml
/docs                   architecture, setup, HA/Tasmota wiring, API, screenshots
```

> *Current layout:* the analysis rework added these folders.
> - `src/Core/Analysis`: pure rules for time and periods, coverage, rollups, quantities and units, totals and category
>   cover, virtual formulas, and the cost calculator.
> - `src/Infrastructure/Analysis`: the reader, the catalog and the virtual upgrade.
> - `src/Infrastructure/Costing`: `CostReader` and `BillRun`.
> - `src/App/Analysis`: the URL contract and the chart, table and attention models.
> - `src/App/Components/Shared/Analysis`: the shared components.
> - Page folders under `src/App/Components/Pages`.
>
> The fixtures live in `sampledata/`. CI is Gitea Actions (`.gitea/workflows/`), publishing to the Gitea container
> registry. `CLAUDE.md` holds the maintained layout.

- **CI (GitHub Actions):** build → test (spin Timescale) → publish Docker image to **GHCR** (amd64; add arm64 if desired) on tag.
- **License:** pick before release — **MIT** (max adoption; matches energietracker/your prior assets) or **AGPL-3.0** (keeps hosted forks open). Default suggestion: **MIT**, unless keeping SaaS forks open-source matters to you.
- **Docs:** a "wire up HA/Tasmota" guide is the highest-leverage doc for adoption.

---

## 12. Milestone roadmap (build order)

- **M0 — Scaffold.** Solution + 3 projects + tests; `docker-compose` with TimescaleDB; EF Core + first migration; `/healthz`. *Exit:* app boots against Timescale in Docker.
- **M1 — Domain & schema.** `energy_type`, `meter`, `meter_source`, `reading` hypertable, `consumption` hypertable + normalization pipeline (register/runtime/swap/reset), `tariff`, seed defaults. *Exit:* insert readings → correct `consumption`, unit-tested incl. swaps.
- **M2 — Manual entry & CSV import.** German-dialect importer (Appendix A), mapping profiles, dry-run + revertible batches; the 4 CSVs import and reconcile. *Exit:* importing the reference CSVs reproduces the spreadsheet's consumption/cost within tolerance (§13).
- **M3 — Live ingestion.** MQTTnet worker + Tasmota field mapping; HA connector (WebSocket + REST poll); idempotency; source status. *Exit:* a Tasmota plug and an HA entity land as readings automatically.
- **M4 — Aggregation & cost engine.** Continuous aggregates (hourly/daily/monthly/yearly); tariff-aware cost view (monthly + prorated); cost categories + manual costs. *Exit:* `GET /cost` and category rollups correct vs fixtures.
- **M5 — Dashboard.** Overview KPIs + Δ; cost breakdown + **difference view**; trends w/ granularity + prev-year overlay; PV panel; oil/consumable panel; meter detail. *Exit:* all §8 views render on imported + live data.
- **M6 — API & auth.** REST + OpenAPI; API keys; reverse-proxy trust. *Exit:* HA can push via `POST /readings`; Swagger published.
- **M7 — Release polish.** i18n (de/en); retention settings; JSON export/import; Unraid template; CI → GHCR; README + wiring guide. *Exit:* `docker compose up` from a clean host yields a working, documented instance.

> *After M7:* the dashboard/analysis rework was built in five phases:
> 1. shared semantics and fixtures;
> 2. virtual evaluation and migration;
> 3. history and navigation;
> 4. Overview and specialized pages;
> 5. integration, performance and documentation.
>
> It replaced M4's continuous aggregates and "monthly + prorated" cost view with rollup tables and the month-by-month
> bill (§5.4, §7.5), and M5's panels with the pages of §8.0. M7's "retention settings" remain display-only (D-57).

---

## 13. Testing strategy

- **Unit:** consumption deltas incl. **water swap** (…861→2) and counter resets; **oil** empirical vs fixed L/h and forecast; tariff time-range resolution + mid-month proration; virtual-meter expression eval; **CSV parsing** of the exact reference dialect (decimal comma, unit suffixes, currency, `DD.MM.YYYY` vs `Monat YYYY`, summary-row skipping, zero-placeholder rows).
- **Integration (Testcontainers + Timescale):** end-to-end ingest → normalize → aggregate → cost; continuous-aggregate refresh; hypertable compression sanity.
- **Golden fixtures:** the four CSVs with expected monthly consumption/cost tables. A regression test asserts computed ≈ spreadsheet (define tolerance for rounding; the sheet rounds to cents / whole kWh).
- **Load smoke (optional):** synthetic 1000-meter × N-year generator to validate aggregate query latency and compression ratio.

> **Current (analysis rework, D-56):** `Core.Tests` has 1,733 tests and `Integration.Tests` 746, all passing at the end
> of the rework. The continuous-aggregate refresh test was replaced by
> `CostReconciliationTests.Monthly_rollup_equals_the_consumption_it_sums` (water Dec 2022 = 14 m³). `SchemaTests`
> pins that the aggregates and their jobs are gone and that the analysis tables cascade with their meter.
>
> - **Frozen clock, pure (Core):**
>   - `PeriodResolverTests`, `ComparisonResolverTests`, `BucketPlannerTests`: local New Year, Berlin DST in spring
>     and autumn (23 h and 25 h days, the repeated hour), 29 Feb, 31 March against February, shorter months, New York
>     (a zone behind UTC).
>   - Coverage, rollups, provenance and freshness: `Coverage*Tests`, `RollupBuilderTests`, `MatchedCoverageTests`,
>     `ProvenanceRulesTests`, `FreshnessRulesTests`.
>   - Totals and categories: `TotalsPolicyTests` (the seeded classification, D-22), `CategoryCoverTests`,
>     `SeparatelyBilledSubmeterTests`.
>   - Virtual formulas: `FormulaParserTests`, `VirtualValidatorTests`, `VirtualEvaluatorTests`, `DependencyGraphTests`,
>     `LegacyVirtualDerivationTests`: A+B, A−B, missing vs zero, nested, loop, division by zero.
>   - Costing: `CostCalculator*Tests`, `CostingTariffBookTests`, `TariffUnitTests`.
>   - Units and changes: `UnitsTests`, `ChangeTests`.
> - **Reader and costs (Testcontainers):**
>   - `AnalysisReaderTests`: the worked examples, local days across DST and in New York, rows after now, new readings
>     and corrections visible on the next read.
>   - `AnalysisDataTests`: rollups written by diff, rows removed behind the tracker's back.
>   - `CostReaderTests`: missing vs zero tariff, price gaps, bucket-independent totals, standing charges once per
>     scope, manual costs once, virtual cost rules. `CostReviewFixTests` adds a category whose members price nothing.
>   - `SeededBillTests`: the D-44 goldens. The yearly bill equals the sheet's `Jahreskosten` within ±0.02 € for 2022,
>     2025 and 2026; 2023 and 2024 are pinned to the documented 3.78 € / 0.46 € differences. It also pins the
>     composition and a manual-cost-only instance.
>   - `CostConsistencyTests`: the same cost change on four pages, and matching standing charges.
>   - `CoverageOfFixturesTests`, and the unchanged reconciliation suites (electricity, water, oil, costs; Netz
>     Einsparung through the new evaluator, D-29).
> - **Contracts and export:** `ApiContractTests` (fields and types of `/consumption`, `/cost`, `/dashboard/summary`,
>   offsets, not-costed meters), `AnalysisExportEndpointTests`, `AnalysisCsvWriterTests`, and `ExportRoundTripTests`
>   (links and virtual definitions remapped, a deleted meter's tariffs not restored).
> - **Pages without a browser:**
>   - Loaders against a database: `MeterAnalysisLoaderTests`, `AnalysisPageLoaderTests`, `EnergyTypePageTests`,
>     `OverviewDataTests`, `SolarServiceTests`, `ConsumableServiceTests`, `MeterDraftPreviewTests`,
>     `VirtualManagementTests`, `MeterDetailServiceTests` (keyset paging, half-open filters).
>   - Pure UI models: `AnalysisQueryTests`, `AnalysisNavigationTests`, `AnalysisChartModelTests`,
>     `AnalysisTableModelTests`, `AttentionItemsTests`, `LoadSequencerTests`, `MeterPageLogicTests`,
>     `EnergyPageTests`, `OverviewLogicTests`, `MeterEditorLogicTests`, `TariffEditingTests`.
>   - Server-rendered HTML in EN and DE: `DashboardRenderTests`, `OverviewPageTests`, `AnalysisComponentRenderTests`,
>     `AdminPagesRenderTests`, `MeterSourcesRenderTests`.
>   - `StringResourceTests`, `EnumDisplayNameTests` and `FormatCultureTests` pin both languages and the currency.
> - **Browser checks:** interactive ApexCharts, browser history, theme switching and responsive layout were checked
>   against the seeded instance. CDP scripts drove Chrome in EN/DE, light/dark, 1440/390/360 px. That acceptance walk
>   is recorded in `docs/ANALYSIS_REPORT.md`. No bUnit or Playwright suite is in the repository.
> - **Performance** (`tests/Integration.Tests/Performance`, trait `Category=Performance`): skipped unless
>   `METERVAULT_PERF=1`. `SyntheticLoadTests` loads a deterministic 1,000-meter × 10-year dataset (≈1.34 M readings;
>   monthly, daily and hourly meters, tanks, roles, links, 20 virtual meters nested up to three levels) through the
>   real pipeline. `ReaderTimingTests` times the reader and cost scenarios (the brief's target: 100 meters, 10 years
>   monthly, under 2 s), counts SQL statements, records query plans, the startup rebuild and per-meter recompute
>   cost. Results are in `docs/ANALYSIS_REPORT.md`.

---

## 14. Open questions & defaults

Pick the **default** and flag it if unsure; only ask when a question isn't listed here.

1. **Virtual meters: compute-on-write or compute-on-read?** *Default:* compute-on-read for dashboards, materialize to `consumption` only if a virtual meter is referenced by cost. (Avoids recompute storms; revisit if slow.) *Decided (D-58):* computed on read everywhere, costs included; nothing is materialized.
2. **Legacy monthly import → daily interpolation or native monthly?** *Default:* offer both per import; interpolation off by default, points marked `interpolated`. *Current (D-57):* native monthly only; interpolation is not offered.
3. **Cost proration when price changes mid-month.** *Default:* day-accurate proration available, but the **displayed** monthly figure uses the month's dominant price to match the spreadsheet unless the user opts into proration. *Decided (D-36, A-16):* one price per local month, the one valid on the 15th (the sheet's convention); no proration mode. Every bucket is priced month by month, so bucket size never changes a total. A reading interval longer than a month is priced as a whole only when its months share one price; otherwise it is unavailable, with an attention item. Standing charges accrue per day (D-40).
4. **Instant-rate (power/flow) integration in v1?** *Default:* schema-supported, worker deferred to post-v1 (Tasmota already gives cumulative `ENERGY.Total`, so it's rarely needed). *Current:* the `instant_rate` normalizer integrates the rate over time (trapezoidal), and a gap longer than max(1 h, 10 × the median sample interval) opens a coverage gap (D-13). There is no rate-specific ingestion worker: rate values arrive through the ordinary sources like any other reading.
5. **Multi-user?** *Default:* single admin + reverse-proxy trust; full accounts post-v1.
6. **.NET version pin.** *Default:* current LTS at implementation time; keep `TargetFramework` in one place.
7. **License.** *Default:* MIT unless you want AGPL's copyleft on hosted forks.
8. **Name.** `MeterVault` is a placeholder — decide before first public tag.

The analysis rework settled these further questions. Each is recorded in `ANALYSIS_IMPLEMENTATION_NOTE.md`.

9. **Raw retention (§5.5).** *Decided (D-57):* not enforced until recompute can start from stored consumption. The
   setting is shown as "Not enforced".
10. **What a type's total and bill count.** *Decided (D-22, D-34):* the non-overlapping topology roots, with
    consumption and generation apart. The bill counts grid import where there is one, otherwise use. Submeters and
    virtual views are never added; the `always` override lets a virtual meter replace its sources (D-23).
11. **Virtual costs.** *Decided (D-39, A-15):* a named cost rule. `sourceCosts` for pure sums, `ownQuantity` for
    linear formulas, `none` otherwise and for generation sums. The default for new virtual meters is "analysis only"
    in the totals.
12. **Default comparison.** *Decided (A-13):* the previous year, at the same elapsed point, measured over what both
    periods cover (D-07).
13. **Default periods.** *Decided (D-02):* the Overview uses month to date. History pages use the last 12 months: 12
    calendar buckets, the current one partial.
14. **Missing tariff.** *Decided (D-38):* "not priced", never 0. An explicit zero tariff is a valid zero.

---

## Appendix A — CSV import dialect (from the reference files)

- **Field separator:** comma; fields quoted when they contain a comma.
- **Decimal separator:** comma (`180,8244706`). **Thousands separator:** dot (`2.940,19`).
- **Currency:** trailing `€` with a space (`120,00 €`); parse to `(amount, currency)`.
- **Unit suffixes on values:** strip and validate (`411kWh` → 411 kWh; `49` cm; `2287` L).
- **Dates:** `Monat YYYY` German month names for monthly tables; `DD.MM.YYYY` for event rows. Support both; store as timestamptz (month tables → first-of-month or period bucket).
- **Skip rows:** summary/label rows — `Total`, `Heute`, `Seitbeginn Tage`, `Seit YYYY`, and any embedded side-tables.
- **No-data rows:** all-zero future placeholders (e.g. Dec 2026) → ignore.
- **Negatives are valid** (savings, grid balance).
- **Column mapping is explicit** (a wizard), because these sheets pack multiple meters and derived columns side by side; ship saved mapping profiles for each of the four sheet shapes.

## Appendix B — Glossary (source ↔ model)

| Sheet term | Model concept |
|------------|---------------|
| Zähler (Haus/Netz/Auto/Solar) | `meter` (`cumulative_`/`generation_counter`) |
| Verbrauch / Erzeugung | `consumption.amount` (kind 0/1) |
| Solar Erzeugung / Eigenverbrauch / Ersparnis / Netz Einsparung | `virtual` meters via expressions (quantities only: Solar Erzeugung = `m(Solar 1) + m(Solar 2)`, Netz Einsparung = `m(Haus) − m(Netz)`). Ersparnis is a price × quantity, which expressions do not support (§7.4). It is the Solar page's savings, or a virtual meter's `ownQuantity` cost. |
| €/kWh, €/m³, €/100l | `tariff.unit_price` (time-ranged) |
| Grundpreis / Abschlag | `tariff.base_price` |
| Betriebststunden | `runtime_counter` meter |
| Verbrauch / Betrieb Stunde | derived L/h (`tank.rate_mode`) |
| Lieferungmenge | `meter_event` `delivery` |
| Füllstand cm / Tank Aktuell | `meter_event` `tank_level` / `tank.cached_balance` |
| Vorraussichtliches Ende | forecast-to-empty |
| Heizung / Strom / Wasser / Pool Betrieb | `cost_category` (Pool via `manual_cost`) |
| Zähler swap (…861→2) | `meter_event` `meter_swap` |