MeterVault/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this repo is

MeterVault is a self-hosted, local-first energy & utility metering platform: it ingests meter data from Home Assistant, Tasmota and MQTT on a schedule, stores every reading timestamped and immutable, normalizes it into consumption, and turns it into cost dashboards. Energy types (electricity, water, heating oil, gas, …) and meters are user-defined, never hardcoded.

Status: implemented (M0–M7) + SDD §8 panels. The full solution is built and green — five projects, ~350 tests, working Docker deploy. docs/SDD.md remains the design reference; the milestone map (§12) matches the git history (M0…M7 commits). The dedicated PV/Solar (/solar), Oil/consumable (/consumables) and meter-detail (/meters/{id}) views (SDD §8.4–§8.6) are implemented as read models in Infrastructure/Dashboard (SolarService, ConsumableService, MeterDetailService) — PV meters are found by Mode == GenerationCounter and grid/load meters by a role tag in Meter.Meta (MeterRoles/MeterMeta), so nothing is hardcoded by name. Admin write-CRUD (SDD §8.7) is implemented as MudBlazor inline-dialog pages: energy types, meters (+ recompute on mode/baseline change), a meter's ingest sources (meter-detail Sources tab), tariffs, cost categories + members, and connectors (ingestion_endpoint, secrets as an env-var reference or typed in and encrypted at rest). Manual readings are entered from the meter-detail Readings tab ("Add reading"): a touch-first dialog prefilled with the meter's last register value and the current local time, with an on-screen keypad for phone entry at the meter, a live parsed-value + delta-since-last readout, and the decrease guard surfaced before saving. It goes through IngestionService.IngestByMeterAsync(quality: Manual), so it is stamped ReadingQuality.Manual and renormalizes inline like any other ingest — the layout of that dialog deliberately reserves fixed space for its verdict line, because anything that reflows moves the keys out from under the user's thumb mid-entry. /admin/settings is a read-only effective-config view (settings are env-driven and reproducible, not DB-stored). Home Assistant reading is configured here: an HA connector (BaseUrl + TokenEnv) + an HA source (entity id) drives HomeAssistantWorker's REST poll, or — with the connector's WebSocket push toggle (HaEndpointConfig.UseWebSocket) — HomeAssistantWebSocketWorker holds a persistent state_changed subscription and ingests in real time (the poll worker skips WS endpoints, so each is served once; HaWebSocketProtocol is the pure, unit-tested handshake/parse logic). HaConnectionTester powers the connector "Test connection" button. Meter topology & flow: MeterLink (a directed from→to edge; a downstream meter is a subsection of an upstream one, multi-parent allowed) drives a per-energy-type page /energy/{id} with a hand-rolled SVG Sankey (SankeyChart.razor, since ApexCharts has no Sankey type) computed by FlowService (link value = downstream consumption, split proportionally across multiple parents; unaccounted remainder → an "Other" node). Upstream meters are wired cycle-safely in the meter editor; the nav lists a link per energy type. CSV mapping wizard (/import/wizard): upload an arbitrary CSV, map columns → meters/roles, dry-run preview, then commit as a revertible import_batch (the /import page lists batches with one-click revert). The instant_rate mode is normalized (InstantRateNormalizer: rate integrated over time, trapezoidal). Meter events from the UI (swap, counter reset, tank level, delivery, note) go through MeterEventService (Infrastructure/Ingestion), never ad-hoc inserts: MeterEventRules.RecordableFor(mode) (Core) decides which events a mode offers (no Correction — nothing reads it), Validate gives the dialog the same verdict the save reaches, and every record/delete recomputes the meter in one transaction. A swap/reset is stored as the event at T plus a manual reading of the new register's start value at exactly T (flagged MeterSwap/CounterReset) — the boundary window is (prevReading, reading], so this books the old tail at T and later readings count from the new start; never write the old final value as the reading at T (double-counts, then rejects every new-register reading). Deleting a swap removes its start reading only while it is still the untouched start value; only Manual readings are deletable in the UI. Navigation conventions: the meter page is the per-meter hub (header actions: primary entry by mode, "Record event" menu, Edit via the shared Shared/MeterEditor.razor, which also owns tank setup); link into it with MeterLinks (/meters/{id}?tab=…&action=…, action consumed once after the interactive render and dropped from the address); the app-bar "Find a meter" dialog offers the same quick entry; NavState tells the per-circuit nav to reload energy types after admin edits. A source that lacks a usable connector detours through /admin/connectors?new=…|edit=…&meter=… and comes back to that source dialog with the connector picked and everything typed restored (the page saves the open dialog to the circuit-scoped DraftStore on dispose) (MeterLinks.Source/NewConnector/EditConnector; the way back is a meter id, never a URL, so it cannot redirect off-site); the connector list shows which meters use each connector. The meter editor owns the meter's own cost-category memberships (type-level ones are only named), import batches list the meters/categories they wrote to, and the dashboard's empty cost panel names the first missing setup step (DashboardService.GetCostSetupAsync → CostSetup.FirstGap). UI language (M7's last item) is now English + German end to end — see Localization below. Set MeterVault__SeedReferenceData=true (compose: METERVAULT_SEED=true) for a one-command populated demo.

Source of truth

docs/SDD.md is the authoritative spec and build brief — read it before implementing anything. Key protocol from §0 that governs all work here:

• Build strictly in milestone order (§12, M0→M7). Each milestone is independently runnable and testable; do not start Mn+1 until Mn's tests pass.
• The four CSVs in sampledata/ are golden fixtures. Every parsing / consumption / cost rule must reconcile against them (§13). If a computed number disagrees with the spreadsheet, the spreadsheet wins unless the discrepancy is a deliberately documented correctness fix.
• When a design decision is ambiguous, check §14 (open questions): if listed, take the stated default and flag it; if not listed, ask before guessing.
• Keep the domain layer free of infrastructure concerns (the domain model and DB schema are UI-agnostic by design).

Committed tech stack (do not re-litigate; see SDD §4.1)

.NET (current LTS — .NET 10, .NET 8 acceptable), C# · ASP.NET Core + Blazor Server · MudBlazor components · ApexCharts (Blazor-ApexCharts) · MQTTnet · PostgreSQL + TimescaleDB · EF Core (Npgsql) for schema/CRUD + Dapper for hot-path time-series reads · BackgroundService hosted services for ingestion/aggregation · xUnit + Testcontainers (Timescale image) · Docker Compose + GHCR.

Project layout

/src/Core            domain entities + enums; pure Normalization engine (mode strategies,
                     expression evaluator); Parsing (German dialect); Costing (TariffResolver)
/src/Infrastructure  MeterVaultDbContext + migrations (relational + raw-SQL Timescale);
                     Import (CsvImporter, profiles, ImportService), Ingestion (MQTT/HA workers,
                     IngestionService), Normalization service, Costing/Dashboard/Backup services
/src/App             ASP.NET Core host: Blazor Server UI (Components/), REST API (Api/), hosted
                     workers, Program.cs (Serilog, migrate+seed on startup, /healthz)
/tests/Core.Tests            unit (no Docker): parsers, normalizers, swap→12, tariff resolver
/tests/Integration.Tests     Testcontainers (Timescale): reconciliation vs the 4 fixtures,
                             import commit/revert, ingestion, cost, CAgg refresh, API, export, render
/deploy              Dockerfile, docker-compose.yml (app + timescaledb), build-and-push.ps1, unraid-template.xml

Central package versions live in Directory.Packages.props; shared build/style in Directory.Build.props + .editorconfig. Snake_case table/column mapping via UseSnakeCaseNamingConvention. EF migrations are exempt from code-style enforcement (see .editorconfig).

Commands

dotnet build                                   # build the solution
dotnet test                                    # all tests (Integration.Tests needs Docker for Testcontainers)
dotnet test tests/Core.Tests                   # unit tests only (no Docker needed)
dotnet test tests/Integration.Tests --filter "FullyQualifiedName~Reconciliation"  # one class/area
dotnet ef migrations add <Name> -p src/Infrastructure -s src/App -o Persistence/Migrations
dotnet run --project src/App                   # run app + workers locally (needs a Timescale DB)
docker compose -f deploy/docker-compose.yml up # app + TimescaleDB together

Timescale-in-EF gotchas (already handled — follow the pattern): hypertable/CAgg DDL lives in raw-SQL migrations; continuous-aggregate creation + policies use migrationBuilder.Sql(..., suppressTransaction: true), one statement each; CAgg policy end_offset must be ≥ one bucket. Tests pause the compression job (historical fixture data would otherwise deadlock imports).

Core architecture (the part that spans multiple files)

Data pipeline — one direction, layered (SDD §4.2, §5, §7):

sources (Tasmota/HA/MQTT/manual/CSV)
  → Ingestion workers write raw `reading` rows (immutable audit truth)
  → Normalization derives append-only `consumption` (deltas in base unit)
  → TimescaleDB continuous aggregates roll consumption to hourly/daily/monthly/yearly
  → Cost engine joins aggregates with time-ranged `tariff`
  → Blazor dashboard + REST API read aggregates + cost views

Invariants that shape everything:

• Raw reading is immutable audit truth. Everything derived (consumption, cost, balances, forecasts) is computed on top and must be reproducible. Never mutate readings to fix a derived number. Live ingestion recomputes the meter inline (IngestionService.RenormalizeAsync) — without it, polled readings never become consumption.
• Consumption is attributed to the months it accrued in (GapAttribution, SDD §7.1). A plain increase whose interval crosses a local month boundary (instance timezone) is divided at those boundaries by elapsed time, each share stamped inside its month (the closing reading keeps its own row when it lies in that month), marked Estimated. Imported monthly-table rows are month-end snapshots, marked by the importer with ReadingFlags.MonthLabel when the date cell named a month (IsMonthLabel = the flag; never infer it from a midnight-on-the-1st stamp — a day-dated "01.08.2026" is an instant). A manual correction keeps the flag; a live/API value or a swap start reading written onto that instant clears it (IngestionService.UpsertAsync). EffectiveTime reads a label as the end of its month; ReadingTimeline (Core) is the one ordering — effective time, then stamp — used by CounterNormalizerBase and RuntimeCounterNormalizer, and via RegisterNeighbours by the ingestion decrease guard and MeterEventService.GetContextAsync, so none of them disagrees about which reading is "previous". Consecutive rows span exactly one month and book unchanged (the golden fixtures reconcile), a live reading after the last imported row counts from that month's end, and a sheet imported after live readings does not double-count. A swap/reset stamped exactly at a label sits at the start of that label's local month (ReadingTimeline.BoundaryTime), and RegisterBoundary.Advance never counts a start value above the reading. StampTime keeps a label's row inside its own local month (zones behind UTC; also used by DirectDeltaNormalizer); NormalizationEngine coalesces any rows that still share a (time, kind) key; GapAttribution.LocalMidnight verifies its answer so contradictory zone data cannot stall the month walk. GapSplittingIsInertOnFixturesTests pins that no fixture interval is divided, in UTC or Berlin. Swaps/resets/decreases are never divided. Readers bucket in the configured MeterVault__TimeZone (CostService, SolarService, ConsumableService, MeterPeriodService take @tz) and turn requested dates into instants with InstanceTimeZone.StartOf (local midnight, not UTC midnight — DashboardService, FlowService, EnergyView too); a hard-coded zone or UTC-midnight range would re-file the divided shares. Program post-configures the zone id to its IANA form (InstanceTimeZone.Canonical) and logs an error when .NET or PostgreSQL does not know it. Stored consumption records normalization_revision and normalization_zone in app_setting; NormalizationUpgrade (startup migration step) first flags month rows of pre-revision-2 imports (reference profiles by name, wizard MonthName batches, and wizard Auto batches whose every row sits on the 1st across ≥2 months — logged per batch), lifting TimescaleDB's decompression cap for that one UPDATE; if flagging fails nothing is rebuilt or recorded. It then rebuilds every non-virtual meter (virtual ones are computed on read) when revision or zone differs, per meter in its own transaction; a failing meter is logged, kept in normalization_pending and retried next start — the upgrade must never crash startup. Bump CurrentRevision whenever the engine books existing readings differently.
• Dashboards and charts read aggregates only — never scan reading. This is what makes 1000 meters × 50 years feasible (§5.5). Raw is kept for a bounded window (default 3y); consumption + aggregates are the long-term source of truth.
• meter.mode (measurement mode) is the central abstraction for how raw readings become consumption (SDD §5.2): cumulative_counter, generation_counter, runtime_counter (Δhours × rate), consumable_balance (tank: deliveries − usage + forecast), direct_delta, instant_rate, virtual (expression over other meters). New ingestion/normalization logic dispatches on mode.
• Nothing domain-specific is hardcoded. Energy types are data. Cost categories are decoupled from energy types (Heizung may be oil today, heat-pump tomorrow). PV self-consumption/savings/net are virtual meters with user-defined expressions, not special-cased code. Tariffs are time-ranged (price history), scoped global / per-type / per-meter.

Timescale vs EF split (SDD §5.3): EF Core migrations own the relational tables. Timescale-specific DDL — create_hypertable, compression policies, continuous aggregates, retention — is not expressible via EF's model builder and must live in raw-SQL migrations. reading and consumption are hypertables.

Time & DST (SDD §10): store UTC everywhere; bucket and display in the instance timezone (default Europe/Berlin). "Daily cost" boundaries are local-midnight — use time_bucket(..., 'Europe/Berlin').

In-app update (UpdateRunner): the dashboard shows a banner when a newer tag exists (UpdateCheckService, cached, never blocks a render). Triggering an update is off by default; MeterVault__AllowInAppUpdate is the only gate — no API key, by explicit owner decision. With it on, anything that can reach the app can trigger a rebuild+restart as root (realistically a DoS, since the build comes from the owner's own repo; RCE if that repo is compromised). The REST endpoint additionally requires an X-MeterVault-Update header — a CSRF guard, not auth, so a foreign page cannot drive it via a LAN browser. Launches detached via systemd-run because the update restarts the service. Treat any change here as security-critical; UpdateRunnerTests pins that the flag defaults off and that API keys alone don't enable it.

Localization (SDD §12, M7 — en + de): UI strings live in src/App/Localization/Strings.resx (neutral = English) and Strings.de.resx. MSBuild generates a strongly-typed Strings class from the neutral resx (see the EmbeddedResource block in MeterVault.App.csproj), aliased as S in _Imports.razor — so components write @S.Common_Save, never a string key, and a stale key is a build error. Loc.F(S.Key, args) formats the {0} ones. Adding a string means editing both resx files: StringResourceTests fails the build on a missing or blank translation, a placeholder mismatch, an orphan, or a key nothing references — resource fallback would otherwise hide a half-translated release. Domain enums stay bare identifiers (they are persisted as text and appear in the REST API); DisplayNames.Display() is the single place that decides how each value is spoken, and EnumDisplayNameTests fails if a value has no wording. Anything from the database (meter names, energy-type display names, category names) is user data and is never translated. Language is per-request: the cookie the /culture/set endpoint writes, else Accept-Language, else MeterVault__Locale (default en). Switching must be a full reload (forceLoad) — a Blazor Server circuit is fixed to the culture of the request that opened it. Format.* formats against CurrentCulture, so numbers and month labels follow the reader; the CSV importer's de-DE parsing is unrelated and unchanged, because that dialect belongs to the files, not the reader.

Secrets (SDD §6.4): broker/HA tokens are never stored in DB plaintext. Two forms, chosen per connector in the admin UI: a reference (token_env/password_env naming an env var or Docker secret path) resolved at runtime, or encrypted at rest (token_enc/password_enc) via SecretProtector over the ASP.NET Core data-protection key ring. Exactly one survives a save; EndpointSecret.Resolve is the single resolution path (encrypted wins). The key ring lives outside the app directory (MeterVault__DataProtectionKeyPath, default /var/lib/metervault/keys) because the LXC updater republishes /opt/metervault. ExportService drops *_enc values — they are bound to the originating key ring.

Reference-data behaviours the code must reproduce (from sampledata/)

These CSVs are the German-dialect Energiebilanz spreadsheet export and define the minimum feature bar (SDD §2, Appendix A). When writing the importer or normalization, honour:

• German number dialect: decimal comma (180,8244706), thousands dot (2.940,19), trailing-€ currency (120,00 €), unit suffixes on values (411kWh, 49 cm, 2287 L) — strip and validate.
• Two date formats: Monat YYYY (German month names, monthly tables) and DD.MM.YYYY (event rows).
• Skip inline summary rows (Total, Heute, Seitbeginn Tage, Seit YYYY) and all-zero future placeholder rows (e.g. Dec 2026) — do not ingest them. Negatives are valid (savings, grid balance).
• Water register swaps mid-series (…861 → 2 → 15): consumption must stay continuous across the boundary via a meter_swap event.
• Electricity has 5 meters (Haus, Netz, Auto, Solar 1, Solar 2) plus derived columns. Verified relations to reproduce: Netz Einsparung = Haus − Netz, Ersparnis = Netz Einsparung × €/kWh, Kosten = Verbrauchskosten − Ersparnis — but implement these as user-definable virtual-meter expressions, not hardcoded formulas.
• Heating oil is the versatility stress test: a consumable/tank model where consumption is derivable two ways — tank-level Δ, or burner runtime × rate (rate fixed from nozzle spec, or empirical = Δlevel ÷ Δhours). Early rows (1997–2004) carry deliveries only (no burner hours yet). Includes cm→litre dipstick calibration and forecast-to-empty.

Git

Remote origin is https://git.finalfactory.de/FinalFactory/MeterVault.git (Gitea; default branch master). CI/release is Gitea Actions under .gitea/workflows/ — edit VERSION on master to tag + publish the image to the Gitea container registry.