5 Commits

Include renames

Author	SHA1	Message	Date
Florian Schmidt	8940ef25c3	Analysis: one selected period, one set of numbers, on every page ... The dashboards told several stories at once. Overview asked for full calendar years, meter detail for a fixed 12-month window that was really 13, Trends for 24 months with an Apply button, and the energy pages for 60. Each page derived "today" from UTC, so the first hours of a local day belonged to yesterday. A missing tariff, a month nobody measured and a genuine zero all rendered as 0. And a virtual meter -- the one thing the spreadsheet leans on hardest -- was excluded from analysis outright: MeterPeriodService returned null for it and the page offered a flow diagram instead. docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58 plus amendments A-01..A-30; code, tests and release notes cite those ids. The analysis layer Core/Analysis holds the pure rules: period presets resolved once in the instance zone into a local date range and a half-open UTC range, bucket plans, calendar-unit comparisons, coverage runs with a resolution class, normalized quantities and units, the totals policy, the virtual formula parser/validator/evaluator, and the cost calculator. "Now" comes from TimeProvider; services never read the clock. Normalization now writes, in the same transaction as consumption and by diff, per-meter rollups by local day and month plus coverage runs and a rollup state (AnalysisDataWriter). AnalysisReader answers a request from those tables -- month rollups for month and year buckets, day rollups otherwise, at most two partial edge days from consumption -- and CostReader prices the result month by month. Pages, /api/v1 and the CSV export read nothing else. The unused continuous aggregates are dropped. The reader's statement count per request is constant whether it covers one meter or a thousand. On a synthetic 1,000-meter, ten-year instance the brief's target request (100 meters, ten years, monthly) takes 374 ms against a two-second target, and the Overview went from 48,244 SQL statements per load to 205. Missing is not zero Every bucket carries a status -- available, partial, missing, unresolved, invalid, pending -- derived from coverage, never from the amount, with provenance and a reason code beside it. A true zero is a number and a bar on the baseline; an unknown bucket is a gap that says why; a month whose data only exists monthly says so instead of inventing daily detail; a scope with no tariff says "not priced" instead of 0. Rows whose interval closes after now are reported separately rather than counted. Virtual meters are analysis subjects A virtual meter stores a canonical definition -- expression over m<id> references, result kind, unit and cost rule -- validated on save and on read for syntax, unknown or self references, loops and unit/kind rules. It is evaluated on read from its sources' rollups over their joint coverage: a missing source makes the bucket missing, an observed zero is a valid input, a non-finite result is invalid with its dependency path, and the page lists each source's contribution. Topology links are topology only and never rewrite a saved calculation; expression-less meters from older installs are converted once at startup. The editor has Sum, Difference and Advanced modes with a live preview. Totals and the bill Per energy type the totals policy separates use, grid import, export, generation and runtime, marks breakdown meters as breakdowns and virtual meters as views, and never adds across units. The bill follows it: grid import where there is one, separately priced subsections at their own price, feed-in only on export meters, standing charges once per scope per local day, manual costs once on their start day, categories as non-overlapping covers whose composition reconciles to the bill. The seeded demo's yearly totals now match the spreadsheet. Pages and navigation The period lives in the URL and every page reads the same contract, so a link, a reload and the browser's Back button keep it. Shared components carry it: page header with breadcrumbs, period toolbar, theme-aware chart with an accessible table beside it, metric cards, comparison and availability states, attention items that each link to the one action that fixes them. Meter detail leads with an Analysis tab and resolves its tabs by key; the energy page has Overview, History, Flow and Meters; the old cost-only Trends page is a general Analysis page over portfolio, type, category, meter or a meter comparison. Records tabs are paged server-side instead of showing the latest 200. Everything is English and German, light and dark, down to 360px. Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and what the first start after the update does (it rebuilds all analysis data before the web server listens). docs/SDD.md and CLAUDE.md describe the system as it now is. Tests: 1,733 Core and 746 integration, all green, plus an opt-in performance suite with a synthetic 1,000-meter generator.	2026-09-20 10:29:13 +02:00
Florian Schmidt	aacdc28d70	Meters: record events from the UI, and book consumption in the months it accrued in ... Two threads that ended up in the same files. One is navigation: a meter swap that happened today had no click path at all, and most per-meter tasks were reachable only by knowing which admin page owned them. The other is attribution: readings on 1 August and 16 September showed six weeks of water under September and nothing under August. Meter events from the UI Swap, counter reset, tank level, delivery and note are recorded through MeterEventService rather than ad-hoc inserts, so the dialog's verdict and the saved result come from the same Validate call, and every record or delete recomputes the meter inside one transaction. MeterEventRules decides which events a mode offers -- a tank has no register to swap, and Correction is offered nowhere because nothing reads it. A swap is stored as the event at T plus a manual reading of the new register's start value at exactly T. That pairing is the whole trick: the boundary window is (previousReading, reading], so the old register's tail books at T and every later reading counts from the new start. Writing the old final value as the reading at T instead -- the obvious thing -- double counts the tail and then rejects every reading the new register produces. Deleting a swap removes that start reading only while it is still the untouched start value, and only Manual readings can be deleted at all. Navigation The meter page is now the hub: primary entry by mode, a "Record event" menu, and Edit through a shared MeterEditor that also owns tank setup. Other pages link into it with MeterLinks (/meters/{id}?tab=...&action=...), whose action is consumed once after the interactive render and dropped from the address -- the reverse order flashes the dialog and closes it, because a circuit's first location change dismisses every open dialog. The app bar gains a "Find a meter" dialog with the same quick entry. A source that has no usable connector now links to creating (or enabling) one and comes back to the same source dialog with the connector picked and everything typed still there; the draft survives in a circuit-scoped DraftStore, and the way back is a meter id rather than a URL, so the page cannot be made to redirect anywhere else. The connector list shows which meters use each connector, import batches list the meters and categories they wrote to, the meter editor owns the meter's own cost categories, and the dashboard's empty cost panel names the first missing step instead of listing every admin page. Months A reading is an instant, and what it measures accrued over the time since the previous one. Booking the whole delta at the closing reading misfiles it whenever the interval crosses a month boundary, so a plain increase is now divided at local month boundaries in proportion to elapsed time, each share stamped inside its month and marked estimated: the meter recorded a total, not a shape. The parts always sum to the original. Imported monthly tables are the exception that keeps the golden fixtures reconciling. "Mai 2026" carries the register at the end of May but is stamped on the 1st, so the importer -- the only place that still knows whether the date cell named a month or a day -- flags it MonthLabel, and the engine reads it as the end of its month. Inferring that from the stamp instead would catch day-dated rows: a sheet with "01.08.2026" in it is not a monthly table, and reading it as one moves two thirds of July into August. ReadingTimeline is the single ordering built on that: effective time, then stamp. The register normalizers walk it, and so do the decrease guard and the event dialog, which is what stops them disagreeing about which reading is "previous" -- a sheet imported after live readings of the same month used to count that month twice, and a mid-month reading below the month's end value was rejected as a drop. A swap detected in a monthly table applies from the start of that local month, i.e. to the first reading in it, and a recorded start value never counts above the reading it lands on. Every reader buckets in the configured timezone rather than a hardcoded one, and turns a requested date into that zone's local midnight, so the divided shares are read back under the months they were stamped in. The zone id is normalised to its IANA form, because .NET accepts a Windows id that PostgreSQL will not bucket by, and both are checked at startup. Stored consumption is derived, so a rule change reaches a meter only at its next reading -- weeks, for a meter read monthly. NormalizationUpgrade records the revision and zone the stored series was built with and rebuilds everything once at startup when either differs, each meter in its own transaction. A meter that fails is logged, kept in normalization_pending and retried at the next start: one bad series must never keep the application down. What an operator sees once Existing charts change on the first start after the update: months that carried a neighbour's use give it back. Rows of earlier imports from monthly tables are marked as such before anything is recomputed, and if that marking fails nothing is rebuilt or recorded, so the upgrade simply runs again next time rather than shifting every imported month by one. A wizard import whose date format was left on auto-detect is treated as a monthly table when all of its rows sit on the 1st across at least two months -- exactly how those rows were attributed before -- and each such batch is named in the log, because a day-dated sheet always read on the 1st looks identical; revert and re-import it with the day format if that is what it was. Tests: 120 unit and 230 integration, including the reference fixtures, which still reconcile month for month.	2026-09-17 21:09:17 +02:00
schmidt.florian	ad896db051	Normalization: apportion a long unread gap across the months it covers ... A counter delta is booked at the reading that closes it. That is right at the reporting cadence -- a monthly series books December against the 1 January reading, which is what the reference sheet does -- and wrong after an outage. Observed on Solar 1: 78 days of generation arrived as one July row, leaving June looking like the array was switched off. An interval containing two or more complete calendar months is now divided across them in proportion to elapsed time. The meter recorded a total, not a shape, so every row a split produces is marked Estimated. The sum is exact: the final segment absorbs the rounding remainder, so a split never creates or destroys energy. Counting whole months *contained* rather than boundaries *crossed* is what makes the rule safe. A monthly series contains exactly one whole month per interval and is untouched, so the golden fixtures keep measuring the normalizer rather than the splitter; and a reading landing hours late cannot tip the rule and hand the new month a sliver. GapSplittingIsInertOnFixturesTests asserts the rule declines to fire on every reference interval, so this cannot drift into the oracle unnoticed. Not apportioned: swap and reset amounts (explicit corrections booked at their event -- apportioning one would rewrite a number the operator supplied), a rejected decrease, and a zero delta, which would otherwise fan out into rows that say nothing. Segments are stamped at their end, keeping the existing convention that a row records the period ending at its timestamp -- so nothing shifts relative to how unsplit intervals are already labelled. Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd	2026-07-18 19:39:42 +02:00
schmidt.florian	e23df37a3f	Connectors: allow secrets to be entered in the UI, encrypted at rest ... Reference-only secrets (SDD §6.4) meant adding a connector required editing a file on the server and restarting the service. In practice that leads to the token being pasted into the env-var *name* field, which fails with "environment variable '<token>' is not set" and gives no hint what went wrong. Add a second storage form, chosen per connector: type the secret in and it is encrypted via ASP.NET Core data protection before it is stored. The env-var reference stays as an equal alternative — this widens the choice rather than replacing it. Exactly one form survives a save, so a stale secret cannot linger and silently win; EndpointSecret.Resolve is the single resolution path. The guarantee that matters is preserved: no plaintext in the database, so pg_dump and JSON exports carry nothing usable. The trust boundary is stated plainly in §6.4 — the key ring is on disk, so this protects against leaked database content, not an attacker who already has the host, which is the same boundary an env var has. Details worth noting: - Key ring defaults to /var/lib/metervault/keys, outside the app directory, because the LXC updater republishes /opt/metervault on every update. Docker gets a named volume. Overridable via MeterVault__DataProtectionKeyPath. - Undecryptable ciphertext (key ring lost) falls back rather than throwing: an ingestion worker on a timer should degrade, not crash. - The stored secret is never sent to the browser; a blank field means "unchanged", not "cleared". - MQTT usernames are stored as-is — §6.4 covers tokens and passwords, and encrypting a username would only blank the field on every edit. - ExportService drops *_enc values: bound to the originating key ring, so useless where an export would be restored. Expect to re-enter after a restore. - HaConnectionTester now takes a resolved token, so the admin UI can test a token that has been typed but not yet saved. SDD §6.4 and §9 updated to describe both forms rather than contradict the code. Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd	2026-07-18 15:41:52 +02:00
schmidt.florian	48a7f5a825	M0: scaffold solution, EF+Timescale schema, /healthz ... Five-project Clean Architecture solution (Core/Infrastructure/App + Core.Tests/ Integration.Tests) on .NET 10 with central package management, snake_case EF mapping, and shared build/style config. - Full domain entity set + EF DbContext for the SDD §5.3 schema (singular table names). - InitialSchema migration (relational) + TimescaleHypertables migration (raw SQL: create_hypertable + compression on reading, hypertable on consumption). - App wiring: Serilog (actually wired, unlike MQTTower), DbContext, migrate-on-startup, /healthz. Serves plain HTTP behind a reverse proxy (no HTTPS redirect). - deploy/Dockerfile (2-stage, ICU-capable) + docker-compose (app + timescaledb). - Integration.Tests: shared TimescaleFixture (Testcontainers) — migrations, hypertables, compression policy, and /healthz all verified green (4/4). Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr	2026-07-13 11:02:21 +02:00