Analysis: one selected period, one set of numbers, on every page
ci / build-test (push) Successful in 2m31s
ci / build-test (push) Successful in 2m31s
The dashboards told several stories at once. Overview asked for full calendar years, meter detail for a fixed 12-month window that was really 13, Trends for 24 months with an Apply button, and the energy pages for 60. Each page derived "today" from UTC, so the first hours of a local day belonged to yesterday. A missing tariff, a month nobody measured and a genuine zero all rendered as 0. And a virtual meter -- the one thing the spreadsheet leans on hardest -- was excluded from analysis outright: MeterPeriodService returned null for it and the page offered a flow diagram instead. docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58 plus amendments A-01..A-30; code, tests and release notes cite those ids. The analysis layer Core/Analysis holds the pure rules: period presets resolved once in the instance zone into a local date range and a half-open UTC range, bucket plans, calendar-unit comparisons, coverage runs with a resolution class, normalized quantities and units, the totals policy, the virtual formula parser/validator/evaluator, and the cost calculator. "Now" comes from TimeProvider; services never read the clock. Normalization now writes, in the same transaction as consumption and by diff, per-meter rollups by local day and month plus coverage runs and a rollup state (AnalysisDataWriter). AnalysisReader answers a request from those tables -- month rollups for month and year buckets, day rollups otherwise, at most two partial edge days from consumption -- and CostReader prices the result month by month. Pages, /api/v1 and the CSV export read nothing else. The unused continuous aggregates are dropped. The reader's statement count per request is constant whether it covers one meter or a thousand. On a synthetic 1,000-meter, ten-year instance the brief's target request (100 meters, ten years, monthly) takes 374 ms against a two-second target, and the Overview went from 48,244 SQL statements per load to 205. Missing is not zero Every bucket carries a status -- available, partial, missing, unresolved, invalid, pending -- derived from coverage, never from the amount, with provenance and a reason code beside it. A true zero is a number and a bar on the baseline; an unknown bucket is a gap that says why; a month whose data only exists monthly says so instead of inventing daily detail; a scope with no tariff says "not priced" instead of 0. Rows whose interval closes after now are reported separately rather than counted. Virtual meters are analysis subjects A virtual meter stores a canonical definition -- expression over m<id> references, result kind, unit and cost rule -- validated on save and on read for syntax, unknown or self references, loops and unit/kind rules. It is evaluated on read from its sources' rollups over their joint coverage: a missing source makes the bucket missing, an observed zero is a valid input, a non-finite result is invalid with its dependency path, and the page lists each source's contribution. Topology links are topology only and never rewrite a saved calculation; expression-less meters from older installs are converted once at startup. The editor has Sum, Difference and Advanced modes with a live preview. Totals and the bill Per energy type the totals policy separates use, grid import, export, generation and runtime, marks breakdown meters as breakdowns and virtual meters as views, and never adds across units. The bill follows it: grid import where there is one, separately priced subsections at their own price, feed-in only on export meters, standing charges once per scope per local day, manual costs once on their start day, categories as non-overlapping covers whose composition reconciles to the bill. The seeded demo's yearly totals now match the spreadsheet. Pages and navigation The period lives in the URL and every page reads the same contract, so a link, a reload and the browser's Back button keep it. Shared components carry it: page header with breadcrumbs, period toolbar, theme-aware chart with an accessible table beside it, metric cards, comparison and availability states, attention items that each link to the one action that fixes them. Meter detail leads with an Analysis tab and resolves its tabs by key; the energy page has Overview, History, Flow and Meters; the old cost-only Trends page is a general Analysis page over portfolio, type, category, meter or a meter comparison. Records tabs are paged server-side instead of showing the latest 200. Everything is English and German, light and dark, down to 360px. Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and what the first start after the update does (it rebuilds all analysis data before the web server listens). docs/SDD.md and CLAUDE.md describe the system as it now is. Tests: 1,733 Core and 746 integration, all green, plus an opt-in performance suite with a synthetic 1,000-meter generator.
This commit is contained in:
Florian Schmidt
@@ -0,0 +1,343 @@
|
||||
using MeterVault.Core.Analysis;
|
||||
using MeterVault.Core.Analysis.Coverage;
|
||||
using static MeterVault.Core.Tests.Analysis.CoverageTestData;
|
||||
|
||||
namespace MeterVault.Core.Tests.Analysis;
|
||||
|
||||
/// <summary>
|
||||
/// Set operations on coverage runs: covered time, capping at now (D-04, D-13), and the joint coverage of a
|
||||
/// virtual meter's sources (D-27) — where time counts only if every source covers it, as coarsely as the
|
||||
/// coarsest source.
|
||||
/// </summary>
|
||||
public sealed class CoverageRunsTests
|
||||
{
|
||||
[Fact]
|
||||
public void Covered_time_unions_overlapping_runs_ignores_gaps_and_reports_the_outer_bounds()
|
||||
{
|
||||
CoverageRun[] runs =
|
||||
[
|
||||
Run(At(2026, 3, 10), At(2026, 3, 20), ResolutionClass.Day),
|
||||
Gap(At(2026, 3, 1), At(2026, 3, 10), CoverageGapReason.SampleGap),
|
||||
Run(At(2026, 3, 15), At(2026, 3, 25), ResolutionClass.Hour),
|
||||
Run(At(2026, 4, 1), At(2026, 4, 3), ResolutionClass.Hour),
|
||||
];
|
||||
|
||||
var covered = CoverageRuns.Covered(runs);
|
||||
|
||||
Assert.Equal(TimeSpan.FromDays(17), covered.Total);
|
||||
Assert.Equal(At(2026, 3, 10), covered.First);
|
||||
Assert.Equal(At(2026, 4, 3), covered.Last);
|
||||
Assert.False(covered.IsEmpty);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Covered_time_inside_a_window_is_clipped_to_it()
|
||||
{
|
||||
var covered = CoverageRuns.Covered([Run(At(2026, 3, 10), At(2026, 3, 20), ResolutionClass.Day)], At(2026, 3, 15), At(2026, 4, 1));
|
||||
|
||||
Assert.Equal(TimeSpan.FromDays(5), covered.Total);
|
||||
Assert.Equal(At(2026, 3, 15), covered.First);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Nothing_covered_is_an_empty_span()
|
||||
{
|
||||
var covered = CoverageRuns.Covered([Gap(At(2026, 3, 1), At(2026, 3, 10), CoverageGapReason.SampleGap)]);
|
||||
|
||||
Assert.True(covered.IsEmpty);
|
||||
Assert.Equal(CoveredSpan.None, covered);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_current_month_label_row_is_recorded_after_now_so_coverage_ends_at_the_month_start()
|
||||
{
|
||||
// D-04: the September row describes all of September and is left out of actuals until it closes.
|
||||
// Clipping at now would leave 1-19 September looking covered by a row nobody reads.
|
||||
var now = At(2026, 9, 19, 10);
|
||||
CoverageRun[] runs =
|
||||
[
|
||||
MonthLabels(2026, 1, 2026, 10),
|
||||
Run(At(2026, 10, 1), At(2026, 11, 1), ResolutionClass.Hour),
|
||||
];
|
||||
|
||||
var capped = CoverageRuns.CapAt(runs, now, Berlin);
|
||||
|
||||
Assert.Equal([MonthLabels(2026, 1, 2026, 9) with { LastIntervalStart = null }], capped);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Runs_that_end_by_now_are_kept_unchanged_and_runs_that_start_after_now_are_dropped()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10);
|
||||
var past = Run(At(2026, 1, 1), now, ResolutionClass.Hour, last: At(2026, 9, 19, 9));
|
||||
|
||||
Assert.Equal([past], CoverageRuns.CapAt([past, Run(now, At(2026, 10, 1), ResolutionClass.Hour)], now, Berlin));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_future_stamped_daily_reading_gives_up_only_the_interval_containing_now()
|
||||
{
|
||||
// m2 #5: read at 06:00 daily, the next reading already stamped tomorrow. The run ends where the
|
||||
// interval containing now starts, not at a midnight or a class limit before it.
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
var daily = Assert.Single(CoverageRuns.CapAt(
|
||||
[Run(At(2026, 9, 1, 6), At(2026, 9, 20, 6), ResolutionClass.Day, divided: true, last: At(2026, 9, 19, 6))], now, Berlin));
|
||||
|
||||
Assert.Equal(Run(At(2026, 9, 1, 6), At(2026, 9, 19, 6), ResolutionClass.Day, divided: true), daily);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Monthly_readings_with_one_in_the_future_keep_every_share_recorded_before_now()
|
||||
{
|
||||
// m2 #5: read on the 15th; the reading of 15 September is stamped in the future (now: 10 September).
|
||||
// Its interval's August share closed on 1 September and is an actual, so August stays covered.
|
||||
var now = At(2026, 9, 10, 10);
|
||||
|
||||
var monthly = Assert.Single(CoverageRuns.CapAt(
|
||||
[Run(At(2026, 5, 15), At(2026, 9, 15), ResolutionClass.Month, divided: true, last: At(2026, 9, 1))], now, Berlin));
|
||||
|
||||
Assert.Equal(At(2026, 9, 1), monthly.To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Weekly_readings_give_up_their_last_week_when_now_falls_inside_it()
|
||||
{
|
||||
// m2 #11: burner hours read every Monday at 10:00, the last reading stamped a week ahead.
|
||||
var stored = Run(At(2026, 8, 3, 10), At(2026, 9, 21, 10), ResolutionClass.Week, divided: true, last: At(2026, 9, 14, 10));
|
||||
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([stored], At(2026, 9, 19, 10), Berlin));
|
||||
|
||||
Assert.Equal(At(2026, 9, 14, 10), capped.To);
|
||||
Assert.Null(capped.LastIntervalStart);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Now_inside_an_earlier_interval_ends_the_run_one_interval_before_now()
|
||||
{
|
||||
// A-14: two readings stamped ahead. Only the final interval's start is stored, so the interval containing
|
||||
// now (14 - 21 September) is not known; it started no earlier than one week-class interval before now, and
|
||||
// claiming nothing after that never counts time whose row is recorded after now.
|
||||
var stored = Run(At(2026, 8, 3, 10), At(2026, 9, 28, 10), ResolutionClass.Week, divided: true, last: At(2026, 9, 21, 10));
|
||||
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([stored], At(2026, 9, 17, 10), Berlin));
|
||||
|
||||
Assert.Equal(At(2026, 9, 17, 10) - ResolutionClassifier.WeekLimit, capped.To);
|
||||
Assert.Null(capped.LastIntervalStart);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_label_run_reaching_past_the_current_month_gives_up_the_current_month()
|
||||
{
|
||||
// Review F4: sheet rows July - October, the October row carrying September's register forward. Now (19
|
||||
// September) falls inside the September row's interval, not the last one; the September row closes after
|
||||
// now, so September must not look covered — or it would read as a true zero.
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([MonthLabels(2026, 7, 2026, 11)], At(2026, 9, 19, 10), Berlin));
|
||||
|
||||
Assert.Equal(At(2026, 9, 1), capped.To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Month_shares_of_a_reading_stamped_weeks_ahead_keep_the_months_before_now()
|
||||
{
|
||||
// Review F4: read on 20 August, the next reading typed as 5 October. The August share closed on 1 September
|
||||
// and is an actual; the September share closes on 30 September, after now.
|
||||
var stored = Run(At(2026, 7, 20, 9), At(2026, 10, 5, 9), ResolutionClass.Month, divided: true, last: At(2026, 10, 1));
|
||||
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([stored], At(2026, 9, 19, 14), Berlin));
|
||||
|
||||
Assert.Equal(At(2026, 9, 1), capped.To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Hourly_readings_with_two_stamped_ahead_give_up_one_hour_before_now_and_stay_up_to_date()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10, 30);
|
||||
var stored = Run(At(2026, 9, 1), At(2026, 9, 19, 13), ResolutionClass.Hour, divided: true, last: At(2026, 9, 19, 12));
|
||||
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([stored], now, Berlin));
|
||||
Assert.Equal(now - ResolutionClassifier.HourLimit, capped.To);
|
||||
|
||||
// A-04: today's bucket still counts as up to date — the coverage reaches within one interval of now.
|
||||
var today = CutAt(Day(2026, 9, 19), now);
|
||||
Assert.Equal(BucketStatus.Available, CoverageEvaluator.Evaluate(today, [stored], Berlin, openingBalanceInBucket: false, now).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_run_that_does_not_know_its_last_interval_is_cut_at_now()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
Assert.Equal(now, Assert.Single(CoverageRuns.CapAt([Run(At(2026, 9, 1), At(2026, 9, 20), ResolutionClass.Hour)], now, Berlin)).To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_undivided_interval_longer_than_a_month_straddling_now_is_dropped_whole()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
Assert.Empty(CoverageRuns.CapAt([Single(At(2026, 8, 5), At(2026, 10, 5), ResolutionClass.Coarse)], now, Berlin));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_divided_interval_straddling_now_keeps_its_shares_of_the_months_before()
|
||||
{
|
||||
// Read on 20 July and (stamped ahead) on 25 September: the September share is the run's last
|
||||
// interval and closes after now; the July and August shares closed before it and are actuals.
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
var capped = CoverageRuns.CapAt([Run(At(2026, 7, 20), At(2026, 9, 25), ResolutionClass.Month, divided: true, last: At(2026, 9, 1))], now, Berlin);
|
||||
|
||||
Assert.Equal([Run(At(2026, 7, 20), At(2026, 9, 1), ResolutionClass.Month, divided: true)], capped);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_gap_straddling_now_is_clipped_at_now()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
var capped = CoverageRuns.CapAt([Gap(At(2026, 9, 18), At(2026, 9, 21), CoverageGapReason.SampleGap) with { LastIntervalStart = At(2026, 9, 18) }], now, Berlin);
|
||||
|
||||
Assert.Equal(now, Assert.Single(capped).To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Zero_length_runs_claim_no_time_and_are_dropped()
|
||||
{
|
||||
// A-01: an opening balance is not a run; nothing zero-length survives capping.
|
||||
var now = At(2026, 9, 19, 10);
|
||||
|
||||
Assert.Empty(CoverageRuns.CapAt([Run(now.AddHours(-1), now.AddHours(-1), ResolutionClass.Hour)], now, Berlin));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Capping_twice_at_the_same_instant_changes_nothing()
|
||||
{
|
||||
var now = At(2026, 9, 19, 10);
|
||||
CoverageRun[] stored =
|
||||
[
|
||||
MonthLabels(2026, 1, 2026, 10),
|
||||
Run(At(2026, 9, 1, 6), At(2026, 9, 20, 6), ResolutionClass.Day, divided: true, last: At(2026, 9, 19, 6)),
|
||||
Run(At(2026, 9, 1), At(2026, 9, 20), ResolutionClass.Hour),
|
||||
];
|
||||
|
||||
var once = CoverageRuns.CapAt(stored, now, Berlin);
|
||||
|
||||
Assert.Equal(once, CoverageRuns.CapAt(once, now, Berlin));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void In_New_York_the_running_months_label_is_given_up_at_the_New_York_month_start()
|
||||
{
|
||||
var now = At(NewYork, 2026, 9, 19, 10);
|
||||
|
||||
var capped = Assert.Single(CoverageRuns.CapAt([MonthLabels(NewYork, 2026, 1, 2026, 10)], now, NewYork));
|
||||
|
||||
Assert.Equal(At(NewYork, 2026, 9, 1), capped.To);
|
||||
Assert.Equal(new DateTimeOffset(2026, 9, 1, 4, 0, 0, TimeSpan.Zero), capped.To);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_joint_coverage_of_a_monthly_and_an_hourly_source_is_their_overlap_at_month_resolution()
|
||||
{
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[MonthLabels(2026, 1, 2026, 7)],
|
||||
[Run(At(2026, 3, 15, 12), At(2026, 9, 1), ResolutionClass.Hour)],
|
||||
]);
|
||||
|
||||
var run = Assert.Single(joint);
|
||||
Assert.Equal(At(2026, 3, 15, 12), run.From);
|
||||
Assert.Equal(At(2026, 7, 1), run.To);
|
||||
Assert.Equal(ResolutionClass.Month, run.Resolution);
|
||||
|
||||
// Only divided when every source was: the hourly samples were never divided.
|
||||
Assert.False(run.DividedAtMonths);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Joint_runs_split_where_the_coarsest_class_changes_and_merge_where_it_does_not()
|
||||
{
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[
|
||||
Run(At(2026, 1, 1), At(2026, 2, 1), ResolutionClass.Hour),
|
||||
Run(At(2026, 2, 1), At(2026, 3, 1), ResolutionClass.Hour),
|
||||
Run(At(2026, 3, 1), At(2026, 4, 1), ResolutionClass.Week),
|
||||
],
|
||||
[Run(At(2026, 1, 1), At(2026, 4, 1), ResolutionClass.Day)],
|
||||
]);
|
||||
|
||||
Assert.Equal(
|
||||
[
|
||||
Run(At(2026, 1, 1), At(2026, 3, 1), ResolutionClass.Day),
|
||||
Run(At(2026, 3, 1), At(2026, 4, 1), ResolutionClass.Week),
|
||||
],
|
||||
joint);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Joint_coverage_is_divided_only_where_every_source_is()
|
||||
{
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[MonthLabels(2025, 1, 2026, 1)],
|
||||
[MonthLabels(2025, 1, 2025, 7), Run(At(2025, 7, 1), At(2026, 1, 1), ResolutionClass.Month)],
|
||||
]);
|
||||
|
||||
// Joint runs do not know where any source's last interval starts.
|
||||
Assert.Equal(
|
||||
[
|
||||
MonthLabels(2025, 1, 2025, 7) with { LastIntervalStart = null },
|
||||
Run(At(2025, 7, 1), At(2026, 1, 1), ResolutionClass.Month),
|
||||
],
|
||||
joint);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_hole_in_one_source_is_a_hole_in_the_joint_coverage_and_keeps_its_reason()
|
||||
{
|
||||
var outage = Gap(At(2026, 3, 10, 14), At(2026, 3, 13, 9), CoverageGapReason.SampleGap);
|
||||
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[Run(At(2026, 3, 1), At(2026, 3, 10, 14), ResolutionClass.Hour), outage, Run(At(2026, 3, 13, 9), At(2026, 4, 1), ResolutionClass.Hour)],
|
||||
[Run(At(2026, 3, 1), At(2026, 4, 1), ResolutionClass.Day)],
|
||||
]);
|
||||
|
||||
Assert.Equal(
|
||||
[
|
||||
Run(At(2026, 3, 1), At(2026, 3, 10, 14), ResolutionClass.Day),
|
||||
outage,
|
||||
Run(At(2026, 3, 13, 9), At(2026, 4, 1), ResolutionClass.Day),
|
||||
],
|
||||
joint);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_source_without_coverage_leaves_no_joint_coverage()
|
||||
{
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[Run(At(2026, 3, 1), At(2026, 4, 1), ResolutionClass.Hour)],
|
||||
[],
|
||||
]);
|
||||
|
||||
Assert.Empty(joint);
|
||||
Assert.Empty(CoverageRuns.Intersect([]));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_single_source_intersects_to_its_own_coverage_with_touching_runs_merged()
|
||||
{
|
||||
var joint = CoverageRuns.Intersect(
|
||||
[
|
||||
[
|
||||
Run(At(2026, 3, 1), At(2026, 3, 15), ResolutionClass.Hour),
|
||||
Run(At(2026, 3, 15), At(2026, 4, 1), ResolutionClass.Hour),
|
||||
],
|
||||
]);
|
||||
|
||||
Assert.Equal([Run(At(2026, 3, 1), At(2026, 4, 1), ResolutionClass.Hour)], joint);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user