FinalFactory/MeterVault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5a6f34a4676bc52d6a228ddd4e539fcd5b1dd845
MeterVault/tests/Core.Tests/Analysis/EngineIntervalTests.cs
T
Florian Schmidt 8940ef25c3
ci / build-test (push) Successful in 2m31s
Details
Analysis: one selected period, one set of numbers, on every page 
The dashboards told several stories at once. Overview asked for full
calendar years, meter detail for a fixed 12-month window that was really
13, Trends for 24 months with an Apply button, and the energy pages for
60. Each page derived "today" from UTC, so the first hours of a local day
belonged to yesterday. A missing tariff, a month nobody measured and a
genuine zero all rendered as 0. And a virtual meter -- the one thing the
spreadsheet leans on hardest -- was excluded from analysis outright:
MeterPeriodService returned null for it and the page offered a flow
diagram instead.

docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it
left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58
plus amendments A-01..A-30; code, tests and release notes cite those ids.

The analysis layer

Core/Analysis holds the pure rules: period presets resolved once in the
instance zone into a local date range and a half-open UTC range, bucket
plans, calendar-unit comparisons, coverage runs with a resolution class,
normalized quantities and units, the totals policy, the virtual formula
parser/validator/evaluator, and the cost calculator. "Now" comes from
TimeProvider; services never read the clock.

Normalization now writes, in the same transaction as consumption and by
diff, per-meter rollups by local day and month plus coverage runs and a
rollup state (AnalysisDataWriter). AnalysisReader answers a request from
those tables -- month rollups for month and year buckets, day rollups
otherwise, at most two partial edge days from consumption -- and
CostReader prices the result month by month. Pages, /api/v1 and the CSV
export read nothing else. The unused continuous aggregates are dropped.

The reader's statement count per request is constant whether it covers one
meter or a thousand. On a synthetic 1,000-meter, ten-year instance the
brief's target request (100 meters, ten years, monthly) takes 374 ms
against a two-second target, and the Overview went from 48,244 SQL
statements per load to 205.

Missing is not zero

Every bucket carries a status -- available, partial, missing, unresolved,
invalid, pending -- derived from coverage, never from the amount, with
provenance and a reason code beside it. A true zero is a number and a bar
on the baseline; an unknown bucket is a gap that says why; a month whose
data only exists monthly says so instead of inventing daily detail; a
scope with no tariff says "not priced" instead of 0. Rows whose interval
closes after now are reported separately rather than counted.

Virtual meters are analysis subjects

A virtual meter stores a canonical definition -- expression over m<id>
references, result kind, unit and cost rule -- validated on save and on
read for syntax, unknown or self references, loops and unit/kind rules.
It is evaluated on read from its sources' rollups over their joint
coverage: a missing source makes the bucket missing, an observed zero is
a valid input, a non-finite result is invalid with its dependency path,
and the page lists each source's contribution. Topology links are
topology only and never rewrite a saved calculation; expression-less
meters from older installs are converted once at startup. The editor has
Sum, Difference and Advanced modes with a live preview.

Totals and the bill

Per energy type the totals policy separates use, grid import, export,
generation and runtime, marks breakdown meters as breakdowns and virtual
meters as views, and never adds across units. The bill follows it: grid
import where there is one, separately priced subsections at their own
price, feed-in only on export meters, standing charges once per scope per
local day, manual costs once on their start day, categories as
non-overlapping covers whose composition reconciles to the bill. The
seeded demo's yearly totals now match the spreadsheet.

Pages and navigation

The period lives in the URL and every page reads the same contract, so a
link, a reload and the browser's Back button keep it. Shared components
carry it: page header with breadcrumbs, period toolbar, theme-aware chart
with an accessible table beside it, metric cards, comparison and
availability states, attention items that each link to the one action
that fixes them. Meter detail leads with an Analysis tab and resolves its
tabs by key; the energy page has Overview, History, Flow and Meters; the
old cost-only Trends page is a general Analysis page over portfolio, type,
category, meter or a meter comparison. Records tabs are paged server-side
instead of showing the latest 200. Everything is English and German,
light and dark, down to 360px.

Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and
what the first start after the update does (it rebuilds all analysis data
before the web server listens). docs/SDD.md and CLAUDE.md describe the
system as it now is.

Tests: 1,733 Core and 746 integration, all green, plus an opt-in
performance suite with a synthetic 1,000-meter generator.
2026-09-20 10:29:13 +02:00

762 lines
36 KiB
C#
Raw Blame History

using MeterVault.Core.Analysis;
using MeterVault.Core.Domain;
using MeterVault.Core.Normalization;
using MeterVault.Core.Normalization.Normalizers;
using static MeterVault.Core.Tests.Analysis.AnalysisTestTime;
using static MeterVault.Core.Tests.TestData;
namespace MeterVault.Core.Tests.Analysis;
/// <summary>
/// Every normalized row says which stretch of time it accrued over (D-10), and a row that closes at a
/// local midnight is booked in the day it closes (D-11). Coverage, rollups and bucket status are built
/// on these intervals, so each mode is pinned here — including the first reading, whose start is only
/// known from a month label or an install date.
/// </summary>
public sealed class EngineIntervalTests
{
private readonly INormalizationEngine _engine = NormalizationEngine.CreateDefault();
private static Reading Manual(DateTimeOffset time, double value, int meterId = 1) =>
new() { MeterId = meterId, Time = time, Value = value, Quality = ReadingQuality.Manual };
private static Reading Measured(DateTimeOffset time, double value, int meterId = 1) =>
new() { MeterId = meterId, Time = time, Value = value, Quality = ReadingQuality.Measured };
private static NormalizationContext Context(
MeterMode mode, TimeZoneInfo zone, IReadOnlyList<Reading> readings, IReadOnlyList<MeterEvent>? events = null, DateOnly? installedAt = null) =>
new()
{
Meter = new MeterConfig { MeterId = 1, Mode = mode, Unit = "kWh", InstalledAt = installedAt },
Readings = readings,
Events = events ?? [],
TimeZone = zone,
};
private static TimeZoneInfo Zone(string id) => id == "UTC" ? TimeZoneInfo.Utc : TimeZoneInfo.FindSystemTimeZoneById(id);
// ---- Registers -------------------------------------------------------------------------------
[Fact]
public void Each_register_row_covers_the_time_since_the_previous_reading()
{
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[
Manual(InBerlin(2026, 9, 2, 8), 100),
Manual(InBerlin(2026, 9, 10, 20), 110),
Manual(InBerlin(2026, 9, 20, 6), 125),
]));
Assert.Equal(3, result.Count);
Assert.Equal([InBerlin(2026, 9, 2, 8), InBerlin(2026, 9, 10, 20)], result.Skip(1).Select(c => c.IntervalStart!.Value));
Assert.Equal([InBerlin(2026, 9, 10, 20), InBerlin(2026, 9, 20, 6)], result.Skip(1).Select(c => c.IntervalEnd!.Value));
Assert.All(result.Skip(1), c =>
{
Assert.False(c.Divided);
Assert.False(c.OpeningBalance);
Assert.Equal(CoverageGapReason.None, c.Gap);
});
}
[Fact]
public void A_first_reading_without_a_label_or_install_date_is_an_opening_balance_with_an_unknown_start()
{
var first = Assert.Single(_engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin, [Manual(InBerlin(2026, 9, 2, 8), 300)])));
Assert.True(first.OpeningBalance);
Assert.Equal(300, first.Amount);
Assert.Equal(InBerlin(2026, 9, 2, 8), first.IntervalStart);
Assert.Equal(InBerlin(2026, 9, 2, 8), first.IntervalEnd);
Assert.Equal(InBerlin(2026, 9, 2, 8), first.Time);
}
[Fact]
public void A_first_reading_starts_at_the_local_midnight_of_the_install_date()
{
var first = Assert.Single(_engine.Normalize(Context(
MeterMode.CumulativeCounter, Berlin, [Manual(InBerlin(2026, 9, 2, 8), 300)], installedAt: new DateOnly(2026, 8, 15))));
Assert.False(first.OpeningBalance);
Assert.False(first.Divided); // the baseline delta is booked at its reading, as it always was
Assert.Equal(Utc(2026, 8, 14, 22), first.IntervalStart); // 15 August 00:00 in Berlin
Assert.Equal(InBerlin(2026, 9, 2, 8), first.IntervalEnd);
Assert.Equal(InBerlin(2026, 9, 2, 8), first.Time);
}
[Fact]
public void An_install_date_after_the_first_reading_says_nothing_about_its_start()
{
var first = Assert.Single(_engine.Normalize(Context(
MeterMode.CumulativeCounter, Berlin, [Manual(InBerlin(2026, 9, 2, 8), 300)], installedAt: new DateOnly(2026, 9, 5))));
Assert.True(first.OpeningBalance);
Assert.Equal(first.IntervalEnd, first.IntervalStart);
}
[Fact]
public void A_first_month_row_covers_the_local_month_it_names()
{
// Zähler Auto's first sheet row, "Mai 2023" = 3755: the whole register booked as May's figure.
var first = Assert.Single(_engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin, [Reading(1, Month(2023, 5), 3755)])));
Assert.False(first.OpeningBalance);
Assert.Equal(InBerlin(2023, 5, 1), first.IntervalStart);
Assert.Equal(InBerlin(2023, 6, 1), first.IntervalEnd);
Assert.Equal(Month(2023, 5), first.Time);
}
[Theory]
[InlineData("UTC")]
[InlineData("Europe/Berlin")]
public void Consecutive_month_rows_keep_their_stamps_and_span_exactly_their_months(string zoneId)
{
// The golden-fixture shape. A label closes at a local midnight (its month's end) but is a month
// figure, not a reading at that instant: it keeps its stamp.
var zone = Zone(zoneId);
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, zone,
[
Reading(1, Month(2022, 9), 0),
Reading(1, Month(2022, 10), 411),
Reading(1, Month(2022, 11), 1153),
Reading(1, Month(2022, 12), 1968),
]));
Assert.Equal([Month(2022, 9), Month(2022, 10), Month(2022, 11), Month(2022, 12)], result.Select(c => c.Time));
Assert.Equal([0, 411, 742, 815], result.Select(c => c.Amount).ToArray());
DateTimeOffset MonthStart(int month) => GapAttribution.LocalMidnight(new DateOnly(2022, month, 1), zone);
Assert.Equal([MonthStart(9), MonthStart(10), MonthStart(11), MonthStart(12)], result.Select(c => c.IntervalStart!.Value));
Assert.Equal([MonthStart(10), MonthStart(11), MonthStart(12), GapAttribution.LocalMidnight(new DateOnly(2023, 1, 1), zone)],
result.Select(c => c.IntervalEnd!.Value));
Assert.DoesNotContain(result, c => c.Divided || c.OpeningBalance || c.Gap != CoverageGapReason.None);
}
[Fact]
public void An_interval_divided_at_a_month_boundary_gives_each_share_its_own_segment()
{
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[
Manual(InBerlin(2026, 8, 1, 9), 700),
Manual(InBerlin(2026, 9, 16, 18), 746),
]));
var shares = result.Skip(1).ToList();
Assert.Equal(2, shares.Count);
Assert.All(shares, c =>
{
Assert.True(c.Divided);
Assert.Equal(ReadingQuality.Estimated, c.Quality);
});
Assert.Equal(InBerlin(2026, 8, 1, 9), shares[0].IntervalStart);
Assert.Equal(InBerlin(2026, 9, 1), shares[0].IntervalEnd);
Assert.Equal(InBerlin(2026, 8, 31, 23, 59, 59), shares[0].Time);
Assert.Equal(InBerlin(2026, 9, 1), shares[1].IntervalStart);
Assert.Equal(InBerlin(2026, 9, 16, 18), shares[1].IntervalEnd);
Assert.Equal(InBerlin(2026, 9, 16, 18), shares[1].Time);
Assert.Equal(46, shares.Sum(c => c.Amount), 9);
}
[Fact]
public void Attribution_segments_tile_the_interval_at_local_month_starts()
{
var from = InBerlin(2026, 5, 20, 7);
var to = InBerlin(2026, 8, 3, 21);
var segments = GapAttribution.Attribute(from, to, to, 1000, Berlin);
Assert.Equal([from, InBerlin(2026, 6, 1), InBerlin(2026, 7, 1), InBerlin(2026, 8, 1)], segments.Select(s => s.From));
Assert.Equal([InBerlin(2026, 6, 1), InBerlin(2026, 7, 1), InBerlin(2026, 8, 1), to], segments.Select(s => s.To));
Assert.All(segments, s => Assert.True(s.Time >= s.From && s.Time <= s.To, $"{s.Time:O} lies outside its segment"));
}
[Fact]
public void An_unexplained_decrease_marks_its_interval_as_a_gap()
{
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, TimeZoneInfo.Utc,
[Reading(1, Month(2023, 1), 100), Reading(1, Month(2023, 2), 60), Reading(1, Month(2023, 3), 70)]));
Assert.Equal([CoverageGapReason.None, CoverageGapReason.UnexplainedDecrease, CoverageGapReason.None], result.Select(c => c.Gap));
Assert.Equal(Month(2023, 2), result[1].IntervalStart);
Assert.Equal(Month(2023, 3), result[1].IntervalEnd);
Assert.Equal(0, result[1].Amount);
}
[Theory]
[InlineData(null, CoverageGapReason.ResetWithoutPrevious)]
[InlineData(170d, CoverageGapReason.None)]
public void A_reset_is_a_gap_only_when_it_does_not_say_where_the_old_register_stopped(double? prevValue, CoverageGapReason expected)
{
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig { MeterId = 1, Mode = MeterMode.CumulativeCounter, Unit = "kWh", InitialBaseline = 90 },
Readings = [Reading(1, Month(2023, 1), 100), Reading(1, Month(2023, 2), 150), Reading(1, Month(2023, 3), 30)],
Events = [Reset(1, Month(2023, 3), newValue: 0, prevValue: prevValue)],
});
Assert.Equal(expected, result[^1].Gap);
Assert.Equal(Month(2023, 3), result[^1].IntervalStart);
}
[Fact]
public void A_swap_with_its_amount_is_covered_time()
{
// The water register …861 → 2 books 12 m³ across the swap month: no hole.
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig { MeterId = 1, Mode = MeterMode.CumulativeCounter, Unit = "m³" },
Readings = [Reading(1, Month(2023, 1), 848), Reading(1, Month(2023, 2), 861), Reading(1, Month(2023, 3), 2), Reading(1, Month(2023, 4), 15)],
Events = [Swap(1, Month(2023, 3), prevValue: 861, newValue: 2, amount: 12)],
});
Assert.DoesNotContain(result, c => c.Gap != CoverageGapReason.None);
Assert.Equal(Month(2023, 3), result[2].IntervalStart);
Assert.Equal(Month(2023, 4), result[2].IntervalEnd);
}
[Fact]
public void Every_row_names_the_whole_reading_interval_it_came_from()
{
// A-03: shares are classified and told apart by their source interval, not by their own segment.
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[
Manual(InBerlin(2026, 8, 2, 9), 700),
Manual(InBerlin(2026, 8, 10, 9), 720),
Manual(InBerlin(2026, 9, 16, 18), 766),
]));
var inside = result.Single(c => c.IntervalEnd == InBerlin(2026, 8, 10, 9));
var shares = result.Where(c => c.Divided).ToList();
Assert.Equal((inside.IntervalStart, inside.IntervalEnd), (inside.SourceStart, inside.SourceEnd));
Assert.Equal(2, shares.Count);
Assert.All(shares, c =>
{
Assert.Equal(InBerlin(2026, 8, 10, 9), c.SourceStart);
Assert.Equal(InBerlin(2026, 9, 16, 18), c.SourceEnd);
});
Assert.NotEqual(shares[0].IntervalEnd, shares[1].IntervalEnd);
}
[Fact]
public void A_register_that_stands_still_across_a_month_boundary_is_one_row_exact_in_every_month()
{
// m7 #2, A-02: nothing moved from 20 June to 20 July. One row, zero, flagged as month-exact, and not
// estimated — nothing about it was inferred.
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[Manual(InBerlin(2026, 6, 20, 12), 130), Manual(InBerlin(2026, 7, 20, 12), 130)]));
var still = result[^1];
Assert.Equal(2, result.Count);
Assert.Equal(0, still.Amount);
Assert.True(still.Divided);
Assert.Equal(ReadingQuality.Manual, still.Quality);
Assert.Equal(InBerlin(2026, 6, 20, 12), still.IntervalStart);
Assert.Equal(InBerlin(2026, 7, 20, 12), still.IntervalEnd);
}
[Fact]
public void A_standstill_inside_one_month_or_a_rejected_decrease_is_not_flagged()
{
var inside = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[Manual(InBerlin(2026, 6, 2, 12), 130), Manual(InBerlin(2026, 6, 20, 12), 130)]));
var decrease = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[Manual(InBerlin(2026, 6, 20, 12), 130), Manual(InBerlin(2026, 7, 20, 12), 90)]));
Assert.False(inside[^1].Divided);
Assert.False(decrease[^1].Divided);
Assert.Equal(CoverageGapReason.UnexplainedDecrease, decrease[^1].Gap);
}
[Theory]
[InlineData(MeterMode.RuntimeCounter)]
[InlineData(MeterMode.DirectDelta)]
[InlineData(MeterMode.GenerationCounter)]
public void Zero_across_a_month_boundary_is_exact_in_every_month_for_every_register_like_mode(MeterMode mode)
{
var first = mode == MeterMode.DirectDelta ? 4 : 130;
var second = mode == MeterMode.DirectDelta ? 0 : 130;
var result = _engine.Normalize(Context(mode, Berlin,
[Manual(InBerlin(2026, 6, 20, 12), first), Manual(InBerlin(2026, 7, 20, 12), second)]));
Assert.Equal(0, result[^1].Amount);
Assert.True(result[^1].Divided);
Assert.Equal(ReadingQuality.Manual, result[^1].Quality);
}
[Fact]
public void Burner_hours_that_moved_across_a_month_boundary_stay_undivided()
{
var result = _engine.Normalize(Context(MeterMode.RuntimeCounter, Berlin,
[Manual(InBerlin(2026, 6, 20, 12), 130), Manual(InBerlin(2026, 7, 20, 12), 150)]));
Assert.False(result[^1].Divided);
Assert.Equal(20, result[^1].Amount);
}
// ---- Midnight stamps (D-11) ------------------------------------------------------------------
[Theory]
[InlineData("Europe/Berlin")]
[InlineData("America/New_York")]
[InlineData("UTC")]
public void A_daily_snapshot_at_local_midnight_is_booked_in_the_day_it_closes(string zoneId)
{
var zone = Zone(zoneId);
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, zone,
[
Measured(Local(zone, 2026, 9, 10), 100),
Measured(Local(zone, 2026, 9, 11), 105),
Measured(Local(zone, 2026, 9, 12), 112),
]));
// The opening balance describes no time before its midnight, so it stays on it.
Assert.Equal(Local(zone, 2026, 9, 10), result[0].Time);
Assert.Equal([Local(zone, 2026, 9, 10, 23, 59, 59), Local(zone, 2026, 9, 11, 23, 59, 59)], result.Skip(1).Select(c => c.Time));
Assert.Equal([new DateOnly(2026, 9, 10), new DateOnly(2026, 9, 11)], result.Skip(1).Select(c => LocalDate(c.Time, zone)));
Assert.Equal([5d, 7d], result.Skip(1).Select(c => c.Amount));
Assert.Equal([Local(zone, 2026, 9, 11), Local(zone, 2026, 9, 12)], result.Skip(1).Select(c => c.IntervalEnd!.Value));
Assert.All(result.Skip(1), c => Assert.Equal(ReadingQuality.Measured, c.Quality));
}
[Fact]
public void A_daily_snapshot_after_the_long_autumn_day_is_stamped_inside_that_day()
{
// 25 October 2026 has 25 hours in Berlin.
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[Measured(InBerlin(2026, 10, 25), 100), Measured(InBerlin(2026, 10, 26), 110)]));
Assert.Equal(new DateOnly(2026, 10, 25), LocalDate(result[1].Time, Berlin));
Assert.Equal(TimeSpan.FromHours(25), result[1].IntervalEnd - result[1].IntervalStart);
}
[Theory]
[InlineData(MeterMode.CumulativeCounter)]
[InlineData(MeterMode.RuntimeCounter)]
[InlineData(MeterMode.DirectDelta)]
[InlineData(MeterMode.InstantRate)]
public void The_midnight_that_ends_a_month_books_its_row_on_that_months_last_day(MeterMode mode)
{
var result = _engine.Normalize(Context(mode, Berlin,
[Manual(InBerlin(2026, 8, 20, 12), 100), Manual(InBerlin(2026, 9, 1), 130)]));
var closing = result[^1];
Assert.Equal(InBerlin(2026, 8, 31, 23, 59, 59), closing.Time);
Assert.Equal(InBerlin(2026, 8, 20, 12), closing.IntervalStart);
Assert.Equal(InBerlin(2026, 9, 1), closing.IntervalEnd);
}
[Fact]
public void A_divided_interval_that_closes_at_a_midnight_stamps_its_last_share_in_the_day_before()
{
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, Berlin,
[Manual(InBerlin(2026, 8, 20, 12), 100), Manual(InBerlin(2026, 9, 10), 142)]));
var shares = result.Skip(1).ToList();
Assert.Equal([InBerlin(2026, 8, 31, 23, 59, 59), InBerlin(2026, 9, 9, 23, 59, 59)], shares.Select(c => c.Time));
Assert.Equal([InBerlin(2026, 9, 1), InBerlin(2026, 9, 10)], shares.Select(c => c.IntervalEnd!.Value));
Assert.All(shares, c => Assert.True(c.Divided));
}
[Fact]
public void A_day_dated_import_at_utc_midnight_closes_the_day_before_only_where_that_is_local_midnight()
{
// The importer stamps "20.07.2026" at 00:00 UTC. In a UTC instance that is the midnight that ends
// 19 July; in Berlin it is 02:00 on the 20th, an instant inside the day.
Reading[] readings = [DayReading(1, Utc(2026, 7, 15), 600), DayReading(1, Utc(2026, 7, 20), 650)];
var inUtc = _engine.Normalize(Context(MeterMode.RuntimeCounter, TimeZoneInfo.Utc, readings));
var inBerlin = _engine.Normalize(Context(MeterMode.RuntimeCounter, Berlin, readings));
Assert.Equal(Utc(2026, 7, 20).AddSeconds(-1), inUtc[^1].Time);
Assert.Equal(Utc(2026, 7, 20), inBerlin[^1].Time);
Assert.Equal(Utc(2026, 7, 20), inUtc[^1].IntervalEnd);
}
[Fact]
public void Where_midnight_happens_twice_only_the_first_starts_the_day()
{
// Havana leaves daylight time at 01:00 on 1 November 2026: 00:00 is at 04:00 and again at 05:00 UTC.
var havana = TimeZoneInfo.FindSystemTimeZoneById("America/Havana");
Assert.True(GapAttribution.IsLocalMidnight(Utc(2026, 11, 1, 4), havana));
Assert.False(GapAttribution.IsLocalMidnight(Utc(2026, 11, 1, 5), havana));
Assert.True(GapAttribution.IsLocalMonthStart(Utc(2026, 11, 1, 4), havana));
}
[Fact]
public void A_midnight_stamp_never_leaves_an_interval_shorter_than_a_second()
{
var end = InBerlin(2026, 9, 1);
var start = end.AddMilliseconds(-400);
var stamp = GapAttribution.CloseStamp(start, end, Berlin);
Assert.True(stamp > start && stamp < end, $"{stamp:O} lies outside the interval");
Assert.Equal(end, GapAttribution.CloseStamp(end, end, Berlin));
Assert.Equal(InBerlin(2026, 9, 1, 0, 0, 1), GapAttribution.CloseStamp(start, InBerlin(2026, 9, 1, 0, 0, 1), Berlin));
}
// ---- Runtime ---------------------------------------------------------------------------------
[Fact]
public void Burner_rows_run_from_the_previous_reading_and_a_long_silence_stays_one_interval()
{
// The seeded burner: 0 h on 18.10.2010, then nothing until 7758 h on 14.10.2022.
var result = _engine.Normalize(Context(MeterMode.RuntimeCounter, Berlin,
[
DayReading(1, Utc(2010, 10, 18), 0),
DayReading(1, Utc(2022, 10, 14), 7758),
DayReading(1, Utc(2022, 11, 28), 7758),
]));
Assert.Equal(3, result.Count);
Assert.True(result[0].OpeningBalance);
Assert.Equal(Utc(2010, 10, 18), result[1].IntervalStart);
Assert.Equal(Utc(2022, 10, 14), result[1].IntervalEnd);
Assert.Equal(7758, result[1].Amount);
Assert.Equal(Utc(2022, 10, 14), result[1].Time);
Assert.Equal(Utc(2022, 10, 14), result[2].IntervalStart);
// The twelve years of hours are not divided; the burner that then stood still across 1 November is
// exactly zero in October and November alike (A-02).
Assert.False(result[1].Divided);
Assert.True(result[2].Divided);
Assert.Equal(0, result[2].Amount);
}
[Fact]
public void Burner_hours_that_go_backwards_or_reset_blind_are_gaps()
{
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig { MeterId = 1, Mode = MeterMode.RuntimeCounter, Unit = "h" },
Readings = [Reading(1, Month(2023, 1), 100), Reading(1, Month(2023, 2), 40), Reading(1, Month(2023, 3), 50), Reading(1, Month(2023, 4), 10)],
Events = [Reset(1, Month(2023, 4), newValue: 0)],
});
Assert.Equal(
[CoverageGapReason.None, CoverageGapReason.UnexplainedDecrease, CoverageGapReason.None, CoverageGapReason.ResetWithoutPrevious],
result.Select(c => c.Gap));
Assert.Equal([100d, 0d, 10d, 10d], result.Select(c => c.Amount));
}
[Fact]
public void A_first_burner_reading_covers_its_label_month_or_starts_at_the_install_date()
{
var label = Assert.Single(_engine.Normalize(Context(MeterMode.RuntimeCounter, Berlin, [Reading(1, Month(2023, 1), 7952)])));
var installed = Assert.Single(_engine.Normalize(Context(
MeterMode.RuntimeCounter, Berlin, [Manual(InBerlin(2023, 1, 20, 10), 12)], installedAt: new DateOnly(2023, 1, 2))));
Assert.Equal(InBerlin(2023, 1, 1), label.IntervalStart);
Assert.Equal(InBerlin(2023, 2, 1), label.IntervalEnd);
Assert.False(label.OpeningBalance);
Assert.Equal(InBerlin(2023, 1, 2), installed.IntervalStart);
Assert.False(installed.OpeningBalance);
}
// ---- Tank ------------------------------------------------------------------------------------
[Fact]
public void A_tank_row_covers_the_time_between_two_dipsticks_and_early_deliveries_cover_nothing()
{
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig
{
MeterId = 30,
Mode = MeterMode.ConsumableBalance,
Unit = "L",
Tank = new TankConfig { Capacity = 7000, Calibration = new CalibrationCurve(7000d / 150d) },
},
Events =
[
Delivery(30, Utc(2020, 9, 7), 3500),
TankLevelCm(30, Utc(2022, 9, 8), 35),
TankLevelCm(30, Utc(2022, 10, 14), 34),
Delivery(30, Utc(2022, 12, 5), 3000),
TankLevelCm(30, Utc(2022, 12, 5), 85),
],
TimeZone = Berlin,
});
Assert.Equal(2, result.Count);
Assert.Equal([Utc(2022, 9, 8), Utc(2022, 10, 14)], result.Select(c => c.IntervalStart!.Value));
Assert.Equal([Utc(2022, 10, 14), Utc(2022, 12, 5)], result.Select(c => c.IntervalEnd!.Value));
Assert.Equal([Utc(2022, 10, 14), Utc(2022, 12, 5)], result.Select(c => c.Time));
Assert.DoesNotContain(result, c => c.OpeningBalance || c.Divided || c.Gap != CoverageGapReason.None);
}
[Fact]
public void A_dipstick_at_local_midnight_is_booked_in_the_day_it_closes()
{
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig { MeterId = 30, Mode = MeterMode.ConsumableBalance, Unit = "L" },
Events =
[
new MeterEvent { MeterId = 30, Time = InBerlin(2026, 8, 15, 10), EventType = MeterEventType.TankLevel, Amount = 3000, Unit = "L" },
new MeterEvent { MeterId = 30, Time = InBerlin(2026, 9, 1), EventType = MeterEventType.TankLevel, Amount = 2800, Unit = "L" },
],
TimeZone = Berlin,
});
var row = Assert.Single(result);
Assert.Equal(InBerlin(2026, 8, 31, 23, 59, 59), row.Time);
Assert.Equal(InBerlin(2026, 9, 1), row.IntervalEnd);
Assert.Equal(200, row.Amount);
}
[Fact]
public void A_tank_nobody_drew_from_across_a_month_boundary_is_exact_in_every_month_but_a_level_that_rose_is_not()
{
MeterEvent Level(DateTimeOffset time, double litres) =>
new() { MeterId = 30, Time = time, EventType = MeterEventType.TankLevel, Amount = litres, Unit = "L" };
var result = _engine.Normalize(new NormalizationContext
{
Meter = new MeterConfig { MeterId = 30, Mode = MeterMode.ConsumableBalance, Unit = "L" },
Events = [Level(InBerlin(2026, 6, 20, 10), 3000), Level(InBerlin(2026, 7, 20, 10), 3000), Level(InBerlin(2026, 8, 20, 10), 3100)],
TimeZone = Berlin,
});
Assert.Equal([true, false], result.Select(c => c.Divided));
Assert.Equal([ReadingQuality.Manual, ReadingQuality.Estimated], result.Select(c => c.Quality));
Assert.All(result, c => Assert.Equal(0, c.Amount));
}
// ---- Instant rate ----------------------------------------------------------------------------
[Fact]
public void A_silence_far_longer_than_the_sensors_rhythm_is_a_sample_gap_that_keeps_its_integral()
{
// 2 kW sampled every five minutes from 10:00 to 11:00, then nothing until 14:00, then 14:05.
var samples = Enumerable.Range(0, 13).Select(i => Measured(Utc(2024, 6, 1, 10).AddMinutes(5 * i), 2))
.Append(Measured(Utc(2024, 6, 1, 14), 2))
.Append(Measured(Utc(2024, 6, 1, 14, 5), 2))
.ToList();
var result = _engine.Normalize(Context(MeterMode.InstantRate, TimeZoneInfo.Utc, samples));
var gap = Assert.Single(result, c => c.Gap != CoverageGapReason.None);
Assert.Equal(CoverageGapReason.SampleGap, gap.Gap);
Assert.Equal(ReadingQuality.Estimated, gap.Quality);
Assert.Equal(6, gap.Amount, 9); // 2 kW over three hours, still counted
Assert.Equal(Utc(2024, 6, 1, 11), gap.IntervalStart);
Assert.Equal(Utc(2024, 6, 1, 14), gap.IntervalEnd);
Assert.Equal(13, result.Count(c => c.Gap == CoverageGapReason.None && c.Quality == ReadingQuality.Measured));
Assert.Equal(CoverageGapReason.None, result.Single(c => c.IntervalStart == Utc(2024, 6, 1, 14)).Gap);
}
[Fact]
public void Each_sample_interval_is_a_row_interval_and_the_first_sample_only_seeds_the_integral()
{
var result = _engine.Normalize(Context(MeterMode.InstantRate, TimeZoneInfo.Utc,
[Measured(Utc(2024, 6, 1, 10), 0), Measured(Utc(2024, 6, 1, 11), 2), Measured(Utc(2024, 6, 1, 12), 4)]));
Assert.Equal([Utc(2024, 6, 1, 10), Utc(2024, 6, 1, 11)], result.Select(c => c.IntervalStart!.Value));
Assert.Equal([Utc(2024, 6, 1, 11), Utc(2024, 6, 1, 12)], result.Select(c => c.IntervalEnd!.Value));
Assert.DoesNotContain(result, c => c.OpeningBalance || c.Gap != CoverageGapReason.None);
}
[Theory]
[InlineData(new[] { 1, 1, 1, 1 }, 61)] // a fast sensor: ten minutes would be too eager, an hourly interval is the floor
[InlineData(new[] { 60, 60, 60 }, 600)] // hourly: a gap is longer than ten hours
[InlineData(new[] { 5, 5, 5, 180 }, 61)] // the gap itself does not raise the threshold: the median stays 5
[InlineData(new[] { 10, 10, 20, 20 }, 150)] // an even count takes the middle of the two middle steps
public void The_sample_gap_threshold_is_ten_median_intervals_but_never_under_an_hourly_interval(int[] stepMinutes, int expectedMinutes)
{
var time = Utc(2024, 6, 1);
var samples = new List<Reading> { Measured(time, 1) };
foreach (var step in stepMinutes)
{
time = time.AddMinutes(step);
samples.Add(Measured(time, 1));
}
var thresholds = InstantRateNormalizer.SampleGapThresholds(samples);
Assert.Equal(stepMinutes.Length, thresholds.Length);
Assert.All(thresholds, t => Assert.Equal(TimeSpan.FromMinutes(expectedMinutes), t));
}
[Fact]
public void A_single_sample_has_no_interval_to_judge()
{
Assert.Empty(InstantRateNormalizer.SampleGapThresholds([Measured(Utc(2024, 6, 1), 1)]));
}
[Fact]
public void Hourly_polls_a_few_seconds_late_after_fast_samples_are_not_gaps()
{
// m7 #6: ten-second samples for a while, then a poller every hour, five seconds late each time.
var samples = Enumerable.Range(0, 360).Select(i => Measured(Utc(2024, 6, 1, 8).AddSeconds(10 * i), 2)).ToList();
var last = samples[^1].Time;
samples.AddRange(Enumerable.Range(1, 30).Select(i => Measured(last.AddSeconds(3605 * i), 2)));
var result = _engine.Normalize(Context(MeterMode.InstantRate, TimeZoneInfo.Utc, samples));
Assert.DoesNotContain(result, c => c.Gap != CoverageGapReason.None);
Assert.Equal(2 * (samples[^1].Time - samples[0].Time).TotalHours, result.Sum(c => c.Amount), 6);
}
[Fact]
public void A_silence_barely_over_an_hour_between_fast_samples_is_within_the_poll_slack()
{
var samples = Enumerable.Range(0, 30).Select(i => Measured(Utc(2024, 6, 1, 8).AddSeconds(10 * i), 2)).ToList();
var silent = samples[^1].Time;
samples.Add(Measured(silent.AddSeconds(3630), 2));
samples.Add(Measured(silent.AddSeconds(3640), 2));
samples.Add(Measured(silent.AddSeconds(3640 + 3662), 2));
samples.AddRange(Enumerable.Range(1, 30).Select(i => Measured(silent.AddSeconds(3640 + 3662 + (10 * i)), 2)));
var result = _engine.Normalize(Context(MeterMode.InstantRate, TimeZoneInfo.Utc, samples));
// An hour and thirty seconds is an hourly interval; an hour and a minute and two seconds is not.
var gap = Assert.Single(result, c => c.Gap == CoverageGapReason.SampleGap);
Assert.Equal(silent.AddSeconds(3640), gap.IntervalStart);
}
[Fact]
public void An_outage_is_judged_by_the_rhythm_around_it_not_by_the_meters_whole_history()
{
// A year of hourly polls would put the whole-history threshold at ten hours and hide a five-hour
// outage of the five-minute sensor that replaced them; the rhythm around the outage finds it.
var hourly = Enumerable.Range(0, 200).Select(i => Measured(Utc(2024, 1, 1).AddHours(i), 1));
var fastStart = Utc(2024, 1, 1).AddHours(199);
var fast = Enumerable.Range(1, 60).Select(i => Measured(fastStart.AddMinutes(5 * i), 1)).ToList();
var resumed = fast[^1].Time.AddHours(5);
var after = Enumerable.Range(0, 60).Select(i => Measured(resumed.AddMinutes(5 * i), 1));
var result = _engine.Normalize(Context(MeterMode.InstantRate, TimeZoneInfo.Utc, [.. hourly, .. fast, .. after]));
var gap = Assert.Single(result, c => c.Gap == CoverageGapReason.SampleGap);
Assert.Equal(fast[^1].Time, gap.IntervalStart);
Assert.Equal(resumed, gap.IntervalEnd);
}
// ---- Direct delta ----------------------------------------------------------------------------
[Fact]
public void Direct_delta_rows_cover_the_time_since_the_previous_report()
{
var result = _engine.Normalize(Context(MeterMode.DirectDelta, Berlin,
[
Manual(InBerlin(2026, 9, 1, 6), 5),
Manual(InBerlin(2026, 9, 1, 12), 3),
Manual(InBerlin(2026, 9, 2), 4),
]));
Assert.True(result[0].OpeningBalance);
Assert.Equal(result[0].IntervalEnd, result[0].IntervalStart);
Assert.Equal([InBerlin(2026, 9, 1, 6), InBerlin(2026, 9, 1, 12)], result.Skip(1).Select(c => c.IntervalStart!.Value));
Assert.Equal(InBerlin(2026, 9, 1, 23, 59, 59), result[2].Time);
Assert.Equal([5d, 3d, 4d], result.Select(c => c.Amount));
}
[Fact]
public void A_first_direct_delta_is_an_opening_balance_even_when_the_install_date_is_known()
{
// m7 #1: the first increment covers one reporting step nobody recorded, not the years since the meter
// was installed — otherwise 0.05 kWh would claim coverage all the way back to 2020.
var first = Assert.Single(_engine.Normalize(Context(
MeterMode.DirectDelta, Berlin, [Measured(InBerlin(2026, 9, 19, 10), 0.05)], installedAt: new DateOnly(2020, 1, 1))));
Assert.True(first.OpeningBalance);
Assert.Equal(InBerlin(2026, 9, 19, 10), first.IntervalStart);
Assert.Equal(first.IntervalEnd, first.IntervalStart);
Assert.Empty(MeterVault.Core.Analysis.Coverage.CoverageBuilder.Build([first], Berlin));
}
[Fact]
public void A_register_still_starts_its_first_reading_at_the_install_date()
{
var register = Assert.Single(_engine.Normalize(Context(
MeterMode.CumulativeCounter, Berlin, [Manual(InBerlin(2026, 9, 1, 6), 5)], installedAt: new DateOnly(2026, 8, 31))));
Assert.False(register.OpeningBalance);
Assert.Equal(InBerlin(2026, 8, 31), register.IntervalStart);
}
[Fact]
public void A_direct_delta_month_row_covers_its_month_whatever_came_before_it()
{
// Behind UTC the month rows are stamped at the local month start, and still cover whole months.
var result = _engine.Normalize(Context(MeterMode.DirectDelta, NewYork,
[
Manual(Local(NewYork, 2026, 6, 20, 12), 7),
Reading(1, Month(2026, 7), 960),
Reading(1, Month(2026, 8), 1000),
]));
var months = result.Where(c => c.Amount >= 960).ToList();
Assert.Equal([Local(NewYork, 2026, 7, 1), Local(NewYork, 2026, 8, 1)], months.Select(c => c.IntervalStart!.Value));
Assert.Equal([Local(NewYork, 2026, 8, 1), Local(NewYork, 2026, 9, 1)], months.Select(c => c.IntervalEnd!.Value));
Assert.Equal([Local(NewYork, 2026, 7, 1), Local(NewYork, 2026, 8, 1)], months.Select(c => c.Time));
Assert.DoesNotContain(months, c => c.OpeningBalance);
}
// ---- Coalesce --------------------------------------------------------------------------------
[Fact]
public void Rows_merged_onto_one_stamp_describe_both_intervals_and_keep_every_warning()
{
var at = Utc(2026, 8, 31, 21, 59);
var merged = Assert.Single(NormalizationEngine.Coalesce(
[
new Consumption
{
MeterId = 1, Time = at, Amount = 1, IntervalStart = at.AddHours(-2), IntervalEnd = at, Divided = true,
SourceStart = at.AddDays(-2), SourceEnd = at.AddMinutes(1),
},
new Consumption
{
MeterId = 1, Time = at, Amount = 2, IntervalStart = at.AddHours(-5), IntervalEnd = at.AddHours(1),
SourceStart = at.AddHours(-5), SourceEnd = at.AddHours(1), OpeningBalance = true, Gap = CoverageGapReason.SampleGap,
},
]));
Assert.Equal(3, merged.Amount);
Assert.Equal(at.AddHours(-5), merged.IntervalStart);
Assert.Equal(at.AddHours(1), merged.IntervalEnd);
Assert.Equal(at.AddDays(-2), merged.SourceStart);
Assert.Equal(at.AddHours(1), merged.SourceEnd);
Assert.False(merged.Divided);
Assert.True(merged.OpeningBalance);
Assert.Equal(CoverageGapReason.SampleGap, merged.Gap);
Assert.Equal(ReadingQuality.Estimated, merged.Quality);
}
[Fact]
public void Two_divided_shares_merged_onto_one_stamp_stay_a_divided_share()
{
var at = Utc(2026, 8, 31, 21, 59);
var merged = Assert.Single(NormalizationEngine.Coalesce(
[
new Consumption { MeterId = 1, Time = at, Amount = 1, IntervalStart = at.AddDays(-3), IntervalEnd = at.AddSeconds(1), Divided = true },
new Consumption { MeterId = 1, Time = at, Amount = 1, IntervalStart = at.AddDays(-1), IntervalEnd = at.AddSeconds(1), Divided = true },
]));
Assert.True(merged.Divided);
Assert.Equal(at.AddDays(-3), merged.IntervalStart);
}
[Fact]
public void A_live_reading_merged_with_a_month_row_spans_the_whole_month()
{
// New York: HA polled at exactly the local midnight the "Juli" row is stamped at. The two rows share
// one key; the merged row covers July.
var julyMidnight = Local(NewYork, 2026, 7, 1);
var result = _engine.Normalize(Context(MeterMode.CumulativeCounter, NewYork,
[
Reading(1, Month(2026, 6), 1000),
Reading(1, Month(2026, 7), 1300),
Measured(julyMidnight, 1005),
]));
var july = Assert.Single(result, c => c.Time == julyMidnight);
Assert.Equal(300, july.Amount, 9);
Assert.Equal(julyMidnight, july.IntervalStart);
Assert.Equal(Local(NewYork, 2026, 8, 1), july.IntervalEnd);
}
}
Reference in New Issue View Git Blame Copy Permalink