Analysis: one selected period, one set of numbers, on every page
ci / build-test (push) Successful in 2m31s

The dashboards told several stories at once. Overview asked for full
calendar years, meter detail for a fixed 12-month window that was really
13, Trends for 24 months with an Apply button, and the energy pages for
60. Each page derived "today" from UTC, so the first hours of a local day
belonged to yesterday. A missing tariff, a month nobody measured and a
genuine zero all rendered as 0. And a virtual meter -- the one thing the
spreadsheet leans on hardest -- was excluded from analysis outright:
MeterPeriodService returned null for it and the page offered a flow
diagram instead.

docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it
left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58
plus amendments A-01..A-30; code, tests and release notes cite those ids.

The analysis layer

Core/Analysis holds the pure rules: period presets resolved once in the
instance zone into a local date range and a half-open UTC range, bucket
plans, calendar-unit comparisons, coverage runs with a resolution class,
normalized quantities and units, the totals policy, the virtual formula
parser/validator/evaluator, and the cost calculator. "Now" comes from
TimeProvider; services never read the clock.

Normalization now writes, in the same transaction as consumption and by
diff, per-meter rollups by local day and month plus coverage runs and a
rollup state (AnalysisDataWriter). AnalysisReader answers a request from
those tables -- month rollups for month and year buckets, day rollups
otherwise, at most two partial edge days from consumption -- and
CostReader prices the result month by month. Pages, /api/v1 and the CSV
export read nothing else. The unused continuous aggregates are dropped.

The reader's statement count per request is constant whether it covers one
meter or a thousand. On a synthetic 1,000-meter, ten-year instance the
brief's target request (100 meters, ten years, monthly) takes 374 ms
against a two-second target, and the Overview went from 48,244 SQL
statements per load to 205.

Missing is not zero

Every bucket carries a status -- available, partial, missing, unresolved,
invalid, pending -- derived from coverage, never from the amount, with
provenance and a reason code beside it. A true zero is a number and a bar
on the baseline; an unknown bucket is a gap that says why; a month whose
data only exists monthly says so instead of inventing daily detail; a
scope with no tariff says "not priced" instead of 0. Rows whose interval
closes after now are reported separately rather than counted.

Virtual meters are analysis subjects

A virtual meter stores a canonical definition -- expression over m<id>
references, result kind, unit and cost rule -- validated on save and on
read for syntax, unknown or self references, loops and unit/kind rules.
It is evaluated on read from its sources' rollups over their joint
coverage: a missing source makes the bucket missing, an observed zero is
a valid input, a non-finite result is invalid with its dependency path,
and the page lists each source's contribution. Topology links are
topology only and never rewrite a saved calculation; expression-less
meters from older installs are converted once at startup. The editor has
Sum, Difference and Advanced modes with a live preview.

Totals and the bill

Per energy type the totals policy separates use, grid import, export,
generation and runtime, marks breakdown meters as breakdowns and virtual
meters as views, and never adds across units. The bill follows it: grid
import where there is one, separately priced subsections at their own
price, feed-in only on export meters, standing charges once per scope per
local day, manual costs once on their start day, categories as
non-overlapping covers whose composition reconciles to the bill. The
seeded demo's yearly totals now match the spreadsheet.

Pages and navigation

The period lives in the URL and every page reads the same contract, so a
link, a reload and the browser's Back button keep it. Shared components
carry it: page header with breadcrumbs, period toolbar, theme-aware chart
with an accessible table beside it, metric cards, comparison and
availability states, attention items that each link to the one action
that fixes them. Meter detail leads with an Analysis tab and resolves its
tabs by key; the energy page has Overview, History, Flow and Meters; the
old cost-only Trends page is a general Analysis page over portfolio, type,
category, meter or a meter comparison. Records tabs are paged server-side
instead of showing the latest 200. Everything is English and German,
light and dark, down to 360px.

Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and
what the first start after the update does (it rebuilds all analysis data
before the web server listens). docs/SDD.md and CLAUDE.md describe the
system as it now is.

Tests: 1,733 Core and 746 integration, all green, plus an opt-in
performance suite with a synthetic 1,000-meter generator.
This commit is contained in:
Florian Schmidt
2026-09-20 10:29:13 +02:00
parent c0f52dbb6f
commit 8940ef25c3
384 changed files with 82753 additions and 4518 deletions
+241
View File
@@ -0,0 +1,241 @@
namespace MeterVault.Core.Analysis;
/// <summary>
/// Cuts a resolved period into chart/table buckets (D-05): day, week (Monday start), month or year, each
/// bounded by local midnights and clipped to the period, or chosen automatically.
/// </summary>
/// <remarks>
/// <para>
/// A chart has one bucket size for all of its series. <see cref="BucketSize.Auto"/> therefore takes a
/// default from the length of the range the period <em>names</em> (<see cref="PeriodResolver.NominalLastDay"/>,
/// A-06) — day up to 62 days, week up to 26 weeks, otherwise month — so "year to date" charts by month in
/// February as in November, and one URL renders the same way all year. It never goes finer than the
/// coarsest resolution any plotted series can resolve (<c>coarsestNeeded</c>): a monthly import asked by
/// day would only produce "unresolved" buckets. It then coarsens until the buckets that actually exist — up
/// to now — fit the caller's point limit, so centuries of history end up in years.
/// </para>
/// <para>
/// An explicit size is honoured, even finer than the data (those buckets read as unresolved, D-14), but
/// never silently truncated: over the point limit it is refused with the finest coarser size that fits.
/// The limit is checked arithmetically before any bucket is built, so a thousand-year day request costs
/// nothing (D-15).
/// </para>
/// <para>
/// Buckets tile <c>[From, To)</c> without gaps: the first may start mid-week or mid-month, the last ends
/// at the period's <c>To</c> — now, for a to-date period, so "last 12 months" is exactly 12 buckets with no
/// future month. A to-date period at the instant it begins (month to date at 00:00 on the 1st) has one
/// empty bucket for today, like the current month of "last 12 months" at that instant; a period that has
/// not started, or has no history, has none.
/// </para>
/// <para>
/// A last bucket that stops before the end of its calendar unit because the period is cut at now carries
/// the unit's end in <see cref="AnalysisBucket.NominalEndDay"/> (clipped to the named range), so drilling
/// into the current month opens the whole month and compares like month to date (D-51).
/// </para>
/// </remarks>
public static class BucketPlanner
{
/// <summary>The most points a series may have (D-05).</summary>
public const int DefaultMaxPoints = 400;
/// <summary>Plans the buckets of <paramref name="period"/>.</summary>
/// <param name="period">
/// The resolved period. For a comparison period, pass <see cref="ComparisonPeriod.ToResolvedPeriod"/>
/// with the current plan's <see cref="BucketPlan.Size"/>, or pair the current buckets with
/// <see cref="ComparisonResolver.PairBuckets"/> to chart one against the other.
/// </param>
/// <param name="size">The requested size; <see cref="BucketSize.Auto"/> to let the planner choose.</param>
/// <param name="coarsestNeeded">
/// The coarsest resolution among the plotted series (for a run divided at month boundaries, pass
/// <see cref="ResolutionClass.Month"/>). Auto never goes finer; a refusal's suggestion neither.
/// </param>
/// <param name="maxPoints">The point limit; at least 1.</param>
public static BucketPlan Plan(ResolvedPeriod period, BucketSize size, ResolutionClass? coarsestNeeded = null, int maxPoints = DefaultMaxPoints)
{
ArgumentNullException.ThrowIfNull(period);
ArgumentOutOfRangeException.ThrowIfLessThan(maxPoints, 1);
var floor = coarsestNeeded is { } need ? MinimumSizeFor(need) : BucketSize.Day;
if (size == BucketSize.Auto)
{
var chosen = Coarsest(DefaultFor(period), floor);
while (CountBuckets(period, chosen) > maxPoints && Coarser(chosen) is { } coarser)
{
chosen = coarser;
}
var count = CountBuckets(period, chosen);
return count > maxPoints
? new BucketPlan(BucketSize.Auto, chosen, [], count, Refused: true, Suggested: null)
: new BucketPlan(BucketSize.Auto, chosen, Build(period, chosen), count, Refused: false, Suggested: null);
}
if (!Enum.IsDefined(size))
{
throw new ArgumentOutOfRangeException(nameof(size), size, "Unknown bucket size.");
}
var points = CountBuckets(period, size);
if (points <= maxPoints)
{
return new BucketPlan(size, size, Build(period, size), points, Refused: false, Suggested: null);
}
BucketSize? suggestion = null;
for (var candidate = Coarser(size); candidate is { } c; candidate = Coarser(c))
{
if (c >= floor && CountBuckets(period, c) <= maxPoints)
{
suggestion = c;
break;
}
}
return new BucketPlan(size, size, [], points, Refused: true, Suggested: suggestion);
}
/// <summary>
/// How many buckets <paramref name="size"/> cuts the period into, without building them — for disabling
/// toolbar options that would exceed the limit. Zero for a period with nothing to plan.
/// </summary>
public static int CountBuckets(ResolvedPeriod period, BucketSize size)
{
ArgumentNullException.ThrowIfNull(period);
if (!TryGetDays(period, out var first, out var last))
{
return 0;
}
return size switch
{
BucketSize.Day => last.DayNumber - first.DayNumber + 1,
BucketSize.Week => ((LocalCalendar.WeekStart(last).DayNumber - LocalCalendar.WeekStart(first).DayNumber) / 7) + 1,
BucketSize.Month => LocalCalendar.MonthsSpanned(first, last),
BucketSize.Year => last.Year - first.Year + 1,
_ => throw new ArgumentOutOfRangeException(nameof(size), size, "Auto has no bucket count; plan it first."),
};
}
/// <summary>The finest bucket a series of this resolution can fill without leaving buckets unresolved.</summary>
public static BucketSize MinimumSizeFor(ResolutionClass resolution) => resolution switch
{
ResolutionClass.Hour or ResolutionClass.Day => BucketSize.Day,
ResolutionClass.Week => BucketSize.Week,
ResolutionClass.Month => BucketSize.Month,
ResolutionClass.Coarse => BucketSize.Year,
_ => throw new ArgumentOutOfRangeException(nameof(resolution), resolution, "Unknown resolution class."),
};
/// <summary>The next coarser bucket size, or null after <see cref="BucketSize.Year"/>.</summary>
public static BucketSize? Coarser(BucketSize size) => size switch
{
BucketSize.Day => BucketSize.Week,
BucketSize.Week => BucketSize.Month,
BucketSize.Month => BucketSize.Year,
BucketSize.Year => null,
_ => throw new ArgumentOutOfRangeException(nameof(size), size, "Auto has no coarser size."),
};
/// <summary>The first day of the bucket after the one containing <paramref name="day"/>: the end of its calendar unit.</summary>
internal static DateOnly NextStart(DateOnly day, BucketSize size) => size switch
{
BucketSize.Day => day.AddDays(1),
BucketSize.Week => LocalCalendar.WeekStart(day).AddDays(7),
BucketSize.Month => LocalCalendar.MonthStart(day).AddMonths(1),
BucketSize.Year => new DateOnly(day.Year + 1, 1, 1),
_ => throw new ArgumentOutOfRangeException(nameof(size), size, "Auto has no bucket boundaries."),
};
/// <summary>
/// The local days that carry buckets: from <see cref="ResolvedPeriod.FirstDay"/> to the last day actuals
/// reach. A to-date period keeps today even at the instant of midnight, so "last 12 months" is 12 buckets
/// and month to date one bucket at every moment (the current one may then be empty); a period that has
/// not started has none.
/// </summary>
private static bool TryGetDays(ResolvedPeriod period, out DateOnly first, out DateOnly last)
{
first = period.FirstDay;
last = period.EffectiveLastDay();
return last >= first;
}
/// <summary>
/// The length-based default, from the named range rather than the elapsed part (A-06): a year to date is
/// a year, whether it is February or November. Whether months fit is not decided here but by the
/// caller's point limit on the buckets that exist, so a limit of 1,000 keeps 501 months as months.
/// </summary>
private static BucketSize DefaultFor(ResolvedPeriod period)
{
var first = period.FirstDay;
var last = period.NominalLastDay();
if (last < first)
{
return BucketSize.Day;
}
return (last.DayNumber - first.DayNumber + 1) switch
{
<= 62 => BucketSize.Day,
<= 26 * 7 => BucketSize.Week,
_ => BucketSize.Month,
};
}
private static BucketSize Coarsest(BucketSize a, BucketSize b) => a >= b ? a : b;
private static List<AnalysisBucket> Build(ResolvedPeriod period, BucketSize size)
{
var buckets = new List<AnalysisBucket>();
if (!TryGetDays(period, out var first, out var last))
{
return buckets;
}
// A bucket cut at now still belongs to its whole unit — but never to more than the period names: the
// last month of a custom range ending on the 25th is the 1st to the 25th, cut at now or not.
var namedEnd = period.NominalLastDay().AddDays(1);
var start = first;
var from = period.From;
while (start <= last)
{
var next = NextStart(start, size);
var endDay = next <= last ? next : last.AddDays(1);
var end = LocalCalendar.Midnight(endDay, period.Zone);
var to = end < period.To ? end : period.To;
if (to < from)
{
to = from;
}
var unitEnd = next < namedEnd ? next : namedEnd;
DateOnly? nominalEnd = unitEnd > endDay ? unitEnd : null;
buckets.Add(new AnalysisBucket(start, endDay, from, to, size, nominalEnd));
start = endDay;
from = to;
}
return buckets;
}
}
/// <summary>
/// The outcome of <see cref="BucketPlanner.Plan"/>: the buckets of the chosen size, or a refusal because the
/// requested size would exceed the point limit (D-05).
/// </summary>
/// <param name="Requested">The size that was asked for (possibly <see cref="BucketSize.Auto"/>).</param>
/// <param name="Size">The size the buckets have — for Auto, the size it chose; never Auto.</param>
/// <param name="Buckets">The buckets, oldest first, tiling the period; empty when refused or when the period has nothing to plan.</param>
/// <param name="PointCount">How many buckets the size produces (or would have produced, when refused).</param>
/// <param name="Refused">True when the size exceeds the limit; nothing is truncated.</param>
/// <param name="Suggested">The finest coarser size within the limit, offered with a refusal; null if none fits.</param>
public sealed record BucketPlan(
BucketSize Requested,
BucketSize Size,
IReadOnlyList<AnalysisBucket> Buckets,
int PointCount,
bool Refused,
BucketSize? Suggested);