Analysis: bound the freshness query, let window sums skip chunks, share the Overview's catalog
ci / build-test (push) Successful in 2m34s
ci / build-test (push) Successful in 2m34s
Three things the 1,000-meter x 10-year measurement found, each proved by EXPLAIN or a statement count before and after. No tally moves. Freshness had two jobs in one unbounded query. The mark -- when a meter last delivered -- is now stored on meter_rollup_state and maintained by every recompute, with a one-pass backfill in the migration, so an import-only meter keeps its years-old last activity without reading a single raw row. The rhythm that decides stale versus live is sampled inside a 90-day window and only for meters that actually have a live source; a source silent for longer than that is re-read unbounded, so it is still called stale by its own rhythm rather than by a default. The portfolio query went from 13.8 ms planning plus 36.1 ms execution across all 123 reading chunks to 0.58 plus 0.44 ms across four. Window sums took their time bounds only from the unnest join, so the planner could not exclude chunks: a 1,960-window case scanned 1.39 M rows in parallel and spilled a 45 MB sort. Repeating the overall min and max as constants makes it five chunks and nested-loop index scans, 121.5 ms to 8.7 ms. The Overview read the catalog three times, once for the quantities and once for each of its two bills. One context and one catalog snapshot now feed all three: 31 statements per load to 23. The final timings on an idle machine are in docs/ANALYSIS_REPORT.md: the brief's target request (100 meters, ten years, monthly) is 286 ms against two seconds, and a startup rebuild of 1,000 meters is 279 s.
This commit is contained in:
@@ -10,6 +10,7 @@ using MeterVault.Infrastructure.Ingestion;
|
||||
using MeterVault.Infrastructure.Normalization;
|
||||
using MeterVault.Infrastructure.Options;
|
||||
using MeterVault.Infrastructure.Persistence;
|
||||
using MeterVault.Integration.Tests.Performance;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using static MeterVault.Integration.Tests.Reconciliation.ReconciliationSupport;
|
||||
|
||||
@@ -694,6 +695,49 @@ public sealed class AnalysisReaderTests(TimescaleFixture fx) : IAsyncLifetime
|
||||
Assert.DoesNotContain(result.Problems, p => p.Kind == AnalysisProblemKind.StaleSource && p.MeterId != stale);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_import_only_meter_keeps_its_years_old_mark_without_reading_the_raw_series()
|
||||
{
|
||||
// D-18, A-40: the mark is the meter's own last reading — for an import-only meter as old as its data — and it
|
||||
// is read from the stored rollup state, so the request never plans across the raw chunks to find it.
|
||||
var type = await TypeAsync();
|
||||
var historical = await MeterAsync(type, MeterMode.CumulativeCounter, "kWh", installedAt: new DateOnly(2018, 1, 1));
|
||||
await ReadingsAsync(historical, (Local(Berlin, 2018, 3, 5, 9), 100), (Local(Berlin, 2018, 4, 5, 9), 140));
|
||||
|
||||
using var counter = CommandCounter.Start();
|
||||
var result = await Reader().ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(historical), Resolve(PeriodPreset.MonthToDate)));
|
||||
|
||||
var freshness = result.SeriesFor(historical)!.Freshness;
|
||||
Assert.Equal(FreshnessState.Historical, freshness.State);
|
||||
Assert.Equal(Local(Berlin, 2018, 4, 5, 9), freshness.LastActivity);
|
||||
Assert.DoesNotContain(counter.Statements, s => s.Contains("SELECT m.id, r.time", StringComparison.Ordinal));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_live_source_silent_for_longer_than_the_recent_window_is_still_stale_by_its_own_rhythm()
|
||||
{
|
||||
// A-40: the bounded sample finds nothing for a meter that has been quiet for months, so its whole history is
|
||||
// read for those few meters — the verdict and the mark are the same as before the bound existed.
|
||||
var type = await TypeAsync();
|
||||
var meter = await MeterAsync(type, MeterMode.CumulativeCounter, "kWh", installedAt: new DateOnly(2026, 1, 1));
|
||||
await using (var db = fx.CreateContext())
|
||||
{
|
||||
db.MeterSources.Add(new MeterSource { MeterId = meter, SourceType = SourceType.Mqtt, Config = """{"topic":"tele/d/SENSOR"}""" });
|
||||
await db.SaveChangesAsync();
|
||||
}
|
||||
|
||||
// Hourly for a day in February, then nothing: seven months of silence, far outside FreshnessRules.RecentWindow.
|
||||
var start = Local(Berlin, 2026, 2, 1);
|
||||
await ReadingsAsync(meter, [.. Enumerable.Range(0, 25).Select(h => (start.AddHours(h), (double)h))]);
|
||||
|
||||
var result = await Reader().ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(meter), Resolve(PeriodPreset.YearToDate)));
|
||||
|
||||
var freshness = result.SeriesFor(meter)!.Freshness;
|
||||
Assert.Equal(FreshnessState.Stale, freshness.State);
|
||||
Assert.Equal(TimeSpan.FromHours(3), freshness.StaleAfter);
|
||||
Assert.Equal(start.AddHours(24), freshness.LastActivity);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------------ limits, availability
|
||||
|
||||
[Fact]
|
||||
|
||||
Reference in New Issue
Block a user