Analysis: one selected period, one set of numbers, on every page
ci / build-test (push) Successful in 2m31s
ci / build-test (push) Successful in 2m31s
The dashboards told several stories at once. Overview asked for full calendar years, meter detail for a fixed 12-month window that was really 13, Trends for 24 months with an Apply button, and the energy pages for 60. Each page derived "today" from UTC, so the first hours of a local day belonged to yesterday. A missing tariff, a month nobody measured and a genuine zero all rendered as 0. And a virtual meter -- the one thing the spreadsheet leans on hardest -- was excluded from analysis outright: MeterPeriodService returned null for it and the page offered a flow diagram instead. docs/DASHBOARD_ANALYSIS_CHANGE_BRIEF.md is the work order. Every choice it left open is settled in docs/ANALYSIS_IMPLEMENTATION_NOTE.md as D-01..D-58 plus amendments A-01..A-30; code, tests and release notes cite those ids. The analysis layer Core/Analysis holds the pure rules: period presets resolved once in the instance zone into a local date range and a half-open UTC range, bucket plans, calendar-unit comparisons, coverage runs with a resolution class, normalized quantities and units, the totals policy, the virtual formula parser/validator/evaluator, and the cost calculator. "Now" comes from TimeProvider; services never read the clock. Normalization now writes, in the same transaction as consumption and by diff, per-meter rollups by local day and month plus coverage runs and a rollup state (AnalysisDataWriter). AnalysisReader answers a request from those tables -- month rollups for month and year buckets, day rollups otherwise, at most two partial edge days from consumption -- and CostReader prices the result month by month. Pages, /api/v1 and the CSV export read nothing else. The unused continuous aggregates are dropped. The reader's statement count per request is constant whether it covers one meter or a thousand. On a synthetic 1,000-meter, ten-year instance the brief's target request (100 meters, ten years, monthly) takes 374 ms against a two-second target, and the Overview went from 48,244 SQL statements per load to 205. Missing is not zero Every bucket carries a status -- available, partial, missing, unresolved, invalid, pending -- derived from coverage, never from the amount, with provenance and a reason code beside it. A true zero is a number and a bar on the baseline; an unknown bucket is a gap that says why; a month whose data only exists monthly says so instead of inventing daily detail; a scope with no tariff says "not priced" instead of 0. Rows whose interval closes after now are reported separately rather than counted. Virtual meters are analysis subjects A virtual meter stores a canonical definition -- expression over m<id> references, result kind, unit and cost rule -- validated on save and on read for syntax, unknown or self references, loops and unit/kind rules. It is evaluated on read from its sources' rollups over their joint coverage: a missing source makes the bucket missing, an observed zero is a valid input, a non-finite result is invalid with its dependency path, and the page lists each source's contribution. Topology links are topology only and never rewrite a saved calculation; expression-less meters from older installs are converted once at startup. The editor has Sum, Difference and Advanced modes with a live preview. Totals and the bill Per energy type the totals policy separates use, grid import, export, generation and runtime, marks breakdown meters as breakdowns and virtual meters as views, and never adds across units. The bill follows it: grid import where there is one, separately priced subsections at their own price, feed-in only on export meters, standing charges once per scope per local day, manual costs once on their start day, categories as non-overlapping covers whose composition reconciles to the bill. The seeded demo's yearly totals now match the spreadsheet. Pages and navigation The period lives in the URL and every page reads the same contract, so a link, a reload and the browser's Back button keep it. Shared components carry it: page header with breadcrumbs, period toolbar, theme-aware chart with an accessible table beside it, metric cards, comparison and availability states, attention items that each link to the one action that fixes them. Meter detail leads with an Analysis tab and resolves its tabs by key; the energy page has Overview, History, Flow and Meters; the old cost-only Trends page is a general Analysis page over portfolio, type, category, meter or a meter comparison. Records tabs are paged server-side instead of showing the latest 200. Everything is English and German, light and dark, down to 360px. Some figures change on purpose; docs/RELEASE_NOTES.md lists each one and what the first start after the update does (it rebuilds all analysis data before the web server listens). docs/SDD.md and CLAUDE.md describe the system as it now is. Tests: 1,733 Core and 746 integration, all green, plus an opt-in performance suite with a synthetic 1,000-meter generator.
This commit is contained in:
Florian Schmidt
@@ -0,0 +1,108 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>
|
||||
/// Counts the SQL commands one async flow sends — through EF Core and through the reader's plain Npgsql commands
|
||||
/// alike — by listening to Npgsql's own tracing (<c>ActivitySource "Npgsql"</c>). Only commands of the flow that
|
||||
/// started the counter are counted, so other work in the process does not leak in. While no counter is open, no
|
||||
/// listener exists and Npgsql creates no activities: the timed runs are not slowed down by it.
|
||||
/// </summary>
|
||||
internal sealed class CommandCounter : IDisposable
|
||||
{
|
||||
private static readonly AsyncLocal<CommandCounter?> Current = new();
|
||||
|
||||
private readonly ActivityListener _listener;
|
||||
private readonly List<string> _statements = [];
|
||||
private readonly CommandCounter? _previous;
|
||||
private TimeSpan _duration;
|
||||
|
||||
private CommandCounter()
|
||||
{
|
||||
_previous = Current.Value;
|
||||
_listener = new ActivityListener
|
||||
{
|
||||
ShouldListenTo = source => source.Name.StartsWith("Npgsql", StringComparison.Ordinal),
|
||||
Sample = (ref ActivityCreationOptions<ActivityContext> _) => ActivitySamplingResult.AllDataAndRecorded,
|
||||
ActivityStopped = Stopped,
|
||||
};
|
||||
ActivitySource.AddActivityListener(_listener);
|
||||
}
|
||||
|
||||
/// <summary>The commands counted so far.</summary>
|
||||
public int Count
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_statements)
|
||||
{
|
||||
return _statements.Count;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The summed duration of the counted commands: from execution to the reader's close, so it includes reading the
|
||||
/// rows (and whatever the caller does per row while reading them).
|
||||
/// </summary>
|
||||
public TimeSpan Duration
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_statements)
|
||||
{
|
||||
return _duration;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Every counted command in order: its duration and the first line of its text.</summary>
|
||||
public IReadOnlyList<string> Statements
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_statements)
|
||||
{
|
||||
return [.. _statements];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Starts counting the commands of the calling async flow.</summary>
|
||||
public static CommandCounter Start()
|
||||
{
|
||||
var counter = new CommandCounter();
|
||||
Current.Value = counter;
|
||||
return counter;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
Current.Value = _previous;
|
||||
_listener.Dispose();
|
||||
}
|
||||
|
||||
private void Stopped(Activity activity)
|
||||
{
|
||||
if (!ReferenceEquals(Current.Value, this))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// A command activity carries its text (db.query.text since Npgsql 10; db.statement before).
|
||||
var text = activity.GetTagItem("db.query.text") as string ?? activity.GetTagItem("db.statement") as string;
|
||||
if (text is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var first = text.TrimStart().Split('\n', 2)[0].Trim();
|
||||
lock (_statements)
|
||||
{
|
||||
_statements.Add(string.Create(
|
||||
CultureInfo.InvariantCulture, $"{activity.Duration.TotalMilliseconds,8:F1} ms {(first.Length > 140 ? first[..140] + "…" : first)}"));
|
||||
_duration += activity.Duration;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,281 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
using System.Text.Json;
|
||||
using MeterVault.Core.Normalization;
|
||||
using MeterVault.Infrastructure.Normalization;
|
||||
using MeterVault.Infrastructure.Options;
|
||||
using MeterVault.Infrastructure.Persistence;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Npgsql;
|
||||
using Testcontainers.PostgreSql;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>How the measured database was prepared: loaded now or reused, and what each step took.</summary>
|
||||
internal sealed record PreparedDatabase(
|
||||
DatasetManifest Manifest,
|
||||
bool Loaded,
|
||||
RebuildStats? Rebuild,
|
||||
double? CompressSeconds,
|
||||
int CompressedChunks);
|
||||
|
||||
/// <summary>The startup rebuild (NormalizationUpgrade) over the whole dataset.</summary>
|
||||
internal sealed record RebuildStats(double Seconds, int Meters, int Rebuilt, bool MeasuredInThisRun);
|
||||
|
||||
/// <summary>
|
||||
/// The database of one performance run: a fresh TimescaleDB container (the image the fixture pins), or an existing
|
||||
/// database from <c>METERVAULT_PERF_DB</c>. Prepares it the way a real instance gets there — migrations, the default
|
||||
/// seed, the raw dataset, the startup rebuild (<see cref="NormalizationUpgrade"/>), and the compression policy's work
|
||||
/// on raw chunks older than 30 days — and hands out contexts for the readers.
|
||||
/// </summary>
|
||||
internal sealed class PerfDatabase : IDbContextFactory<MeterVaultDbContext>, IAsyncDisposable
|
||||
{
|
||||
/// <summary>The pinned image of <see cref="TimescaleFixture"/>.</summary>
|
||||
public const string Image = "timescale/timescaledb:2.17.2-pg16";
|
||||
|
||||
/// <summary>The frozen "now" the measured dataset ends at: 19 September 2026, 14:37 Berlin, as the cost tests use.</summary>
|
||||
public static readonly DateTimeOffset FrozenNow = new DateTimeOffset(2026, 9, 19, 14, 37, 0, TimeSpan.FromHours(2)).ToUniversalTime();
|
||||
|
||||
private const string RebuildKey = "perf_rebuild";
|
||||
|
||||
private readonly PostgreSqlContainer? _container;
|
||||
|
||||
private PerfDatabase(PostgreSqlContainer? container, string connectionString)
|
||||
{
|
||||
_container = container;
|
||||
|
||||
// Long statements are allowed: a 1,000-meter read on a busy machine must be measured, not time out.
|
||||
ConnectionString = new NpgsqlConnectionStringBuilder(connectionString) { CommandTimeout = 600, IncludeErrorDetail = true }.ConnectionString;
|
||||
}
|
||||
|
||||
public string ConnectionString { get; }
|
||||
|
||||
/// <summary>Where the database came from, for the report.</summary>
|
||||
public string Origin => _container is null ? "existing database (METERVAULT_PERF_DB)" : $"fresh container ({Image})";
|
||||
|
||||
public static async Task<PerfDatabase> OpenAsync(PerfSettings settings, PerfLog log)
|
||||
{
|
||||
if (settings.Database is { } external)
|
||||
{
|
||||
log.Write("Using the database from METERVAULT_PERF_DB");
|
||||
return new PerfDatabase(null, external);
|
||||
}
|
||||
|
||||
log.Write($"Starting a {Image} container…");
|
||||
var container = new PostgreSqlBuilder(Image)
|
||||
.WithDatabase("metervault")
|
||||
.WithUsername("metervault")
|
||||
.WithPassword("metervault")
|
||||
.Build();
|
||||
await container.StartAsync();
|
||||
return new PerfDatabase(container, container.GetConnectionString());
|
||||
}
|
||||
|
||||
public MeterVaultDbContext CreateDbContext()
|
||||
{
|
||||
var options = new DbContextOptionsBuilder<MeterVaultDbContext>()
|
||||
.UseNpgsql(ConnectionString, npgsql => npgsql.MigrationsAssembly(typeof(MeterVaultDbContext).Assembly.FullName))
|
||||
.UseSnakeCaseNamingConvention()
|
||||
.Options;
|
||||
return new MeterVaultDbContext(options);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Migrates and seeds, loads the dataset unless the database already holds it, rebuilds what is not built yet, and
|
||||
/// compresses raw chunks older than 30 days (what the compression policy does on a running instance).
|
||||
/// </summary>
|
||||
public async Task<PreparedDatabase> PrepareAsync(PerfSettings settings, PerfLog log)
|
||||
{
|
||||
await using (var db = CreateDbContext())
|
||||
{
|
||||
db.Database.SetCommandTimeout(TimeSpan.FromMinutes(10));
|
||||
await db.Database.MigrateAsync();
|
||||
await DatabaseSeeder.SeedAsync(db);
|
||||
|
||||
// As in TimescaleFixture: the scheduled compression job must not race the load and rebuild. Compression
|
||||
// is applied explicitly below, once, the way the policy would have left the chunks.
|
||||
await db.Database.ExecuteSqlRawAsync(
|
||||
"SELECT alter_job(job_id, scheduled => false) FROM timescaledb_information.jobs " +
|
||||
"WHERE proc_name IN ('policy_compression', 'policy_columnstore');");
|
||||
}
|
||||
|
||||
await using var connection = new NpgsqlConnection(ConnectionString);
|
||||
await connection.OpenAsync();
|
||||
|
||||
var manifest = await SyntheticDataset.ReadMarkerAsync(connection);
|
||||
var loaded = false;
|
||||
if (manifest is null)
|
||||
{
|
||||
var zone = TimeZoneInfo.FindSystemTimeZoneById("Europe/Berlin");
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Loading the synthetic dataset (scale {settings.Scale}, now {FrozenNow:O})…"));
|
||||
manifest = await SyntheticDataset.LoadAsync(ConnectionString, FrozenNow, zone, settings.Scale, log.Write);
|
||||
loaded = true;
|
||||
}
|
||||
else
|
||||
{
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Reusing the loaded dataset ({manifest.MeterCount} meters, now {manifest.Now:O})"));
|
||||
}
|
||||
|
||||
var rebuild = await RebuildAsync(connection, manifest, log);
|
||||
|
||||
var (compressSeconds, compressed) = await CompressAsync(connection, manifest.Now, log);
|
||||
if (loaded || rebuild?.MeasuredInThisRun == true || compressed > 0)
|
||||
{
|
||||
log.Write("VACUUM ANALYZE…");
|
||||
await ExecuteAsync(connection, "VACUUM ANALYZE");
|
||||
}
|
||||
|
||||
return new PreparedDatabase(manifest, loaded, rebuild, compressSeconds, compressed);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Runs the startup rebuild when stored consumption is missing or outdated — timed, with a frozen clock at the
|
||||
/// dataset's "now" — and remembers its duration in the database, so a later reuse can still report it.
|
||||
/// </summary>
|
||||
private async Task<RebuildStats?> RebuildAsync(NpgsqlConnection connection, DatasetManifest manifest, PerfLog log)
|
||||
{
|
||||
var meters = Convert.ToInt32(await ScalarAsync(connection, "SELECT count(*) FROM meter"), CultureInfo.InvariantCulture);
|
||||
var built = Convert.ToInt32(
|
||||
await ScalarAsync(connection, $"SELECT count(*) FROM meter_rollup_state WHERE revision >= {NormalizationUpgrade.CurrentRevision}"),
|
||||
CultureInfo.InvariantCulture);
|
||||
var revision = await ScalarAsync(connection, $"SELECT value::text FROM app_setting WHERE key = '{NormalizationUpgrade.SettingKey}'");
|
||||
if (built >= meters && revision is not null)
|
||||
{
|
||||
return await ScalarAsync(connection, $"SELECT value::text FROM app_setting WHERE key = '{RebuildKey}'") is string json
|
||||
? JsonSerializer.Deserialize<RebuildStats>(json)! with { MeasuredInThisRun = false }
|
||||
: null;
|
||||
}
|
||||
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Rebuilding {meters} meters through NormalizationUpgrade…"));
|
||||
var watch = Stopwatch.StartNew();
|
||||
int rebuilt;
|
||||
await using (var db = CreateDbContext())
|
||||
{
|
||||
var normalization = Normalization(db, manifest);
|
||||
var upgrade = new NormalizationUpgrade(db, normalization, new PerfLogger<NormalizationUpgrade>(log));
|
||||
rebuilt = await upgrade.RunAsync();
|
||||
}
|
||||
|
||||
watch.Stop();
|
||||
var stats = new RebuildStats(watch.Elapsed.TotalSeconds, meters, rebuilt, MeasuredInThisRun: true);
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Rebuilt {rebuilt} of {meters} meters in {stats.Seconds:F1} s"));
|
||||
|
||||
await using var command = new NpgsqlCommand(
|
||||
"INSERT INTO app_setting (key, value) VALUES (@key, @value) ON CONFLICT (key) DO UPDATE SET value = excluded.value", connection);
|
||||
command.Parameters.AddWithValue("key", RebuildKey);
|
||||
command.Parameters.Add(new NpgsqlParameter("value", NpgsqlTypes.NpgsqlDbType.Jsonb) { Value = JsonSerializer.Serialize(stats) });
|
||||
await command.ExecuteNonQueryAsync();
|
||||
return stats;
|
||||
}
|
||||
|
||||
/// <summary>A normalization service as the app builds it, in the dataset's zone, on a clock frozen at its "now".</summary>
|
||||
public static NormalizationService Normalization(MeterVaultDbContext db, DatasetManifest manifest) =>
|
||||
new(db, NormalizationEngine.CreateDefault(),
|
||||
Microsoft.Extensions.Options.Options.Create(new MeterVaultOptions { TimeZone = manifest.Zone }),
|
||||
new FixedTimeProvider(manifest.Now));
|
||||
|
||||
/// <summary>Compresses the raw chunks the 30-day policy would have compressed by the dataset's "now".</summary>
|
||||
private static async Task<(double? Seconds, int Chunks)> CompressAsync(NpgsqlConnection connection, DateTimeOffset now, PerfLog log)
|
||||
{
|
||||
await using var pending = new NpgsqlCommand(
|
||||
"""
|
||||
SELECT count(*) FROM timescaledb_information.chunks
|
||||
WHERE hypertable_name = 'reading' AND NOT is_compressed AND range_end <= @horizon
|
||||
""", connection);
|
||||
pending.Parameters.AddWithValue("horizon", now.AddDays(-30).UtcDateTime);
|
||||
var count = Convert.ToInt32(await pending.ExecuteScalarAsync(), CultureInfo.InvariantCulture);
|
||||
if (count == 0)
|
||||
{
|
||||
return (null, 0);
|
||||
}
|
||||
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Compressing {count} raw chunks older than 30 days…"));
|
||||
var watch = Stopwatch.StartNew();
|
||||
await using var compress = new NpgsqlCommand(
|
||||
"SELECT count(compress_chunk(c, if_not_compressed => true)) FROM show_chunks('reading', older_than => @horizon) c", connection)
|
||||
{
|
||||
CommandTimeout = 0,
|
||||
};
|
||||
compress.Parameters.AddWithValue("horizon", now.AddDays(-30).UtcDateTime);
|
||||
await compress.ExecuteScalarAsync();
|
||||
return (watch.Elapsed.TotalSeconds, count);
|
||||
}
|
||||
|
||||
public static async Task<object?> ScalarAsync(NpgsqlConnection connection, string sql)
|
||||
{
|
||||
await using var command = new NpgsqlCommand(sql, connection) { CommandTimeout = 0 };
|
||||
var value = await command.ExecuteScalarAsync();
|
||||
return value is DBNull ? null : value;
|
||||
}
|
||||
|
||||
public static async Task ExecuteAsync(NpgsqlConnection connection, string sql)
|
||||
{
|
||||
await using var command = new NpgsqlCommand(sql, connection) { CommandTimeout = 0 };
|
||||
await command.ExecuteNonQueryAsync();
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_container is not null)
|
||||
{
|
||||
await _container.DisposeAsync();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Timestamped progress to <c>perf-<label>.log</c> in the output directory and to the test output.</summary>
|
||||
internal sealed class PerfLog : IDisposable
|
||||
{
|
||||
private readonly StreamWriter _file;
|
||||
private readonly ITestOutputHelper? _output;
|
||||
private readonly Stopwatch _clock = Stopwatch.StartNew();
|
||||
|
||||
public PerfLog(PerfSettings settings, string name, ITestOutputHelper? output)
|
||||
{
|
||||
Directory.CreateDirectory(settings.OutputDirectory);
|
||||
Path = System.IO.Path.Combine(settings.OutputDirectory, $"{name}-{settings.Label}.log");
|
||||
_file = new StreamWriter(Path, append: false) { AutoFlush = true };
|
||||
_output = output;
|
||||
}
|
||||
|
||||
public string Path { get; }
|
||||
|
||||
public void Write(string message)
|
||||
{
|
||||
var line = string.Create(CultureInfo.InvariantCulture, $"[{DateTimeOffset.Now:HH:mm:ss} +{_clock.Elapsed.TotalSeconds,7:F1}s] {message}");
|
||||
lock (_file)
|
||||
{
|
||||
_file.WriteLine(line);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
_output?.WriteLine(line);
|
||||
}
|
||||
catch (InvalidOperationException)
|
||||
{
|
||||
// The test has finished; the file still has it.
|
||||
}
|
||||
}
|
||||
|
||||
public void Dispose() => _file.Dispose();
|
||||
}
|
||||
|
||||
/// <summary>Routes the upgrade's progress ("Rebuilt 100 of 1000 meter(s)") into the perf log.</summary>
|
||||
internal sealed class PerfLogger<T>(PerfLog log) : ILogger<T>
|
||||
{
|
||||
public IDisposable? BeginScope<TState>(TState state)
|
||||
where TState : notnull => null;
|
||||
|
||||
public bool IsEnabled(LogLevel logLevel) => logLevel >= LogLevel.Information;
|
||||
|
||||
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter)
|
||||
{
|
||||
if (IsEnabled(logLevel))
|
||||
{
|
||||
log.Write($"{typeof(T).Name}: {formatter(state, exception)}{(exception is null ? string.Empty : " — " + exception.Message)}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
using System.Runtime.InteropServices;
|
||||
using System.Text;
|
||||
using Npgsql;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>The timings of one scenario: every measured run, and the SQL a separate counted run sent.</summary>
|
||||
internal sealed record ScenarioTiming(
|
||||
string Id,
|
||||
string Description,
|
||||
IReadOnlyList<double> Milliseconds,
|
||||
int Commands,
|
||||
IReadOnlyList<string> Statements,
|
||||
string Shape)
|
||||
{
|
||||
/// <summary>The summed duration of the SQL commands in the counted run (execution to reader close).</summary>
|
||||
public double SqlMilliseconds { get; init; }
|
||||
|
||||
public double Median => Percentile(0.5);
|
||||
|
||||
public double P95 => Percentile(0.95);
|
||||
|
||||
public double Min => Milliseconds.Min();
|
||||
|
||||
public double Max => Milliseconds.Max();
|
||||
|
||||
/// <summary>Nearest-rank percentile; the median of an even count is the mean of the middle two.</summary>
|
||||
private double Percentile(double p)
|
||||
{
|
||||
var sorted = Milliseconds.Order().ToArray();
|
||||
if (p == 0.5 && sorted.Length % 2 == 0)
|
||||
{
|
||||
return (sorted[(sorted.Length / 2) - 1] + sorted[sorted.Length / 2]) / 2;
|
||||
}
|
||||
|
||||
var rank = (int)Math.Ceiling(p * sorted.Length);
|
||||
return sorted[Math.Clamp(rank - 1, 0, sorted.Length - 1)];
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Markdown building and the facts about the machine and database a result was taken on.</summary>
|
||||
internal static class PerfReport
|
||||
{
|
||||
public static string F(double value, int digits = 1) => value.ToString("F" + digits.ToString(CultureInfo.InvariantCulture), CultureInfo.InvariantCulture);
|
||||
|
||||
public static string N(long value) => value.ToString("N0", CultureInfo.InvariantCulture);
|
||||
|
||||
/// <summary>A markdown table cell: pipes escaped, newlines flattened.</summary>
|
||||
public static string Cell(string text) => text.Replace("|", "\\|", StringComparison.Ordinal).Replace('\n', ' ');
|
||||
|
||||
/// <summary>CPU model, cores, memory, OS and runtime.</summary>
|
||||
public static IReadOnlyList<(string Name, string Value)> Machine()
|
||||
{
|
||||
var memory = GC.GetGCMemoryInfo().TotalAvailableMemoryBytes;
|
||||
return
|
||||
[
|
||||
("CPU", CpuModel()),
|
||||
("Logical processors", Environment.ProcessorCount.ToString(CultureInfo.InvariantCulture)),
|
||||
("Memory visible to .NET", F(memory / 1024d / 1024 / 1024) + " GiB"),
|
||||
("OS", RuntimeInformation.OSDescription),
|
||||
(".NET", RuntimeInformation.FrameworkDescription),
|
||||
("Docker", Docker()),
|
||||
];
|
||||
}
|
||||
|
||||
/// <summary>The share of all logical processors busy over <paramref name="window"/> (Windows only): how loaded the machine was.</summary>
|
||||
public static async Task<string> HostLoadAsync(TimeSpan window)
|
||||
{
|
||||
if (!OperatingSystem.IsWindows() || !NativeMethods.GetSystemTimes(out var idle1, out var kernel1, out var user1))
|
||||
{
|
||||
return "n/a";
|
||||
}
|
||||
|
||||
await Task.Delay(window);
|
||||
if (!NativeMethods.GetSystemTimes(out var idle2, out var kernel2, out var user2))
|
||||
{
|
||||
return "n/a";
|
||||
}
|
||||
|
||||
// Kernel time includes idle time.
|
||||
var total = (kernel2 - kernel1) + (user2 - user1);
|
||||
var busy = total - (idle2 - idle1);
|
||||
return total <= 0 ? "n/a" : F(100d * busy / total, 0) + " % of all logical processors busy (sampled over " + F(window.TotalSeconds, 0) + " s)";
|
||||
}
|
||||
|
||||
/// <summary>PostgreSQL/Timescale versions and the settings that shape plans.</summary>
|
||||
public static async Task<IReadOnlyList<(string Name, string Value)>> DatabaseAsync(NpgsqlConnection connection)
|
||||
{
|
||||
var rows = new List<(string, string)>
|
||||
{
|
||||
("PostgreSQL", (await PerfDatabase.ScalarAsync(connection, "SHOW server_version"))?.ToString() ?? "?"),
|
||||
("TimescaleDB", (await PerfDatabase.ScalarAsync(connection, "SELECT extversion FROM pg_extension WHERE extname = 'timescaledb'"))?.ToString() ?? "?"),
|
||||
};
|
||||
|
||||
await using var command = new NpgsqlCommand(
|
||||
"""
|
||||
SELECT name, setting || coalesce(' ' || unit, '') FROM pg_settings
|
||||
WHERE name IN ('shared_buffers', 'work_mem', 'effective_cache_size', 'max_parallel_workers_per_gather', 'max_worker_processes',
|
||||
'jit', 'random_page_cost', 'timescaledb.max_tuples_decompressed_per_dml_transaction')
|
||||
ORDER BY name
|
||||
""", connection);
|
||||
await using var reader = await command.ExecuteReaderAsync();
|
||||
while (await reader.ReadAsync())
|
||||
{
|
||||
rows.Add((reader.GetString(0), reader.GetString(1)));
|
||||
}
|
||||
|
||||
return rows;
|
||||
}
|
||||
|
||||
/// <summary>Row counts and on-disk sizes of the tables the pipeline and the reader use.</summary>
|
||||
public static async Task<IReadOnlyList<(string Table, long Rows, string Size, string Note)>> TablesAsync(NpgsqlConnection connection)
|
||||
{
|
||||
var result = new List<(string, long, string, string)>();
|
||||
(string Table, bool Hypertable)[] tables =
|
||||
[
|
||||
("reading", true), ("consumption", true), ("consumption_rollup", false), ("consumption_rollup_month", false),
|
||||
("meter_coverage", false), ("meter_rollup_state", false), ("meter_event", false), ("meter", false), ("tariff", false),
|
||||
("manual_cost", false), ("meter_link", false), ("cost_category_member", false),
|
||||
];
|
||||
foreach (var (table, hypertable) in tables)
|
||||
{
|
||||
var rows = Convert.ToInt64(await PerfDatabase.ScalarAsync(connection, $"SELECT count(*) FROM {table}"), CultureInfo.InvariantCulture);
|
||||
var bytes = Convert.ToInt64(
|
||||
await PerfDatabase.ScalarAsync(connection, hypertable ? $"SELECT hypertable_size('{table}')" : $"SELECT pg_total_relation_size('{table}')"),
|
||||
CultureInfo.InvariantCulture);
|
||||
var note = string.Empty;
|
||||
if (hypertable)
|
||||
{
|
||||
note = (await PerfDatabase.ScalarAsync(
|
||||
connection,
|
||||
$"SELECT count(*) || ' chunks, ' || count(*) FILTER (WHERE is_compressed) || ' compressed' FROM timescaledb_information.chunks WHERE hypertable_name = '{table}'"))
|
||||
?.ToString() ?? string.Empty;
|
||||
}
|
||||
|
||||
result.Add((table, rows, F(bytes / 1024d / 1024) + " MiB", note));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private static string CpuModel()
|
||||
{
|
||||
try
|
||||
{
|
||||
if (OperatingSystem.IsWindows())
|
||||
{
|
||||
using var key = Microsoft.Win32.Registry.LocalMachine.OpenSubKey(@"HARDWARE\DESCRIPTION\System\CentralProcessor\0");
|
||||
if (key?.GetValue("ProcessorNameString") is string name)
|
||||
{
|
||||
return name.Trim();
|
||||
}
|
||||
}
|
||||
else if (File.Exists("/proc/cpuinfo"))
|
||||
{
|
||||
var line = File.ReadLines("/proc/cpuinfo").FirstOrDefault(l => l.StartsWith("model name", StringComparison.Ordinal));
|
||||
if (line is not null)
|
||||
{
|
||||
return line[(line.IndexOf(':', StringComparison.Ordinal) + 1)..].Trim();
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or System.Security.SecurityException)
|
||||
{
|
||||
return "unknown (" + ex.Message + ")";
|
||||
}
|
||||
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
private static string Docker()
|
||||
{
|
||||
try
|
||||
{
|
||||
using var process = Process.Start(new ProcessStartInfo("docker", "info --format \"{{.ServerVersion}}|{{.OperatingSystem}}|{{.NCPU}} CPUs|{{.MemTotal}}\"")
|
||||
{
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
UseShellExecute = false,
|
||||
});
|
||||
if (process is null)
|
||||
{
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
var output = process.StandardOutput.ReadToEnd().Trim();
|
||||
process.WaitForExit(10_000);
|
||||
var parts = output.Split('|');
|
||||
if (parts.Length == 4 && long.TryParse(parts[3], CultureInfo.InvariantCulture, out var bytes))
|
||||
{
|
||||
parts[0] = "Engine " + parts[0];
|
||||
parts[3] = F(bytes / 1024d / 1024 / 1024) + " GiB";
|
||||
return string.Join(", ", parts);
|
||||
}
|
||||
|
||||
return output.Length > 0 ? output : "unknown";
|
||||
}
|
||||
catch (Exception ex) when (ex is System.ComponentModel.Win32Exception or InvalidOperationException)
|
||||
{
|
||||
return "unknown (" + ex.Message + ")";
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Appends a markdown table.</summary>
|
||||
public static void Table(StringBuilder md, IReadOnlyList<string> header, IEnumerable<IReadOnlyList<string>> rows)
|
||||
{
|
||||
md.Append("| ").AppendJoin(" | ", header).AppendLine(" |");
|
||||
md.Append('|').AppendJoin("|", header.Select(_ => "---")).AppendLine("|");
|
||||
foreach (var row in rows)
|
||||
{
|
||||
md.Append("| ").AppendJoin(" | ", row.Select(Cell)).AppendLine(" |");
|
||||
}
|
||||
|
||||
md.AppendLine();
|
||||
}
|
||||
|
||||
private static class NativeMethods
|
||||
{
|
||||
[DllImport("kernel32.dll", SetLastError = true)]
|
||||
[DefaultDllImportSearchPaths(DllImportSearchPath.System32)]
|
||||
[return: MarshalAs(UnmanagedType.Bool)]
|
||||
public static extern bool GetSystemTimes(out long idleTime, out long kernelTime, out long userTime);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
using System.Globalization;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>
|
||||
/// The environment switches of the performance measurement (brief §9.10, D-56). Everything is opt-in: without
|
||||
/// <c>METERVAULT_PERF=1</c> the performance facts are skipped, so the normal suite neither runs nor slows down.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <list type="table">
|
||||
/// <item><term>METERVAULT_PERF</term><description><c>1</c> runs the performance facts.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_LABEL</term><description>Names the result files (<c>results-<label>.md</c>); default <c>local</c>.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_OUT</term><description>Where results go; default <c>%TEMP%/mv_analysis/perf</c>.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_RUNS</term><description>Measured runs per scenario (default 10), after <c>METERVAULT_PERF_WARMUP</c> warm-ups (default 2).</description></item>
|
||||
/// <item><term>METERVAULT_PERF_DB</term><description>A connection string to use instead of a fresh container. A database that already holds
|
||||
/// the dataset is reused as it is (no load, no rebuild), so a re-run on a quiet machine only measures.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_LOAD_DB</term><description>For the loader fact: an app-migrated database (old or new schema) to load the raw
|
||||
/// dataset into, for the page-level comparison.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_NOW</term><description>For the loader fact: the instant the dataset ends at (ISO 8601); default the current time.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_SCALE</term><description>Scales every meter group (default 1 = 1,000 meters); only for quick trial runs.</description></item>
|
||||
/// <item><term>METERVAULT_PERF_NOTE</term><description>Free text copied into the report (e.g. "machine busy with other builds").</description></item>
|
||||
/// </list>
|
||||
/// </remarks>
|
||||
internal sealed record PerfSettings(
|
||||
bool Enabled,
|
||||
string Label,
|
||||
string OutputDirectory,
|
||||
int Runs,
|
||||
int Warmup,
|
||||
string? Database,
|
||||
string? LoadDatabase,
|
||||
DateTimeOffset? Now,
|
||||
double Scale,
|
||||
string? Note)
|
||||
{
|
||||
public const string EnabledVariable = "METERVAULT_PERF";
|
||||
|
||||
public const string LoadDatabaseVariable = "METERVAULT_PERF_LOAD_DB";
|
||||
|
||||
public static PerfSettings Current { get; } = Read();
|
||||
|
||||
private static PerfSettings Read()
|
||||
{
|
||||
static string? Env(string name) => Environment.GetEnvironmentVariable(name) is { Length: > 0 } value ? value.Trim() : null;
|
||||
|
||||
static int Int(string name, int fallback) =>
|
||||
int.TryParse(Env(name), NumberStyles.Integer, CultureInfo.InvariantCulture, out var value) && value >= 0 ? value : fallback;
|
||||
|
||||
var scale = double.TryParse(Env("METERVAULT_PERF_SCALE"), NumberStyles.Float, CultureInfo.InvariantCulture, out var s) && s > 0 ? s : 1;
|
||||
DateTimeOffset? now = DateTimeOffset.TryParse(Env("METERVAULT_PERF_NOW"), CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out var n)
|
||||
? n.ToUniversalTime()
|
||||
: null;
|
||||
|
||||
return new PerfSettings(
|
||||
Enabled: Env(EnabledVariable) == "1",
|
||||
Label: Env("METERVAULT_PERF_LABEL") ?? "local",
|
||||
OutputDirectory: Env("METERVAULT_PERF_OUT") ?? Path.Combine(Path.GetTempPath(), "mv_analysis", "perf"),
|
||||
Runs: Math.Max(1, Int("METERVAULT_PERF_RUNS", 10)),
|
||||
Warmup: Int("METERVAULT_PERF_WARMUP", 2),
|
||||
Database: Env("METERVAULT_PERF_DB"),
|
||||
LoadDatabase: Env(LoadDatabaseVariable),
|
||||
Now: now,
|
||||
Scale: scale,
|
||||
Note: Env("METERVAULT_PERF_NOTE"));
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>A fact that runs only with <c>METERVAULT_PERF=1</c>; otherwise it is reported as skipped.</summary>
|
||||
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
|
||||
public sealed class PerfFactAttribute : FactAttribute
|
||||
{
|
||||
public PerfFactAttribute()
|
||||
{
|
||||
if (!PerfSettings.Current.Enabled)
|
||||
{
|
||||
Skip = $"Performance measurement; set {PerfSettings.EnabledVariable}=1 to run it.";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A fact that runs only with <c>METERVAULT_PERF=1</c> and a target database in <c>METERVAULT_PERF_LOAD_DB</c>: it loads
|
||||
/// the synthetic raw dataset into a database an app has migrated, for the page-level before/after comparison.
|
||||
/// </summary>
|
||||
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
|
||||
public sealed class PerfLoadFactAttribute : FactAttribute
|
||||
{
|
||||
public PerfLoadFactAttribute()
|
||||
{
|
||||
if (!PerfSettings.Current.Enabled || PerfSettings.Current.LoadDatabase is null)
|
||||
{
|
||||
Skip = $"Loads the synthetic dataset into an existing database; set {PerfSettings.EnabledVariable}=1 and {PerfSettings.LoadDatabaseVariable}.";
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,557 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using MeterVault.Core.Analysis;
|
||||
using MeterVault.Core.Analysis.Rollups;
|
||||
using MeterVault.Core.Normalization;
|
||||
using MeterVault.Infrastructure.Analysis;
|
||||
using MeterVault.Infrastructure.Costing;
|
||||
using MeterVault.Infrastructure.Options;
|
||||
using Npgsql;
|
||||
using static MeterVault.Integration.Tests.Performance.PerfReport;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>
|
||||
/// Reader and cost timings on the synthetic 1,000-meter decade (brief §9.10, D-56): the brief's target (cached metadata
|
||||
/// plus a ten-year monthly request for 100 meters within two seconds), portfolio, energy-type, meter, virtual and bill
|
||||
/// requests, the refusals a 1,000-meter selection meets before any SQL, the SQL each request sends, and the plans of
|
||||
/// the reader's main queries. Writes <c>results-<label>.md</c> (and <c>.json</c>) to the output directory.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Skipped unless <c>METERVAULT_PERF=1</c> (see <see cref="PerfSettings"/>). Run it alone, in Release, on a quiet machine:
|
||||
/// <c>dotnet test tests/Integration.Tests -c Release --filter "FullyQualifiedName~Performance.ReaderTimingTests"</c>.
|
||||
/// The numbers are measured, not asserted; only correctness of the measured requests (and the zero-SQL refusals) is.
|
||||
/// </remarks>
|
||||
[Trait("Category", "Performance")]
|
||||
public sealed class ReaderTimingTests(ITestOutputHelper output)
|
||||
{
|
||||
/// <summary>The brief's proposed review target for (a).</summary>
|
||||
private const double TargetMilliseconds = 2000;
|
||||
|
||||
[PerfFact]
|
||||
public async Task Reader_and_cost_timings_on_a_synthetic_1000_meter_decade()
|
||||
{
|
||||
var settings = PerfSettings.Current;
|
||||
using var log = new PerfLog(settings, "reader", output);
|
||||
var loadBefore = await HostLoadAsync(TimeSpan.FromSeconds(3));
|
||||
log.Write($"Host load before: {loadBefore}");
|
||||
|
||||
await using var database = await PerfDatabase.OpenAsync(settings, log);
|
||||
var prepared = await database.PrepareAsync(settings, log);
|
||||
var manifest = prepared.Manifest;
|
||||
|
||||
var zone = TimeZoneInfo.FindSystemTimeZoneById(manifest.Zone);
|
||||
var options = Microsoft.Extensions.Options.Options.Create(new MeterVaultOptions { TimeZone = manifest.Zone, Currency = "EUR" });
|
||||
var reader = new AnalysisReader(database, options);
|
||||
var costs = new CostReader(database, reader, options);
|
||||
var now = manifest.Now;
|
||||
var today = DateOnly.FromDateTime(TimeZoneInfo.ConvertTime(now, zone).DateTime);
|
||||
var currentMonth = new DateOnly(today.Year, today.Month, 1);
|
||||
var firstMonth = currentMonth.AddMonths(-119);
|
||||
|
||||
ResolvedPeriod Custom(DateOnly first, DateOnly last) => PeriodResolver.Resolve(PeriodPreset.Custom, first, last, now, zone);
|
||||
ResolvedPeriod Preset(PeriodPreset preset) => PeriodResolver.Resolve(preset, null, null, now, zone);
|
||||
|
||||
var tenYears = Custom(firstMonth, currentMonth.AddMonths(1).AddDays(-1));
|
||||
var lastYearByDay = Custom(today.AddDays(-364), today);
|
||||
var last12 = Preset(PeriodPreset.Last12Months);
|
||||
var selection = manifest.Selection100;
|
||||
var all = manifest.AllMeterIds;
|
||||
var electricity = manifest.EnergyTypes["electricity"];
|
||||
var samples = manifest.Samples;
|
||||
|
||||
// The catalog is the metadata a page holds for its lifetime: loaded once, then every request reads data only.
|
||||
var catalog = await reader.LoadCatalogAsync();
|
||||
var invalid = catalog.Meters.Values.Where(m => m.IsVirtual && m.VirtualStatus != VirtualMeterStatus.Valid).Select(m => $"{m.Name}: {m.VirtualStatus}").ToList();
|
||||
log.Write(invalid.Count == 0 ? "All virtual meters validate" : "Virtual meters that do not validate: " + string.Join("; ", invalid));
|
||||
|
||||
async Task<AnalysisResult> Cached(AnalysisRequest request)
|
||||
{
|
||||
await using var db = database.CreateDbContext();
|
||||
return await reader.ReadAsync(db, catalog, request, CancellationToken.None);
|
||||
}
|
||||
|
||||
var scenarios = new List<(string Id, string Description, Func<Task<string>> Run)>
|
||||
{
|
||||
("a", "100 selected meters (60 monthly, 25 daily, 10 live, 5 virtual), 10 years by month, catalog cached — the brief's target",
|
||||
async () => Describe(await Cached(new AnalysisRequest(AnalysisScope.ForMeters(selection), tenYears) { Bucket = BucketSize.Month, MaxSeries = selection.Length }))),
|
||||
("a'", "Same through the public API (catalog loaded by the request)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeters(selection), tenYears) { Bucket = BucketSize.Month, MaxSeries = selection.Length }))),
|
||||
("b", "Portfolio, last 12 months by month (measures only, as the overview)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.Portfolio, last12) { Bucket = BucketSize.Month }))),
|
||||
("b'", "Portfolio, last 12 months by month, with one series per meter",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.Portfolio, last12) { Bucket = BucketSize.Month, IncludeMeterSeries = true }))),
|
||||
("b''", "Portfolio, last 12 months by month, compared with the previous year",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.Portfolio, last12)
|
||||
{
|
||||
Bucket = BucketSize.Month,
|
||||
Comparison = new ComparisonRequest(ComparisonKind.PreviousYear),
|
||||
}))),
|
||||
("c", "Electricity type (≈420 meters), 10 years by month (measures only)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForEnergyType(electricity), tenYears) { Bucket = BucketSize.Month }))),
|
||||
("c'", "Electricity type, 10 years by month, with one series per meter (the type page's table)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForEnergyType(electricity), tenYears) { Bucket = BucketSize.Month, IncludeMeterSeries = true }))),
|
||||
("d", "One daily meter, 10 years by week (point limit raised to 600)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["dailyMeter"]), tenYears) { Bucket = BucketSize.Week, MaxPoints = 600 }))),
|
||||
("d'", "One monthly meter, 10 years by week (point limit raised to 600)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["monthlyMeter"]), tenYears) { Bucket = BucketSize.Week, MaxPoints = 600 }))),
|
||||
("d''", "One daily meter, the last 365 days by day",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["dailyMeter"]), lastYearByDay) { Bucket = BucketSize.Day }))),
|
||||
("d'''", "One live (hourly) meter, the last 365 days by day",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["liveMeter"]), lastYearByDay) { Bucket = BucketSize.Day }))),
|
||||
("e", "Portfolio bill, last 12 months by month, with categories",
|
||||
async () => Describe(await costs.ReadAsync(new CostAnalysisRequest(CostScope.Portfolio, last12) { Bucket = BucketSize.Month, IncludeCategories = true }))),
|
||||
("e'", "Portfolio bill, 10 years by month",
|
||||
async () => Describe(await costs.ReadAsync(new CostAnalysisRequest(CostScope.Portfolio, tenYears) { Bucket = BucketSize.Month }))),
|
||||
("f", "Virtual meter nested three levels (over 15 monthly, 10 daily, 2 differenced sources), 10 years by month",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["virtualNested"]), tenYears) { Bucket = BucketSize.Month }))),
|
||||
("f'", "Virtual difference of a 40-meter sum and an 8-meter live sum, 10 years by month",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeter(samples["virtualBigDifference"]), tenYears) { Bucket = BucketSize.Month }))),
|
||||
("g", "1,000-meter selection with the chart limit (6 series): refused",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeters(all), last12) { Bucket = BucketSize.Month }))),
|
||||
("g'", "Portfolio by day over 10 years: refused (point limit)",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.Portfolio, tenYears) { Bucket = BucketSize.Day }))),
|
||||
("g''", "Portfolio bill by day over 10 years: refused (point limit)",
|
||||
async () => Describe(await costs.ReadAsync(new CostAnalysisRequest(CostScope.Portfolio, tenYears) { Bucket = BucketSize.Day }))),
|
||||
("g'''", "Catalog load: 1,000 meters, tanks, links, rollup states; virtual definitions validated, totals classified",
|
||||
async () => $"{(await reader.LoadCatalogAsync()).Meters.Count} meters"),
|
||||
("g''''", "1,000-meter selection with the limit raised (as an export may), last 12 months by month",
|
||||
async () => Describe(await reader.ReadAsync(new AnalysisRequest(AnalysisScope.ForMeters(all), last12) { Bucket = BucketSize.Month, MaxSeries = int.MaxValue }))),
|
||||
};
|
||||
|
||||
var loadDuring = HostLoadAsync(TimeSpan.FromSeconds(3));
|
||||
var timings = new List<ScenarioTiming>();
|
||||
foreach (var (id, description, run) in scenarios)
|
||||
{
|
||||
timings.Add(await MeasureAsync(id, description, run, settings, log));
|
||||
}
|
||||
|
||||
// What the measured requests must hold, whatever they took.
|
||||
var target = timings.Single(t => t.Id == "a");
|
||||
Assert.True(target.Commands > 0, "The command counter saw no SQL for a request that reads data; Npgsql tracing did not reach it.");
|
||||
foreach (var refused in timings.Where(t => t.Id.StartsWith('g') && t.Shape.StartsWith("refused", StringComparison.Ordinal)))
|
||||
{
|
||||
if (refused.Commands != 0)
|
||||
{
|
||||
Assert.Fail($"{refused.Id} was refused but sent {refused.Commands} SQL command(s): {string.Join(" / ", refused.Statements)}");
|
||||
}
|
||||
}
|
||||
|
||||
Assert.StartsWith("refused", timings.Single(t => t.Id == "g").Shape, StringComparison.Ordinal);
|
||||
Assert.StartsWith("refused", timings.Single(t => t.Id == "g'").Shape, StringComparison.Ordinal);
|
||||
Assert.StartsWith("refused", timings.Single(t => t.Id == "g''").Shape, StringComparison.Ordinal);
|
||||
Assert.Contains($"{selection.Length} series", target.Shape, StringComparison.Ordinal);
|
||||
|
||||
var recomputes = await RecomputeTimingsAsync(database, manifest, log);
|
||||
var plans = await PlansAsync(database, catalog, manifest, zone, firstMonth, currentMonth, today, log);
|
||||
|
||||
await using var connection = new NpgsqlConnection(database.ConnectionString);
|
||||
await connection.OpenAsync();
|
||||
var markdown = await WriteReportAsync(
|
||||
settings, database, connection, prepared, timings, recomputes, plans, invalid, loadBefore, await loadDuring, catalog.Meters.Count);
|
||||
log.Write($"Results written to {markdown}");
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------------ measuring
|
||||
|
||||
private static async Task<ScenarioTiming> MeasureAsync(string id, string description, Func<Task<string>> run, PerfSettings settings, PerfLog log)
|
||||
{
|
||||
for (var i = 0; i < settings.Warmup; i++)
|
||||
{
|
||||
await run();
|
||||
}
|
||||
|
||||
// One counted run, untimed: the listener is only attached while counting.
|
||||
int commands;
|
||||
IReadOnlyList<string> statements;
|
||||
string shape;
|
||||
double sql;
|
||||
using (var counter = CommandCounter.Start())
|
||||
{
|
||||
shape = await run();
|
||||
commands = counter.Count;
|
||||
statements = counter.Statements;
|
||||
sql = counter.Duration.TotalMilliseconds;
|
||||
}
|
||||
|
||||
var times = new List<double>();
|
||||
for (var i = 0; i < settings.Runs; i++)
|
||||
{
|
||||
var watch = Stopwatch.StartNew();
|
||||
await run();
|
||||
times.Add(watch.Elapsed.TotalMilliseconds);
|
||||
}
|
||||
|
||||
var timing = new ScenarioTiming(id, description, times, commands, statements, shape) { SqlMilliseconds = sql };
|
||||
log.Write($"({id}) median {F(timing.Median)} ms, p95 {F(timing.P95)} ms, {commands} SQL taking {F(sql)} ms — {shape}");
|
||||
return timing;
|
||||
}
|
||||
|
||||
private static string Describe(AnalysisResult result)
|
||||
{
|
||||
if (result.Refusal != AnalysisRefusal.None)
|
||||
{
|
||||
return $"refused ({result.Refusal})";
|
||||
}
|
||||
|
||||
var values = result.Series.Concat(result.Measures).SelectMany(s => s.Values).ToList();
|
||||
var statuses = values.GroupBy(v => v.Status).OrderBy(g => g.Key).Select(g => string.Create(CultureInfo.InvariantCulture, $"{g.Key} {g.Count()}"));
|
||||
return string.Create(
|
||||
CultureInfo.InvariantCulture,
|
||||
$"{result.Series.Count} series · {result.Measures.Count} measures · {result.Plan.Buckets.Count} {result.Plan.Size} buckets · {string.Join(", ", statuses)}");
|
||||
}
|
||||
|
||||
private static string Describe(CostAnalysis bill)
|
||||
{
|
||||
if (bill.Refusal != CostRefusal.None)
|
||||
{
|
||||
return $"refused ({bill.Refusal})";
|
||||
}
|
||||
|
||||
return string.Create(
|
||||
CultureInfo.InvariantCulture,
|
||||
$"{bill.Plan.Buckets.Count} {bill.Plan.Size} buckets · total {bill.Total.Cost ?? double.NaN:N0} {bill.Currency} ({bill.Total.Status}) · {bill.Lines.Count} lines · {bill.EnergyTypes.Count} types · categories {(bill.Composition is null ? "no" : "yes")}");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// How long one meter's full recompute takes (D-57: it still runs per live reading): delete and rewrite its
|
||||
/// consumption, diff its rollups and coverage — as <c>IngestionService</c> does after each reading.
|
||||
/// </summary>
|
||||
private static async Task<List<(string Meter, int Readings, ScenarioTiming Timing)>> RecomputeTimingsAsync(
|
||||
PerfDatabase database, DatasetManifest manifest, PerfLog log)
|
||||
{
|
||||
var result = new List<(string, int, ScenarioTiming)>();
|
||||
(string Name, int Id)[] meters =
|
||||
[
|
||||
("monthly counter", manifest.Samples["monthlyMeter"]),
|
||||
("imported month labels", manifest.Samples["labelMeter"]),
|
||||
("daily counter", manifest.Samples["dailyMeter"]),
|
||||
("live meter (hourly for a year)", manifest.Samples["liveMeter"]),
|
||||
("tank", manifest.Samples["tankMeter"]),
|
||||
("virtual (purge + state only)", manifest.Samples["virtualNested"]),
|
||||
];
|
||||
|
||||
await using var connection = new NpgsqlConnection(database.ConnectionString);
|
||||
await connection.OpenAsync();
|
||||
foreach (var (name, id) in meters)
|
||||
{
|
||||
var readings = Convert.ToInt32(
|
||||
await PerfDatabase.ScalarAsync(connection, string.Create(CultureInfo.InvariantCulture, $"SELECT count(*) FROM reading WHERE meter_id = {id}")),
|
||||
CultureInfo.InvariantCulture);
|
||||
var times = new List<double>();
|
||||
for (var i = 0; i < 4; i++)
|
||||
{
|
||||
await using var db = database.CreateDbContext();
|
||||
await using var tx = await db.Database.BeginTransactionAsync();
|
||||
var watch = Stopwatch.StartNew();
|
||||
await PerfDatabase.Normalization(db, manifest).RecomputeMeterAsync(id, batchId: null);
|
||||
await db.SaveChangesAsync();
|
||||
await tx.CommitAsync();
|
||||
if (i > 0)
|
||||
{
|
||||
times.Add(watch.Elapsed.TotalMilliseconds);
|
||||
}
|
||||
}
|
||||
|
||||
var timing = new ScenarioTiming(name, name, times, 0, [], string.Empty);
|
||||
log.Write($"Recompute {name} ({readings} readings): median {F(timing.Median)} ms");
|
||||
result.Add((name, readings, timing));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------------ query plans
|
||||
|
||||
/// <summary>
|
||||
/// EXPLAIN (ANALYZE, BUFFERS) of the reader's queries (copied from <c>AnalysisQueries</c>) with the parameters request
|
||||
/// (a) — and, for the portfolio and day cases, (b) and (d'') — would send.
|
||||
/// </summary>
|
||||
private static async Task<List<(string Title, string Sql, string Plan)>> PlansAsync(
|
||||
PerfDatabase database, AnalysisCatalog catalog, DatasetManifest manifest, TimeZoneInfo zone,
|
||||
DateOnly firstMonth, DateOnly currentMonth, DateOnly today, PerfLog log)
|
||||
{
|
||||
var plans = new List<(string, string, string)>();
|
||||
await using var connection = new NpgsqlConnection(database.ConnectionString);
|
||||
await connection.OpenAsync();
|
||||
|
||||
var leaves = catalog.PhysicalLeaves(manifest.Selection100).Order().ToArray();
|
||||
var virtualSources = manifest.Selection100.Where(id => catalog.Find(id)?.IsVirtual == true)
|
||||
.SelectMany(id => catalog.PhysicalLeaves([id])).ToHashSet();
|
||||
var portfolio = catalog.Meters.Values.Where(m => !m.IsVirtual).Select(m => m.Id).Order().ToArray();
|
||||
var now = manifest.Now.UtcDateTime;
|
||||
DateTime Midnight(DateOnly day) => GapAttribution.LocalMidnight(day, zone).UtcDateTime;
|
||||
|
||||
const string month = """
|
||||
SELECT r.meter_id, r.month, r.kind, r.amount, r.measured, r.manual, r.imported, r.estimated, r.rows, r.flags,
|
||||
r.max_interval_end
|
||||
FROM unnest(@ids, @froms, @tos) AS w(meter_id, from_month, to_month)
|
||||
JOIN consumption_rollup_month r ON r.meter_id = w.meter_id AND r.month >= w.from_month AND r.month < w.to_month
|
||||
""";
|
||||
const string day = """
|
||||
SELECT r.meter_id, r.day, r.kind, r.amount, r.measured, r.manual, r.imported, r.estimated, r.rows, r.flags,
|
||||
r.max_interval_end, false AS after_now
|
||||
FROM unnest(@ids, @froms, @tos) AS w(meter_id, from_day, to_day)
|
||||
JOIN consumption_rollup r ON r.meter_id = w.meter_id AND r.day >= w.from_day AND r.day < w.to_day
|
||||
UNION ALL
|
||||
SELECT r.meter_id, r.day, r.kind, r.amount, r.measured, r.manual, r.imported, r.estimated, r.rows, r.flags,
|
||||
r.max_interval_end, true AS after_now
|
||||
FROM consumption_rollup r
|
||||
WHERE r.meter_id = ANY(@after_ids) AND r.day >= @after_from AND r.day < @after_to AND r.max_interval_end > @now
|
||||
""";
|
||||
const string raw = "SELECT meter_id, time, amount, quality FROM consumption WHERE meter_id = ANY(@i0) AND time >= @f0 AND time < @t0";
|
||||
const string windows = """
|
||||
SELECT w.idx, count(*)::int, sum(c.amount),
|
||||
coalesce(sum(c.amount) FILTER (WHERE c.quality = 0), 0),
|
||||
coalesce(sum(c.amount) FILTER (WHERE c.quality = 2), 0),
|
||||
coalesce(sum(c.amount) FILTER (WHERE c.quality = 3), 0),
|
||||
coalesce(sum(c.amount) FILTER (WHERE c.quality NOT IN (0, 2, 3)), 0)
|
||||
FROM unnest(@ids, @froms, @tos) WITH ORDINALITY AS w(meter_id, from_time, to_time, idx)
|
||||
JOIN consumption c ON c.meter_id = w.meter_id AND c.time >= w.from_time AND c.time < w.to_time
|
||||
GROUP BY w.idx
|
||||
""";
|
||||
const string coverage = """
|
||||
SELECT meter_id, span_from, span_to, resolution_class, divided_at_months, gap_reason, last_interval_start
|
||||
FROM meter_coverage
|
||||
WHERE meter_id = ANY(@ids)
|
||||
ORDER BY meter_id, span_from
|
||||
""";
|
||||
const string opening = """
|
||||
SELECT m.id, c.time
|
||||
FROM unnest(@ids) AS m(id)
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT r.flags FROM consumption_rollup r WHERE r.meter_id = m.id ORDER BY r.day LIMIT 1) f
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT c.time FROM consumption c WHERE c.meter_id = m.id ORDER BY c.time LIMIT 1) c
|
||||
WHERE (f.flags & @flag) <> 0
|
||||
""";
|
||||
const string recent = """
|
||||
SELECT m.id, r.time
|
||||
FROM unnest(@ids) AS m(id)
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT time FROM reading WHERE meter_id = m.id ORDER BY time DESC LIMIT @count) r
|
||||
""";
|
||||
const string events = "SELECT meter_id, max(time) FROM meter_event WHERE meter_id = ANY(@ids) GROUP BY meter_id";
|
||||
|
||||
async Task PlanAsync(string title, string sql, params (string Name, object Value)[] parameters)
|
||||
{
|
||||
try
|
||||
{
|
||||
await using var command = new NpgsqlCommand("EXPLAIN (ANALYZE, BUFFERS, SETTINGS) " + sql, connection) { CommandTimeout = 0 };
|
||||
foreach (var (name, value) in parameters)
|
||||
{
|
||||
command.Parameters.AddWithValue(name, value);
|
||||
}
|
||||
|
||||
var lines = new List<string>();
|
||||
await using var reader = await command.ExecuteReaderAsync();
|
||||
while (await reader.ReadAsync())
|
||||
{
|
||||
lines.Add(reader.GetString(0));
|
||||
}
|
||||
|
||||
plans.Add((title, sql, string.Join('\n', lines)));
|
||||
}
|
||||
catch (PostgresException ex)
|
||||
{
|
||||
plans.Add((title, sql, "EXPLAIN failed: " + ex.MessageText));
|
||||
}
|
||||
}
|
||||
|
||||
var dayFroms = leaves.Select(id => virtualSources.Contains(id) ? firstMonth : currentMonth).ToArray();
|
||||
var afterTo = currentMonth.AddMonths(1);
|
||||
log.Write("Capturing query plans…");
|
||||
|
||||
await PlanAsync($"(a) month rollups — {leaves.Length} physical leaves × {firstMonth:yyyy-MM} … {currentMonth.AddMonths(-1):yyyy-MM}", month,
|
||||
("ids", leaves), ("froms", leaves.Select(_ => firstMonth).ToArray()), ("tos", leaves.Select(_ => currentMonth).ToArray()));
|
||||
await PlanAsync($"(a) day rollups — the current month's complete days for every leaf, every day for virtual sources ({virtualSources.Count}), plus the after-now block", day,
|
||||
("ids", leaves), ("froms", dayFroms), ("tos", leaves.Select(_ => today).ToArray()),
|
||||
("after_ids", leaves), ("after_from", today.AddDays(1)), ("after_to", afterTo), ("now", now));
|
||||
await PlanAsync($"(a) partial edge day — today ({today:yyyy-MM-dd}) from consumption", raw,
|
||||
("i0", leaves), ("f0", Midnight(today)), ("t0", Midnight(today.AddDays(1))));
|
||||
await PlanAsync("(a) coverage runs", coverage, ("ids", leaves));
|
||||
await PlanAsync("(a) opening balances", opening, ("ids", leaves), ("flag", (int)RollupFlags.OpeningBalance));
|
||||
await PlanAsync("(a) freshness — the latest readings per meter (reading is compressed after 30 days)", recent, ("ids", leaves), ("count", FreshnessRules.RecentReadingCount));
|
||||
await PlanAsync("(a) freshness — the latest event per meter", events, ("ids", leaves));
|
||||
await PlanAsync($"(b) freshness — the latest readings of every physical meter ({portfolio.Length}), as a portfolio request loads them", recent,
|
||||
("ids", portfolio), ("count", FreshnessRules.RecentReadingCount));
|
||||
|
||||
// Not the reader's SQL: the same statement with a constant lower time bound, to show what chunk exclusion saves.
|
||||
var daily = manifest.Samples["dailyMeter"];
|
||||
const string recentBounded = """
|
||||
SELECT m.id, r.time
|
||||
FROM unnest(@ids) AS m(id)
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT time FROM reading WHERE meter_id = m.id AND time >= @since ORDER BY time DESC LIMIT @count) r
|
||||
""";
|
||||
await PlanAsync("(d) freshness — one daily meter, as every single-meter request sends it", recent,
|
||||
("ids", new[] { daily }), ("count", FreshnessRules.RecentReadingCount));
|
||||
await PlanAsync("(comparison, not the reader's SQL) the same for one daily meter with a constant 90-day lower bound", recentBounded,
|
||||
("ids", new[] { daily }), ("since", now.AddDays(-90)), ("count", FreshnessRules.RecentReadingCount));
|
||||
await PlanAsync($"(b) month rollups — portfolio, {portfolio.Length} physical meters × the 11 complete months of the last 12", month,
|
||||
("ids", portfolio), ("froms", portfolio.Select(_ => currentMonth.AddMonths(-11)).ToArray()), ("tos", portfolio.Select(_ => currentMonth).ToArray()));
|
||||
|
||||
var yearAgo = today.AddYears(-1);
|
||||
var nowLocal = TimeZoneInfo.ConvertTimeFromUtc(now, zone);
|
||||
var yearAgoCut = TimeZoneInfo.ConvertTimeToUtc(yearAgo.ToDateTime(TimeOnly.FromDateTime(nowLocal)), zone);
|
||||
var single = manifest.Samples["monthlyMeter"];
|
||||
await PlanAsync("(b'') window sums — one meter's two partial days (today, and its image a year ago): the small case", windows,
|
||||
("ids", new[] { single, single }),
|
||||
("froms", new[] { Midnight(today), Midnight(yearAgo) }),
|
||||
("tos", new[] { now, yearAgoCut }));
|
||||
await PlanAsync(
|
||||
$"(b'') window sums — stress case, not what (b'') sends (see its SQL list): today's partial day and its image a year ago for all {portfolio.Length} meters",
|
||||
windows,
|
||||
("ids", portfolio.Concat(portfolio).ToArray()),
|
||||
("froms", portfolio.Select(_ => Midnight(today)).Concat(portfolio.Select(_ => Midnight(yearAgo))).ToArray()),
|
||||
("tos", portfolio.Select(_ => now).Concat(portfolio.Select(_ => yearAgoCut)).ToArray()));
|
||||
|
||||
await PlanAsync("(d'') day rollups — one daily meter, the last 365 days", day,
|
||||
("ids", new[] { daily }), ("froms", new[] { today.AddDays(-364) }), ("tos", new[] { today }),
|
||||
("after_ids", new[] { daily }), ("after_from", today.AddDays(1)), ("after_to", today.AddDays(1)), ("now", now));
|
||||
return plans;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------------ report
|
||||
|
||||
private static async Task<string> WriteReportAsync(
|
||||
PerfSettings settings,
|
||||
PerfDatabase database,
|
||||
NpgsqlConnection connection,
|
||||
PreparedDatabase prepared,
|
||||
List<ScenarioTiming> timings,
|
||||
List<(string Meter, int Readings, ScenarioTiming Timing)> recomputes,
|
||||
List<(string Title, string Sql, string Plan)> plans,
|
||||
List<string> invalidVirtuals,
|
||||
string loadBefore,
|
||||
string loadDuring,
|
||||
int catalogMeters)
|
||||
{
|
||||
var manifest = prepared.Manifest;
|
||||
var md = new StringBuilder();
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"# Analysis reader performance — {settings.Label}");
|
||||
md.AppendLine();
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"Measured {DateTimeOffset.Now:yyyy-MM-dd HH:mm zzz}. Brief §9.10 / D-56. Dataset \"now\": {manifest.Now:O} ({manifest.Zone}).");
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"{settings.Warmup} warm-up run(s), then {settings.Runs} measured runs per scenario; the SQL count comes from one extra untimed run.");
|
||||
if (settings.Note is { } note)
|
||||
{
|
||||
md.AppendLine().AppendLine(CultureInfo.InvariantCulture, $"> Note: {note}");
|
||||
}
|
||||
|
||||
md.AppendLine();
|
||||
md.AppendLine("## Machine");
|
||||
md.AppendLine();
|
||||
var machine = Machine().Select(m => (IReadOnlyList<string>)[m.Name, m.Value]).ToList();
|
||||
machine.Add(["Host load before the run", loadBefore]);
|
||||
machine.Add(["Host load while measuring", loadDuring]);
|
||||
machine.Add(["Database", database.Origin]);
|
||||
foreach (var (name, value) in await DatabaseAsync(connection))
|
||||
{
|
||||
machine.Add([name, value]);
|
||||
}
|
||||
|
||||
Table(md, ["", "Value"], machine);
|
||||
|
||||
md.AppendLine("## Dataset");
|
||||
md.AppendLine();
|
||||
md.AppendLine(CultureInfo.InvariantCulture,
|
||||
$"{manifest.MeterCount} meters (scale {F(manifest.Scale, 2)}): {string.Join(", ", manifest.Groups.Select(g => $"{g.Value.Length} {g.Key}"))}. " +
|
||||
$"{manifest.EnergyTypes.Count} energy types, {manifest.Links} links, {manifest.Tariffs} tariff rows, {manifest.CategoryMembers} category members, {manifest.ManualCosts} manual costs, {N(manifest.Events)} events.");
|
||||
md.AppendLine();
|
||||
md.AppendLine("Monthly counters mix readings at local midnight on the 1st, imported month labels (00:00 UTC, flagged) and irregular manual readings " +
|
||||
"(divided at month ends); some have no install date (opening balance), start late, retire in 2022, or have a register swap. Daily counters read at 06:xx, " +
|
||||
"live meters hourly for the last year (monthly before), generation counters in the evening; the electricity type and two extra sites have grid import/export roles. " +
|
||||
"Twenty virtual meters are sums and differences, nested up to three levels. The cost setup has yearly unit-price changes (a mid-2022 spike for electricity and gas), " +
|
||||
"standing charges, feed-in, five meter prices on linked subsections, four categories (one overlapping) and manual costs.");
|
||||
md.AppendLine();
|
||||
Table(md, ["Table", "Rows", "Size", "Chunks"], (await TablesAsync(connection)).Select(t => (IReadOnlyList<string>)[t.Table, N(t.Rows), t.Size, t.Note]));
|
||||
|
||||
md.AppendLine("## Loading and rebuild");
|
||||
md.AppendLine();
|
||||
var steps = new List<IReadOnlyList<string>>();
|
||||
steps.Add(prepared.Loaded
|
||||
? ["Raw load (binary COPY of readings, plus configuration)", F(manifest.LoadSeconds) + " s", $"{N(manifest.Readings)} readings; COPY itself {F(manifest.CopySeconds)} s"]
|
||||
: ["Raw load", "reused", $"{N(manifest.Readings)} readings loaded earlier (COPY {F(manifest.CopySeconds)} s, total {F(manifest.LoadSeconds)} s)"]);
|
||||
if (prepared.Rebuild is { } rebuild)
|
||||
{
|
||||
steps.Add([
|
||||
"Startup rebuild (NormalizationUpgrade: consumption, rollups, coverage, state)",
|
||||
F(rebuild.Seconds) + " s",
|
||||
$"{rebuild.Rebuilt} of {rebuild.Meters} meters, {F(rebuild.Meters / rebuild.Seconds)} meters/s{(rebuild.MeasuredInThisRun ? string.Empty : " (measured when the dataset was loaded)")}",
|
||||
]);
|
||||
}
|
||||
|
||||
steps.Add(prepared.CompressSeconds is { } compress
|
||||
? ["Compression of raw chunks older than 30 days", F(compress) + " s", $"{prepared.CompressedChunks} chunks"]
|
||||
: ["Compression of raw chunks older than 30 days", "—", "already compressed"]);
|
||||
Table(md, ["Step", "Time", "Detail"], steps);
|
||||
|
||||
md.AppendLine("One meter's full recompute (what every ingested reading triggers, D-57), median of 3 after a warm-up:");
|
||||
md.AppendLine();
|
||||
Table(md, ["Meter", "Readings", "Median ms", "Min ms", "Max ms"],
|
||||
recomputes.Select(r => (IReadOnlyList<string>)[r.Meter, N(r.Readings), F(r.Timing.Median), F(r.Timing.Min), F(r.Timing.Max)]));
|
||||
|
||||
md.AppendLine("## Reader and cost timings");
|
||||
md.AppendLine();
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"Catalog: {catalogMeters} meters. {(invalidVirtuals.Count == 0 ? "All virtual meters validate." : "Virtual meters that do not validate: " + string.Join("; ", invalidVirtuals))}");
|
||||
md.AppendLine();
|
||||
md.AppendLine("*SQL* is the number of commands one extra, untimed run sent, and *SQL ms* their summed duration (Npgsql activity: execution " +
|
||||
"until the reader is closed, so row materialization is included); the rest of the median is in-process work.");
|
||||
md.AppendLine();
|
||||
Table(md, ["", "Request", "Median ms", "p95 ms", "Min ms", "Max ms", "SQL", "SQL ms", "Result"],
|
||||
timings.Select(t => (IReadOnlyList<string>)[
|
||||
t.Id, t.Description, F(t.Median), F(t.P95), F(t.Min), F(t.Max), t.Commands.ToString(CultureInfo.InvariantCulture), F(t.SqlMilliseconds), t.Shape]));
|
||||
|
||||
var target = timings.Single(t => t.Id == "a");
|
||||
md.AppendLine(CultureInfo.InvariantCulture,
|
||||
$"**Target (a):** median {F(target.Median)} ms, p95 {F(target.P95)} ms against {F(TargetMilliseconds, 0)} ms — {(target.P95 < TargetMilliseconds ? "met" : target.Median < TargetMilliseconds ? "met at the median, not at p95" : "not met")}.");
|
||||
md.AppendLine();
|
||||
md.AppendLine("**Limits:** the refused requests (g, g', g'') sent " +
|
||||
string.Join(", ", timings.Where(t => t.Shape.StartsWith("refused", StringComparison.Ordinal)).Select(t => $"{t.Id}: {t.Commands}")) +
|
||||
" SQL commands — the limits are checked before any SQL runs.");
|
||||
md.AppendLine();
|
||||
|
||||
md.AppendLine("### SQL sent per request");
|
||||
md.AppendLine();
|
||||
foreach (var timing in timings.Where(t => t.Commands > 0))
|
||||
{
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"<details><summary>({timing.Id}) {timing.Commands} commands, {F(timing.SqlMilliseconds)} ms</summary>");
|
||||
md.AppendLine();
|
||||
md.AppendLine("```sql");
|
||||
foreach (var statement in timing.Statements)
|
||||
{
|
||||
md.AppendLine(statement);
|
||||
}
|
||||
|
||||
md.AppendLine("```");
|
||||
md.AppendLine("</details>");
|
||||
md.AppendLine();
|
||||
}
|
||||
|
||||
md.AppendLine("## Query plans");
|
||||
md.AppendLine();
|
||||
md.AppendLine("EXPLAIN (ANALYZE, BUFFERS, SETTINGS) of the reader's statements (copied from `AnalysisQueries`) with the parameters of the request named.");
|
||||
md.AppendLine();
|
||||
foreach (var (title, sql, plan) in plans)
|
||||
{
|
||||
md.AppendLine(CultureInfo.InvariantCulture, $"### {title}");
|
||||
md.AppendLine();
|
||||
md.AppendLine("```sql").AppendLine(sql.Trim()).AppendLine("```");
|
||||
md.AppendLine("```").AppendLine(plan).AppendLine("```");
|
||||
md.AppendLine();
|
||||
}
|
||||
|
||||
Directory.CreateDirectory(settings.OutputDirectory);
|
||||
var path = Path.Combine(settings.OutputDirectory, $"results-{settings.Label}.md");
|
||||
await File.WriteAllTextAsync(path, md.ToString());
|
||||
await File.WriteAllTextAsync(
|
||||
Path.Combine(settings.OutputDirectory, $"results-{settings.Label}.json"),
|
||||
JsonSerializer.Serialize(
|
||||
new { settings.Label, manifest.Now, prepared.Rebuild, Timings = timings.Select(t => new { t.Id, t.Description, t.Median, t.P95, t.Min, t.Max, t.Commands, t.SqlMilliseconds, t.Shape, t.Milliseconds }) },
|
||||
DatasetManifest.Json));
|
||||
return path;
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,49 @@
|
||||
using System.Globalization;
|
||||
using System.Text.Json;
|
||||
using Npgsql;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace MeterVault.Integration.Tests.Performance;
|
||||
|
||||
/// <summary>
|
||||
/// Loads the synthetic raw dataset into a database an app has already migrated — the pre-rework app (c0f52db) or the
|
||||
/// current one — for the page-level before/after comparison (brief §9.10). Nothing derived is written: the next start
|
||||
/// of that app rebuilds consumption (and, in the new app, rollups and coverage) itself, because the load clears the
|
||||
/// stored normalization revision. Writes <c>manifest-<label>.json</c> with the meter and type ids the page
|
||||
/// timings address.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <c>METERVAULT_PERF=1 METERVAULT_PERF_LOAD_DB="Host=…;Port=…;Database=metervault;Username=metervault;Password=metervault"
|
||||
/// dotnet test tests/Integration.Tests --filter "FullyQualifiedName~Performance.SyntheticLoadTests"</c>. The dataset ends at
|
||||
/// <c>METERVAULT_PERF_NOW</c>, or at the current time, so the pages see recent live data.
|
||||
/// </remarks>
|
||||
[Trait("Category", "Performance")]
|
||||
public sealed class SyntheticLoadTests(ITestOutputHelper output)
|
||||
{
|
||||
[PerfLoadFact]
|
||||
public async Task Load_the_synthetic_dataset_into_an_app_migrated_database()
|
||||
{
|
||||
var settings = PerfSettings.Current;
|
||||
using var log = new PerfLog(settings, "load", output);
|
||||
var target = settings.LoadDatabase!;
|
||||
|
||||
await using (var connection = new NpgsqlConnection(target))
|
||||
{
|
||||
await connection.OpenAsync();
|
||||
var migrated = await PerfDatabase.ScalarAsync(connection, "SELECT to_regclass('meter') IS NOT NULL AND to_regclass('app_setting') IS NOT NULL");
|
||||
Assert.True(migrated is true, "The target database is not migrated: start the app against it once first.");
|
||||
var schema = await PerfDatabase.ScalarAsync(connection, "SELECT to_regclass('consumption_rollup') IS NOT NULL") is true ? "analysis rework" : "pre-rework (c0f52db)";
|
||||
log.Write($"Target schema: {schema}");
|
||||
}
|
||||
|
||||
// The current minute, so the hourly live meters end just before the pages are timed.
|
||||
var now = settings.Now ?? DateTimeOffset.UtcNow.AddSeconds(-DateTimeOffset.UtcNow.Second);
|
||||
var zone = TimeZoneInfo.FindSystemTimeZoneById("Europe/Berlin");
|
||||
var manifest = await SyntheticDataset.LoadAsync(target, now, zone, settings.Scale, log.Write);
|
||||
|
||||
Directory.CreateDirectory(settings.OutputDirectory);
|
||||
var path = Path.Combine(settings.OutputDirectory, $"manifest-{settings.Label}.json");
|
||||
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(manifest, DatasetManifest.Json));
|
||||
log.Write(string.Create(CultureInfo.InvariantCulture, $"Loaded {manifest.MeterCount} meters and {manifest.Readings:N0} readings; manifest {path}"));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user