feat(observability): add diagnostic dashboards (#27)
quality-gate / quality (push) Failing after 1m31s
quality-gate / container (push) Has been skipped

This commit is contained in:
KyuubiYoru
2026-07-17 00:22:46 +02:00
parent 06c4ecf8f3
commit 99885f8c8c
30 changed files with 3173 additions and 18 deletions
@@ -2,9 +2,10 @@
This runbook defines the production signals and privileged controls for the
Rendezvous service. The service emits `System.Diagnostics.Metrics` instruments
from the `FinalFactory.Rendezvous` meter and distributed-tracing activities from
`FinalFactory.Rendezvous.Server`. Connect those sources to the deployment's
OpenTelemetry or equivalent collector. Do not add identifiers to metric labels.
from the `FinalFactory.Rendezvous` meter, distributed-tracing activities from
`FinalFactory.Rendezvous.Server`, and an optional bearer-protected Prometheus
endpoint. Connect only a private collector network. Do not add identifiers to
metric labels.
Concrete detect/contain/recover/verify procedures for abuse, key compromise,
targeted revocation, restart, rollback, saturation, privacy incidents, and
@@ -38,6 +39,25 @@ dependency upgrades are in the [incident and change runbooks](incident-runbooks.
| `rendezvous.store.expiry_churn` | Cumulative natural expiry activity | none |
| `rendezvous.store.available` | Store health (`1` available, `0` unavailable) | none |
The Prometheus exporter additionally exposes active/fresh/awaiting store state,
drain state, SSE subscriber/tenant/replay gauges, bounded HTTP and UDP
histograms, UDP ingress and conservative admitted-response budgets, signing-key
state/window gauges, and process/.NET pressure. Its only labels are the fixed
operation, status, result, transport, partition, action, outcome, elapsed-bucket,
key-state, and GC-generation dimensions. Unknown or unsafe values normalize to
`other`; identifiers, metadata, addresses, endpoints, tokens, and capabilities
are never labels.
The exporter is disabled by default. Enabling `Rendezvous:Metrics:Enabled`
requires `BearerTokenSecretReference` to be an external `env:` or absolute
`file:` secret containing 32-128 visible ASCII bytes. Unauthorized requests get
the same `404` as a disabled endpoint, and accepted responses are `no-store`.
Expose `/metrics` only to the private collector network, rotate its token as a
deployment secret, and never place the token in a URL, Compose environment
value, dashboard, log, or issue. The checked-in Prometheus/Grafana provisioning
and its verification procedure are in
[diagnostic dashboards](diagnostic-dashboards.md).
HTTP responses include `X-Rendezvous-Correlation-ID`. It is a generated trace ID
or random value, never a caller-supplied session or player identifier. UDP and
HTTP activities contain operation-level data only. Logs and traces must not add