147 lines
8.1 KiB
Markdown
147 lines
8.1 KiB
Markdown
# ADR 0003: state, privacy, availability, and safety budgets
|
|
|
|
- Status: Accepted
|
|
- Date: 2026-07-16
|
|
- Tracking: #2
|
|
|
|
## Context
|
|
|
|
V1 needs safe defaults before contracts and stores make them difficult to
|
|
change. The initial deployment is deliberately single-active and in-memory, so
|
|
its restart and availability behavior must be honest.
|
|
|
|
## Decision
|
|
|
|
### State and lifecycle
|
|
|
|
All directory, lease, presence, attempt, capability, ticket-consumption, and
|
|
rate-limit state is ephemeral and held behind atomic store interfaces. V1 has
|
|
one active writer/service instance. A second instance may be a cold standby but
|
|
must not accept public traffic concurrently.
|
|
|
|
```mermaid
|
|
stateDiagram-v2
|
|
[*] --> Registered: authenticated register
|
|
Registered --> Visible: fresh lease and fresh UDP presence
|
|
Visible --> Registered: presence becomes stale
|
|
Visible --> Visible: lease renew + presence refresh
|
|
Registered --> Expired: lease expires
|
|
Visible --> Expired: lease expires
|
|
Registered --> Revoked: host or operator revokes
|
|
Visible --> Revoked: host or operator revokes
|
|
Expired --> [*]
|
|
Revoked --> [*]
|
|
```
|
|
|
|
Restart loses all ephemeral state, used capabilities, and listings. Readiness is
|
|
false until HTTP, UDP, policy, key material, and the state store are ready. SDK
|
|
publishers use jittered backoff and re-register after a restart; old credentials
|
|
remain invalid. The service drains by refusing new registrations/attempts,
|
|
allowing a bounded completion window, then cancelling remaining work.
|
|
|
|
No horizontal scale is supported until shared atomic state and deterministic
|
|
mediator routing exist. A shared-state design is triggered when any of these is
|
|
true:
|
|
|
|
- one measured supported node cannot sustain 150% of the 30-day peak load;
|
|
- the approved availability target exceeds what single-active operation can meet;
|
|
- planned maintenance without listing loss becomes a product requirement; or
|
|
- a region needs more than one active mediator endpoint.
|
|
|
|
Relay remains independently triggered only when a representative real-network
|
|
canary shows direct-connect failure high enough to justify its privacy, abuse,
|
|
bandwidth, and operating cost.
|
|
|
|
### Initial time and size budgets
|
|
|
|
These are enforceable v1 ceilings, not suggestions. Contract issue #4 may lower
|
|
them but must not raise them without security review.
|
|
|
|
| Budget | V1 ceiling |
|
|
| --- | --- |
|
|
| HTTP request body | 16 KiB after content decoding; compressed request bodies are rejected in v1 |
|
|
| Listing metadata | 4 KiB encoded JSON, at most 32 keys; key 64 UTF-8 bytes; scalar value 256 UTF-8 bytes; nesting depth 3 |
|
|
| Browser page | 100 listings and 256 KiB encoded response; opaque cursor; stable bounded sort |
|
|
| UDP datagram accepted | 1,200 bytes; oversized or fragmented application payloads are dropped without response |
|
|
| Opaque HTTP credential | 1,024 bytes encoded |
|
|
| UDP capability or connection ticket | 192 base64url characters; NAT punch capabilities also remain below LiteNetLib's 256-character token ceiling; complete datagram at most 1,200 bytes |
|
|
| Clock skew | 30 seconds maximum when validating issued/not-before/expiry times |
|
|
| Lease lifetime | 60 seconds; renewal accepted from 30 seconds; no client-selected extension |
|
|
| Host presence freshness | 20 seconds |
|
|
| Join attempt lifetime | 30 seconds |
|
|
| Punch capability lifetime | 30 seconds and one successful use per role |
|
|
| Connection ticket lifetime | 20 seconds and one successful host consumption |
|
|
| Graceful drain | 30 seconds maximum |
|
|
|
|
All work queues are bounded. Initial per-instance ceilings are 1,024 concurrent
|
|
HTTP requests, 4,096 queued UDP datagrams, and 10,000 active join attempts.
|
|
Overflow is rejected or dropped early with a metric; it never creates an
|
|
unbounded task, allocation, log entry, or retry loop.
|
|
|
|
For an endpoint that has not proved possession of a valid capability, the UDP
|
|
mediator sends no response. Once both valid peer contributions exist,
|
|
authenticated mediation sends at most one introduction datagram to each peer.
|
|
The combined response bytes caused by the completing contribution must be no
|
|
more than twice that contribution's bytes, giving zero unverified amplification
|
|
and at most 2.0 verified byte amplification. Protocol padding or a smaller
|
|
response enforces the byte ratio. Responses are sent only to endpoints observed
|
|
from the corresponding authenticated gameplay socket, never to an arbitrary
|
|
HTTP-supplied address.
|
|
|
|
### Supported and capacity profiles
|
|
|
|
The development profile is functional, not a production capacity claim. The
|
|
initial production candidate is one Linux instance with 2 vCPU and 2 GiB RAM,
|
|
targeting 25,000 visible listings, 10,000 active attempts, 200 HTTP requests per
|
|
second, and 2,000 UDP datagrams per second while staying below 70% sustained CPU
|
|
and 75% memory. Issue #18 must measure and publish the actual supported profile;
|
|
production is blocked if the target is not met or the documented profile is not
|
|
reduced accordingly.
|
|
|
|
The initial single-active service objective, after the real-network canary, is
|
|
99.5% monthly successful availability for valid in-profile requests, excluding
|
|
announced maintenance. In-profile latency objectives are p95 <= 200 ms for HTTP
|
|
and p95 <= 100 ms from the second valid UDP contribution to both introduction
|
|
datagrams. These are service objectives, not guarantees of NAT traversal.
|
|
|
|
### Data classification and retention
|
|
|
|
| Data | Classification | Retention and handling |
|
|
| --- | --- | --- |
|
|
| Raw public/local endpoints | Sensitive network data | In memory only while the lease/attempt requires it, then deleted within 10 minutes; never logged or exported as metric labels |
|
|
| Listing display metadata | Public-untrusted or unlisted-untrusted | In memory for the active lease; audit stores only schema/result and a listing ID, not metadata values |
|
|
| Lease/capability/ticket/key material | Secret | Opaque random credentials are retained only as keyed digests; signed credentials retain verification keys and consumption IDs, not issued plaintext; plaintext is returned only at creation and is never logged or traced |
|
|
| Principal and tenant IDs | Internal identifiers | Audit retention 30 days; access-controlled and never used as high-cardinality metric labels |
|
|
| Security/audit event | Confidential operations data | 30 days online, access-controlled; contains action, coarse result, tenant, principal, and correlation ID, but no raw endpoint or secret |
|
|
| Diagnostic attempt record | Sensitive diagnostic data | Disabled by default; when explicitly enabled, redacted record retained at most 24 hours; raw endpoints remain excluded |
|
|
| Aggregate outcome/capacity metrics | Operational aggregate | 13 months; only bounded dimensions such as game, environment, region, trust mode, and typed outcome |
|
|
|
|
Logs use allowlisted fields rather than after-the-fact redaction. Correlation IDs
|
|
are random and are not credentials. Error responses are stable and do not reveal
|
|
whether a cross-tenant resource exists.
|
|
|
|
## Owner decisions required before production
|
|
|
|
Implementation can proceed with the baseline above. Production remains blocked
|
|
until the owner records:
|
|
|
|
- the actual secret-provider and key-custody system for each environment;
|
|
- which games may enable anonymous unlisted player hosting;
|
|
- deployment regions, data-processing jurisdiction, and approval of the stated
|
|
30-day audit/13-month aggregate retention periods;
|
|
- the per-game dedicated fallback endpoint policy.
|
|
|
|
Issue #18 measured and ratified the original 2-vCPU/2-GiB, 25,000-listing,
|
|
10,000-attempt core-state candidate profile and retained the 99.5% single-active
|
|
topology. It does not claim that core measurements prove public HTTP/UDP SLOs.
|
|
The versioned evidence, RTO, failure domains, and explicit signals that trigger
|
|
shared-state/high-availability work are recorded in the
|
|
[capacity and resilience gate](../operations/capacity-and-resilience.md). The
|
|
real-network canary in #23 must confirm that the proposed regional launch load
|
|
fits this profile and validate the public SLOs; it may lower the launch cap but
|
|
may not silently enable a
|
|
second active instance.
|
|
|
|
These are configuration and launch decisions, not permission to weaken the
|
|
tenant, replay, endpoint-verification, or secret-handling controls.
|