feat(server): harden hostile input and overload behavior (#15)
quality-gate / quality (push) Failing after 1m5s
quality-gate / quality (push) Failing after 1m5s
This commit is contained in:
@@ -0,0 +1,98 @@
|
||||
# Hostile-input and overload protection
|
||||
|
||||
Tracking: #15
|
||||
|
||||
Rendezvous treats every public HTTP request and UDP datagram as hostile. The
|
||||
server applies bounded fixed-window request budgets and concurrency ceilings in
|
||||
two stages so malformed input is discarded before expensive work while valid
|
||||
traffic is also isolated by its authenticated scope.
|
||||
|
||||
## Enforcement order
|
||||
|
||||
1. Kestrel and the HTTP abuse middleware cap request bodies at 16 KiB. A known
|
||||
oversized body receives a typed `413` response before endpoint dispatch.
|
||||
2. Every HTTP request consumes global, source-prefix, and operation budgets and
|
||||
acquires the corresponding concurrency leases. IPv4 sources share a `/24`
|
||||
budget and IPv6 sources share a `/56` budget; raw addresses are not retained.
|
||||
Non-lease operations also consume a smaller optional-work budget, leaving a
|
||||
configured global and source-prefix reserve for renew, update, and delete
|
||||
operations during shedding.
|
||||
Health probes use their own source-prefix budget so public API overload cannot
|
||||
make a healthy instance fail its orchestrator probes, while health traffic is
|
||||
still bounded.
|
||||
3. Once an endpoint has safely derived identities, it also acquires applicable
|
||||
tenant, principal or capability, and listing/attempt budgets. Secret
|
||||
capabilities are represented only by bounded SHA-256 fingerprints.
|
||||
4. Every UDP envelope consumes global, source-prefix, and wire-operation
|
||||
budgets before decoding. A structurally and cryptographically valid request
|
||||
then consumes capability, role, and mediation-handle budgets before state
|
||||
mutation or introduction.
|
||||
5. HTTP overload returns the stable `RateLimited` error, status `429`, and a
|
||||
bounded `Retry-After` value in both the header and response contract. UDP
|
||||
overload and every invalid UDP input are silently dropped.
|
||||
|
||||
The same HTTP identity budget is computed whether or not a listing or attempt
|
||||
exists. Rejection therefore does not disclose resource existence. Publisher
|
||||
authentication also completes before any tenant/resource operation, while the
|
||||
pre-authentication source budget prevents invalid credentials from bypassing
|
||||
load shedding.
|
||||
|
||||
## Bounded state and recovery
|
||||
|
||||
`Rendezvous:AbuseProtection:MaxTrackedKeys` is a hard combined ceiling for rate
|
||||
and active-concurrency keys. General HTTP and UDP traffic cannot consume the
|
||||
configured `CriticalTrackedKeyReserve`; lease operations and health probes may
|
||||
use that reserve but never exceed the hard ceiling. A request that would exceed
|
||||
its applicable ceiling fails closed without adding state. Fixed-window rate keys
|
||||
are cleared at the next window boundary; concurrency keys are removed as their
|
||||
request leases finish. HTTP and UDP trackers have separate locks and cardinality
|
||||
partitions, so a UDP flood cannot block HTTP admission on a shared lock or
|
||||
consume HTTP key capacity. This gives
|
||||
deterministic burst recovery and prevents an attacker from growing a permanent
|
||||
high-cardinality address, credential, or resource table.
|
||||
|
||||
The complete default profile is checked into
|
||||
`src/FinalFactory.Rendezvous.Server/appsettings.json`. Operators may lower or
|
||||
tune limits for a measured deployment profile, but must preserve all dimensions
|
||||
and leave the tracker ceiling above the maximum simultaneous key set. A rolling
|
||||
deployment should use the same profile on every instance. These per-process
|
||||
limits are a final service boundary; an edge proxy may add stricter distributed
|
||||
limits but is not a substitute for them.
|
||||
|
||||
When an HTTP reverse proxy is used, every immediate proxy address must be
|
||||
allowlisted in `Rendezvous:AbuseProtection:TrustedProxyAddresses` (or indexed
|
||||
environment variables such as
|
||||
`Rendezvous__AbuseProtection__TrustedProxyAddresses__0`). Only one forwarded
|
||||
hop is accepted. With an empty allowlist, forwarded headers are ignored and the
|
||||
direct TCP peer is the source. Never add a broad network range or accept
|
||||
untrusted `X-Forwarded-For` input: that would let a caller choose its own rate
|
||||
partition.
|
||||
|
||||
## Reflection, disclosure, and logging rules
|
||||
|
||||
- UDP sends nothing for malformed, oversized, unauthenticated, stale,
|
||||
replayed, wrong-role, or rate-limited input.
|
||||
- Introductions are emitted only after both role-scoped capabilities bind to
|
||||
their observed gameplay-socket sources. HTTP never supplies a public
|
||||
introduction target.
|
||||
- Private candidates must be same-family private unicast addresses and are used
|
||||
only for peers observed behind the same public address.
|
||||
- Abuse keys, exceptions, and responses never include bearer credentials,
|
||||
capabilities, tickets, raw endpoints, metadata values, or hostile markup.
|
||||
- Endpoint and capability values are not used as metric labels or log fields.
|
||||
|
||||
## Verification
|
||||
|
||||
The deterministic test corpora use the recorded seeds `0x152026`, `0x154A50`,
|
||||
and `0x1557A7E`. They exercise 10,000 arbitrary UDP envelopes through the
|
||||
production decoder, 5,000 arbitrary HTTP/credential parser inputs, and 1,000
|
||||
mutated state transitions, including the oversized and configured-capacity
|
||||
boundaries.
|
||||
Focused tests cover IPv4 and IPv6 prefix
|
||||
partitioning, tenant/principal/resource concurrency, tracker exhaustion,
|
||||
window recovery, wire-operation isolation, a steady-state allocation ceiling,
|
||||
typed `429`/`413` responses, secret fingerprint redaction, and silent
|
||||
authenticated UDP shedding. The existing state, contract, HTTP, client,
|
||||
and mediator suites continue to cover cross-tenant access, replay, role swaps,
|
||||
credential rotation, bounded metadata, endpoint validation, and one-shot
|
||||
amplification behavior.
|
||||
@@ -11,7 +11,7 @@ backlog where the control is implemented and verified.
|
||||
| Per-game credentials and signing keys | Provisioned principals and versioned keys are scoped to game/environment; secrets come from a provider and never a public binary. (#5) | Cross-tenant authorization tests, rotation/overlap/revocation tests, and secret scans. |
|
||||
| Short-lived, single-purpose tokens resistant to replay | Issuer fixes audience, tenant, attempt, role, issued/expiry times, nonce, and key ID; store atomically consumes nonce/ticket. (#4, #6, #10) | Golden vectors; expired, future, mutated, wrong-role, wrong-tenant, and concurrent replay tests. |
|
||||
| Strict payload, metadata, and token size limits | ADR 0003 ceilings are checked before allocation/deserialization and again at domain construction. (#4, #15) | Boundary/property tests, malformed corpus, and allocation-aware fuzzing. |
|
||||
| Registration, query, and introduction rate limits | Layered per-address, principal, tenant, and global token buckets with bounded queues and stable retry guidance. (#15) | Limit partition/isolation tests and overload/soak profiles. |
|
||||
| Registration, query, and introduction rate limits | Layered fixed-window budgets and concurrency leases cover global, operation, IPv4 `/24` or IPv6 `/56`, tenant, principal/capability, and listing/attempt dimensions with a bounded key table and stable retry guidance. (#15) | Deterministic partition, concurrency, tracker-exhaustion, recovery, typed-overload, and silent-UDP-shedding tests. |
|
||||
| Lease expiry removes abandoned servers | Visibility and join eligibility atomically require a fresh lease and fresh authenticated presence. (#6, #7) | Fake-clock expiry, renew/expire race, restart, and stale-host join tests. |
|
||||
| Validate game, environment, room, and protocol boundaries | Every identifier is a validated type; store keys and authorization decisions include server-derived tenant scope; protocol is exact-match in v1. (#4-#10) | Contract, tenant-isolation, incompatible-version, and confused-deputy tests. |
|
||||
| Structured audit events without secrets or reusable credentials | Allowlisted audit schema excludes metadata values, raw endpoints, tokens, and key material; event volume is bounded. (#16) | Captured-log/audit assertions and credential canary scans. |
|
||||
|
||||
Reference in New Issue
Block a user