docs: add integration guides and incident runbooks (#20)
quality-gate / quality (push) Failing after 1m28s
quality-gate / container (push) Has been skipped

This commit is contained in:
KyuubiYoru
2026-07-16 18:25:10 +02:00
parent cc5793f935
commit 7fb85059fb
11 changed files with 1082 additions and 104 deletions
+227
View File
@@ -0,0 +1,227 @@
# Game integration seams
Tracking: #20
Use the [TestClient start-to-finish guide](test-client.md) before integrating a
game. It proves the service and network path without engine or game code. This
page documents only the seams that the diagnostic cannot choose for a game:
package/version policy, ownership of the gameplay socket, host admission,
metadata, credential custody, and deployment compatibility.
## Packages and compatibility
Consume `FinalFactory.Rendezvous.Client` and
`FinalFactory.Rendezvous.Contracts` from the approved Gitea NuGet source and pin
both to the same exact released version. Do not use a floating version range.
The current release matrix is machine-readable in
[`compatibility.json`](../releases/compatibility.json); the same window is
available from authenticated `GET /v1/operator/status`.
The authoritative package feed is
`https://git.finalfactory.de/api/packages/HeiKyu/nuget/index.json`. Add it to the
consumer's `NuGet.config` and map only Rendezvous packages to it; retain the
consumer's existing NuGet.org mapping for other dependencies:
```xml
<packageSources>
<add key="FinalFactory" value="https://git.finalfactory.de/api/packages/HeiKyu/nuget/index.json" />
</packageSources>
<packageSourceMapping>
<packageSource key="FinalFactory">
<package pattern="FinalFactory.Rendezvous.*" />
</packageSource>
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
</packageSourceMapping>
```
When the feed is anonymously readable, no reader credential is needed. If
registry policy requires authentication, use the platform's NuGet credential
provider or a protected per-user/CI NuGet configuration populated by the secret
manager. Never put a registry token in the project file, repository,
package-source URL, or `dotnet` command argument.
```xml
<ItemGroup>
<PackageReference Include="FinalFactory.Rendezvous.Client" Version="1.0.0" />
<PackageReference Include="FinalFactory.Rendezvous.Contracts" Version="1.0.0" />
</ItemGroup>
```
Run `dotnet restore`, then `dotnet list package --include-transitive` and verify
that Client and Contracts resolve to the same exact version and LiteNetLib to the
release matrix version before compiling the game.
Release 1.0.0 targets `netstandard2.1`, requires LiteNetLib `2.1.4`, speaks HTTP,
UDP, and connection-ticket contract version `1`, and requires an exact
tenant-configured gameplay protocol match. A package patch does not silently
change a wire version. Follow the [release and migration policy](../releases/README.md)
when changing any dimension, and validate the generated
[OpenAPI v1 document](../api/rendezvous-v1.json) rather than hand-building HTTP.
## One caller-owned gameplay socket
Create the game's LiteNetLib manager through `RendezvousNetListener`; do not open
a separate NAT socket. The game owns start, stop, and disposal. A coordinator
owns polling while it is active, so call its `Poll()` once from the game/network
thread and do not also call `NetManager.PollEvents()` during that period.
```csharp
RendezvousNetListener networkEvents = new();
NetManager gameplayNetwork = networkEvents.CreateManager();
if (!gameplayNetwork.Start(gameplayPort))
{
throw new InvalidOperationException("Gameplay UDP socket could not start.");
}
using RendezvousHostCoordinator host = new(
gameplayNetwork,
networkEvents,
mediatorEndPoint,
publishedSession,
joinClient);
host.Poll(); // call each game frame while this coordinator owns polling
```
Register normal game callbacks on `networkEvents.GameplayEvents`. Rendezvous
reserves only its authenticated direct requests and forwards other callbacks.
The same socket sends host presence, punches through the mediator, establishes
the peer, and then carries gameplay. A NAT introduction is not success; accept a
peer only after the coordinator reports the typed `Connected` outcome.
`Poll()` does not fetch new invitations. Schedule
`RefreshJoinAttemptsAsync` repeatedly for the entire hosting lifetime using a
bounded caller-owned timer (the diagnostic uses 250 ms), never allow two refreshes
to overlap, and inspect each typed result. The refresh performs HTTP work and
queues a snapshot; it does not call the LiteNetLib manager. Continue calling
`Poll()` on the manager's owning thread so the queued snapshot, presence traffic,
and callbacks are processed. Run the lease maintainer concurrently and cancel
both loops before disposing the coordinator.
For example, start one sequential refresh loop when hosting begins and await it
during shutdown:
```csharp
static async Task RefreshInvitationsAsync(
RendezvousHostCoordinator host,
CancellationToken cancellationToken)
{
using PeriodicTimer timer = new(TimeSpan.FromMilliseconds(250));
do
{
RendezvousClientResult<int> result =
await host.RefreshJoinAttemptsAsync(cancellationToken);
if (!result.IsSuccess)
{
ObserveBoundedHostRefreshFailure(result.Error);
}
}
while (await timer.WaitForNextTickAsync(cancellationToken));
}
```
On the joining side, create an attempt through `RendezvousJoinClient`, then give
the issued attempt to `RendezvousClientCoordinator` using the same manager and
listener. Cancellation, outcome reporting, bounded deadlines, fallback, and
lease-maintainer examples are in the packaged
[`FinalFactory.Rendezvous.Client` README](../../src/FinalFactory.Rendezvous.Client/README.md).
## Host admission remains game-owned
The coordinator privately validates and consumes the signed one-time connection
ticket before accepting the LiteNetLib transport request. Do not create a second
`ConnectionTicketValidator` beside it: the coordinator deliberately does not
expose the expected or presented ticket. A connected transport proves only that
Rendezvous authorized one attempt; it does not prove player identity,
entitlement, capacity, ban status, or gameplay compatibility.
Treat `AttemptCompleted` with a successful outcome and non-null `Peer` as the
start of game-owned admission. Keep that peer outside authoritative gameplay
until the game's normal authentication and admission exchange succeeds; disconnect
it on rejection or timeout:
```csharp
host.AttemptCompleted += (_, completed) =>
{
if (!completed.Outcome.IsSuccess || completed.Peer is null)
{
return;
}
BeginBoundedGameAuthentication(
completed.Peer,
onAccepted: AdmitToAuthoritativeGameplay,
onRejected: peer => peer.Disconnect());
};
```
Revoke an attempt when the game cancels it. Never log a ticket or capability. A
successful Rendezvous check must not bypass the game's authentication or
authoritative server rules. `ConnectionTicketValidator` is a lower-level
primitive for a custom transport integration that owns the complete request
acceptance path; it is not an extra gate for `RendezvousHostCoordinator`.
## Provision each game and environment
Provision game/environment scope before issuing credentials. The policy fixes
enabled regions, exact gameplay protocols, visibility and publisher trust modes,
metadata schema and byte budgets, quotas, and whether a dedicated fallback may
be published. Unknown or disabled scope fails closed. Follow
[game provisioning and signing-key lifecycle](../security/provisioning.md) for
the complete schema, principal kinds, secret providers, overlap, and revocation.
Dedicated publisher credentials belong only on trusted hosting infrastructure.
Never ship one in a player build, repository, image layer, appsettings file, URL,
argument, log, crash report, or analytics event. Issue a short-lived credential
scoped to one game/environment and its allowed regions from the trusted
deployment boundary. Player-host grants are issued to an authenticated player
session at runtime and are never embedded in the build. Player-host grants,
dedicated publishers, anonymous unlisted hosts, and operators are separate
principal kinds; do not interchange them.
Rotate signing keys with an overlap:
1. install a new authorized key inside its `NotBefore`/`SignUntil` window;
2. begin issuing with it while the old key remains verify-only;
3. wait at least the maximum credential lifetime plus allowed clock skew;
4. retire the old verifier after `VerifyUntil` and preserve custody records.
A suspected compromise is not routine rotation: stop issuance, revoke the exact
key through the protected operator route, remove or replace it in provisioning,
invalidate affected credentials, and follow the
[key-compromise runbook](../operations/incident-runbooks.md#signing-key-or-issuer-compromise).
## Metadata is public and policy-owned
Treat listing metadata as untrusted public input. Define a small allowlist in
each provisioned game's `MetadataValueMaxBytes`, set `RequiredMetadataKeys`, and
keep `MetadataMaxKeys` and `MetadataMaxBytes` to the smallest useful values. Values
must be display data only—for example a bounded map or ruleset identifier. Never
publish player identity, free-form chat, secrets, access tokens, internal
addresses, world state, or data needed for authoritative gameplay.
The platform contract caps metadata at 32 keys, 256 UTF-8 bytes per value, and
4096 encoded bytes total; tenant policy can and should be smaller. Build version
and display name are separately bounded public fields. Games must escape metadata
for their UI and must not infer trust from a listing being present.
## Local, staging, and production path
Use the checked-in Compose profile only for the local TestClient guide. For a
real environment:
1. provision the game/environment policy and externally held signing keys;
2. deploy one active service behind the source-preserving HTTPS/UDP topology in
[secure single-active Linux deployment](../deployment/linux.md);
3. install matching exact package versions in the game and set its service and
mediator endpoints through environment-specific configuration;
4. pass the TestClient health/publish/browse/punch/direct-traffic smoke using a
short-lived diagnostic credential;
5. run the topology harness and representative consumer-network trials; and
6. monitor typed outcomes and bounded metrics before broad rollout.
Rendezvous v1 has no relay, account system, matchmaking engine, server-browser
UI, gameplay authority, or durable session database. A game owns player-facing
recovery and an explicit fallback. Do not describe direct traversal as guaranteed.
+193 -68
View File
@@ -1,97 +1,222 @@
# Diagnostic TestClient integration guide
# Start-to-finish TestClient guide
Tracking: #25
Tracking: #20, #25
`FinalFactory.Rendezvous.TestClient` is the smallest supported public-SDK consumer.
It exists for integration development, CI smoke checks, deployment verification,
and operator diagnosis. It is intentionally not a production game client, game
server, matchmaking UI, or relay.
`FinalFactory.Rendezvous.TestClient` is the supported executable proof that a
consumer can publish, browse, authorize, punch, connect, exchange direct traffic,
and diagnose a failure using only the public Client and Contracts packages. It is
intentionally thin: a polished server browser and player-facing connection UI
belong in each game repository.
The automated scenario matrix, privileged Linux namespace run, and topology
limitations are documented in the [deterministic topology harness](topology-harness.md).
> **Traversal boundary:** Rendezvous v1 is not a relay and cannot guarantee a
> connection through symmetric NAT, carrier-grade NAT, restrictive firewalls,
> VPNs, or platform policy. It provides no accounts, social system, skill-based
> matchmaking, gameplay server, gameplay authority, or gameplay transport.
## Prerequisites
The automated scenario matrix, privileged Linux namespace run, and simulation
limits are in the [deterministic topology harness](topology-harness.md). The
[SDK seam guide](sdk-seams.md) covers the few integration details that this
executable cannot show.
Start a configured Rendezvous service and note both its HTTP base URL and UDP
mediator endpoint. The host needs a tenant-scoped publisher credential from the
deployment secret boundary. Put it in an environment variable and pass only that
variable's name when the default is unsuitable:
## First local connection from a clean checkout
Prerequisites are the pinned .NET SDK, Docker with Compose, OpenSSL, Python 3,
`curl`, and `jq`. Run these commands from the repository root. The generated key
and credential are disposable local fixtures, not production provisioning.
```bash
export RENDEZVOUS_PUBLISHER_CREDENTIAL='<deployment-supplied value>'
install -d -m 0700 deploy/compose/secrets
umask 077
openssl rand -out deploy/compose/secrets/signing-key 32
export RENDEZVOUS_UID="$(id -u)"
export RENDEZVOUS_GID="$(id -g)"
test "$RENDEZVOUS_UID" -ne 0
docker compose -f deploy/compose/compose.yaml up --build --detach
ready=false
for attempt in {1..45}; do
if curl --fail --silent http://127.0.0.1:8080/health/ready >/dev/null; then
ready=true
break
fi
sleep 1
done
test "$ready" = true
curl --fail http://127.0.0.1:8080/health/live
curl --fail http://127.0.0.1:8080/health/ready
dotnet build src/FinalFactory.Rendezvous.TestClient --configuration Release
```
Never put the credential in a command argument, URL, checked-in configuration,
shell trace, or captured test fixture. The development server's signing material
is process-ephemeral; credentials from a prior development process are invalid.
## Manual three-terminal flow
Start the host:
Only the host terminal needs a publisher credential. Disable shell tracing before
capturing it; the helper prints the credential on stdout so command substitution
can place it directly in the environment without writing it to disk.
```bash
dotnet run --project src/FinalFactory.Rendezvous.TestClient -- \
host --service http://127.0.0.1:5000/ --mediator 127.0.0.1:9050 \
--game space-game --environment development --region local --protocol 1
set +x
export RENDEZVOUS_PUBLISHER_CREDENTIAL="$(./scripts/mint-local-publisher-credential.sh)"
```
Browse from another terminal:
The helper accepts no arguments, reads the ignored `0600` local Compose key, and
mints only `space-game` / `smoke` / `local` / protocol `1` for ten minutes. It is
not a reusable issuer or an example for production. Never put the result in a
command argument, URL, shell history, log, screenshot, support ticket, captured
fixture, or source file.
In terminal 1, publish a host. It stays alive for at most 60 seconds and exits
after a joining peer completes the authenticated echo exchange:
```bash
dotnet run --project src/FinalFactory.Rendezvous.TestClient -- \
browse --service http://127.0.0.1:5000/ \
--game space-game --environment development --region local --protocol 1
dotnet run --project src/FinalFactory.Rendezvous.TestClient \
--configuration Release --no-build -- \
host --service http://127.0.0.1:8080/ --mediator 127.0.0.1:9050 \
--game space-game --environment smoke --region local --protocol 1 \
--display-name "Local diagnostic" --timeout-seconds 60 --run-seconds 60 \
--exit-after-echo
```
Join from a third terminal. Omit `--listing` for an interactive choice:
Copy the public listing ID printed by the host, or discover it from terminal 2:
```bash
dotnet run --project src/FinalFactory.Rendezvous.TestClient -- \
join --service http://127.0.0.1:5000/ --mediator 127.0.0.1:9050 \
--game space-game --environment development --region local --protocol 1 \
--listing 00000000-0000-0000-0000-000000000000
dotnet run --project src/FinalFactory.Rendezvous.TestClient \
--configuration Release --no-build -- \
browse --service http://127.0.0.1:8080/ \
--game space-game --environment smoke --region local --protocol 1
```
Replace the sample UUID with the public listing ID printed by host or browse.
Host and join each create one caller-owned LiteNetLib manager. That same socket
sends presence/punch traffic, establishes the authenticated direct connection,
and carries the ping/echo/ack/completion payload. The final completion confirms
that the host received the reliable acknowledgement; none of this traffic passes through the HTTP
service or UDP mediator.
In terminal 3, either omit `--listing` and select interactively, or provide the
copied ID for deterministic selection:
## CI and deployment smoke flow
```bash
dotnet run --project src/FinalFactory.Rendezvous.TestClient \
--configuration Release --no-build -- \
join --service http://127.0.0.1:8080/ --mediator 127.0.0.1:9050 \
--game space-game --environment smoke --region local --protocol 1 \
--listing REPLACE_WITH_LISTING_UUID --timeout-seconds 30
```
Use `--script --json`, set `--listing` when deterministic selection matters, and
check the documented process exit code. `--timeout-seconds` bounds each startup,
traversal, or direct-traffic stage; a script host also uses it as its total runtime
unless `--run-seconds` is explicit. A host can add `--exit-after-echo` so it
terminates after the joining peer acknowledges direct traffic and receives the
host's completion confirmation. Every wait is
bounded by coordinator state and `--timeout-seconds`; no orchestration should use
an unbounded sleep.
Success means the joiner prints `join.connected` and verified direct traffic,
and the host prints verified direct traffic before deregistering. The host and
joiner each create one caller-owned LiteNetLib manager. The same UDP socket sends
presence and punch traffic, accepts the authenticated peer, and carries the
ping/echo/ack/completion payload; direct traffic does not pass through the HTTP
service or mediator.
The normal test suite contains a real process gate that starts the built Server,
host TestClient, and join TestClient, waits for readiness and versioned events,
and verifies direct traffic, cleanup, JSON shape, and secret canaries. Process
trees are force-terminated in the test cleanup path if normal shutdown fails.
Clean up secrets and the disposable service when finished:
Useful success events are:
```bash
unset RENDEZVOUS_PUBLISHER_CREDENTIAL
docker compose -f deploy/compose/compose.yaml down
rm deploy/compose/secrets/signing-key
```
- `host.registered`, `host.ready`, `host.direct-traffic`, and `host.deregistered`;
- `browse.completed` and `browse.session`; and
- `join.connected`, `join.direct-traffic`, and `join.outcome-report`.
## Script and JSON automation
Failure events preserve stable typed phases and outcomes. When a terminal outcome
contains a configured dedicated endpoint, `join.fallback` reports `available`
with endpoint type `dedicated`; no raw address is printed and no fallback is
started implicitly.
`--script` forbids prompts and selects the first compatible listing unless
`--listing UUID` fixes the choice. `--json` emits one JSON object per line with
`version: 1`. New optional properties may be added, but event names and exit
codes are stable automation contracts. Informational events use stdout and
failures use stderr.
## What the proof does and does not establish
The deployment smoke performs the full health, publish, join, mediation, direct
traffic, outcome-report, and cleanup flow using bounded waits:
The deterministic loopback test proves the complete service/host/client protocol,
ticket admission, and peer-to-peer payload path. Loopback is not evidence that all
consumer routers, carrier-grade NATs, symmetric NATs, firewalls, VPNs, IPv6 paths,
or platform policies permit hole punching. Same-LAN, separated observed endpoints,
network namespaces/containers, mediator restart, and adverse topology coverage
belong to the topology harness tracked by #14. Production rollout still requires
tests from representative networks and a game-owned fallback policy.
```bash
dotnet build src/FinalFactory.Rendezvous.TestClient --configuration Release
./scripts/smoke-deployment.sh
```
For custom automation, capture JSON and preserve the process status separately:
```bash
set +e
dotnet run --project src/FinalFactory.Rendezvous.TestClient \
--configuration Release --no-build -- \
browse --service http://127.0.0.1:8080/ \
--game space-game --environment smoke --region local --protocol 1 \
--script --json >browse.jsonl
status=$?
set -e
jq -e 'select(.version == 1 and .event == "browse.completed")' browse.jsonl
test "$status" -eq 0
```
Never use an unbounded sleep to orchestrate processes. Wait for versioned events
such as `host.ready` and apply a deadline. Useful success events are
`host.registered`, `host.ready`, `host.direct-traffic`, `host.deregistered`,
`browse.completed`, `browse.session`, `join.connected`, `join.direct-traffic`,
and `join.outcome-report`.
| Exit | Meaning |
| ---: | --- |
| `0` | Requested diagnostic flow completed successfully |
| `2` | Invalid command or options |
| `3` | Missing or invalid local configuration |
| `10` | HTTP, registration, browser, lease, or socket failure |
| `11` | No compatible session was available or selected |
| `12` | Authorization or traversal reached a typed terminal failure |
| `13` | Direct connection succeeded but the direct traffic proof failed |
| `130` | Caller cancellation or Ctrl+C |
## Observe a safe failure
Run this after the protocol-1 browse in terminal 2 and before the terminal-3
join (or restart terminal 1 first). The preceding browse proves that one
protocol-1 host is present. Now browse for deliberately incompatible protocol
`999`. The command emits a successful directory response with
`browse.completed`, `count: 0`, then exits `11` to distinguish compatibility
from a service outage:
```bash
set +e
dotnet run --project src/FinalFactory.Rendezvous.TestClient \
--configuration Release --no-build -- \
browse --service http://127.0.0.1:8080/ \
--game space-game --environment smoke --region local --protocol 999 \
--script --json >incompatible.jsonl
status=$?
set -e
jq -e 'select(.event == "browse.completed" and .phase == "directory" and .count == 0)' \
incompatible.jsonl
test "$status" -eq 11
```
This is a diagnostic failure drill, not a bypass: unknown tenant scope and
protocols still fail closed, and the local helper cannot mint a credential for
them.
## Diagnose by phase, not by guesswork
Start with the exit code, then the last versioned event and its `phase`, `status`,
and typed `outcome`. Endpoint categories may be
reported as `loopback`, `private`, or `public`; raw endpoints, credentials,
capabilities, metadata, and player identities are never emitted.
| Symptom or last event | Distinction | Check next |
| --- | --- | --- |
| `host.configuration`, exit `3` | Local credential variable is missing or malformed before any request | Confirm the named environment variable exists, tracing is off, and the credential has not expired |
| `host.registration`, exit `10` | Publisher authentication, tenant policy, metadata, quota, or HTTP failure | Use the typed status; compare credential scope with game/environment/region and the provisioned policy, then correlate protected server telemetry by operation and time |
| `browse.sessions`, exit `10` | Directory request failed | Check HTTP reachability, `/health/ready`, rate limiting, and contract compatibility |
| `browse.completed` count `0`, or `join.selection` empty, exit `11` | Healthy directory but no compatible visible listing | Match game, environment, region, and exact gameplay protocol; then confirm a host lease is still active |
| Exact `join.selection` failure, exit `10` | Listing disappeared, is hidden, or scope no longer matches | Browse again; do not retry an old listing ID forever |
| `join.authorization`, exit `12` | Service rejected the attempt before NAT traversal | Inspect typed category/outcome for policy, capacity, stale host, or active-attempt limits |
| `join.punch` / `join.traversal`, exit `12` | Mediation or NAT traversal did not establish a peer | Confirm UDP endpoint/reply path, host presence, clocks, firewall/NAT behavior, and topology; use a game-owned fallback if policy supplies one |
| `join.direct-connect`, exit `12` | Introduction occurred but authenticated direct admission failed | Confirm host is polling the same socket, the one-time ticket is current, and game admission did not reject capacity, identity, or bans |
| `join.connected` followed by exit `13` | Peer connected but the direct gameplay-like echo did not finish | Inspect the peer lifecycle and caller polling; this is not an HTTP/directory failure |
Stopping a host without deregistration may leave its listing visible only until
the bounded lease expires. During that window, a join can produce a typed stale
host or traversal outcome; it must not be interpreted as a healthy host. Restarting
the single-active service intentionally loses all ephemeral listings and attempts,
so hosts re-register and clients browse again.
If a terminal outcome reports an authoritative dedicated fallback,
`join.fallback` exposes only availability and endpoint type. TestClient never
connects to it automatically. The game owns the decision, authentication, and
connection policy. If no fallback is present, Rendezvous v1 offers no relay.
## Production use
Do not copy a production signing key to a diagnostic host. Supply a short-lived,
least-scope publisher credential from the deployment secret boundary and set the
external service, mediator, and matching scope variables described in the
[secure Linux deployment smoke](../deployment/linux.md#http-and-udp-smoke).
Run representative external-network tests; loopback success is not NAT coverage.