Rendezvous/docs/deployment/linux.md

Secure single-active Linux deployment

Tracking: #17

Rendezvous v1 stores listings, observed endpoints, join attempts, replay markers, and runtime revocations only in the process that accepted them. Deploy exactly one active instance. A second live replica would have a different directory and replay boundary; SingleActiveInstance=false is therefore rejected rather than presented as high availability.

Pinned container

The root Dockerfile uses a multi-stage .NET 10 build and pins both Microsoft base images by multi-architecture manifest digest. The runtime is the chiseled ASP.NET image, contains only the published server, runs as UID/GID 1654, exposes TCP 8080 and UDP 9050 explicitly, and does not require a writable application directory. Supply a small writable /tmp tmpfs because runtime libraries can legitimately need temporary space; keep the root filesystem read-only.

From a clean checkout:

docker build --pull=false --tag finalfactory/rendezvous:local .
docker inspect --format '{{.Config.User}}' finalfactory/rendezvous:local

The reported user must be 1654:1654. Digest pins make a rebuild reproducible; updating .NET is an explicit reviewed change to the tag, digest, SDK pin, and lock files together. Do not replace the digest with latest in production.

The local Compose example applies a read-only root, non-root user, no Linux capabilities, no-new-privileges, bounded PIDs/files/memory/CPU, and a shutdown grace period longer than the service drain deadline:

install -d -m 0700 deploy/compose/secrets
umask 077
openssl rand -out deploy/compose/secrets/signing-key 32
export RENDEZVOUS_UID="$(id -u)"
export RENDEZVOUS_GID="$(id -g)"
test "$RENDEZVOUS_UID" -ne 0
docker compose -f deploy/compose/compose.yaml up --build --detach

deploy/compose/appsettings.Production.json is an isolated loopback smoke profile, not an Internet template: it deliberately opts into private advertised endpoints and has no TLS proxy. Its random key is ignored by Git and must be deleted after use. Its deliberately long key window only keeps this disposable local fixture usable; production keys require short, reviewed rotation windows. Production configuration must use its real public names and must leave AllowPrivatePublicEndpoints false.

Production topology

Use one active service behind a source-preserving edge:

clients -- HTTPS/443 --> TLS reverse proxy -- HTTP/8080 --> Rendezvous
clients -- UDP/9050 -------------------------------------> Rendezvous

• Give the HTTPS origin and UDP endpoint stable DNS names. Set PublicHttpBaseUrl to the exact external HTTPS origin and PublicUdpHost / PublicUdpPort to the endpoint given to game clients.
• Terminate TLS 1.2 or newer at a maintained reverse proxy. Bind internal HTTP only to the private proxy network. Restrict AllowedHosts to the public HTTP host; wildcard host filtering is rejected.
• Put only the proxy's exact literal addresses in Rendezvous:AbuseProtection:TrustedProxyAddresses. Rendezvous ignores forwarded headers from every other source. Keep the last proxy from replacing the original client address and prevent direct access to TCP 8080.
• Forward UDP as UDP, without an HTTP proxy. NAT, load balancer, firewall, and return routing must preserve the client's source IP/port and must send replies from the same advertised IP/port. Many HTTP load balancers, Kubernetes ingress controllers, rootless container port proxies, anycast products, and generic L7 services cannot guarantee this. Do not deploy through one unless an actual host/join smoke proves both observed source and reply path. A load balancer must have exactly one healthy Rendezvous target.
• Permit inbound TCP 443 to the TLS proxy and UDP 9050 to Rendezvous. Permit the proxy to reach TCP 8080. Permit DNS, time synchronization, image/telemetry destinations as required by local policy, and UDP replies to client endpoints. Deny public TCP 8080 and every unused inbound port.

Readiness is the load-balancer gate; liveness is only a process-health signal. Remove a draining instance from new traffic when /health/ready becomes 503. Do not use liveness failure to start a second active process while the old one still owns the public UDP address.

Required production configuration

Production startup validates all of these before binding listeners:

• an absolute path-free HTTPS PublicHttpBaseUrl;
• an unambiguous public PublicUdpHost and port;
• SingleActiveInstance=true, an explicit non-wildcard AllowedHosts, and at least one exact trusted TLS-proxy address;
• a 1-30 second drain deadline whose minimum observation interval is shorter;
• at least one enabled game policy and an active scoped signing key.

Missing values produce an actionable startup error. The checked-in base file is intentionally unsafe for Production so an accidental bare launch fails closed.

Signing keys support two external references:

• env:NAME reads 1-4096 bytes encoded as base64 from NAME;
• file:/absolute/path reads 1-4096 raw bytes from a non-symlink file.

Prefer a read-only container secret owned by the configured container identity. For systemd, use a root-owned, rendezvous-group-owned 0440 file (or an equivalent narrow ACL) so the non-root process can read but not replace it. A signing key must contain at least 32 random bytes. Never put the key, publisher/operator credential, or secret value in JSON, a command argument, an image layer, Compose environment, logs, metrics, or source control. Configuration contains only the reference and non-secret lifecycle metadata. A vault/KMS adapter can replace the provider where local policy requires it.

Keep the host clock synchronized with authenticated NTP. Credential and key windows use wall time; lease, timeout, drain, and rate-limit deadlines use a monotonic clock. Alert on clock synchronization loss before rotating keys.

The checked-in Compose limits (one CPU and 512 MiB) are for its isolated smoke profile, not a production capacity claim. The measured core-state candidate uses 2 vCPU and 2 GiB with the same 128-PID/4096-descriptor ceilings; see the capacity and resilience gate. Measure real traffic, then change resource limits and server budgets together. Memory pressure or CPU throttling must not extend orchestrator termination past DrainDeadlineSeconds plus five seconds.

Graceful shutdown

SIGTERM and the authenticated operator drain both stop new registrations and join attempts immediately. On process shutdown, HTTP and UDP remain available long enough for existing join attempts to finish. The service exits as soon as the minimum drain interval has elapsed and no attempts remain, or forcibly clears all ephemeral state at the configured deadline. It then stops UDP and HTTP listeners and exits. Configure Docker/systemd/Kubernetes termination grace strictly longer than the service deadline; the examples use 40 seconds for a 30-second drain.

Never use SIGKILL for a normal rollout. After stopping, verify the process is gone and neither 8080/tcp nor 9050/udp is bound before starting its replacement on the same host. A crashed or force-killed process cannot drain; clients recover through bounded retries and hosts re-register.

systemd alternative

Publish the server for Linux, install the immutable output at /opt/rendezvous, place production configuration beside the application read-only, place key files below /etc/rendezvous, and install deploy/systemd/rendezvous.service:

dotnet publish src/FinalFactory.Rendezvous.Server \
  --configuration Release --runtime linux-x64 --self-contained false \
  --output publish/rendezvous
systemd-analyze verify deploy/systemd/rendezvous.service
sudo systemctl daemon-reload
sudo systemctl enable --now rendezvous.service

Create the dedicated rendezvous user without a login shell. Keep /opt/rendezvous and /etc/rendezvous root-owned and non-writable by that user; install each required key with root:rendezvous ownership and mode 0440. The unit applies the measured 2-vCPU/2-GiB core-state candidate profile plus the same filesystem, privilege, network-family, and shutdown hardening as Compose.

HTTP and UDP smoke

Build the diagnostic once, then exercise the actual published HTTP and UDP paths. The test creates a public listing, sends authenticated presence and punch traffic through UDP 9050, establishes peer-to-peer traffic, reports the outcome, and deregisters cleanly:

dotnet build src/FinalFactory.Rendezvous.TestClient --configuration Release
./scripts/smoke-deployment.sh

For the local Compose profile, the script derives a ten-minute diagnostic publisher credential from the ignored local key without printing either secret. The fixed-scope helper used by the smoke can also support the manual TestClient local flow; it is deliberately not a production issuer. For production, do not copy the signing key to the smoke host. Instead inject a short-lived, region-scoped credential through RENDEZVOUS_PUBLISHER_CREDENTIAL, and set the external endpoints:

export RENDEZVOUS_PUBLISHER_CREDENTIAL='<short-lived deployment credential>'
export RENDEZVOUS_SMOKE_HTTP_URL='https://rendezvous.your-company.tld/'
export RENDEZVOUS_SMOKE_UDP_ENDPOINT='rendezvous-udp.your-company.tld:9050'
export RENDEZVOUS_SMOKE_GAME_ID='<credential game ID>'
export RENDEZVOUS_SMOKE_ENVIRONMENT_ID='<credential environment ID>'
export RENDEZVOUS_SMOKE_REGION='<credential region>'
export RENDEZVOUS_SMOKE_PROTOCOL_VERSION='<enabled protocol version>'
./scripts/smoke-deployment.sh

Those four scope values must match both the short-lived credential and an enabled server policy. The defaults (space-game, smoke, local, protocol 1) are only for the checked-in local Compose profile.

The smoke fails unless both health endpoints and the complete authenticated UDP mediation/direct-traffic flow succeed. It does not prove every consumer NAT; run the topology harness and representative external-network tests as well.

Restart, upgrade, rollback, and backup

Rendezvous has no durable runtime database. Restarting intentionally loses all listings, observed endpoints, attempts, replay markers, and runtime-only revocations. Hosts must treat registration as a renewable lease and re-register after service recovery. Clients must re-browse and start a new bounded attempt.

Back up only reviewed configuration, policy, secret references, key material and its custody/lifecycle records, deployment manifests, and image digest. Never claim a backup contains live sessions or endpoints. Restore keys only through the secret system, not into the image or repository.

For an upgrade:

1. Build and test the new pinned digest; validate configuration without starting a second active instance.
2. Drain and stop the current process, verify both sockets are released, then start the replacement on the same public endpoints.
3. Require live/readiness and HTTP+UDP smoke success; monitor host re-registration, error rate, and direct-connect outcomes.

For rollback, repeat the same stop-before-start sequence with the previously recorded image digest and compatible configuration/key set. Never run old and new versions concurrently to avoid split ephemeral state. If a wire-incompatible change ever becomes necessary, use a new API/protocol version rather than a rolling two-version replica set.