diff --git a/docs/deploy/topologies.md b/docs/deploy/topologies.md new file mode 100644 index 0000000..a69e550 --- /dev/null +++ b/docs/deploy/topologies.md @@ -0,0 +1,105 @@ +# Deployment topologies — one binary, three blessed shapes + +The FOB never decomposes; topology is a configuration property, not a code property. The same +binary runs in every shape below; only `RUTSTER_*` env and what is deployed alongside it +differ. Ratified (Proposed) in [ADR-0011](../adr/0011-deployment-topology.md). + +## Decision guide + +| You are… | Use | Doc | +|---|---|---| +| Trying rutster for the first time; one box, one domain, a public IP | **T1 — Solo** | [quickstart-docker.md](quickstart-docker.md) | +| Running production and want independently upgradable parts | **T2 — Modular** (the reference deployment) | [quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack) | +| Behind CGNAT / no public IP (homelab) | T1/T2 + tunnel (dev, **PSTN-only**) or VPS forwarder (production) | [homelab.md](homelab.md) | +| Already running nginx / HAProxy / Traefik | T2 minus the `caddy` service | [reverse-proxies.md](reverse-proxies.md) | +| On EC2 and want ALB + ACM to terminate TLS | T2/T3 behind an ALB | [aws.md](aws.md) | +| Outgrowing one node | **T3 — Fleet** — **paper only today** | [ADR-0011](../adr/0011-deployment-topology.md) | + +## T1 — Solo (all-in-one container) + +One image, `rutster-allinone`, with s6-overlay supervising four processes: + +| Process | Role | Binding | +|---|---|---| +| `caddy` | Edge: ACME issuance/renewal, TLS termination, WS proxy | `:443` / `:80` public | +| `rutster` (the FOB) | Engine — signaling, trunk webhook, media, reflexes | `127.0.0.1:8080` behind Caddy; **media UDP direct** | +| `rutster-brain-realtime` | Brain | `127.0.0.1:8082` loopback tap | +| `valkey-server` | Event stream (`EventSink`); ledger/directory later | `127.0.0.1:6379` | + +Two volumes, **both non-negotiable**: + +- `/data` — Caddy's cert/ACME state. Losing it can lock your domain out of Let's Encrypt for + up to a week (duplicate-certificate limit) — a **total inbound outage**. See + [certificates.md](certificates.md). +- `/var/lib/valkey` — stream/state persistence. + +WebRTC media (UDP) goes **direct to the FOB, never through Caddy**: run with host networking, +or publish `RUTSTER_MEDIA_PORT_RANGE` and set `RUTSTER_MEDIA_ADVERTISED_IP` to your public IP +(NAT 1:1 model — no STUN). + +Copy-paste path from zero to first call: [quickstart-docker.md](quickstart-docker.md). + +## T2 — Modular (compose stack) — the reference deployment + +Four services in `deploy/compose.yaml`: + +| Service | Image | Notes | +|---|---|---| +| `caddy` | `rutster-edge` | Same custom Caddy build + same Caddyfile as T1 | +| `engine` | `rutster-engine` | The FOB; plaintext `:8080` on the compose network; media UDP published | +| `brain` | `rutster-brain` | `network_mode: "service:engine"` — shares the engine's netns so the loopback-only tap posture survives unchanged | +| `valkey` | `valkey/valkey` (upstream) | Own service; network-near | + +Operational contract: + +- `stop_grace_period: 660s` paired with `RUTSTER_DRAIN_DEADLINE_SECS=600` — grace **must** + exceed drain, or the orchestrator kills calls the engine was gracefully draining. +- Bring-your-own-proxy: disable the `caddy` service; the engine keeps its plaintext `:8080` + mode. Tuned-timeout configs: [reverse-proxies.md](reverse-proxies.md). +- Upgrades: `docker compose pull && docker compose up -d` — the drain window lets in-flight + calls finish (calls are non-migratable; a killed engine kills its calls, honestly stated). + +Copy-paste path from zero to first call: +[quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack). + +## T3 — Fleet (paper only — no shipped artifacts in this epoch) + +**Honest statement: you cannot deploy T3 today by copy-paste; there are no fleet artifacts +yet.** The design is ratified on paper in +[ADR-0011](../adr/0011-deployment-topology.md) so single-node deployments grow toward it, not +away from it: + +- N symmetric nodes (the T1 process set minus valkey) + a shared green zone: one Valkey + (presence + directory + spend ledger), object storage. +- Wildcard DNS `*.pbx.domain`; per-node certs — the traps live in + [certificates.md](certificates.md). +- Answer-time placement via Valkey presence; node-addressed `wss://node-K.pbx.domain/...` + Stream URLs; no router tier. +- Per-node admission cap = the benchmark number (`RUTSTER_MAX_SESSIONS`, default 64, is a + placeholder until then). +- Scale-in is drain-then-terminate only. No spot instances for the engine tier. + +## What is deliberately not supported + +- **Splitting the FOB into services** — permanent rejection, not a roadmap item. +- **Serverless** — non-migratable multi-hour calls, per-session UDP sockets, in-process media + state. Categorically unfit. +- **Tunnels in production** — plaintext audio at the tunnel vendor's edge + documented + mid-call WS kills. Dev/demo only; the whole story is in [homelab.md](homelab.md). +- **NLB TLS listeners** — see [aws.md](aws.md) for the 350-second reason. +- **k8s manifests/Helm** — compose-first. If you run k8s anyway: set + `terminationGracePeriodSeconds` above `RUTSTER_DRAIN_DEADLINE_SECS`, same grace-exceeds-drain + rule as compose. + +## Ports & volumes reference (all shapes) + +| Surface | Where | Notes | +|---|---|---| +| `:80` / `:443` TCP | Caddy, public | ACME HTTP-01 + TLS + all HTTPS/WSS traffic | +| `:8080` TCP | FOB, behind the edge | Signaling + `/v1/trunk/webhook` + `/twilio/media-stream` WS — one listener (`RUTSTER_HTTP_BIND`) | +| `RUTSTER_MEDIA_PORT_RANGE` UDP | FOB, **public, direct** | WebRTC media; never proxied | +| `:8082` TCP | Brain, loopback only | The tap; non-loopback rejected until step 6 | +| `:6379` TCP | Valkey, private | Loopback (T1) / compose network (T2) / green zone (T3) | +| `:9090` TCP | FOB `/metrics`, internal | `RUTSTER_METRICS_BIND`; never routed through Caddy | +| `/data` volume | Caddy | Cert/ACME state — loss class: total inbound outage | +| `/var/lib/valkey` volume | Valkey | Event stream persistence |