rutster/docs/deploy/topologies.md

# Deployment topologies — one binary, three blessed shapes

The FOB never decomposes; topology is a configuration property, not a code property. The same
binary runs in every shape below; only `RUTSTER_*` env and what is deployed alongside it
differ. Ratified (Proposed) in [ADR-0011](../adr/0011-deployment-topology.md).

## Decision guide

| You are… | Use | Doc |
|---|---|---|
| Trying rutster for the first time; one box, one domain, a public IP | **T1 — Solo** | [quickstart-docker.md](quickstart-docker.md) |
| Running production and want independently upgradable parts | **T2 — Modular** (the reference deployment) | [quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack) |
| Behind CGNAT / no public IP (homelab) | T1/T2 + tunnel (dev, **PSTN-only**) or VPS forwarder (production) | [homelab.md](homelab.md) |
| Already running nginx / HAProxy / Traefik | T2 minus the `caddy` service | [reverse-proxies.md](reverse-proxies.md) |
| On EC2 and want ALB + ACM to terminate TLS | T2/T3 behind an ALB | [aws.md](aws.md) |
| Outgrowing one node | **T3 — Fleet** — **paper only today** | [ADR-0011](../adr/0011-deployment-topology.md) |

## T1 — Solo (all-in-one container)

One image, `rutster-allinone`, with s6-overlay supervising four processes:

| Process | Role | Binding |
|---|---|---|
| `caddy` | Edge: ACME issuance/renewal, TLS termination, WS proxy | `:443` / `:80` public |
| `rutster` (the FOB) | Engine — signaling, trunk webhook, media, reflexes | `127.0.0.1:8080` behind Caddy; **media UDP direct** |
| `rutster-brain-realtime` | Brain | `127.0.0.1:8082` loopback tap |
| `valkey-server` | Event stream (`EventSink`); ledger/directory later | `127.0.0.1:6379` |

Two volumes, **both non-negotiable**:

- `/data` — Caddy's cert/ACME state. Losing it can lock your domain out of Let's Encrypt for
  up to a week (duplicate-certificate limit) — a **total inbound outage**. See
  [certificates.md](certificates.md).
- `/var/lib/valkey` — stream/state persistence.

WebRTC media (UDP) goes **direct to the FOB, never through Caddy**: run with host networking,
or publish `RUTSTER_MEDIA_PORT_RANGE` and set `RUTSTER_MEDIA_ADVERTISED_IP` to your public IP
(NAT 1:1 model — no STUN).

Copy-paste path from zero to first call: [quickstart-docker.md](quickstart-docker.md).

## T2 — Modular (compose stack) — the reference deployment

Four services in `deploy/compose.yaml`:

| Service | Image | Notes |
|---|---|---|
| `caddy` | `rutster-edge` | Same custom Caddy build + same Caddyfile as T1 |
| `engine` | `rutster-engine` | The FOB; plaintext `:8080` on the compose network; media UDP published |
| `brain` | `rutster-brain` | `network_mode: "service:engine"` — shares the engine's netns so the loopback-only tap posture survives unchanged |
| `valkey` | `valkey/valkey` (upstream) | Own service; network-near |

Operational contract:

- `stop_grace_period: 660s` paired with `RUTSTER_DRAIN_DEADLINE_SECS=600` — grace **must**
  exceed drain, or the orchestrator kills calls the engine was gracefully draining.
- Bring-your-own-proxy: disable the `caddy` service; the engine keeps its plaintext `:8080`
  mode. Tuned-timeout configs: [reverse-proxies.md](reverse-proxies.md).
- Upgrades: `docker compose pull && docker compose up -d` — the drain window lets in-flight
  calls finish (calls are non-migratable; a killed engine kills its calls, honestly stated).

Copy-paste path from zero to first call:
[quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack).

## T3 — Fleet (paper only — no shipped artifacts in this epoch)

**Honest statement: you cannot deploy T3 today by copy-paste; there are no fleet artifacts
yet.** The design is ratified on paper in
[ADR-0011](../adr/0011-deployment-topology.md) so single-node deployments grow toward it, not
away from it:

- N symmetric nodes (the T1 process set minus valkey) + a shared green zone: one Valkey
  (presence + directory + spend ledger), object storage.
- Wildcard DNS `*.pbx.domain`; per-node certs — the traps live in
  [certificates.md](certificates.md).
- Answer-time placement via Valkey presence; node-addressed `wss://node-K.pbx.domain/...`
  Stream URLs; no router tier.
- Per-node admission cap = the benchmark number (`RUTSTER_MAX_SESSIONS`, default 64, is a
  placeholder until then).
- Scale-in is drain-then-terminate only. No spot instances for the engine tier.

## What is deliberately not supported

- **Splitting the FOB into services** — permanent rejection, not a roadmap item.
- **Serverless** — non-migratable multi-hour calls, per-session UDP sockets, in-process media
  state. Categorically unfit.
- **Tunnels in production** — plaintext audio at the tunnel vendor's edge + documented
  mid-call WS kills. Dev/demo only; the whole story is in [homelab.md](homelab.md).
- **NLB TLS listeners** — see [aws.md](aws.md) for the 350-second reason.
- **k8s manifests/Helm** — compose-first. If you run k8s anyway: set
  `terminationGracePeriodSeconds` above `RUTSTER_DRAIN_DEADLINE_SECS`, same grace-exceeds-drain
  rule as compose.

## Ports & volumes reference (all shapes)

| Surface | Where | Notes |
|---|---|---|
| `:80` / `:443` TCP | Caddy, public | ACME HTTP-01 + TLS + all HTTPS/WSS traffic |
| `:8080` TCP | FOB, behind the edge | Signaling + `/v1/trunk/webhook` + `/twilio/media-stream` WS — one listener (`RUTSTER_HTTP_BIND`) |
| `RUTSTER_MEDIA_PORT_RANGE` UDP | FOB, **public, direct** | WebRTC media; never proxied |
| `:8082` TCP | Brain, loopback only | The tap; non-loopback rejected until step 6 |
| `:6379` TCP | Valkey, private | Loopback (T1) / compose network (T2) / green zone (T3) |
| `:9090` TCP | FOB `/metrics`, internal | `RUTSTER_METRICS_BIND`; never routed through Caddy |
| `/data` volume | Caddy | Cert/ACME state — loss class: total inbound outage |
| `/var/lib/valkey` volume | Valkey | Event stream persistence |