Plan A: engine hygiene (TCP_NODELAY via axum 0.7.9 Serve::tcp_nodelay, trunk WS pings, webhook-base TwiML derivation, trusted-proxy posture). Plan B: packaging + CI (four images, s6 all-in-one, compose, Caddyfile, smoke suite incl. reload-during-call, slice-F publish workflow). Plan C: in-binary features (rustls Phase 1 BYO-cert, /metrics, ValkeyEventSink + smoke hook fill). Plan G: docs/deploy tree + ADR-0011 drafting. Authored by parallel plan-writers (this session + omo rescue after a session-limit interruption); verified: placeholder scan, cited-path existence, cross-plan image/env-var consistency, seam-gate compliance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012QndwfhjyTiZcUYp87dwW8 Signed-off-by: Aaron D. Lee <himself@adlee.work>
1611 lines
76 KiB
Markdown
1611 lines
76 KiB
Markdown
# Slice G — `docs/deploy/` tree + ADR-0011 (deployment topology) — Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use `superpowers:subagent-driven-development` (recommended) or `superpowers:executing-plans` to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Draft ADR-0011 (deployment topology, status Proposed) from spec §2–§4 and ship the
|
||
six-file `docs/deploy/` documentation tree, where **every topology doc contains a copy-paste
|
||
path from zero to first call — or the honest statement of what cannot work there** (spec §10
|
||
acceptance bar), then point `docs/QUICKSTART.md` at the new Docker fast path.
|
||
|
||
**Architecture:** Docs-only slice — no code, no artifacts. ADR-0011 is drafted verbatim-faithful
|
||
to the [deployment-topology spec](../specs/2026-07-05-deployment-topology-design.md) §2 (three
|
||
shapes + rejected shapes), §3 (edge doctrine incl. the rustls Phase-2 triggers verbatim), §4
|
||
(node-addressed placement), in the house ADR format established by ADR-0007/ADR-0008
|
||
(bullet-list header → Context → Decision → Consequences → References, heavy relative
|
||
cross-linking). The `docs/deploy/` tree sources every factual claim from the spec + the
|
||
[TLS/edge decision brief](../specs/2026-07-05-tls-edge-decision-brief.md) and cites the brief's
|
||
external URLs wherever a doc asserts a third-party fact.
|
||
|
||
**Tech Stack:** Markdown only. No new dependencies, no build steps. Verification is
|
||
`bash` content-greps + a relative-link resolver check per task.
|
||
|
||
## Global Constraints
|
||
|
||
House rules (every task's requirements implicitly include this section):
|
||
|
||
- **License:** GPL-3.0-or-later on every crate manifest (ADR-0004). No crates are touched in
|
||
this slice; the rule stands for any task that would.
|
||
- **DCO:** every commit signed off — `git commit -s` (AGENTS.md Git workflow).
|
||
- **SEAM GATE:** `crates/rutster-media/src/loop_driver.rs` and
|
||
`crates/rutster-media/src/rtc_session.rs` stay byte-identical (CI pinned-blob). **NO task may
|
||
touch them.** This slice touches only `docs/` — trivially satisfied, still checked in the
|
||
final sweep.
|
||
- **Hot-path policy:** never `?`-propagate on the 20 ms tick; no `unwrap()`/`expect()` outside
|
||
tests. (No code in this slice; stated because every plan carries it.)
|
||
- **Code style:** `cargo fmt --all --check` + `cargo clippy --all --all-targets -- -D warnings`
|
||
must stay green (docs changes cannot break them; run them in the final sweep anyway).
|
||
- **MSRV:** CI matrix runs stable + 1.85; docs must not instruct anything that contradicts the
|
||
pinned `rust-toolchain.toml`.
|
||
- **Workspace dep pinning:** new deps go in the ROOT `Cargo.toml` `[workspace.dependencies]`;
|
||
members reference with `dep.workspace = true`. (None added here.)
|
||
- **cargo-deny license gate** stays green (unaffected; final sweep runs it).
|
||
- **sim-bench CI job** runs `--test-threads=1` (unaffected).
|
||
|
||
Slice-G-specific constraints:
|
||
|
||
- **Acceptance bar (spec §10):** every topology doc contains a copy-paste path from zero to
|
||
first call, **or** the honest statement of what cannot work there (e.g. CGNAT WebRTC). Each
|
||
doc task's verification step greps for the load-bearing statements.
|
||
- **Artifact names:** plan B (`docs/superpowers/plans/2026-07-05-deploy-b-packaging-ci.md`)
|
||
has landed and settled the registry namespace to **`git.adlee.work/alee/rutster-*`**
|
||
(matches the `alee` tea login per AGENTS.md + the root `Cargo.toml`'s
|
||
`repository = "https://git.adlee.work/alee/rutster"`). Images: `rutster-allinone`,
|
||
`rutster-engine`, `rutster-brain`, `rutster-edge`. `deploy/` tree = `Dockerfile`,
|
||
`compose.yaml`, `Caddyfile`, `.env.example`. Every doc task still carries a **cross-check
|
||
step**: if `deploy/` exists in the worktree by execution time, reconcile env-var names and
|
||
compose service names against the real files before committing.
|
||
- **Env vars:** grounded in `crates/rutster/src/config.rs` for what exists today
|
||
(`RUTSTER_HTTP_BIND`, `RUTSTER_MEDIA_BIND_IP`, `RUTSTER_MEDIA_ADVERTISED_IP`,
|
||
`RUTSTER_MEDIA_PORT_RANGE`, `RUTSTER_MAX_SESSIONS`, `RUTSTER_DRAIN_DEADLINE_SECS`, the four
|
||
`RUTSTER_TWILIO_*` vars) and in spec §5.7 for what sibling slices A/C/D/E add
|
||
(`RUTSTER_TRUSTED_PROXIES`, `RUTSTER_TLS_CERT`/`RUTSTER_TLS_KEY`, `RUTSTER_WS_PING_SECS`,
|
||
`RUTSTER_METRICS_BIND`, `RUTSTER_VALKEY_URL`). The docs document the **post-epoch** artifact
|
||
set — slice G is sequenced last (spec §11) — so referencing §5.7 vars is correct, not
|
||
speculative.
|
||
- **Citations:** wherever a `docs/deploy/` file makes a factual claim about third-party
|
||
behavior, cite the TLS brief's external source URL inline (Twilio docs/error pages, the
|
||
Caddy/cloudflared/axum/livekit issue URLs, etc.).
|
||
- **Ports (grounded):** one FOB listener `:8080` (`RUTSTER_HTTP_BIND`;
|
||
`crates/rutster/src/main.rs` merges the trunk WS router into the main app — the
|
||
`/twilio/media-stream` WS rides the same listener); brain tap `127.0.0.1:8082`; valkey
|
||
`:6379`; metrics `127.0.0.1:9090` (spec §5.5); Caddy `:80`/`:443` public; WebRTC media =
|
||
UDP `RUTSTER_MEDIA_PORT_RANGE` direct to the FOB, never through Caddy.
|
||
- **Do NOT restyle existing docs.** Task 8 (QUICKSTART) is a minimal-diff pointer insertion,
|
||
nothing else.
|
||
|
||
## File Structure
|
||
|
||
| Path | Action | Responsibility |
|
||
|---|---|---|
|
||
| `docs/adr/0011-deployment-topology.md` | Create | The topology ADR: three blessed shapes, edge doctrine, fleet placement paradigm, rejected shapes. Status: Proposed. |
|
||
| `docs/deploy/topologies.md` | Create | The three shapes + decision-guide table; entry point of the tree. |
|
||
| `docs/deploy/quickstart-docker.md` | Create | T1 `docker run` + T2 `compose up`, zero to first call (browser + PSTN). |
|
||
| `docs/deploy/homelab.md` | Create | The CGNAT truth: PSTN-only behind tunnels; ngrok blessed dev path; Tailscale Funnel; VPS+WireGuard graduation with TLS at home. |
|
||
| `docs/deploy/aws.md` | Create | ALB appendix: three mandatory attribute overrides, 100-TG cap, never-NLB-TLS. |
|
||
| `docs/deploy/reverse-proxies.md` | Create | BYO nginx/HAProxy/Traefik tuned-timeout snippets; the universal 60 s footgun; Traefik version pinning. |
|
||
| `docs/deploy/certificates.md` | Create | ACME paths, wildcard traps, LE duplicate-cert lockout, `/data` persistence, per-node vs wildcard fleet patterns. |
|
||
| `docs/QUICKSTART.md` | Modify (minimal) | One blockquote pointing at `deploy/quickstart-docker.md` as the fast path. |
|
||
|
||
## Task ordering
|
||
|
||
- **Task 1 (ADR-0011) has NO dependency — execute immediately.** It is drafted from the spec,
|
||
not from artifacts.
|
||
- **Tasks 2–7** are order-independent among themselves and parallelizable; each carries the
|
||
slice-B cross-check step (spec §11: G depends on B "for accuracy").
|
||
- **Task 8** depends on Task 3 (the file it links to must exist).
|
||
|
||
Each task = one commit. Verification per task is a bash content-check that fails before the
|
||
file is written and passes after.
|
||
|
||
---
|
||
|
||
### Task 1: ADR-0011 — `docs/adr/0011-deployment-topology.md`
|
||
|
||
**Files:**
|
||
- Create: `docs/adr/0011-deployment-topology.md`
|
||
- Test: bash content-check (Step 1/Step 3 below)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §2/§3/§4 (`docs/superpowers/specs/2026-07-05-deployment-topology-design.md`),
|
||
the TLS brief (`docs/superpowers/specs/2026-07-05-tls-edge-decision-brief.md`), the house ADR
|
||
format (read `docs/adr/0007-trunk-rented-transport.md` and `docs/adr/0008-fob-and-green-zone.md`
|
||
before writing — the header bullet-list, Context/Decision/Consequences/References structure,
|
||
and relative cross-link style below match them exactly).
|
||
- Produces: the ADR every `docs/deploy/` file cross-links as
|
||
`../adr/0011-deployment-topology.md` (Tasks 2–7 rely on this exact path).
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'Status:\*\* Proposed' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'dns-persist-01' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q '307' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'on-demand TLS' docs/adr/0011-deployment-topology.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL` (file does not exist).
|
||
|
||
- [ ] **Step 2: Write the complete ADR**
|
||
|
||
Write `docs/adr/0011-deployment-topology.md` with exactly this content:
|
||
|
||
````markdown
|
||
# ADR-0011 — Deployment topology: one binary, three blessed shapes
|
||
|
||
- **Status:** Proposed
|
||
- **Date:** 2026-07-05
|
||
- **Closes:** the deployment-topology ADR reserved by [ADR-0008](0008-fob-and-green-zone.md)'s
|
||
worked example ("the deployment topology itself may get its own ADR once that design closes").
|
||
- **Origin:** [2026-07-05 deployment-topology spec](../superpowers/specs/2026-07-05-deployment-topology-design.md)
|
||
§2–§4. The TLS/edge fork was researched via a six-family survey — see the companion
|
||
[TLS/edge decision brief](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md).
|
||
- **Related:** [ADR-0002](0002-north-star-and-fused-core.md) (the fused per-call vertical no
|
||
topology may split), [ADR-0005](0005-event-bus.md) (Valkey = bus + KV + presence; media never
|
||
rides the bus; the bus is never billing source-of-truth), [ADR-0006](0006-ingress-posture.md)
|
||
(two inbound protocols only; the tap is egress-only), [ADR-0007](0007-trunk-rented-transport.md)
|
||
(rented transport ⇒ public HTTPS/WSS reachability is non-negotiable),
|
||
[ADR-0008](0008-fob-and-green-zone.md) (the FOB/green-zone rule this ADR applies to
|
||
deployment), [ADR-0009](0009-spend-gate-honest-rescope.md) (ledger trait: in-memory
|
||
single-node, Valkey-backed fleets), [ADR-0010](0010-spearhead-benchmark-sim-harness.md)
|
||
(sequencing this ADR must not reorder), README pillar 6 ("one binary, one bus, one deploy").
|
||
|
||
## Context
|
||
|
||
[ADR-0008](0008-fob-and-green-zone.md) made the build-vs-reuse doctrine physical in a worked
|
||
example — the all-in-one container — and reserved the topology ADR for when the design closed.
|
||
Two things have since closed it: the rented transport
|
||
([ADR-0007](0007-trunk-rented-transport.md)) is merged, so every deployment must now present a
|
||
public HTTPS/WSS edge to a CPaaS; and slice-5/seams landed the operational primitives a
|
||
deployment story needs (drain lifecycle, `/readyz`, admission cap, `MediaAddressConfig`,
|
||
`EventSink`).
|
||
|
||
The CPaaS side imposes a hard external spec on any edge rutster ships (researched in the
|
||
[decision brief](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md); restated here
|
||
because these invariants are load-bearing for every decision below):
|
||
|
||
1. Publicly-CA-trusted cert; **no self-signed path exists** (Twilio error 31910). Auto-renewal
|
||
is availability-critical.
|
||
2. wss:// on 443, open to the whole internet: Twilio publishes no egress IPs and offers no
|
||
mTLS. Auth is application-layer only (HMAC signatures).
|
||
3. One WS per call, up to 24 h, 50 msg/s/direction at 20 ms cadence; zero tolerance for frame
|
||
buffering, connection-lifetime caps, or write-side idle timers. Neither Twilio nor Telnyx
|
||
documents WS keepalive — keepalive is entirely our job.
|
||
4. `<Stream>` URLs carry **no query strings** — wss routing keys on hostname/path only.
|
||
5. `X-Twilio-Signature` is HMAC over the URL *as Twilio saw it*: the edge must forward
|
||
`X-Forwarded-Proto/Host` honestly and the FOB must reconstruct the public URL.
|
||
6. Webhook budget: sub-second for UX, 15 s hard cap — no blocking issuance in the handshake
|
||
path.
|
||
7. TLS 1.2+1.3, mainstream ECDHE, never 1.3-only; never pin Twilio certs.
|
||
|
||
Design for Twilio; Telnyx/Vonage come for free (strictly less demanding, plus optional
|
||
source-IP allowlisting as extra hardening Twilio can't use).
|
||
|
||
On the other side, the FOB's own physics constrain every shape: per-call state is in-process
|
||
and non-migratable (the fused vertical, [ADR-0002](0002-north-star-and-fused-core.md)), WebRTC
|
||
media is per-session UDP direct to the process, and calls run for hours. Any topology that
|
||
assumes migratable, short-lived, connectionless workloads is disqualified before it starts.
|
||
|
||
## Decision
|
||
|
||
**The FOB never decomposes; topology is a configuration property, not a code property.** The
|
||
same binary runs in every blessed shape; only `RUTSTER_*` env and what is deployed alongside it
|
||
differ. "Microservices" in rutster means *green-zone services get their own
|
||
processes/containers* — never that the per-call vertical splits
|
||
([ADR-0002](0002-north-star-and-fused-core.md)).
|
||
|
||
### The three blessed shapes
|
||
|
||
#### T1 — Solo (all-in-one container) — shipped artifact
|
||
|
||
One image (`rutster-allinone`), s6-overlay as PID 1 supervising four processes:
|
||
|
||
| Process | Role | Binding |
|
||
|---|---|---|
|
||
| `caddy` | Edge: ACME, TLS termination, WS proxy to FOB | :443/:80 public |
|
||
| `rutster` (FOB) | Engine | 127.0.0.1:8080 (behind Caddy); media UDP direct |
|
||
| `rutster-brain-realtime` | Brain | 127.0.0.1:8082 loopback tap (as today) |
|
||
| `valkey-server` | Dark on day one except `EventSink` | 127.0.0.1:6379 |
|
||
|
||
Two volumes, both non-negotiable: `/data` (Caddy cert/ACME state — loss risks the Let's
|
||
Encrypt duplicate-cert lockout, a total-inbound-outage class of failure) and `/var/lib/valkey`
|
||
(stream/state persistence). WebRTC media UDP goes **direct to the FOB**, never through Caddy:
|
||
host networking recommended; otherwise published `RUTSTER_MEDIA_PORT_RANGE` +
|
||
`RUTSTER_MEDIA_ADVERTISED_IP` (NAT 1:1 model, no STUN — unchanged from slice-5/seams).
|
||
|
||
Valkey ships bundled from v1 **before most consumers exist**: the operator contract (volumes,
|
||
ports, upgrade shape) is stable from day one, so the spend ledger
|
||
([ADR-0009](0009-spend-gate-honest-rescope.md)) and the fleet directory land later without
|
||
changing the deployment shape. Cost accepted: ~15 MB and one mostly-idle process.
|
||
|
||
#### T2 — Modular (compose stack) — shipped artifact, the reference deployment
|
||
|
||
Services: `caddy` (image `rutster-edge`), `engine` (`rutster-engine`), `brain`
|
||
(`rutster-brain`), `valkey` (upstream `valkey/valkey`).
|
||
|
||
- **The brain shares the engine's network namespace** (`network_mode: "service:engine"`) so
|
||
the loopback-only tap posture (`resolve_tap_url` rejects non-loopback until step 6) survives
|
||
unchanged. The brain graduates to its own netns when the wss:// tap lands (step 6).
|
||
- Valkey is its own service on the compose network — network-near, satisfying
|
||
[ADR-0005](0005-event-bus.md)'s sub-ms rule.
|
||
- `stop_grace_period: 660s` paired with `RUTSTER_DRAIN_DEADLINE_SECS=600` (grace > drain).
|
||
- BYO-proxy supported: disable the `caddy` service; the engine keeps its plaintext `:8080`
|
||
mode; tuned-timeout snippets for nginx/HAProxy/Traefik ship in
|
||
[`docs/deploy/reverse-proxies.md`](../deploy/reverse-proxies.md).
|
||
|
||
#### T3 — Fleet — ADR chapter only (paper; no code in this epoch)
|
||
|
||
N **symmetric** nodes, each running the T1 process set *minus* valkey, plus a shared green
|
||
zone: one Valkey (presence + directory + the [ADR-0009](0009-spend-gate-honest-rescope.md)
|
||
spend ledger) and object storage (named as required, still deferred). Wildcard DNS
|
||
`*.pbx.domain`; per-node certs (see the fleet cert story below). The per-node admission cap is
|
||
the [ADR-0010](0010-spearhead-benchmark-sim-harness.md) benchmark number (replacing the
|
||
`RUTSTER_MAX_SESSIONS=64` placeholder). Scale-in is drain-then-terminate only; spot instances
|
||
are rejected for the engine tier (2026-07-04 review — a reclaimed node kills every
|
||
non-migratable call on it).
|
||
|
||
### Edge doctrine
|
||
|
||
#### Bundled edge: Caddy (both shipped artifacts, same build, same Caddyfile)
|
||
|
||
Custom xcaddy build (the `rutster-edge` image): Caddy core (Apache-2.0) + a curated DNS-01
|
||
plugin set — cloudflare, route53, porkbun, hetzner, desec (duckdns excluded: no license file).
|
||
Zero-downtime in-memory cert renewal; verified protocol-unaware WS tunneling (no frame
|
||
buffering). Config is a ~6-line Caddyfile with tuned timeouts, honest `X-Forwarded-*`, and
|
||
`stream_close_delay` above max call duration — **plus a CI e2e case for
|
||
config-reload-during-live-call, because the upstream mitigation has an open bug trail
|
||
(caddy #6420/#7222) and is not trusted untested.**
|
||
|
||
#### In-process TLS: staged — the ratified end-state, not adopted wholesale now
|
||
|
||
[ADR-0008](0008-fob-and-green-zone.md) claims the trust-domain edge as security-constitutive
|
||
FOB territory, and str0m already terminates DTLS-SRTP in-process on the same aws-lc-rs
|
||
provider — "keep TLS out of the FOB" is not the objection. The objection is economics: owning
|
||
an ACME renewal state machine forever, in a 45-day-cert world.
|
||
|
||
- **Phase 1 (this epoch):** `axum-server` + rustls with operator-supplied certs
|
||
(`RUTSTER_TLS_CERT`/`RUTSTER_TLS_KEY`), hot-reload without dropping live WS.
|
||
Plaintext-behind-Caddy stays the artifact default.
|
||
- **Phase 2 (in-binary ACME) triggers, named:** (a) Let's Encrypt dns-persist-01 confirmed GA
|
||
(closes the wildcard/CGNAT issuance gap; instant-acme already parses it); (b) a field
|
||
incident where Caddy reload/`stream_close_delay` drops live calls; (c) fleet or
|
||
VPS-forwarder topology making encrypted-to-the-binary a selling point; (d) maintenance
|
||
fatigue with the curated xcaddy plugin build. Until a trigger fires, Phase 2 is deliberately
|
||
not built — no turnkey ACME crate fits the tree today (rustls-acme conflicts with the
|
||
deny.toml rcgen/x509-parser posture).
|
||
|
||
#### Fleet cert story
|
||
|
||
Per-node public TLS names are CPaaS-imposed (invariant 4 + node-addressed placement). Two
|
||
blessed patterns:
|
||
|
||
1. **Wildcard `*.pbx.domain` via DNS-01** with central issuance/distribution.
|
||
2. **Per-node distinct certs** (`node-N.domain`) where the wildcard private key must not live
|
||
on every node — one compromised node otherwise burns the whole namespace.
|
||
|
||
Traps, stated so nobody re-discovers them in production: N nodes independently requesting the
|
||
*identical* wildcard hit the Let's Encrypt duplicate-cert limit (5/week) — distinct names or
|
||
central distribution only. **Caddy on-demand TLS is rejected for node names**: the first
|
||
handshake to a fresh name blocks seconds on issuance, colliding with the sub-second webhook
|
||
budget (invariant 6), and its rate-limit knobs are deprecated. Node names known at provision
|
||
time get pre-issued certs.
|
||
|
||
### Fleet placement paradigm (the trunk webhook handler and session API grow *toward* this; no fleet code in this epoch)
|
||
|
||
**Node-addressed URLs + Valkey presence; no router tier.** (Decision 2026-07-05 over the two
|
||
rejected alternatives below.)
|
||
|
||
- Nodes heartbeat capacity into Valkey (TTL'd presence keys — the
|
||
[ADR-0005](0005-event-bus.md) presence layer).
|
||
- Placement happens **at answer time** by whichever node the request lands on (`api.domain`,
|
||
DNS round-robin): read fleet capacity → pick owner → return TwiML whose Stream URL is
|
||
`wss://node-K.pbx.domain/...`. The media WS dials the owning node directly — zero extra
|
||
hops, consistent with connection-follows-ownership. Twilio's answer-time Stream URL is
|
||
purpose-shaped for this: invariant 4 permits hostname routing.
|
||
- WebRTC: the create-session response carries the owner's node-addressed base URL + advertised
|
||
media IP (UDP is already direct).
|
||
- Misdirected control ops (e.g. a DELETE hitting a non-owner) consult the Valkey directory and
|
||
**307-redirect** to the owner.
|
||
- **Degraded mode (Valkey dark): the answering node self-places if it has capacity**; presence
|
||
heals via TTL on recovery. Fail-safe degradation, consistent with
|
||
[ADR-0009](0009-spend-gate-honest-rescope.md)'s posture.
|
||
- Single-node is the degenerate case of the same code: a one-entry directory.
|
||
|
||
**Rejected: a routing tier that proxies everything.** It would put every 20 ms media frame
|
||
through an extra userspace hop and re-create exactly the SPOF the symmetric design avoids —
|
||
and it buys nothing, because the CPaaS already lets the answer-time URL do the routing.
|
||
|
||
**Rejected: a distinguished control node.** It breaks node symmetry (two artifacts, two
|
||
failure stories, a promotion protocol) to solve a problem the presence directory already
|
||
solves; placement needs fleet state either way, and Valkey is already the fleet-state home
|
||
([ADR-0005](0005-event-bus.md)).
|
||
|
||
### Rejected shapes
|
||
|
||
- **Decomposing the FOB into services** — rejected **permanently**, not deferred.
|
||
Media/reflex/tap/spend stay one process, one trust domain
|
||
([ADR-0002](0002-north-star-and-fused-core.md)).
|
||
- **Serverless.** Lambdas are categorically unfit for the engine: non-migratable multi-hour
|
||
calls, per-session UDP sockets, in-process media state. Webhook-on-Lambda buys nothing
|
||
because placement needs fleet state anyway.
|
||
- **Tunnels in production.** Cloudflare Tunnel and ngrok see plaintext audio at the vendor
|
||
edge (an unconsented subprocessor — DPA/BAA/PCI failure), document mid-call WS kills, and
|
||
cloudflared has an open Twilio handshake bug (cloudflared #1465). ngrok is the blessed
|
||
5-minute demo path (the only tunnel with a proven Media Streams record); Tailscale Funnel is
|
||
the privacy-clean single-user demo (TLS terminates on-node), unsizable beyond that. **No
|
||
tunnel carries inbound UDP → homelab behind CGNAT is PSTN-only, period.** The production
|
||
graduation path is a cheap VPS + WireGuard as a dumb TCP forwarder with TLS terminating at
|
||
home, or running the engine on the VPS — see
|
||
[`docs/deploy/homelab.md`](../deploy/homelab.md).
|
||
- **NLB TLS listeners — never.** Fixed 350 s idle timeout that silently drops flows, and no
|
||
HTTP context, so `X-Forwarded-*` headers for signature validation (invariant 5) don't exist.
|
||
The ALB path is documented as a docs-only appendix
|
||
([`docs/deploy/aws.md`](../deploy/aws.md)), never a shipped artifact.
|
||
- **k8s manifests/Helm.** Compose-first (README pillar 6). k8s notes live in docs only (drain
|
||
vs `terminationGracePeriodSeconds`); manifests are a later rung if demand appears.
|
||
|
||
### Deferred, named (so the boundary is deliberate)
|
||
|
||
- **T3 fleet implementation** (presence heartbeats, directory redirects, placement code).
|
||
- **The zero-egress / air-gap profile** (SBC layer-2 ingress per
|
||
[ADR-0007](0007-trunk-rented-transport.md) layer 2 + a self-hosted brain): a defined
|
||
profile, no code this epoch. T1 is about one-command simplicity, not isolation.
|
||
- **In-binary ACME (rustls Phase 2)** — behind the four named triggers above.
|
||
- **Durable CDR pipeline**: `ValkeyEventSink` is evidence preservation on a capped stream —
|
||
explicitly NOT the billing ledger ([ADR-0005](0005-event-bus.md) source-of-truth rule).
|
||
CDR → object storage stays deferred.
|
||
- **Multi-tenancy** (needs its own ADR before schemas ossify); wss:// tap + brain fleet +
|
||
resume tokens (step 6); media-thread sharding graduation; arm64 images.
|
||
|
||
## Consequences
|
||
|
||
- **Positive:** one binary and one mental model across all three shapes — an operator's
|
||
knowledge transfers from `docker run` to a fleet; the CPaaS invariants are satisfied by a
|
||
decade-hardened ACME loop (Caddy) rather than a DIY renewal state machine; the fleet
|
||
paradigm needs zero new infrastructure beyond what [ADR-0005](0005-event-bus.md) already
|
||
bundles; the rejected shapes are written down, so nobody burns a week discovering that
|
||
Lambda or NLB-TLS can't work.
|
||
- **Negative:** the curated xcaddy DNS-plugin set is a permanent maintenance surface until
|
||
dns-persist-01 GA; Caddy config-reload behavior vs live calls is mitigated
|
||
(`stream_close_delay` + CI e2e) but has an open upstream bug trail; homelab WebRTC behind
|
||
CGNAT is unsolved **by design** — only the VPS graduation fixes it, and the docs must say so
|
||
loudly or operators will read "homelab supported" as including browser calls.
|
||
- **Mitigation:** the Phase-2 triggers convert both negatives into a planned exit; the
|
||
`docs/deploy/` tree ships with a copy-paste-to-first-call acceptance bar so the honest
|
||
limits are in the operator's face, not in a footnote.
|
||
|
||
## References
|
||
|
||
- [2026-07-05 deployment-topology spec](../superpowers/specs/2026-07-05-deployment-topology-design.md) — §2–§4 are this ADR's source
|
||
- [TLS/edge decision brief](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md) — six-family survey + the CPaaS ground truth (external citations live there)
|
||
- [ADR-0002](0002-north-star-and-fused-core.md) · [ADR-0005](0005-event-bus.md) · [ADR-0006](0006-ingress-posture.md) · [ADR-0007](0007-trunk-rented-transport.md) · [ADR-0008](0008-fob-and-green-zone.md) · [ADR-0009](0009-spend-gate-honest-rescope.md) · [ADR-0010](0010-spearhead-benchmark-sim-harness.md)
|
||
- The `docs/deploy/` tree: [topologies](../deploy/topologies.md) · [quickstart-docker](../deploy/quickstart-docker.md) · [homelab](../deploy/homelab.md) · [aws](../deploy/aws.md) · [reverse-proxies](../deploy/reverse-proxies.md) · [certificates](../deploy/certificates.md)
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'Status:\*\* Proposed' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'dns-persist-01' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q '307' docs/adr/0011-deployment-topology.md \
|
||
&& grep -q 'on-demand TLS' docs/adr/0011-deployment-topology.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
# Relative links resolve (the ../deploy/ links will report BROKEN until Tasks 2–7 land —
|
||
# acceptable inside this slice; re-run in the final sweep where all must resolve):
|
||
grep -oE '\]\([^)#]+\)' docs/adr/0011-deployment-topology.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/adr/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; link check reports only `../deploy/*.md` as BROKEN if Tasks 2–7
|
||
haven't run yet (nothing else).
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/adr/0011-deployment-topology.md
|
||
git commit -s -m "docs(adr): ADR-0011 deployment topology — one binary, three blessed shapes (Proposed) (slice-G)
|
||
|
||
Drafted from the 2026-07-05 deployment-topology spec §2–§4: T1 solo /
|
||
T2 modular / T3 fleet (paper), the edge doctrine (bundled Caddy; rustls
|
||
Phase 1 now, Phase 2 behind four named triggers), the fleet cert story
|
||
(wildcard vs per-node; on-demand TLS rejected), node-addressed placement
|
||
via Valkey presence (routing tier + distinguished control node rejected),
|
||
and the rejected shapes (FOB decomposition, serverless, tunnels-in-prod,
|
||
NLB-TLS, k8s manifests). Status Proposed — ratification per repo process.
|
||
Closes the topology ADR reserved by ADR-0008's worked example."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: `docs/deploy/topologies.md` — the three shapes + decision guide
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/topologies.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: ADR-0011 (Task 1) as `../adr/0011-deployment-topology.md`; spec §2, §7, §8.
|
||
- Produces: the tree's entry point; Tasks 3–7 link to it as `topologies.md` (same dir).
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/topologies.md \
|
||
&& grep -q 'decision guide' docs/deploy/topologies.md \
|
||
&& grep -qi 'paper' docs/deploy/topologies.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/topologies.md` with exactly this content:
|
||
|
||
````markdown
|
||
# Deployment topologies — one binary, three blessed shapes
|
||
|
||
The FOB never decomposes; topology is a configuration property, not a code property. The same
|
||
binary runs in every shape below; only `RUTSTER_*` env and what is deployed alongside it
|
||
differ. Ratified (Proposed) in [ADR-0011](../adr/0011-deployment-topology.md).
|
||
|
||
## Decision guide
|
||
|
||
| You are… | Use | Doc |
|
||
|---|---|---|
|
||
| Trying rutster for the first time; one box, one domain, a public IP | **T1 — Solo** | [quickstart-docker.md](quickstart-docker.md) |
|
||
| Running production and want independently upgradable parts | **T2 — Modular** (the reference deployment) | [quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack) |
|
||
| Behind CGNAT / no public IP (homelab) | T1/T2 + tunnel (dev, **PSTN-only**) or VPS forwarder (production) | [homelab.md](homelab.md) |
|
||
| Already running nginx / HAProxy / Traefik | T2 minus the `caddy` service | [reverse-proxies.md](reverse-proxies.md) |
|
||
| On EC2 and want ALB + ACM to terminate TLS | T2/T3 behind an ALB | [aws.md](aws.md) |
|
||
| Outgrowing one node | **T3 — Fleet** — **paper only today** | [ADR-0011](../adr/0011-deployment-topology.md) |
|
||
|
||
## T1 — Solo (all-in-one container)
|
||
|
||
One image, `rutster-allinone`, with s6-overlay supervising four processes:
|
||
|
||
| Process | Role | Binding |
|
||
|---|---|---|
|
||
| `caddy` | Edge: ACME issuance/renewal, TLS termination, WS proxy | `:443` / `:80` public |
|
||
| `rutster` (the FOB) | Engine — signaling, trunk webhook, media, reflexes | `127.0.0.1:8080` behind Caddy; **media UDP direct** |
|
||
| `rutster-brain-realtime` | Brain | `127.0.0.1:8082` loopback tap |
|
||
| `valkey-server` | Event stream (`EventSink`); ledger/directory later | `127.0.0.1:6379` |
|
||
|
||
Two volumes, **both non-negotiable**:
|
||
|
||
- `/data` — Caddy's cert/ACME state. Losing it can lock your domain out of Let's Encrypt for
|
||
up to a week (duplicate-certificate limit) — a **total inbound outage**. See
|
||
[certificates.md](certificates.md).
|
||
- `/var/lib/valkey` — stream/state persistence.
|
||
|
||
WebRTC media (UDP) goes **direct to the FOB, never through Caddy**: run with host networking,
|
||
or publish `RUTSTER_MEDIA_PORT_RANGE` and set `RUTSTER_MEDIA_ADVERTISED_IP` to your public IP
|
||
(NAT 1:1 model — no STUN).
|
||
|
||
Copy-paste path from zero to first call: [quickstart-docker.md](quickstart-docker.md).
|
||
|
||
## T2 — Modular (compose stack) — the reference deployment
|
||
|
||
Four services in `deploy/compose.yaml`:
|
||
|
||
| Service | Image | Notes |
|
||
|---|---|---|
|
||
| `caddy` | `rutster-edge` | Same custom Caddy build + same Caddyfile as T1 |
|
||
| `engine` | `rutster-engine` | The FOB; plaintext `:8080` on the compose network; media UDP published |
|
||
| `brain` | `rutster-brain` | `network_mode: "service:engine"` — shares the engine's netns so the loopback-only tap posture survives unchanged |
|
||
| `valkey` | `valkey/valkey` (upstream) | Own service; network-near |
|
||
|
||
Operational contract:
|
||
|
||
- `stop_grace_period: 660s` paired with `RUTSTER_DRAIN_DEADLINE_SECS=600` — grace **must**
|
||
exceed drain, or the orchestrator kills calls the engine was gracefully draining.
|
||
- Bring-your-own-proxy: disable the `caddy` service; the engine keeps its plaintext `:8080`
|
||
mode. Tuned-timeout configs: [reverse-proxies.md](reverse-proxies.md).
|
||
- Upgrades: `docker compose pull && docker compose up -d` — the drain window lets in-flight
|
||
calls finish (calls are non-migratable; a killed engine kills its calls, honestly stated).
|
||
|
||
Copy-paste path from zero to first call:
|
||
[quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack).
|
||
|
||
## T3 — Fleet (paper only — no shipped artifacts in this epoch)
|
||
|
||
**Honest statement: you cannot deploy T3 today by copy-paste; there are no fleet artifacts
|
||
yet.** The design is ratified on paper in
|
||
[ADR-0011](../adr/0011-deployment-topology.md) so single-node deployments grow toward it, not
|
||
away from it:
|
||
|
||
- N symmetric nodes (the T1 process set minus valkey) + a shared green zone: one Valkey
|
||
(presence + directory + spend ledger), object storage.
|
||
- Wildcard DNS `*.pbx.domain`; per-node certs — the traps live in
|
||
[certificates.md](certificates.md).
|
||
- Answer-time placement via Valkey presence; node-addressed `wss://node-K.pbx.domain/...`
|
||
Stream URLs; no router tier.
|
||
- Per-node admission cap = the benchmark number (`RUTSTER_MAX_SESSIONS`, default 64, is a
|
||
placeholder until then).
|
||
- Scale-in is drain-then-terminate only. No spot instances for the engine tier.
|
||
|
||
## What is deliberately not supported
|
||
|
||
- **Splitting the FOB into services** — permanent rejection, not a roadmap item.
|
||
- **Serverless** — non-migratable multi-hour calls, per-session UDP sockets, in-process media
|
||
state. Categorically unfit.
|
||
- **Tunnels in production** — plaintext audio at the tunnel vendor's edge + documented
|
||
mid-call WS kills. Dev/demo only; the whole story is in [homelab.md](homelab.md).
|
||
- **NLB TLS listeners** — see [aws.md](aws.md) for the 350-second reason.
|
||
- **k8s manifests/Helm** — compose-first. If you run k8s anyway: set
|
||
`terminationGracePeriodSeconds` above `RUTSTER_DRAIN_DEADLINE_SECS`, same grace-exceeds-drain
|
||
rule as compose.
|
||
|
||
## Ports & volumes reference (all shapes)
|
||
|
||
| Surface | Where | Notes |
|
||
|---|---|---|
|
||
| `:80` / `:443` TCP | Caddy, public | ACME HTTP-01 + TLS + all HTTPS/WSS traffic |
|
||
| `:8080` TCP | FOB, behind the edge | Signaling + `/v1/trunk/webhook` + `/twilio/media-stream` WS — one listener (`RUTSTER_HTTP_BIND`) |
|
||
| `RUTSTER_MEDIA_PORT_RANGE` UDP | FOB, **public, direct** | WebRTC media; never proxied |
|
||
| `:8082` TCP | Brain, loopback only | The tap; non-loopback rejected until step 6 |
|
||
| `:6379` TCP | Valkey, private | Loopback (T1) / compose network (T2) / green zone (T3) |
|
||
| `:9090` TCP | FOB `/metrics`, internal | `RUTSTER_METRICS_BIND`; never routed through Caddy |
|
||
| `/data` volume | Caddy | Cert/ACME state — loss class: total inbound outage |
|
||
| `/var/lib/valkey` volume | Valkey | Event stream persistence |
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/topologies.md \
|
||
&& grep -qi 'decision guide' docs/deploy/topologies.md \
|
||
&& grep -q 'paper only' docs/deploy/topologies.md \
|
||
&& grep -q 'cannot deploy T3 today' docs/deploy/topologies.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/topologies.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; BROKEN lines only for sibling docs whose tasks haven't run yet.
|
||
|
||
- [ ] **Step 4: Cross-check against slice B (if landed)**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
ls deploy/ 2>/dev/null && grep -n 'image:' deploy/compose.yaml
|
||
```
|
||
|
||
If `deploy/` exists and any image/service name differs from this doc's tables, fix the doc to
|
||
match reality before committing. If `deploy/` does not exist, the doc stands on spec §6.1
|
||
names — no change.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/topologies.md
|
||
git commit -s -m "docs(deploy): topologies.md — three blessed shapes + decision guide (slice-G)
|
||
|
||
The docs/deploy/ tree entry point. T1/T2 process+service tables, the
|
||
grace-exceeds-drain rule, the honest T3 statement (paper only, no
|
||
copy-paste deploy exists), the not-supported list, and the shared
|
||
ports/volumes reference. Acceptance bar: copy-paste paths delegate to
|
||
quickstart-docker.md; T3 states plainly what cannot be done today."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: `docs/deploy/quickstart-docker.md` — zero to first call
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/quickstart-docker.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §2.1/§2.2/§5.7/§6, image names from spec §6.1, env vars grounded in
|
||
`crates/rutster/src/config.rs` (the four `RUTSTER_TWILIO_*` vars are all-or-none — the
|
||
parser rejects partial config), routes grounded in `crates/rutster/src/routes.rs`
|
||
(`/healthz`, `/readyz`, `/v1/trunk/webhook`).
|
||
- Produces: the file `docs/QUICKSTART.md` links to in Task 8 (exact path
|
||
`docs/deploy/quickstart-docker.md`).
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/quickstart-docker.md \
|
||
&& grep -q 'rutster-allinone' docs/deploy/quickstart-docker.md \
|
||
&& grep -q 'docker compose up' docs/deploy/quickstart-docker.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/quickstart-docker.md` with exactly this content:
|
||
|
||
````markdown
|
||
# Docker quickstart — zero to first call
|
||
|
||
Two shapes, one binary ([topologies.md](topologies.md)). T1 is one `docker run`; T2 is the
|
||
reference `compose up`. Both end the same way: a browser call at your domain, and (with Twilio
|
||
credentials) a real phone call.
|
||
|
||
> **Artifact names note:** images are published to the `git.adlee.work/alee/` registry
|
||
> namespace by the release CI (plan B / slice F settled this). If your Gitea registry uses a
|
||
> different owner segment, substitute accordingly — check `deploy/compose.yaml` in your
|
||
> checkout, which is authoritative.
|
||
|
||
## Prerequisites (both shapes)
|
||
|
||
1. A Linux host with Docker, a **public IPv4**, and inbound `80/tcp`, `443/tcp`, and your
|
||
chosen media UDP range open. (No public IP? → [homelab.md](homelab.md).)
|
||
2. A domain with an A record pointing at that host, e.g. `pbx.example.com`. Certificates are
|
||
issued automatically via ACME — no self-signed path exists, the CPaaS refuses it
|
||
([certificates.md](certificates.md)).
|
||
3. For the phone call: a [Twilio](https://www.twilio.com/) account + a Voice-capable number
|
||
(trial works).
|
||
|
||
## T1 — Solo (all-in-one)
|
||
|
||
```bash
|
||
docker run -d --name rutster --restart unless-stopped \
|
||
--network host \
|
||
-v rutster-caddy-data:/data \
|
||
-v rutster-valkey:/var/lib/valkey \
|
||
-e RUTSTER_DOMAIN=pbx.example.com \
|
||
-e RUTSTER_ACME_EMAIL=you@example.com \
|
||
-e RUTSTER_MEDIA_ADVERTISED_IP=203.0.113.7 \
|
||
-e RUTSTER_MEDIA_PORT_RANGE=49152-49407 \
|
||
-e RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
|
||
-e RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token \
|
||
-e RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081 \
|
||
-e RUTSTER_TWILIO_WEBHOOK_BASE=https://pbx.example.com \
|
||
git.adlee.work/alee/rutster-allinone:latest
|
||
```
|
||
|
||
Notes:
|
||
|
||
- `--network host` is the recommended mode: WebRTC media UDP goes **direct to the engine**,
|
||
never through Caddy. If you can't use host networking, publish `-p 80:80 -p 443:443
|
||
-p 49152-49407:49152-49407/udp` instead and keep `RUTSTER_MEDIA_ADVERTISED_IP` = the host's
|
||
public IP (NAT 1:1, no STUN).
|
||
- The two volumes are **non-negotiable**. `/data` in particular: recreating the container
|
||
without it re-requests certificates and can hit Let's Encrypt's duplicate-certificate
|
||
limit — a week-long total inbound outage ([certificates.md](certificates.md)).
|
||
- The four `RUTSTER_TWILIO_*` vars are all-or-none — the engine refuses partial Twilio config
|
||
at startup. Omit all four to run WebRTC-only.
|
||
- `RUTSTER_DOMAIN` / `RUTSTER_ACME_EMAIL` feed the bundled Caddyfile. Domains that can't do
|
||
HTTP-01 (no port 80, wildcard needs) use DNS-01 — [certificates.md](certificates.md).
|
||
|
||
Verify it's up:
|
||
|
||
```bash
|
||
curl -fsS https://pbx.example.com/healthz && echo OK # liveness
|
||
curl -fsS https://pbx.example.com/readyz && echo READY # can accept a new call
|
||
```
|
||
|
||
### First call (browser, WebRTC)
|
||
|
||
Open `https://pbx.example.com/`, click **Start call**, grant the microphone. You're on a call
|
||
with the engine.
|
||
|
||
### First call (phone, PSTN)
|
||
|
||
1. Twilio Console → Phone Numbers → your number → **A call comes in** → Webhook,
|
||
`POST https://pbx.example.com/v1/trunk/webhook`.
|
||
2. Dial your Twilio number from any phone. Twilio hits the webhook, the engine answers with
|
||
TwiML pointing Twilio's Media Streams at `wss://pbx.example.com/twilio/media-stream`, and
|
||
the call's audio enters the same reflex loop a WebRTC call uses.
|
||
|
||
## T2 — Modular (compose stack)
|
||
|
||
```bash
|
||
git clone https://git.adlee.work/alee/rutster.git
|
||
cd rutster/deploy
|
||
cp .env.example .env
|
||
$EDITOR .env # set DOMAIN, ACME email, RUTSTER_MEDIA_ADVERTISED_IP, RUTSTER_TWILIO_*
|
||
docker compose up -d
|
||
```
|
||
|
||
What comes up: `caddy` (edge, `rutster-edge`), `engine` (`rutster-engine`), `brain`
|
||
(`rutster-brain`, sharing the engine's network namespace), `valkey` (upstream image). The
|
||
same two first-call paths as T1 apply verbatim.
|
||
|
||
Operational notes:
|
||
|
||
- `.env.example` ships `RUTSTER_DRAIN_DEADLINE_SECS=600` and the stack sets
|
||
`stop_grace_period: 660s` — grace exceeds drain, so `docker compose down` lets in-flight
|
||
calls finish (up to 10 minutes) instead of killing them.
|
||
- Upgrade: `docker compose pull && docker compose up -d`.
|
||
- Bring your own proxy instead of Caddy: [reverse-proxies.md](reverse-proxies.md).
|
||
|
||
## Troubleshooting
|
||
|
||
| Symptom | Cause / fix |
|
||
|---|---|
|
||
| Twilio error **31910** on calls | Your cert isn't publicly trusted or expired — check `/data` volume survived, see [certificates.md](certificates.md) ([twilio.com/docs/api/errors/31910](https://www.twilio.com/docs/api/errors/31910)) |
|
||
| Webhook signature validation fails | The edge isn't forwarding `X-Forwarded-Proto/Host` honestly, or `RUTSTER_TRUSTED_PROXIES` doesn't include your proxy — the engine ignores forwarded headers from unlisted sources |
|
||
| Browser call connects, no audio | Media UDP blocked or `RUTSTER_MEDIA_ADVERTISED_IP` wrong — the SDP advertises that IP; callers send UDP straight to it |
|
||
| `curl /readyz` returns 503 | Node is draining or at the admission cap (`RUTSTER_MAX_SESSIONS`) — liveness (`/healthz`) stays 200 |
|
||
| Let's Encrypt rate-limit errors in Caddy logs | You recreated the container without the `/data` volume — [certificates.md](certificates.md) |
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/quickstart-docker.md \
|
||
&& grep -q 'rutster-allinone' docs/deploy/quickstart-docker.md \
|
||
&& grep -q 'docker compose up' docs/deploy/quickstart-docker.md \
|
||
&& grep -q '/v1/trunk/webhook' docs/deploy/quickstart-docker.md \
|
||
&& grep -q 'all-or-none' docs/deploy/quickstart-docker.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/quickstart-docker.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; BROKEN only for sibling docs not yet written.
|
||
|
||
- [ ] **Step 4: Cross-check against slice B (if landed)**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
ls deploy/ 2>/dev/null && { grep -n 'image:' deploy/compose.yaml; grep -n '^[A-Z]' deploy/.env.example; }
|
||
```
|
||
|
||
Reconcile: image names, the `RUTSTER_DOMAIN`/`RUTSTER_ACME_EMAIL` variable names (these two
|
||
are artifact-level names owned by slice B's Caddyfile — the spec does not fix them), the
|
||
compose service names, and the git clone URL. If `deploy/` does not exist yet, keep the doc as
|
||
written and leave the artifact-names note box in place.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/quickstart-docker.md
|
||
git commit -s -m "docs(deploy): quickstart-docker.md — T1 docker run + T2 compose, zero to first call (slice-G)
|
||
|
||
The acceptance-bar doc: a copy-paste path from a bare Linux host with a
|
||
domain to (1) a browser WebRTC call and (2) a real PSTN call via the
|
||
Twilio webhook. Volumes marked non-negotiable with the LE-lockout reason;
|
||
all-or-none RUTSTER_TWILIO_* contract stated (matches config.rs); host
|
||
networking recommended for direct media UDP."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: `docs/deploy/homelab.md` — the CGNAT truth
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/homelab.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §3.5, brief §3(d) + §2 invariant 9 + §5 risks 3/6, the brief's URLs
|
||
(cloudflared #1465/#1282, livekit/agents #3379).
|
||
- Produces: the doc `topologies.md` and ADR-0011 link as `homelab.md` /
|
||
`../deploy/homelab.md`.
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/homelab.md \
|
||
&& grep -q 'PSTN-only' docs/deploy/homelab.md \
|
||
&& grep -q 'WireGuard' docs/deploy/homelab.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/homelab.md` with exactly this content:
|
||
|
||
````markdown
|
||
# Homelab & CGNAT — the honest story
|
||
|
||
**The truth first: no tunnel carries inbound UDP. WebRTC callers are unreachable behind any
|
||
tunnel, so a homelab behind CGNAT is PSTN-only, period.** The engine's WebRTC media is
|
||
UDP-direct to an advertised IP (no STUN, no TURN); the CPaaS side offers no relay or
|
||
rendezvous either, and intermediaries in the media path are a documented frame-dropping hazard
|
||
([livekit/agents#3379](https://github.com/livekit/agents/issues/3379)). Nothing on this page
|
||
makes CGNAT production-grade for free. Three tiers, worst to best:
|
||
|
||
## Tier 1 — dev/demo: ngrok (the blessed 5-minute path)
|
||
|
||
ngrok is the **only** tunnel with a proven Twilio Media Streams record. Do **not** use
|
||
Cloudflare Tunnel even for dev: cloudflared has an open Twilio 31920 handshake bug
|
||
([cloudflared#1465](https://github.com/cloudflare/cloudflared/issues/1465)), recurring 1006
|
||
closures ([cloudflared#1282](https://github.com/cloudflare/cloudflared/issues/1282)), and
|
||
Cloudflare documents killing WebSockets on edge code releases.
|
||
|
||
```bash
|
||
# 1. Run the engine, plaintext :8080 (no Caddy needed — ngrok terminates TLS):
|
||
docker run -d --name rutster-engine --network host \
|
||
-e RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
|
||
-e RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token \
|
||
-e RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081 \
|
||
-e RUTSTER_TWILIO_WEBHOOK_BASE=https://REPLACE-ME.ngrok-free.app \
|
||
-e RUTSTER_TRUSTED_PROXIES=127.0.0.1/32 \
|
||
git.adlee.work/alee/rutster-engine:latest
|
||
# (or from source: cargo run — see docs/QUICKSTART.md)
|
||
|
||
# 2. Tunnel it:
|
||
ngrok http 8080
|
||
# 3. Put the printed https://xxxx.ngrok-free.app URL into RUTSTER_TWILIO_WEBHOOK_BASE
|
||
# (restart the container), and into the Twilio number's webhook:
|
||
# POST https://xxxx.ngrok-free.app/v1/trunk/webhook
|
||
# 4. Dial your Twilio number.
|
||
```
|
||
|
||
**Free-tier arithmetic** (why this is a demo, not a deployment — research: [TLS brief §3(d)](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)): a Media Streams call is
|
||
8 kHz µ-law — 8 kB/s of audio per direction, base64-encoded inside a JSON envelope at 50
|
||
messages/s, both directions ≈ **~140 MB per call-hour** on the wire. ngrok's free 1 GB/month
|
||
data cap ([ngrok pricing](https://ngrok.com/pricing)) therefore buys roughly **seven call-hours a month**. And the audio transits ngrok's
|
||
edge **in plaintext** (they terminate TLS) — an unconsented subprocessor, which is a
|
||
DPA/BAA/PCI failure. Dev only. Never production.
|
||
|
||
## Tier 2 — single-user demo: Tailscale Funnel
|
||
|
||
Privacy-clean variant: TLS terminates **on your node**, so Funnel relays ciphertext it cannot
|
||
read.
|
||
|
||
```bash
|
||
tailscale funnel 8080
|
||
# webhook base = https://<your-node>.<tailnet>.ts.net
|
||
```
|
||
|
||
Still PSTN-only (no inbound UDP), bandwidth cap undisclosed, one user. Unsizable beyond a
|
||
personal demo.
|
||
|
||
## Tier 3 — production graduation: cheap VPS + WireGuard, TLS at home
|
||
|
||
The recommended path. The VPS is a **dumb layer-4 forwarder**: TLS terminates at home (your
|
||
Caddy, DNS-01 certs), so the forwarder **physically cannot read the audio** — the strongest
|
||
privacy topology available. Bonus: forwarding the media UDP range over the same tunnel
|
||
restores WebRTC, which no tunnel product can do. Cost: one small VPS (~$5/mo) and one extra
|
||
network hop of media latency — pick a VPS region near home.
|
||
|
||
Point DNS at the VPS: `pbx.example.com A <VPS-public-IP>`.
|
||
|
||
**VPS `/etc/wireguard/wg0.conf`:**
|
||
|
||
```ini
|
||
[Interface]
|
||
Address = 10.88.0.1/24
|
||
ListenPort = 51820
|
||
PrivateKey = <vps-private-key>
|
||
|
||
[Peer]
|
||
PublicKey = <home-public-key>
|
||
AllowedIPs = 10.88.0.2/32
|
||
```
|
||
|
||
**Home box `/etc/wireguard/wg0.conf`** (home initiates — CGNAT-friendly; the keepalive holds
|
||
the NAT mapping):
|
||
|
||
```ini
|
||
[Interface]
|
||
Address = 10.88.0.2/24
|
||
PrivateKey = <home-private-key>
|
||
|
||
[Peer]
|
||
PublicKey = <vps-public-key>
|
||
Endpoint = <VPS-public-IP>:51820
|
||
AllowedIPs = 10.88.0.0/24
|
||
PersistentKeepalive = 25
|
||
```
|
||
|
||
**VPS forwarding** (`sysctl -w net.ipv4.ip_forward=1`, persist it, then nftables):
|
||
|
||
```nft
|
||
table ip rutster-fwd {
|
||
chain prerouting {
|
||
type nat hook prerouting priority dstnat;
|
||
iifname "eth0" tcp dport { 80, 443 } dnat to 10.88.0.2
|
||
iifname "eth0" udp dport 49152-49407 dnat to 10.88.0.2
|
||
}
|
||
chain postrouting {
|
||
type nat hook postrouting priority srcnat;
|
||
oifname "wg0" masquerade
|
||
}
|
||
}
|
||
```
|
||
|
||
**Home side:** run T1 or T2 exactly per [quickstart-docker.md](quickstart-docker.md), with:
|
||
|
||
- `RUTSTER_MEDIA_ADVERTISED_IP=<VPS-public-IP>` — callers send media UDP to the VPS; the DNAT
|
||
delivers it home through the tunnel.
|
||
- `RUTSTER_MEDIA_PORT_RANGE=49152-49407` (must match the nftables rule).
|
||
- Certificates: DNS-01 is the robust choice here ([certificates.md](certificates.md));
|
||
HTTP-01 also works since `:80` is forwarded.
|
||
|
||
First call: same two paths as [quickstart-docker.md](quickstart-docker.md) — browser at
|
||
`https://pbx.example.com/` (WebRTC now works — the UDP range rides the tunnel) and the Twilio
|
||
webhook at `https://pbx.example.com/v1/trunk/webhook`.
|
||
|
||
**Or skip the tunnel entirely:** run the engine *on* the VPS (that is just
|
||
[T1](quickstart-docker.md) on rented hardware). You trade at-home media for zero forwarding
|
||
complexity.
|
||
|
||
## Explicitly unsupported for production
|
||
|
||
Cloudflare Tunnel or ngrok in the live audio path: plaintext audio at the vendor edge
|
||
(unconsented subprocessor — DPA/BAA/PCI failure), documented mid-call WS terminations, zero
|
||
SLA, and Cloudflare's discretionary "disproportionate audio" ToS clause aimed at exactly this
|
||
traffic profile (research basis: [TLS brief §4](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)). Ratified in [ADR-0011](../adr/0011-deployment-topology.md).
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/homelab.md \
|
||
&& grep -q 'PSTN-only, period' docs/deploy/homelab.md \
|
||
&& grep -q 'seven call-hours' docs/deploy/homelab.md \
|
||
&& grep -q 'Tailscale Funnel' docs/deploy/homelab.md \
|
||
&& grep -q 'physically cannot read the audio' docs/deploy/homelab.md \
|
||
&& grep -q 'cloudflared/issues/1465' docs/deploy/homelab.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/homelab.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; BROKEN only for sibling docs not yet written.
|
||
|
||
- [ ] **Step 4: Cross-check against slice B (if landed)** — same command as Task 3 Step 4;
|
||
reconcile the `rutster-engine` image name and env-var names if `deploy/` exists.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/homelab.md
|
||
git commit -s -m "docs(deploy): homelab.md — CGNAT truth: PSTN-only behind tunnels, VPS+WireGuard graduation (slice-G)
|
||
|
||
Acceptance bar honored by honesty: the doc leads with what cannot work
|
||
(no tunnel carries inbound UDP -> WebRTC unreachable behind CGNAT). Three
|
||
tiers: ngrok blessed dev path with the free-tier arithmetic (~140 MB/
|
||
call-hour -> ~7 call-hours/month), Tailscale Funnel single-user demo
|
||
(ciphertext-only), and the VPS+WireGuard graduation with TLS terminating
|
||
at home (forwarder physically cannot read audio; media UDP DNAT restores
|
||
WebRTC). cloudflared explicitly excluded even for dev (#1465, #1282)."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: `docs/deploy/aws.md` — the ALB appendix
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/aws.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §3.5 (AWS appendix bullet), brief §3(c) + §4 (ALB row, NLB rejection).
|
||
- Produces: the doc `topologies.md` and ADR-0011 link as `aws.md` / `../deploy/aws.md`.
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/aws.md \
|
||
&& grep -q 'idle_timeout' docs/deploy/aws.md \
|
||
&& grep -q '350' docs/deploy/aws.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/aws.md` with exactly this content:
|
||
|
||
````markdown
|
||
# AWS appendix — ALB + ACM (docs only, never a shipped artifact)
|
||
|
||
Scope: operators **already on EC2**. There, an ALB is trust-neutral (the engine is on AWS
|
||
infrastructure regardless) and ACM renewal is genuinely zero-touch. This page is an appendix,
|
||
not a recommendation to move to AWS — the shipped edges are Caddy
|
||
([topologies.md](topologies.md)) and BYO proxies ([reverse-proxies.md](reverse-proxies.md)).
|
||
All ALB/NLB facts below are grounded in the [TLS/edge decision brief §3(c)](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md).
|
||
|
||
## Never NLB-TLS
|
||
|
||
Do not put the engine behind an NLB TLS listener. Two disqualifiers, ratified in
|
||
[ADR-0011](../adr/0011-deployment-topology.md):
|
||
|
||
1. **Fixed 350-second idle timeout** that silently drops flows — any quiet WebSocket (hold,
|
||
parked call, unidirectional stream) dies mid-call with no close frame.
|
||
2. **No HTTP context**: an NLB adds no `X-Forwarded-Proto/Host`, so the engine cannot
|
||
reconstruct the public URL that `X-Twilio-Signature` is HMAC'd over
|
||
([Twilio's SSL-termination guidance](https://www.twilio.com/en-us/blog/developers/tutorials/building-blocks/handle-ssl-termination-twilio-node-js-helper-library))
|
||
— genuine traffic fails validation.
|
||
|
||
## The three mandatory ALB attribute overrides
|
||
|
||
The defaults kill telephony. All three overrides are **mandatory**, not tuning:
|
||
|
||
| Attribute | Value | Why |
|
||
|---|---|---|
|
||
| `idle_timeout.timeout_seconds` | `4000` | The default 60 s kills any quiet WS. 4000 is the ALB maximum — still finite, so the engine's app-level WS pings (`RUTSTER_WS_PING_SECS`, default 20) remain load-bearing here. |
|
||
| `client_keep_alive.seconds` | `604800` | The 3600 s default can terminate the client connection under a **>1-hour call**. Set to the 7-day maximum. |
|
||
| `routing.http.preserve_host_header.enabled` | `true` | Signature validation reconstructs the URL *as Twilio saw it* — the engine must receive the honest public `Host`, and its `RUTSTER_TRUSTED_PROXIES` must include the ALB's subnet CIDRs. |
|
||
|
||
## Copy-paste path (CLI)
|
||
|
||
Assumes: engine node(s) running per [quickstart-docker.md](quickstart-docker.md) with the
|
||
`caddy` service disabled (plaintext `:8080`), a VPC, two subnets, and a security group
|
||
allowing `:443` from anywhere and `:8080` from the ALB.
|
||
|
||
```bash
|
||
VPC=vpc-0123456789abcdef0
|
||
SUBNETS="subnet-0aaa subnet-0bbb"
|
||
SG=sg-0ccc
|
||
|
||
ALB_ARN=$(aws elbv2 create-load-balancer --name rutster-edge --type application \
|
||
--subnets $SUBNETS --security-groups $SG \
|
||
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
|
||
|
||
# THE THREE MANDATORY OVERRIDES — do not skip:
|
||
aws elbv2 modify-load-balancer-attributes --load-balancer-arn "$ALB_ARN" --attributes \
|
||
Key=idle_timeout.timeout_seconds,Value=4000 \
|
||
Key=client_keep_alive.seconds,Value=604800 \
|
||
Key=routing.http.preserve_host_header.enabled,Value=true
|
||
|
||
TG_ARN=$(aws elbv2 create-target-group --name rutster-node-1 \
|
||
--protocol HTTP --port 8080 --vpc-id "$VPC" \
|
||
--health-check-path /readyz \
|
||
--query 'TargetGroups[0].TargetGroupArn' --output text)
|
||
aws elbv2 register-targets --target-group-arn "$TG_ARN" --targets Id=i-0node1
|
||
|
||
CERT_ARN=$(aws acm request-certificate --domain-name '*.pbx.example.com' \
|
||
--validation-method DNS --query CertificateArn --output text)
|
||
# Create the DNS validation CNAME ACM reports, wait for ISSUED, then:
|
||
|
||
LISTENER_ARN=$(aws elbv2 create-listener --load-balancer-arn "$ALB_ARN" \
|
||
--protocol HTTPS --port 443 \
|
||
--ssl-policy ELBSecurityPolicy-TLS13-1-2-2021-06 \
|
||
--certificates CertificateArn="$CERT_ARN" \
|
||
--default-actions Type=forward,TargetGroupArn="$TG_ARN" \
|
||
--query 'Listeners[0].ListenerArn' --output text)
|
||
|
||
# Fleet growth: one target group + one host-header rule per node
|
||
# (node-addressed placement — ADR-0011):
|
||
aws elbv2 create-rule --listener-arn "$LISTENER_ARN" --priority 10 \
|
||
--conditions Field=host-header,Values=node-1.pbx.example.com \
|
||
--actions Type=forward,TargetGroupArn="$TG_ARN"
|
||
```
|
||
|
||
Point `*.pbx.example.com` (alias record) at the ALB. First call: identical to
|
||
[quickstart-docker.md](quickstart-docker.md) — browser at your domain, Twilio webhook at
|
||
`https://pbx.example.com/v1/trunk/webhook`.
|
||
|
||
The `--ssl-policy` above serves TLS 1.2+1.3 — never pick a 1.3-only policy (Twilio's TLS 1.3
|
||
client support is undocumented; see the invariants in
|
||
[certificates.md](certificates.md)).
|
||
|
||
## WebRTC media does not ride the ALB
|
||
|
||
The ALB carries HTTPS/WSS only. WebRTC media UDP goes **direct to each node**: give every
|
||
engine node an Elastic IP, open the media UDP range in the node's security group, and set
|
||
`RUTSTER_MEDIA_ADVERTISED_IP` to that EIP.
|
||
|
||
## Caps and traps
|
||
|
||
- **Hard cap: 100 target groups per ALB.** With one TG per node (host-header routing), that
|
||
is your fleet ceiling per ALB.
|
||
- **AWS fleet rotation can still cut a multi-hour call** — ALB infrastructure is replaced
|
||
under maintenance without regard for your WS lifetimes. Accepted residual risk; there is no
|
||
attribute for it.
|
||
- Health checks target `/readyz` (503 while draining or at the admission cap) — scale-in must
|
||
be drain-then-terminate: deregister the target, wait out
|
||
`RUTSTER_DRAIN_DEADLINE_SECS`, then terminate the instance.
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/aws.md \
|
||
&& grep -q 'idle_timeout.timeout_seconds' docs/deploy/aws.md \
|
||
&& grep -q '604800' docs/deploy/aws.md \
|
||
&& grep -q 'preserve_host_header' docs/deploy/aws.md \
|
||
&& grep -q '100 target groups' docs/deploy/aws.md \
|
||
&& grep -q '350-second' docs/deploy/aws.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/aws.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; BROKEN only for sibling docs not yet written.
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/aws.md
|
||
git commit -s -m "docs(deploy): aws.md — ALB appendix: three mandatory overrides, never NLB-TLS (slice-G)
|
||
|
||
Docs-only appendix per ADR-0011 (never a shipped artifact). The three
|
||
mandatory ALB attribute overrides (idle_timeout=4000,
|
||
client_keep_alive=604800, preserve_host_header=true) with the telephony
|
||
reasons; the never-NLB-TLS rule with the 350s-silent-drop + no-HTTP-
|
||
context rationale; 100-target-group fleet ceiling; per-node EIPs for
|
||
direct media UDP; copy-paste aws elbv2 CLI path to first call."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: `docs/deploy/reverse-proxies.md` — BYO nginx/HAProxy/Traefik
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/reverse-proxies.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §2.2 (BYO-proxy seam) + §10, brief §3(b) + §4 (runner-up table: HAProxy
|
||
`timeout tunnel`/#2280/`hard-stop-after`; Traefik pin-exact-versions #10601 / v2.11.2 /
|
||
#11405; nginx not-recommended rationale).
|
||
- Produces: the doc `topologies.md`/`quickstart-docker.md` link as `reverse-proxies.md`.
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/reverse-proxies.md \
|
||
&& grep -q 'HAProxy' docs/deploy/reverse-proxies.md \
|
||
&& grep -qi 'pin' docs/deploy/reverse-proxies.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/reverse-proxies.md` with exactly this content:
|
||
|
||
````markdown
|
||
# Bring your own reverse proxy — tuned-timeout configs
|
||
|
||
Supported: disable the `caddy` service in the compose stack
|
||
([quickstart-docker.md](quickstart-docker.md#t2--modular-compose-stack)); the engine keeps its
|
||
plaintext `:8080` listener. Everything below terminates TLS in *your* proxy and forwards to
|
||
the engine.
|
||
|
||
## The universal 60-second footgun
|
||
|
||
**nginx, HAProxy, and Traefik all default their idle/read timeouts to ~60–180 seconds — every
|
||
one of them will kill a quiet, perfectly healthy call WebSocket.** A Media Streams WS can
|
||
legitimately be near-silent in one direction for hours, and neither Twilio nor Telnyx sends
|
||
protocol-level keepalives. The engine ships app-level WS pings (`RUTSTER_WS_PING_SECS`,
|
||
default 20) as belt-and-braces, but your proxy timeouts must still exceed the maximum call
|
||
duration. Every snippet below does that.
|
||
|
||
## The contract any proxy must honor
|
||
|
||
1. **Publicly-trusted cert**, auto-renewed — no self-signed path exists
|
||
([certificates.md](certificates.md)).
|
||
2. **WS upgrade with no frame buffering and no connection-lifetime cap** — 20 ms audio frames,
|
||
calls up to 24 h.
|
||
3. **Honest `X-Forwarded-Proto` and `X-Forwarded-Host`** (including on the WS upgrade
|
||
request): `X-Twilio-Signature` is HMAC over the URL as Twilio saw it
|
||
([Twilio's SSL-termination guidance](https://www.twilio.com/en-us/blog/developers/tutorials/building-blocks/handle-ssl-termination-twilio-node-js-helper-library)).
|
||
4. **Engine-side trust**: set `RUTSTER_TRUSTED_PROXIES` to your proxy's source IP/CIDR —
|
||
forwarded headers from unlisted sources are ignored, and signature validation will fail.
|
||
5. **TLS 1.2+1.3, mainstream ECDHE — never 1.3-only** (Twilio's 1.3 client support is
|
||
undocumented).
|
||
|
||
## nginx (works if you insist — not a recommended edge)
|
||
|
||
Why not recommended: no native DNS-01/wildcard support, and cert renewal requires a reload
|
||
whose old workers linger against hours-long WS connections (research basis: [TLS brief §4](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)). rutster ships this snippet and
|
||
nothing more for nginx.
|
||
|
||
```nginx
|
||
map $http_upgrade $connection_upgrade { default upgrade; '' close; }
|
||
|
||
server {
|
||
listen 443 ssl;
|
||
server_name pbx.example.com;
|
||
ssl_certificate /etc/letsencrypt/live/pbx.example.com/fullchain.pem;
|
||
ssl_certificate_key /etc/letsencrypt/live/pbx.example.com/privkey.pem;
|
||
ssl_protocols TLSv1.2 TLSv1.3; # never 1.3-only
|
||
|
||
location / {
|
||
proxy_pass http://127.0.0.1:8080;
|
||
proxy_http_version 1.1;
|
||
proxy_set_header Upgrade $http_upgrade;
|
||
proxy_set_header Connection $connection_upgrade;
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
proxy_set_header X-Forwarded-Host $host;
|
||
proxy_buffering off; # 20 ms frames must not be coalesced
|
||
proxy_read_timeout 86400s; # the 60s default is the footgun
|
||
proxy_send_timeout 86400s;
|
||
}
|
||
}
|
||
```
|
||
|
||
## HAProxy (3.2 LTS or newer)
|
||
|
||
The best WS mental model of the classic family: after the upgrade, only `timeout tunnel`
|
||
governs the connection — **but only if you set it**; put it in `defaults` or it silently
|
||
doesn't apply to the paths you think it does (see
|
||
[haproxy#2280](https://github.com/haproxy/haproxy/issues/2280)). Two extra notes: `acme.sh`
|
||
can hot-load renewed certs through the stats socket with zero reload; and `hard-stop-after`
|
||
kills calls that are still draining — leave it unset or above your drain deadline.
|
||
|
||
```haproxy
|
||
defaults
|
||
mode http
|
||
timeout connect 5s
|
||
timeout client 75s
|
||
timeout server 75s
|
||
timeout tunnel 24h # governs upgraded WS; unset = the 60s-class footgun
|
||
|
||
frontend fe_rutster
|
||
bind :443 ssl crt /etc/haproxy/certs/ ssl-min-ver TLSv1.2
|
||
http-request set-header X-Forwarded-Proto https
|
||
http-request set-header X-Forwarded-Host %[req.hdr(Host)]
|
||
default_backend be_rutster
|
||
|
||
backend be_rutster
|
||
server fob 127.0.0.1:8080
|
||
```
|
||
|
||
## Traefik v3 — PIN THE EXACT VERSION
|
||
|
||
Traefik has the best built-in DNS-01/wildcard of the classic family **and a track record of
|
||
breaking WebSockets in patch releases**:
|
||
[#10601](https://github.com/traefik/traefik/issues/10601), the v2.11.2 timeout-behavior flip,
|
||
and [#11405](https://github.com/traefik/traefik/issues/11405). **Pin the exact image tag and
|
||
never auto-update the edge.**
|
||
|
||
```yaml
|
||
# traefik.yml (static config)
|
||
entryPoints:
|
||
websecure:
|
||
address: ":443"
|
||
transport:
|
||
respondingTimeouts:
|
||
readTimeout: 0 # 0 disables; the 60s-class default kills quiet WS
|
||
idleTimeout: 0
|
||
writeTimeout: 0
|
||
certificatesResolvers:
|
||
le:
|
||
acme:
|
||
email: you@example.com
|
||
storage: /data/acme.json # persist this volume — certificates.md
|
||
dnsChallenge:
|
||
provider: cloudflare
|
||
```
|
||
|
||
```yaml
|
||
# dynamic config
|
||
http:
|
||
routers:
|
||
rutster:
|
||
rule: "Host(`pbx.example.com`)"
|
||
entryPoints: [websecure]
|
||
service: rutster
|
||
tls: { certResolver: le }
|
||
services:
|
||
rutster:
|
||
loadBalancer:
|
||
servers:
|
||
- url: "http://127.0.0.1:8080"
|
||
```
|
||
|
||
Traefik forwards `X-Forwarded-Proto/Host` by default — you still must list its address in
|
||
`RUTSTER_TRUSTED_PROXIES`.
|
||
|
||
## First call
|
||
|
||
Any of the above in front of the engine, then the same two first-call paths as
|
||
[quickstart-docker.md](quickstart-docker.md): browser at `https://pbx.example.com/`, Twilio
|
||
webhook at `https://pbx.example.com/v1/trunk/webhook`.
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/reverse-proxies.md \
|
||
&& grep -q 'proxy_read_timeout 86400s' docs/deploy/reverse-proxies.md \
|
||
&& grep -q 'timeout tunnel' docs/deploy/reverse-proxies.md \
|
||
&& grep -q 'PIN THE EXACT VERSION' docs/deploy/reverse-proxies.md \
|
||
&& grep -q 'RUTSTER_TRUSTED_PROXIES' docs/deploy/reverse-proxies.md \
|
||
&& grep -q '60' docs/deploy/reverse-proxies.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/reverse-proxies.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; BROKEN only for sibling docs not yet written.
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/reverse-proxies.md
|
||
git commit -s -m "docs(deploy): reverse-proxies.md — BYO nginx/HAProxy/Traefik tuned-timeout configs (slice-G)
|
||
|
||
The universal 60s idle-timeout footgun stated up front; the five-point
|
||
contract any proxy must honor (incl. RUTSTER_TRUSTED_PROXIES + honest
|
||
X-Forwarded-* for signature validation); complete snippets: nginx
|
||
(explicitly not-recommended, snippet-only support), HAProxy (timeout
|
||
tunnel in defaults per #2280; hard-stop-after warning), Traefik (pin the
|
||
exact version — #10601 / v2.11.2 / #11405 WS-breaking patch history)."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: `docs/deploy/certificates.md` — ACME paths, wildcard traps, `/data`
|
||
|
||
**Files:**
|
||
- Create: `docs/deploy/certificates.md`
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: spec §3.1–§3.4, brief §2 invariants 1/7/8, §3(a)/(c), §5 risks 1/2/5; brief URLs
|
||
(twilio security page, errors 31910/11237, caddy #6420/#7222).
|
||
- Produces: the doc every other tree file links as `certificates.md`.
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/certificates.md \
|
||
&& grep -q 'duplicate' docs/deploy/certificates.md \
|
||
&& grep -q '/data' docs/deploy/certificates.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Write the complete file**
|
||
|
||
Write `docs/deploy/certificates.md` with exactly this content:
|
||
|
||
````markdown
|
||
# Certificates & ACME — the availability-critical subsystem
|
||
|
||
**A publicly-CA-trusted certificate is a hard CPaaS requirement; no self-signed path exists.**
|
||
Twilio refuses self-signed certs for webhooks and Media Streams (error
|
||
[31910](https://www.twilio.com/docs/api/errors/31910); see
|
||
[twilio.com/docs/usage/security](https://www.twilio.com/docs/usage/security)) — and the
|
||
console's cert-validation toggle is account-wide, webhook-only, and dev-only, so it is not an
|
||
escape hatch. In a 24/7 product whose calls are non-migratable there is no maintenance window:
|
||
**auto-renewal is availability-critical, not a nicety.**
|
||
|
||
Protocol posture everywhere: TLS 1.2+1.3, mainstream ECDHE suites, **never 1.3-only**
|
||
(Twilio's TLS 1.3 client support is undocumented), and never pin Twilio's certificates.
|
||
|
||
## The `/data` volume — read this before anything else
|
||
|
||
Caddy keeps its certificates and ACME account state in `/data`. If you recreate the container
|
||
without that volume, Caddy re-requests the same certificate — and Let's Encrypt's
|
||
**duplicate-certificate limit (5 per week for an identical name set,
|
||
[letsencrypt.org/docs/rate-limits](https://letsencrypt.org/docs/rate-limits/))** will lock
|
||
your domain out for up to a week. The failure class is **total inbound outage**: 31910 on
|
||
Media Streams, [11237](https://www.twilio.com/docs/api/errors/11237) on webhooks, no calls in
|
||
or out of the trunk. A container recreate loop (crash-looping deploy, CI that recreates on
|
||
every push) burns all five in minutes.
|
||
|
||
- Always mount `/data` on a persistent volume (`-v rutster-caddy-data:/data`).
|
||
- Back it up like state, because it is:
|
||
`docker run --rm -v rutster-caddy-data:/data -v "$PWD":/backup debian:stable-slim tar czf /backup/caddy-data.tgz /data`
|
||
|
||
## ACME challenge paths
|
||
|
||
| Path | When | Notes |
|
||
|---|---|---|
|
||
| **HTTP-01** (default) | Port 80 publicly reachable, single hostname | Zero config beyond the domain + email. |
|
||
| **DNS-01** | Wildcards (`*.pbx.domain`), CGNAT/behind-NAT hosts, no port 80 | The bundled `rutster-edge` Caddy build ships a curated plugin set: **cloudflare, route53, porkbun, hetzner, desec** (duckdns excluded — no license file). Any other DNS provider requires your own xcaddy build. DNS API credentials live in the container env — scope them to the zone. |
|
||
| **BYO-cert** (in-process rustls, Phase 1) | You already have cert distribution (corporate ACME, central wildcard issuance) | Set `RUTSTER_TLS_CERT` / `RUTSTER_TLS_KEY`; the engine serves TLS itself and **hot-reloads on file change without dropping live WS**. You own renewal delivery. |
|
||
|
||
In-binary ACME (Phase 2) is deliberately deferred behind named triggers — among them Let's
|
||
Encrypt's dns-persist-01 reaching GA, which would dissolve the DNS-plugin matrix entirely. See
|
||
[ADR-0011](../adr/0011-deployment-topology.md).
|
||
|
||
## Caddy reload vs live calls
|
||
|
||
Routine renewals are safe: Caddy swaps certs **in memory, zero-downtime — the routine periodic
|
||
event drops nothing.** Config *reloads* (editing the Caddyfile) are the risk: the mitigation
|
||
(`stream_close_delay` above max call duration) has an open upstream bug trail
|
||
([caddy#6420](https://github.com/caddyserver/caddy/issues/6420),
|
||
[caddy#7222](https://github.com/caddyserver/caddy/issues/7222)), which is why the shipped
|
||
artifacts carry a CI e2e test for config-reload-during-live-call. Operator rule: **do not edit
|
||
the Caddyfile while calls are live** unless your image version passed that test; drain first
|
||
(`/readyz` returns 503 while draining).
|
||
|
||
## Fleet certificate patterns (T3)
|
||
|
||
Node-addressed placement means every node needs its own publicly-valid TLS name — this is
|
||
CPaaS-imposed (`<Stream>` URLs route on hostname only), not a proxy choice. Two blessed
|
||
patterns:
|
||
|
||
1. **Wildcard `*.pbx.domain`** via DNS-01, issued **centrally** and distributed to nodes.
|
||
Understand the trade: the wildcard private key on every node means one compromised node
|
||
burns the whole namespace.
|
||
2. **Per-node distinct certs** (`node-1.pbx.domain`, `node-2.…`) — no shared key material.
|
||
Renewals are exempt from Let's Encrypt's per-domain limits, so this scales with fleet size.
|
||
|
||
Traps:
|
||
|
||
- **N nodes independently requesting the identical wildcard** are N duplicate requests — the
|
||
5/week duplicate-cert limit locks the fleet out. Central issuance or distinct names only.
|
||
- **Caddy on-demand TLS is rejected for node names**: the first TLS handshake to a fresh name
|
||
blocks for seconds on issuance, colliding with the sub-second webhook budget (Twilio's hard
|
||
cap is 15 s, UX budget sub-second), and its rate-limit knobs are deprecated. Node names are
|
||
known at provision time — pre-issue their certs.
|
||
|
||
## Troubleshooting
|
||
|
||
| Symptom | Fix |
|
||
|---|---|
|
||
| Twilio 31910 on Media Streams | Cert not publicly trusted / expired / SNI mismatch. Check `openssl s_client -connect pbx.example.com:443 -servername pbx.example.com`. |
|
||
| `rateLimited` / `too many certificates` in Caddy logs | You hit the duplicate-cert limit — almost always a lost `/data` volume. Restore the volume or wait out the window; then fix the volume mount so it never recurs. |
|
||
| DNS-01 fails for your provider | Provider not in the bundled plugin set — use a delegated zone on a supported provider, or build your own xcaddy image. |
|
||
````
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
test -f docs/deploy/certificates.md \
|
||
&& grep -q '5 per week' docs/deploy/certificates.md \
|
||
&& grep -q 'errors/31910' docs/deploy/certificates.md \
|
||
&& grep -q 'errors/11237' docs/deploy/certificates.md \
|
||
&& grep -q 'on-demand TLS is rejected' docs/deploy/certificates.md \
|
||
&& grep -q 'RUTSTER_TLS_CERT' docs/deploy/certificates.md \
|
||
&& grep -q 'never 1.3-only' docs/deploy/certificates.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
grep -oE '\]\([^)#]+\)' docs/deploy/certificates.md | sed 's/](//;s/)//' \
|
||
| grep -v '^http' | while read -r p; do [ -e "docs/deploy/$p" ] || echo "BROKEN: $p"; done
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; no BROKEN lines if Tasks 1–6 have run.
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/deploy/certificates.md
|
||
git commit -s -m "docs(deploy): certificates.md — ACME paths, /data persistence, wildcard traps (slice-G)
|
||
|
||
The availability-critical subsystem: no-self-signed-path (Twilio 31910),
|
||
the /data-loss -> LE 5/week duplicate-cert lockout -> total-inbound-outage
|
||
chain (31910 streams + 11237 webhooks) with backup command; HTTP-01 vs
|
||
DNS-01 (bundled plugin set: cloudflare/route53/porkbun/hetzner/desec,
|
||
duckdns excluded) vs BYO-cert rustls Phase 1 (RUTSTER_TLS_CERT/KEY,
|
||
hot-reload); Caddy renewal-vs-reload distinction with the #6420/#7222
|
||
caveat; fleet patterns (central wildcard vs per-node certs) with the
|
||
identical-wildcard trap and the on-demand-TLS rejection."
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: `docs/QUICKSTART.md` — point at the Docker fast path (minimal diff)
|
||
|
||
**Files:**
|
||
- Modify: `docs/QUICKSTART.md:13-15` (insert one blockquote between the status blockquote and
|
||
the `---` rule; verified against current content — the file currently flows from the status
|
||
blockquote ending `…for the active build target's design.` straight into `---` /
|
||
`## Prerequisites`)
|
||
- Test: bash content-check (Steps 1/3)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `docs/deploy/quickstart-docker.md` (Task 3 — must exist first).
|
||
- Produces: nothing downstream.
|
||
|
||
- [ ] **Step 1: Run the failing content-check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
grep -q 'deploy/quickstart-docker.md' docs/QUICKSTART.md && echo CHECK-PASS || echo CHECK-FAIL
|
||
```
|
||
|
||
Expected: `CHECK-FAIL`.
|
||
|
||
- [ ] **Step 2: Apply the minimal edit**
|
||
|
||
In `docs/QUICKSTART.md`, find this exact text (end of the status blockquote, lines 12–15):
|
||
|
||
```markdown
|
||
> [`docs/superpowers/specs/2026-07-05-slice-5-rented-transport-design.md`](superpowers/specs/2026-07-05-slice-5-rented-transport-design.md)
|
||
> for the active build target's design.
|
||
|
||
---
|
||
```
|
||
|
||
Replace with:
|
||
|
||
```markdown
|
||
> [`docs/superpowers/specs/2026-07-05-slice-5-rented-transport-design.md`](superpowers/specs/2026-07-05-slice-5-rented-transport-design.md)
|
||
> for the active build target's design.
|
||
|
||
> **Fastest path:** if you want a *running deployment* — TLS, a domain, a phone
|
||
> number — skip the source build and use the Docker quickstart:
|
||
> [`docs/deploy/quickstart-docker.md`](deploy/quickstart-docker.md). This page
|
||
> is the from-source developer path.
|
||
|
||
---
|
||
```
|
||
|
||
No other changes to the file.
|
||
|
||
- [ ] **Step 3: Re-run the content-check + link check**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
grep -q 'deploy/quickstart-docker.md' docs/QUICKSTART.md \
|
||
&& test -e docs/deploy/quickstart-docker.md \
|
||
&& echo CHECK-PASS || echo CHECK-FAIL
|
||
git diff --stat docs/QUICKSTART.md # expect: 1 file changed, ~6 insertions(+), 0 deletions(-)
|
||
```
|
||
|
||
Expected: `CHECK-PASS`; the diffstat shows insertions only (minimal diff honored).
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
git add docs/QUICKSTART.md
|
||
git commit -s -m "docs: QUICKSTART points at deploy/quickstart-docker.md as the fast path (slice-G)
|
||
|
||
Minimal diff: one blockquote after the status note. The from-source page
|
||
stays the developer path; the Docker quickstart is now the operator fast
|
||
path (slice-G docs tree)."
|
||
```
|
||
|
||
---
|
||
|
||
## Final acceptance checklist
|
||
|
||
After all 8 tasks:
|
||
|
||
- [ ] **Spec §10 acceptance bar, checked doc by doc:** `quickstart-docker.md` (copy-paste T1 +
|
||
T2 to first browser call and first PSTN call), `homelab.md` (copy-paste ngrok dev path +
|
||
VPS/WireGuard graduation, **plus** the honest CGNAT-WebRTC impossibility statement),
|
||
`aws.md` (copy-paste `aws elbv2` path), `reverse-proxies.md` (three complete configs +
|
||
first-call pointer), `topologies.md` (delegates copy-paste to quickstart; states T3
|
||
cannot be deployed today), `certificates.md` (supporting doc; backup + troubleshooting
|
||
commands are copy-paste).
|
||
- [ ] **All relative links resolve across the whole set** (no BROKEN output):
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
for f in docs/adr/0011-deployment-topology.md docs/deploy/*.md docs/QUICKSTART.md; do
|
||
d=$(dirname "$f")
|
||
grep -oE '\]\([^)#]+\)' "$f" | sed 's/](//;s/)//' | grep -v '^http' \
|
||
| while read -r p; do [ -e "$d/$p" ] || echo "BROKEN in $f: $p"; done
|
||
done
|
||
```
|
||
|
||
- [ ] **ADR-0011 format audit:** header bullet-list (Status/Date/Closes/Origin/Related) →
|
||
Context → Decision → Consequences → References, matching ADR-0007/0008; Status is
|
||
`Proposed` (ratification is the maintainer's move, not this slice's).
|
||
- [ ] **Slice-B reconciliation done or explicitly deferred:** if `deploy/` exists, image
|
||
names, service names, `.env.example` var names, and the registry namespace in the docs
|
||
match it; if not, the artifact-names note box in `quickstart-docker.md` is present.
|
||
- [ ] **Repo gates still green** (docs can't break them; prove it anyway):
|
||
|
||
```bash
|
||
cd /home/alee/Sources/rutster
|
||
cargo fmt --all --check && cargo clippy --all --all-targets -- -D warnings && cargo test --all && cargo deny check
|
||
git diff main --name-only | grep -E 'crates/rutster-media/src/(loop_driver|rtc_session)\.rs' && echo "SEAM VIOLATION" || echo "seam intact"
|
||
```
|
||
|
||
- [ ] Eight commits, each signed off (`git log --format='%h %s' -8` shows the slice-G
|
||
series; `git log -8 | grep -c 'Signed-off-by'` = 8).
|
||
- [ ] PR opened via `tea` per house workflow; the PR description notes: **ADR-0011 lands as
|
||
Proposed — ratification discussion is a separate step**, and names the plan-B
|
||
reconciliation status.
|