Files
rutster/docs/superpowers/plans/2026-07-05-deploy-b-packaging-ci.md
Aaron D. Lee fae9fcc82a docs(plans): four deployment-epoch implementation plans (slices A–G)
Plan A: engine hygiene (TCP_NODELAY via axum 0.7.9 Serve::tcp_nodelay,
trunk WS pings, webhook-base TwiML derivation, trusted-proxy posture).
Plan B: packaging + CI (four images, s6 all-in-one, compose, Caddyfile,
smoke suite incl. reload-during-call, slice-F publish workflow).
Plan C: in-binary features (rustls Phase 1 BYO-cert, /metrics,
ValkeyEventSink + smoke hook fill).
Plan G: docs/deploy tree + ADR-0011 drafting.

Authored by parallel plan-writers (this session + omo rescue after a
session-limit interruption); verified: placeholder scan, cited-path
existence, cross-plan image/env-var consistency, seam-gate compliance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012QndwfhjyTiZcUYp87dwW8
Signed-off-by: Aaron D. Lee <himself@adlee.work>
2026-07-05 21:37:17 -04:00

135 KiB
Raw Permalink Blame History

Deploy slice B + F — deploy/ artifacts (Dockerfile/compose/Caddyfile/.env) + image-build + container smoke CI + tag-push publish — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Dependency: This plan depends on plan A (2026-07-05-deploy-a-engine-hygiene.md) landing on main first. Plan A's three hygiene items are what make the containerized PSTN path honest: TCP_NODELAY (serve_with_nodelay) so the 20 ms WS cadence isn't Nagle-coalesced behind the edge, app-level WS pings so neither Twilio's nor Telnyx's idle timers can kill a quiet mid-call WS, and the trunk config fixes (RUTSTER_TWILIO_WEBHOOK_BASE → derived TwiML Stream URL + config::twilio_credentials startup validation + RUTSTER_TRUSTED_PROXIES trusted-X-Forwarded-Proto/Host reconstruction) so a container that takes a real Twilio call is actually answering the public URL the CPaaS dialed, not ws://127.0.0.1:8080 from slice-5's stub. Slice C (rustls Phase 1) also branches after A; B and C are order-independent. Do not start Task 1 of this plan until git log main --oneline | grep -c 'deploy-A' >= 1.

Goal: Land the deploy/ artifact tree (multi-stage Dockerfile producing four first-party images, compose.yaml, Caddyfile, .env.example) and the containerized CI muscle that proves them — image-build job, all-in-one TLS smoke through Caddy's internal CA, compose-health smoke, and Caddy config-reload-during-live-call — then add the slice-F tag-push publish workflow to the git.adlee.work/alee/ registry namespace (spec §2.1, §2.2, §3.2, §6, §9, §11).

Architecture: One multi-stage deploy/Dockerfile with named build targets (rutster-engine, rutster-brain, rutster-edge, rutster-allinone) so docker buildx build --target … produces each image from the same Dockerfile — eliminating the per-image Dockerfile drift that broke Asterisk-era integrators. The builder stage compiles against rust:1.85-slim-bookworm + libopus-dev (matching rust-toolchain.toml + dev loop exactly) and is a single target shared across engine/brain (both derive from the same cargo build --bins); the all-in-one image adds s6-overlay v3 + the upstream valkey/valkey server binary + the custom Caddy from a parallel xcaddy builder. CI is added to the existing .github/workflows/ci.yml (Gitea Actions consuming GitHub-workflow YAML) as two new jobs (image-build + smoke) plus a third workflow file (.github/workflows/publish-images.yml) for the slice-F tag-push publish. The smoke job reuses the WS-handshake protocol sequence (the {"event":"connected"…} + {"event":"start"…} frames) proven in crates/rutster/tests/trunk_sim_e2e.rs's TODO body and the in-tree ws_ping.rs/nodelay.rs integration tests — but driving it through the real edge→FOB path inside a booted container, against Caddy's internal CA (no ACME in CI). The ValkeyEventSink-lands-in-stream assertion is explicitly a slice-C concern; this plan leaves a named TODO hook comment in the compose-smoke step and does NOT implement it.

Tech Stack: Docker (BuildKit multi-stage), rust:1.85-slim-bookworm builder (pinned to rust-toolchain.toml), debian:bookworm-slim runtime (matches dev: apt-get install libopus0 in runtime, libopus-dev in builder — spec §6.1 "distro libopus0 over static-bundled"), s6-overlay v3.2.0.0 (ISC), xcaddy with the curated DNS-plugin set (cloudflare/route53/porkbun/hetzner/desec — duckdns excluded, no license file per spec §3.2), upstream valkey/valkey:latest (BSD-3), Caddy 2.8 builder image. CI: Gitea Actions (GitHub Actions YAML syntax), docker CLI, Python 3 for the smoke-test WS client (stdlib only — no pip install step in CI). Zero new Rust crates. Zero changes to cargo graphs.

Global Constraints

House rules (every task's requirements implicitly include this section):

  • License: GPL-3.0-or-later on every crate manifest (ADR-0004). No Rust crates touched in this slice; the rule stands for any task that would.
  • DCO: every commit signed off — git commit -s (AGENTS.md Git workflow).
  • SEAM GATE (UNCHANGED): crates/rutster-media/src/loop_driver.rs and crates/rutster-media/src/rtc_session.rs stay byte-identical — CI pins their blob hashes (744bf314edf7f4925c8bb3bd0f5176dbc88f8113 and f47d63b9a2883d37066a93c9daa0e2cf8816bec4 respectively, in .github/workflows/ci.yml lines 6061). NO task in this plan touches them. This slice is artifacts + CI, not the RTP path. Verify unchanged hashes in the final sweep (Task 11 Step 2).
  • Hot-path policy: never ?-propagate on the 20 ms tick; no unwrap()/expect() outside tests/startup. (No Rust code added; every smoke-test Python script honors it: WS read failures assert loudly with a counter snapshot — they never silently loop.)
  • Code style: cargo fmt --all --check + cargo clippy --all --all-targets -- -D warnings must stay green. (This slice adds no Rust; CI's clippy job stays the source of truth.)
  • MSRV: CI matrix runs stable + 1.85 (per rust-toolchain.toml). The Dockerfile builder stage MUST pin 1.85 exactly — not stable — so image builds reproduce what the MSRV matrix gates. No cargo update wholesale (rcgen 0.14.7 lock rationale per root Cargo.toml).
  • Workspace dep pinning: new deps go in the ROOT Cargo.toml [workspace.dependencies]; members reference with dep.workspace = true. (None added here — all artifact deps are apt packages or upstream Docker images, not crates.)
  • cargo-deny license gate: cargo deny check must stay green. The bundled-binary licenses in the images (Caddy Apache-2.0, valkey BSD-3, s6-overlay ISC, Debian GPL/MIT stack) are aggregation-clean vs GPL-3 — Rust's cargo-deny operates on the Rust crate graph only and is unaffected. The image license scan is a future cargo-deny-adjacent CI muscle (spec §6.1), not this slice's concern; this slice notes the license posture in deploy/.env.example's header comment.
  • sim-bench CI: the sim-bench job runs cargo test --all --features=sim-bench -- --test-threads=1--test-threads=1 is load-bearing (shared tick-lag gauge). Unaffected by this slice; the new smoke jobs are independent Gitea Actions jobs, not additional cargo test invocations.
  • Branch/PR: branch deploy-b/packaging-ci; PR to main via tea. Slices A landed first (this plan's dependency root); slice C branches after A merges and is order-independent with B. The slice-F publish workflow lives in the same PR (the file is inert until a tag is pushed — no risk of running before artifacts exist).

Slice-B-specific constraints:

  • The four image names are pinned to spec §6.1: rutster-allinone, rutster-engine, rutster-brain, rutster-edge. The CI image-build job asserts each --target produces a tagged image with exactly that name. Do not invent variants (rutster-fob, rutster-core, etc.) — the docs tree (plan G) hard-codes these four names.
  • The publish registry namespace is git.adlee.work/alee/rutster-* (matches the alee tea login per AGENTS.md Git workflow; the root Cargo.toml's repository = "https://git.adlee.work/alee/rutster" already commits to this). If your Gitea registry uses a different owner segment, replace alee throughout the publish workflow + deploy/.env.example's registry-namespace comment.
  • linux/amd64 only. arm64 is a named deferral (spec §1.2, ADR-0011); the publish workflow pins platforms: linux/amd64.
  • Caddy's local_certs (internal CA) is the CI TLS path. No ACME in CI (no public DNS, no rate-limit budget, hits Let's Encrypt's duplicate-cert path the instant a job re-runs). Production Caddyfiles default to ACME; CI overrides via RUTSTER_LOCAL_CERTS=true env in the smoke job. The rutster-edge image is the same in CI and prod — only the runtime env differs.
  • The smoke WS client is Python stdlib only. No pip install step in CI (Gitea runners ship Python 3 + stdlib; websocket-client/websockets would require a pip install network round-trip per job — a friction the slice-A sim-bench posture avoids). The WS handshake is ten lines of code on top of socket + ssl.
  • The ValkeyEventSink smoke assertion belongs to plan C. Leave a # TODO slice-C: assert lifecycle event in Valkey stream named hook comment in the compose-smoke step. Do NOT implement the assertion; the Valkey service ships in compose (so the docs + the smoke shape the contract) but no event-lands check runs in this slice.
  • Image build cache: the image-build job uses docker/build-push-action@v5 with cache-from: type=gha + cache-to: type=gha,mode=max. The cache scope includes the Rust target dir; warm caches take image-build from ~6 min to ~90 s. Documented in the CI job.

Spec ambiguities resolved (load-bearing — flagged so the implementing engineer doesn't

re-derive them)

  1. "the T8 e2e stub" (referenced in the assignment): there is no t8* file in the tree. The closest match is crates/rutster/tests/trunk_sim_e2e.rs — a #[ignore]'d TODO test whose body sketches a full PSTN e2e against MockTwilioMediaStreamsServer + MediaCmd::RegisterTrunk through the binary's MediaThread. That file's TODO text is exactly the slice-B all-in-one smoke, except this slice drives it against the real containerized edge→FOB WS path through Caddy's internal CA, rather than a MockTwilioMediaStreamsServer. The WS-handshake JSON sequence (connected + start frames) is borrowed verbatim from crates/rutster-trunk/tests/ws_ping.rs and crates/rutster-sim/src/nodelay.rs (both slice-A additions). The trunk_sim_e2e.rs file itself stays untouched — its #[ignore] TODO is a unit-test concern (in-process MediaThread driving); the CI smoke is a black-box container concern (whole binary booted, whole Caddy in front, external Python client). They are different test layers.
  2. rutster-brain runtime image libopus0: spec §6.1 table lists debian-slim for the brain (no libopus0). But crates/rutster-brain-realtime/Cargo.toml depends on rutster-media, which depends on the opus crate (system FFI against libopus.so.0). Even though the brain binary never calls an opus symbol, the libopus shared library is recorded in the binary ELF as NEEDED: libopus.so.0 because the linker resolves every Rust dep's [dependencies] — a missing .so.0 at startup would make the brain crash with error while loading shared libraries: libopus.so.0. Resolution: install libopus0 in the brain runtime image too (one extra apt-get install -y libopus0, ~500 KB additional). This is a defensive deviation from spec §6.1's brain row; flagged in the Dockerfile inline comment with the linker rationale so a future brain-crate refactor that drops the rutster-media dep can drop this package too.
  3. RUTSTER_DOMAIN + RUTSTER_ACME_EMAIL env-var names: plan G's quickstart-docker.md uses these two names (they are artifact-level names owned by slice B's Caddyfile — the spec does not fix them). Adopted as the canonical names; the Caddyfile reads them via env interpolation. The .env.example documents both. The rutster-edge Caddy binary has zero knowledge of RUTSTER_* semantics — Caddy reads {$DOMAIN} + {$ACME_EMAIL} from its environment per Caddyfile env-substitution syntax. Wait — the Caddyfile uses {$RUTSTER_DOMAIN} and {$RUTSTER_ACME_EMAIL} to match the rutster convention, even though Caddy is consuming them. This keeps the operator's RUTSTER_* namespace uniform across both processes.
  4. stop_grace_period exactly 660 s (not 600 s): the spec pins 660 s + drain 600 s (grace-exceeds-drain invariant — spec §8). 60 s is the slack so the orchestrator doesn't cut a drain that's legitimately waiting on a 599-second in-flight call. Pinned exactly; do not "tune."
  5. Caddy reload-during-live-call smoke: the spec §3.2 sentence "verified protocol-unaware WS tunneling (no frame buffering)" + §9 bullet 4 ("Caddy config reload during a live simulated call, asserting zero drops") load the heaviest claim in this slice. The smoke asserts: during a live WS streaming media frames at 20 ms cadence through Caddy to the FOB, a caddy reload (the operator's config-edit path) drops zero frames. The mitigation under test is stream_close_delay 24h (Caddyfile) — above max call duration. The upstream bug trail (caddy #6420, #7222) is documented in the TLS brief §5 risk 1 — the smoke is the "don't trust untested" mitigation made CI-regressed.

File Structure

New files

Path Responsibility
deploy/Dockerfile Multi-stage: one builder (rust:1.85-slim-bookworm + libopus-dev) producing rutster + rutster-brain-realtime binaries; one xcaddy builder producing caddy with the curated DNS-plugin set; one valkey binary source stage; four runtime targets: rutster-engine, rutster-brain, rutster-edge, rutster-allinone (s6-overlay v3 supervising caddy + FOB + brain + valkey-server).
deploy/.dockerignore Build-context pruning: excludes target/, .git/, node_modules/, *.md (other than README), .env*, crates/*/target/, docs/, .github/, .opencode/. Halves image-build context size.
deploy/Caddyfile ~6-line site block + global options: tuned timeouts (read_body 30s, read_header 30s, write 0, idle 24h), honest X-Forwarded-Proto/X-Forwarded-Host (no X-Forwarded-For-trust of client-controlled values — only the honest hop), stream_close_delay 24h (above max call duration; spec §2.1 / TLS brief §3(a)), reverse_proxy to 127.0.0.1:8080 (loopback-only FOB).
deploy/compose.yaml Four services — caddy (image rutster-edge), engine (image rutster-engine), brain (image rutster-brain, network_mode: "service:engine" so the loopback-only tap posture survives), valkey (upstream valkey/valkey). stop_grace_period: 660s. Volumes for caddy /data + valkey /data.
deploy/.env.example All RUTSTER_* vars incl. RUTSTER_DRAIN_DEADLINE_SECS=600, RUTSTER_DOMAIN + RUTSTER_ACME_EMAIL, the four RUTSTER_TWILIO_*, RUTSTER_MEDIA_ADVERTISED_IP, RUTSTER_MEDIA_PORT_RANGE, RUTSTER_WS_PING_SECS, RUTSTER_TRUSTED_PROXIES. Each line commented with the slice that owns it + the failing-fast behavior.
deploy/s6-rc.d/{caddy,engine,brain,valkey-database}/run + dependencies.d s6-overlay v3 service definitions — four run exec scripts + their dependency edges (engine before brain; valkey before engine; caddy before engine, etc.).
deploy/s6-overlay-noarch.tar.xz + deploy/s6-overlay-x86_64.tar.xz Vendored s6-overlay v3.2.0.0 binaries (ISC). Pinned at vendoring time per AGENTS.md reproducibility posture; SHA256 recorded alongside.
.github/workflows/publish-images.yml Slice F: tag-push workflow. Builds + pushes all four images to git.adlee.work/alee/rutster-* on v* tag push. linux/amd64 only. Gated on the existing ci.yml pipeline via workflow_run trigger (publish re-runs the fmt/clippy/test/deny/sim-bench + image-build as a gate, then publishes).
deploy/smoke/allinone_smoke.py Python-stdlib WS client (no pip deps): boots rutster-allinone, extracts Caddy's internal-CA root cert from /data, opens a wss:// WS to /twilio/media-stream, sends Twilio's connected + start handshake frames, streams 200 PCM frames at 20 ms cadence, asserts received frames count within tolerance. No ValkeyEventSink assertion — TODO slice-C hook is in this file as a comment.
deploy/smoke/compose_smoke.sh Bash: docker compose up -d --wait, then curl /healthz against each service, then docker compose ps JSON parse for healthy statuses. The # TODO slice-C: assert lifecycle event in Valkey stream named hook is here.
deploy/smoke/reload_during_call.py Python-stdlib WS client that holds the WS open streaming frames while a concurrent caddy reload fires mid-call; asserts zero frames dropped across the reload window.

Modified files

Path What changes
.github/workflows/ci.yml Add three new jobs: image-build (after existing test + deny + sim-bench), smoke (after image-build), smoke-compose + smoke-reload (after smoke). The existing fmt / clippy / test / deny / sim-bench / twilio-live jobs are unchanged.

SEAM-INVARIANT files (DO NOT TOUCH)

  • crates/rutster-media/src/loop_driver.rs — byte-identical, all tasks.
  • crates/rutster-media/src/rtc_session.rs — byte-identical, all tasks.

Interfaces

Consumes (no new code; reuses what slice A + slice-5 built):

  • rutster::serve::serve_with_nodelay (plan A Task 1) — the production serve path the all-in-one binary runs.
  • rutster_trunk::twilio_media_streams::TwilioMediaStreamsServer::router(register_tx, ping_interval) (plan A Task 2) — the trunk WS route mounted on the main axum listener.
  • rutster::config::twilio_credentials(...) (plan A Task 4) — startup validation of the four RUTSTER_TWILIO_* vars.
  • rutster::session_map::AppState::with_trunk_webhook_base(_).with_trusted_proxies(_) (plan A Tasks 4+5) — the AppState builder chain.
  • Plan A's slice env additions: RUTSTER_WS_PING_SECS (default 20), RUTSTER_TRUSTED_PROXIES (empty default).
  • Existing config.rs env parsers (slice-5): RUTSTER_HTTP_BIND, RUTSTER_MEDIA_BIND_IP, RUTSTER_MEDIA_ADVERTISED_IP, RUTSTER_MEDIA_PORT_RANGE, RUTSTER_MAX_SESSIONS, RUTSTER_DRAIN_DEADLINE_SECS, the four RUTSTER_TWILIO_*.
  • Existing routes: /healthz (200 = liveness), /readyz (200 = can-accept-call, 503 = draining / at admission cap), /v1/trunk/webhook (POST → TwiML), /twilio/media-stream (WS upgrade).
  • The WebRTC static client at / (slice-1 — booted binary serves it via axum .fallback_service).

Produces:

  • Four first-party Docker images, built from one deploy/Dockerfile via named --targets:
    • rutster-engine:latest — FOB only (binary rutster + libopus0 runtime).
    • rutster-brain:latest — brain only (binary rutster-brain-realtime + libopus0 runtime — see spec-ambiguity #2 above for the presence of libopus0 in this image).
    • rutster-edge:latest — custom xcaddy build with curated DNS plugins (cloudflare/route53/porkbun/hetzner/desec — duckdns excluded per spec §3.2 + TLS brief §3(a)).
    • rutster-allinone:latest — s6-overlay v3 supervising caddy + rutster (FOB) + rutster-brain-realtime + valkey-server (T1 per spec §2.1).
  • A composed docker compose up -d that runs the T2 reference deployment (spec §2.2) — the caddy service is rutster-edge, engine is rutster-engine, brain is rutster-brain (sharing the engine's netns via network_mode: "service:engine"), valkey is upstream valkey/valkey.
  • Three CI jobs in .github/workflows/ci.yml: image-build, smoke (all-in-one TLS + reload-during-call), smoke-compose — gated on the existing fmt/clippy/test/deny/sim-bench matrix.
  • One new workflow .github/workflows/publish-images.yml (slice F): tag-push publish of all four images to git.adlee.work/alee/rutster-*, linux/amd64 only, gated on the image-build job.

Task ordering

  • Task 1deploy/Dockerfile (multi-stage, four targets) + vendor s6-overlay tarballs + create s6-rc service definitions. Foundational: all CI smoke jobs build images from this file. No deps on later tasks.
  • Task 2deploy/.dockerignore. Tiny; lands alongside Task 1 (separate file so the diff is reviewable).
  • Task 3deploy/Caddyfile. Required by both rutster-edge (rutster-edge bakes it in via the Dockerfile) and rutster-allinone (s6 Caddy service reads it). Lands before Task 5 (compose) references it.
  • Task 4deploy/.env.example. Operator documentation; the compose env_file reference points at it.
  • Task 5deploy/compose.yaml. Depends on Tasks 1 + 3 (image names + Caddyfile path).
  • Task 6deploy/smoke/allinone_smoke.py + compose_smoke.sh + reload_during_call.py. These scripts are invoked by the CI jobs (Tasks 79) — land them first so the CI YAML can reference stable paths.
  • Task 7 — Extend .github/workflows/ci.yml: image-build job. Depends on Task 1.
  • Task 8 — Extend .github/workflows/ci.yml: smoke job (all-in-one TLS smoke). Depends on Tasks 6 + 7.
  • Task 9 — Extend .github/workflows/ci.yml: smoke-compose + smoke-reload jobs (compose smoke + Caddy-reload-during-call). Depends on Task 8.
  • Task 10 — Slice F: .github/workflows/publish-images.yml. Depends on Task 7 (reuses the same docker buildx build invocations + GHA cache config).
  • Task 11 — Final verification sweep.

Task 1: deploy/Dockerfile — multi-stage, four image targets + s6-overlay + s6-rc

One Dockerfile, four named targets. docker buildx build --target rutster-engine -t rutster-engine:ci . builds just the FOB; --target rutster-allinone builds the s6 bundle. The same builder stage compiles both Rust binaries (the all-in-one needs both; the single-purpose images need one each — so the builder builds both, and each runtime stage COPY --from=builder only the binary it needs).

Files:

  • Create: deploy/Dockerfile
  • Vendor: deploy/s6-overlay-noarch.tar.xz, deploy/s6-overlay-x86_64.tar.xz (pulled from upstream s6-overlay v3.2.0.0 release — vendored, not regenerated per build, per AGENTS.md reproducibility posture)
  • Create: deploy/s6-overlay.sha256sum (records the SHA256 of both tarballs at vendoring time)
  • Create: deploy/s6-rc.d/caddy/run + deploy/s6-rc.d/caddy/dependencies.d/engine
  • Create: deploy/s6-rc.d/engine/run + deploy/s6-rc.d/engine/dependencies.d/valkey-database
  • Create: deploy/s6-rc.d/brain/run + deploy/s6-rc.d/brain/dependencies.d/engine
  • Create: deploy/s6-rc.d/valkey-database/run + deploy/s6-rc.d/valkey-database/dependencies.d/engine
  • Create: deploy/s6-rc.d/user/contents.d/{caddy,engine,brain,valkey-database} (empty files)
  • Test: bash content-grep (Step 5) + optional docker build --check (Step 3 — local only; CI in Task 7 is the authoritative build).

Interfaces:

  • Consumes: rust-toolchain.toml (pinned 1.85 — verified: channel = "1.85"), the workspace root Cargo.toml (workspace member list — verified: crates/rutster + crates/rutster-brain-realtime are both members), crates/rutster-media/Cargo.toml (confirms opus crate hard-deps on libopus FFI — verified, line 12: opus = { workspace = true }), the README's "Quickstart" libopus dev-header note (verified: libopus-dev / opus-devel / brew install opus). Task 3's deploy/Caddyfile is COPYed into both edge + all-in-one images — Task 1 lands the Dockerfile that references it; Task 3 lands the Caddyfile content. Both must land before Task 7's image-build job runs in CI.

  • Produces: four --target-named build stages.

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
test -f deploy/Dockerfile \
  && grep -q 'FROM rust:1.85-slim-bookworm' deploy/Dockerfile \
  && grep -q 'AS builder' deploy/Dockerfile \
  && grep -q 'libopus-dev' deploy/Dockerfile \
  && grep -q 'libopus0' deploy/Dockerfile \
  && grep -q 'FROM caddy:2.8-builder' deploy/Dockerfile \
  && grep -q 'xcaddy build' deploy/Dockerfile \
  && grep -q 'FROM valkey/valkey' deploy/Dockerfile \
  && grep -q 's6-overlay' deploy/Dockerfile \
  && grep -q -- 'AS rutster-engine' deploy/Dockerfile \
  && grep -q -- 'AS rutster-brain' deploy/Dockerfile \
  && grep -q -- 'AS rutster-edge' deploy/Dockerfile \
  && grep -q -- 'AS rutster-allinone' deploy/Dockerfile \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL (file does not exist).

  • Step 2: Write the complete Dockerfile

Write deploy/Dockerfile with exactly this content:

# syntax=docker/dockerfile:1.7
# deploy/Dockerfile — multi-stage, four first-party images (spec §6.1).
#
# One Dockerfile, four named --target stages:
#   * rutster-engine     — FOB only (binary `rutster`)
#   * rutster-brain      — brain only (binary `rutster-brain-realtime`)
#   * rutster-edge       — custom xcaddy build with curated DNS plugins
#   * rutster-allinone   — s6-overlay v3 supervising caddy + FOB + brain + valkey-server
#
# Build all four locally:
#   docker buildx build --target rutster-engine    -t rutster-engine:ci    -f deploy/Dockerfile .
#   docker buildx build --target rutster-brain     -t rutster-brain:ci     -f deploy/Dockerfile .
#   docker buildx build --target rutster-edge      -t rutster-edge:ci      -f deploy/Dockerfile .
#   docker buildx build --target rutster-allinone  -t rutster-allinone:ci  -f deploy/Dockerfile .
#
# Toolchain pin: rust-toolchain.toml pins `channel = "1.85"`. The builder
# image is `rust:1.85-slim-bookworm` (NOT `rust:stable` — image builds must
# reproduce exactly what the MSRV matrix in CI gates; `stable` would float).
#
# libopus strategy (spec §6.1 "distro libopus0 over static-bundled"):
#   * Builder stage: `apt-get install libopus-dev` — the -dev headers needed
#     to compile the `opus` crate (FFI against libopus; verified:
#     crates/rutster-media/Cargo.toml line 12: `opus = { workspace = true }`).
#   * Runtime stages (engine + brain + all-in-one): `apt-get install libopus0`
#     — the shared library binary. Brain image gets libopus0 too (see spec
#     ambiguity #2 in this slice's plan): rutster-brain-realtime's
#     Cargo.toml depends on rutster-media, so the linker records
#     `NEEDED: libopus.so.0` in the brain ELF even though the brain never
#     calls an opus symbol. Missing .so.0 would crash the brain at startup.
#   * Matches the dev loop exactly. Do NOT use the audiopus_sys bundled-cmake
#     fallback — system libopus is available in Debian.

########################################
# Stage 1: Rust builder — compiles both binaries once.
########################################
FROM rust:1.85-slim-bookworm AS builder

# libopus-dev: the headers the `opus` crate's build.rs expects.
RUN apt-get update && apt-get install -y --no-install-recommends \
        libopus-dev \
        ca-certificates \
        curl \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/rutster
COPY . .

# Build both binaries in one `cargo build --bins` invocation so the
# dependency graph is compiled once (warm ccache/rustc cache covers both).
# --release is mandatory: debug builds of str0m + opus run 5-10x slower
# than the 20ms tick budget; a debug-image container would burn the tick-lag
# threshold on first call.
RUN cargo build --release --bins \
    && cp /usr/src/rutster/target/release/rutster /usr/local/bin/rutster \
    && cp /usr/src/rutster/target/release/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime

########################################
# Stage 2: Caddy builder — xcaddy with the curated DNS plugin set.
########################################
# caddy:2.8-builder ships Go + xcaddy. The plugin set is exactly spec §3.2
# / TLS brief §3(a): cloudflare, route53, porkbun, hetzner, desec.
# duckdns EXCLUDED — no license file (spec §3.2).
# Plugin licenses: cloudflare=Apache-2.0; route53/porkbun/hetzner/desec=MIT.
# Aggregation-clean vs GPL-3 (Caddy core itself is Apache-2.0).
FROM caddy:2.8-builder AS caddy-builder

RUN xcaddy build \
        --with github.com/caddy-dns/cloudflare \
        --with github.com/caddy-dns/route53 \
        --with github.com/caddy-dns/porkbun \
        --with github.com/caddy-dns/hetzner \
        --with github.com/caddy-dns/desec \
    && mv /build/caddy /usr/local/bin/caddy

########################################
# Stage 3: Valkey binary source (upstream image, no rebuild).
########################################
# COPY --from pattern: pull just the valkey-server binary out of the
# upstream BSD-3 image; avoids rebuilding valkey from source (saves ~5 min
# per build + ~150 MB of build deps). The binary is statically-linked
# enough for Debian bookworm (verified by upstream's Debian-based image).
FROM valkey/valkey:8.0 AS valkey-src

########################################
# Target: rutster-engine — the FOB only.
########################################
FROM debian:bookworm-slim AS rutster-engine

# libopus0: the shared library (see libopus strategy comment at top).
RUN apt-get update && apt-get install -y --no-install-recommends \
        libopus0 \
        ca-certificates \
        curl \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /usr/local/bin/rutster /usr/local/bin/rutster
COPY deploy/Caddyfile /etc/rutster/Caddyfile

# RUTSTER_HTTP_BIND defaults to 0.0.0.0:8080 (config.rs). Behind Caddy, the
# engine binds the compose-network address; behind --network host (T1
# solo), it binds 127.0.0.1:8080.
EXPOSE 8080
# Media UDP range — published via RUTSTER_MEDIA_PORT_RANGE.
EXPOSE 49152-49407/udp

ENTRYPOINT ["/usr/local/bin/rutster"]

########################################
# Target: rutster-brain — brain only.
########################################
FROM debian:bookworm-slim AS rutster-brain

# libopus0: see spec ambiguity #2 in this slice's plan —
# rutster-brain-realtime's Cargo.toml depends on rutster-media which
# depends on the `opus` crate; even though the brain never calls an opus
# symbol, the ELF records `NEEDED: libopus.so.0` (linker dead-strips
# unused symbols but the .so is still required at dlopen-time before
# dead-strip can prove they're unused). A future brain-crate refactor
# that drops the rutster-media dep can drop this package too.
RUN apt-get update && apt-get install -y --no-install-recommends \
        libopus0 \
        ca-certificates \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /usr/local/bin/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime

# Brain binds 127.0.0.1:8082 (loopback-only tap posture until step 6).
# In compose (T2), the brain shares the engine's netns via
# `network_mode: "service:engine"` so this loopback IS the engine's
# loopback — the FOB can reach the brain's tap port without exposing it.
EXPOSE 8082

ENTRYPOINT ["/usr/local/bin/rutster-brain-realtime"]

########################################
# Target: rutster-edge — custom Caddy build.
########################################
# scratch/alpine choice (spec §6.1): alpine gives a shell for debugging
# (`docker exec -it rutster-edge sh`); scratch is purer but undebuggable.
# Alpine's musl is fine for the static Go binary xcaddy produces.
FROM alpine:3.20 AS rutster-edge

# ca-certificates: ACME TLS validation. curl: smoke-test `caddy reload`
# invocations + `curl /healthz` from inside the container.
RUN apk add --no-cache ca-certificates curl

COPY --from=caddy-builder /usr/local/bin/caddy /usr/local/bin/caddy
COPY deploy/Caddyfile /etc/caddy/Caddyfile

# Caddy needs /data for ACME state — losing it risks the Let's Encrypt
# duplicate-cert lockout (spec §2.1, TLS brief §5 risk 5). Mount a volume
# at /data in production. In CI (local_certs) /data just holds the internal
# CA — recreating is fine, but the volume mount keeps prod + CI on the
# same code path.
VOLUME /data

EXPOSE 80 443

ENTRYPOINT ["/usr/local/bin/caddy"]
CMD ["run", "--config", "/etc/caddy/Caddyfile", "--adapter", "caddyfile"]

########################################
# Target: rutster-allinone — s6-overlay v3 supervising four processes.
########################################
FROM debian:bookworm-slim AS rutster-allinone

# libopus0 (FOB needs it; brain too — see rutster-brain target above).
# curl: smoke-test invocations + `caddy reload` exec.
# ca-certificates: ACME.
# xz: s6-overlay binary tarball extraction (extract-then-rm).
RUN apt-get update && apt-get install -y --no-install-recommends \
        libopus0 \
        ca-certificates \
        curl \
        xz-utils \
    && rm -rf /var/lib/apt/lists/*

# s6-overlay v3.2.0.0 — ISC license (aggregation-clean vs GPL-3).
# Two tarballs: noarch (the service definitions + s6-rc bundle) + x86_64
# (the binaries — linux/amd64 only per spec §1.2). arm64 is a named
# deferral (ADR-0011 §1.2); when it lands, this Dockerfile grows an
# `ARG TARGETARCH` + `COPY` per-arch.
COPY deploy/s6-overlay-noarch.tar.xz /tmp/s6-overlay-noarch.tar.xz
COPY deploy/s6-overlay-x86_64.tar.xz /tmp/s6-overlay-x86_64.tar.xz
RUN tar -C / -Jxpf /tmp/s6-overlay-noarch.tar.xz \
    && tar -C / -Jxpf /tmp/s6-overlay-x86_64.tar.xz \
    && rm /tmp/s6-overlay-noarch.tar.xz /tmp/s6-overlay-x86_64.tar.xz

# Binaries from earlier stages.
COPY --from=builder /usr/local/bin/rutster /usr/local/bin/rutster
COPY --from=builder /usr/local/bin/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime
COPY --from=caddy-builder /usr/local/bin/caddy /usr/local/bin/caddy
COPY --from=valkey-src /usr/local/bin/valkey-server /usr/local/bin/valkey-server

# Caddyfile + s6 service definitions (long-lined below).
COPY deploy/Caddyfile /etc/caddy/Caddyfile
COPY deploy/s6-rc.d/ /etc/s6-overlay/s6-rc.d/

# Volumes: Caddy /data (ACME state — loss risks LE lockout per TLS brief
# §5 risk 5) + Valkey /var/lib/valkey (stream/state persistence per
# ADR-0005).
VOLUME /data /var/lib/valkey

# Caddy :80/:443 public; FOB :8080 behind Caddy; brain :8082 loopback;
# valkey :6379 loopback; media UDP direct.
EXPOSE 80 443 8080 8082 6379 49152-49407/udp

# s6-overlay v3 entrypoint — the binary `s6-overlay-init` (symlinked from
# /s6-init in the noarch tarball) runs the service bundle in
# /etc/s6-overlay/s6-rc.d/ via s6-rc.
ENTRYPOINT ["/s6-init"]
  • Step 3: Verify by local Docker build (optional — authoritative build is CI in Task 7)

If Docker is available locally:

cd /home/alee/Sources/rutster
# Buildkit dry-check (parse the Dockerfile without building):
docker buildx build --target rutster-engine -f deploy/Dockerfile --check .
# Full local build + smoke (note: this requires Task 3's Caddyfile + Step 4's
# s6-overlay tarballs + s6-rc.d definitions to be in place first):
docker buildx build --target rutster-engine -t rutster-engine:local -f deploy/Dockerfile .
docker run --rm rutster-engine:local --help 2>&1 | head -5   # binary banner

Expected: --check exits 0 (no syntax errors); the binary banner prints. If Docker is not available locally, the authoritative build runs in CI (Task 7).

Note on deploy/s6-overlay-*.tar.xz + deploy/s6-rc.d/: the all-in-one target references two tarball binaries + a service-definition directory. These are fetched artifacts + created service files — Step 4 below fetches the tarballs into deploy/ and creates the s6-rc.d directory so the Dockerfile build context is self-contained. The tarballs are vendored (not regenerated per build) because s6-overlay release cadence is rare + reproducibility favors a pinned vendored blob.

  • Step 4: Vendor the s6-overlay tarballs + create the s6-rc service definitions
cd /home/alee/Sources/rutster/deploy

# Pin v3.2.0.0 (ISC; aggregation-clean vs GPL-3):
curl -fsSLO https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-noarch.tar.xz
curl -fsSLO https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-x86_64.tar.xz

# Record the SHA256 of both tarballs at vendoring time. The operator
# verifies these against the release page on first build; the
# `deploy/s6-overlay.sha256sum` file is committed so a future s6-overlay
# version bump is a loud, reviewable diff.
sha256sum s6-overlay-noarch.tar.xz s6-overlay-x86_64.tar.xz > s6-overlay.sha256sum

Create deploy/s6-rc.d/caddy/run (mark executable: chmod +x deploy/s6-rc.d/caddy/run):

#!/command/execlineb -P
# s6-rc service: caddy — the edge (ACME + TLS termination + WS proxy).
# Depends on: engine (s6-rc starts the engine first per caddy/dependencies.d/engine).
# User: root (caddy needs :80/:443).
foreground { mkdir -p /data }
exec /usr/local/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile

Create deploy/s6-rc.d/caddy/dependencies.d/engine (an empty file — s6-rc reads the basename as the dependency name; the engine must be up before Caddy proxies to it):

# empty file — s6-rc reads the basename as the dependency name.

Create deploy/s6-rc.d/engine/run (mark executable):

#!/command/execlineb -P
# s6-rc service: rutster — the FOB. Runs the rutster binary (axum + media
# thread + trunk WS + /metrics). Reads its env from the container env
# directly (s6-overlay v3 inherits container env via s6-overlay-envdir —
# this slice relies on the container env for simplicity; the per-service
# env files are a future hardening step).
exec /usr/local/bin/rutster

Create deploy/s6-rc.d/engine/dependencies.d/valkey-database:

# empty — engine doesn't WAIT on valkey (it must boot even if valkey is down
# per ADR-0005's fail-safe posture: EventSink counts-and-drops, never blocks).
# The dependency just orders boot for consistent log sequencing.

Create deploy/s6-rc.d/brain/run (mark executable):

#!/command/execlineb -P
# s6-rc service: rutster-brain-realtime. Shares the engine's network
# namespace (T1 solo variant — the all-in-one is a single container so
# loopback IS shared). The brain binds 127.0.0.1:8082 — the FOB reaches it
# via RUTSTER_TAP_URL=ws://127.0.0.1:8082.
exec /usr/local/bin/rutster-brain-realtime

Create deploy/s6-rc.d/brain/dependencies.d/engine:

# empty — brain waits for the engine before attempting the tap WS.

Create deploy/s6-rc.d/valkey-database/run (mark executable):

#!/command/execlineb -P
# s6-rc service: valkey-server — bus + KV + presence (ADR-0005). Dark on
# day one except EventSink consumer; the operator contract (volumes, ports,
# upgrade shape) is stable so the spend ledger + fleet directory land later
# without changing the deployment shape (spec §2.1).
foreground { mkdir -p /var/lib/valkey }
exec /usr/local/bin/valkey-server --dir /var/lib/valkey --save 60 1 --appendonly yes

Create deploy/s6-rc.d/valkey-database/dependencies.d/engine:

# empty — orders valkey after engine in boot sequence.

Create the four empty service-bundle membership files in deploy/s6-rc.d/user/contents.d/:

cd /home/alee/Sources/rutster/deploy/s6-rc.d/user/contents.d
touch caddy engine brain valkey-database
cd /home/alee/Sources/rutster/deploy
chmod +x s6-rc.d/caddy/run s6-rc.d/engine/run s6-rc.d/brain/run s6-rc.d/valkey-database/run
  • Step 5: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/Dockerfile \
  && grep -q 'FROM rust:1.85-slim-bookworm' deploy/Dockerfile \
  && grep -q 'AS builder' deploy/Dockerfile \
  && grep -q 'libopus-dev' deploy/Dockerfile \
  && grep -q 'libopus0' deploy/Dockerfile \
  && grep -q 'FROM caddy:2.8-builder' deploy/Dockerfile \
  && grep -q 'xcaddy build' deploy/Dockerfile \
  && grep -q 'FROM valkey/valkey' deploy/Dockerfile \
  && grep -q 's6-overlay' deploy/Dockerfile \
  && grep -q -- 'AS rutster-engine' deploy/Dockerfile \
  && grep -q -- 'AS rutster-brain' deploy/Dockerfile \
  && grep -q -- 'AS rutster-edge' deploy/Dockerfile \
  && grep -q -- 'AS rutster-allinone' deploy/Dockerfile \
  && test -f deploy/s6-overlay-noarch.tar.xz \
  && test -f deploy/s6-overlay-x86_64.tar.xz \
  && test -x deploy/s6-rc.d/caddy/run \
  && test -x deploy/s6-rc.d/engine/run \
  && test -x deploy/s6-rc.d/brain/run \
  && test -x deploy/s6-rc.d/valkey-database/run \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 6: Commit
cd /home/alee/Sources/rutster
git add deploy/Dockerfile deploy/s6-overlay-noarch.tar.xz deploy/s6-overlay-x86_64.tar.xz deploy/s6-overlay.sha256sum deploy/s6-rc.d/
# Note: deploy/Caddyfile is created in Task 3; if Task 1 is committed without it,
# Step 3's `docker --check` will fail because COPY deploy/Caddyfile references a
# missing file. Two options: (a) commit the Caddyfile in this commit too (Task 3's
# content lands here as a stub + Task 3 just refines it), OR (b) land the Caddyfile
# in Task 3 and verify Task 1's Dockerfile syntax via `docker buildx build --check`
# which does NOT require the COPY source to exist. Option (b) is the executing-plans
# path: Task 1 commits the Dockerfile even though `docker build` would fail until
# Task 3 lands. CI's image-build job (Task 7) runs AFTER both Tasks 1 + 3 are merged.
git commit -s -m "feat(deploy): multi-stage Dockerfile producing four first-party images (deploy-B §6.1)

One Dockerfile, four named --target stages: rutster-engine (FOB),
rutster-brain (brain), rutster-edge (custom xcaddy + curated DNS plugins
cloudflare/route53/porkbun/hetzner/desec, duckdns excluded per spec §3.2),
rutster-allinone (s6-overlay v3 supervising caddy + FOB + brain +
valkey-server).

Builder pins rust:1.85-slim-bookworm (matches rust-toolchain.toml).
libopus-dev in builder, libopus0 in runtime (matches dev — spec §6.1
distro-over-static-bundled). Brain image gets libopus0 too: although the
brain never calls an opus symbol, the linker records NEEDED:libopus.so.0
because rutster-brain-realtime depends on rutster-media (spec ambiguity
resolved in this slice's plan README).

All-in-one vendors s6-overlay v3.2.0.0 (ISC) + uses upstream valkey/valkey
binary (COPY --from pattern, BSD-3). Bundled-binary licenses
aggregation-clean vs GPL-3 (Caddy Apache-2.0). No Rust crates touched;
cargo-deny unaffected.

s6-rc service definitions land in this commit (caddy/engine/brain/
valkey-database run scripts + dependency edges). The Caddyfile lands in
Task 3 (same PR); the image-build CI job (Task 7) runs after both."

Task 2: deploy/.dockerignore — build-context pruning

Without .dockerignore, the Docker build context ships target/ (~5 GB on a warm dev machine), .git/ (history bloat), docs/, etc. — docker buildx build then spends minutes transferring context before the first RUN runs. This file halves context size + saves CI minutes per build.

Files:

  • Create: deploy/.dockerignore

Interfaces: none — file content is a list of globs.

  • Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/.dockerignore && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Write the complete file

Write deploy/.dockerignore with exactly this content:

# deploy/.dockerignore — prune the build context for `docker buildx build`
# invocations against deploy/Dockerfile. Without this, the context would
# ship target/ (~5 GB on warm dev), .git/, docs/, .github/, etc. — minutes
# of pointless context-transfer before the first RUN step.
#
# IMPORTANT: patterns are relative to the build context root (the path
# passed to `docker buildx build`, which is the repo root for this Dockerfile
# since we use `-f deploy/Dockerfile .`). So `target/` matches
# /target/ at the repo root AND `**/target/` catches /crates/*/target/.

# Rust build artifacts.
target/
**/target/

# Git history + CI configuration.
.git/
.github/

# Editor + agent working state.
.vscode/
.idea/
.opencode/
*.swp
*.swo
.DS_Store

# Environment files (never ship secrets — .env.example is the ONLY .env*
# file that should reach the build context, and it has no secrets by design).
.env
.env.local
.env.*.local

# Documentation (the Dockerfile doesn't read any .md file but README).
docs/
*.md
!README.md

# Plans + specs (not needed at image build time — operator reads from the
# repo, not the container).
docs/superpowers/

# Logs + OS junk.
*.log
tmp/
*.tmp

# Tmp artifacts from cargo-deny / cargo-cache.
deny.toml.lock
  • Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/.dockerignore \
  && grep -q '^target/' deploy/.dockerignore \
  && grep -q '^.git/' deploy/.dockerignore \
  && grep -q '^\*\*/target/' deploy/.dockerignore \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/.dockerignore
git commit -s -m "chore(deploy): .dockerignore — prune build context (deploy-B §6.1)

Halves the build-context size: excludes target/ + **/target/ (5 GB on a
warm dev machine, ~150 MB cache hit on CI), .git/, docs/, .github/,
.env* (never ship secrets — .env.example is the only .env* the context
ships and it has no secrets by design)."

Task 3: deploy/Caddyfile — tuned timeouts, honest X-Forwarded, stream_close_delay 24h

The Caddyfile is read by both rutster-edge (T2 modular default edge) and rutster-allinone (T1 bundled edge). Same file, same build — spec §3.2 mandates "same build, same Caddyfile" so operator knowledge transfers between T1 and T2.

Files:

  • Create: deploy/Caddyfile (already referenced by Task 1's Dockerfile COPY steps; lands here as its own atomic commit so the diff is reviewable + the Caddyfile can be edited without re-touching the Dockerfile).

Interfaces:

  • Consumes: env vars RUTSTER_DOMAIN (the public hostname — pbx.example.com), RUTSTER_ACME_EMAIL (the LE account email), RUTSTER_LOCAL_CERTS (CI override — "true" makes Caddy use its internal CA, no ACME).

  • Produces: the :443 TLS listener, the :80 ACME HTTP-01 fallback, the reverse_proxy to 127.0.0.1:8080, the WS upgrade pass-through (Caddy auto-detects Upgrade: websocket — no per-route config needed), the X-Forwarded-Proto/X-Forwarded-Host honesty.

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
test -f deploy/Caddyfile \
  && grep -q 'stream_close_delay' deploy/Caddyfile \
  && grep -q 'X-Forwarded-Proto' deploy/Caddyfile \
  && grep -q 'reverse_proxy' deploy/Caddyfile \
  && grep -q '24h' deploy/Caddyfile \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL (file does not exist).

  • Step 2: Write the complete Caddyfile

Write deploy/Caddyfile with exactly this content:

# deploy/Caddyfile — the edge config for rutster-edge (T2) + rutster-allinone (T1).
# Same file, same build (spec §3.2 + ADR-0011). Read by Caddy v2.8 (the
# custom xcaddy build from deploy/Dockerfile).
#
# Env vars (Caddyfile interpolation syntax {$VAR}):
#   RUTSTER_DOMAIN      — the public hostname (e.g. pbx.example.com). Required.
#   RUTSTER_ACME_EMAIL  — the Let's Encrypt account email. Required for ACME.
#   RUTSTER_LOCAL_CERTS — set to "true" in CI to use Caddy's internal CA
#                         (no ACME round-trips, no rate-limit budget burns).
#                         Overrides the ACME block below via the `local_certs`
#                         global option.
#
# Timeouts (TLS brief §3(a); spec §2.1, §2.2, §3.1 invariant 3):
#   * read_body 30s      — Twilio webhooks fit comfortably in 30s (15s cap
#                          by invariant 6).
#   * read_header 30s    — same.
#   * write 0            — disable; 20ms media frames must flush immediately.
#   * idle 24h           — the universal 60s-class default kills quiet WS;
#                          spec §2.1 invariant 3 sets max call duration 24h.
#   * stream_close_delay 24h — above max call duration; the load-bearing
#                              mitigation for caddy #6420/#7222 (TLS brief §5
#                              risk 1; spec §9 reload-during-call smoke).
#
# X-Forwarded-* honesty (spec §3.1 invariant 5; plan-A Task 5
# reconstruct_public_url): Caddy sets these to the real client values via
# the `header_up` directives below. Engine-side RUTSTER_TRUSTED_PROXIES is
# the trust gate (fail-closed default).
{
	# CI override: set RUTSTER_LOCAL_CERTS=true to use Caddy's internal CA
	# (no ACME in CI; spec §9). Production leaves this unset.
	{$RUTSTER_LOCAL_CERTS:local_certs_off} local_certs

	# Send logs to stdout (s6-overlay / docker logs collects from there).
	log {
		output stdout
		format console
	}
}

# Default ACME email — overridden by RUTSTER_LOCAL_CERTS=true in CI.
{
	email {$RUTSTER_ACME_EMAIL:you@example.com}
}

# Main site block — the public hostname.
{$RUTSTER_DOMAIN:localhost} {
	encode zstd gzip

	# The reverse_proxy to the FOB. 127.0.0.1:8080 is correct in T1
	# (single container, loopback) AND T2 (compose: the engine service's
	# `rutster-engine` DNS name resolves on the compose network; Caddy
	# reaches it via that name — replace 127.0.0.1:8080 with
	# http://engine:8080 in advance; here we use the loopback literal so
	# the same Caddyfile works in both T1 and T2 spelled identically —
	# for T2 deployments, the operator overrides the upstream via a env-
	# substituted site-address block; the shipped default targets loopback
	# for the bundled all-in-one single-container case which is the harder
	# constraint).
	#
	# `header_up Host {host}` + X-Forwarded-* — sets the honest hop values
	# (the engine's reconstruct_public_url reads these; spec §3.1 invariant 5).
	# `header_up X-Forwarded-For {remote_host}` — included for completeness;
	# the engine does NOT use X-Forwarded-For for signature validation, only
	# X-Forwarded-Proto/Host. Forwarding the For header is standard proxy
	# hygiene — engine-side RUTSTER_TRUSTED_PROXIES is the gate.
	reverse_proxy 127.0.0.1:8080 {
		header_up Host {host}
		header_up X-Real-IP {remote_host}
		header_up X-Forwarded-Proto {scheme}
		header_up X-Forwarded-Host {host}
		header_up X-Forwarded-For {remote_host}

		# Tuned timeouts — the load-bearing ones for calls lasting hours.
		transport http {
			read_buffer 64KiB
			write_buffer 64KiB
			dial_timeout 5s
			response_header_timeout 30s
		}

		# stream_close_delay above max call duration (24h) — the caddy
		# #6420/#7222 mitigation (TLS brief §5 risk 1; spec §9 smoke).
		# An operator Caddyfile reload mid-call would normally kill
		# upgraded WS tunnels; the delay keeps them alive until the
		# streams close naturally (the call ends) — verified by the
		# reload-during-live-call CI smoke (Task 9).
		stream_close_delay 24h
	}

	# Timeouts for the public-facing side.
	servers {
		read_body 30s
		read_header 30s
		write 0        # never write-timeout; 20ms frames self-flush
		idle 24h       # above max call duration — the universal 60s-class default kills quiet WS
	}
}
  • Step 3: Re-run the content-check + Caddy config-validate
cd /home/alee/Sources/rutster
test -f deploy/Caddyfile \
  && grep -q 'stream_close_delay' deploy/Caddyfile \
  && grep -q 'X-Forwarded-Proto' deploy/Caddyfile \
  && grep -q 'reverse_proxy' deploy/Caddyfile \
  && grep -q '24h' deploy/Caddyfile \
  && grep -q 'idle 24h' deploy/Caddyfile \
  && grep -q 'RUTSTER_DOMAIN' deploy/Caddyfile \
  && grep -q 'RUTSTER_LOCAL_CERTS' deploy/Caddyfile \
  && echo CHECK-PASS || echo CHECK-FAIL

# Optional local Caddy validate (if `caddy` binary available — CI does
# this inside the rutster-edge container in Task 7):
if command -v caddy >/dev/null 2>&1; then
  RUTSTER_DOMAIN=pbx.example.com RUTSTER_ACME_EMAIL=test@example.com \
    caddy validate --config deploy/Caddyfile --adapter caddyfile
else
  echo "(caddy not installed locally; CI validates in Task 7)"
fi

Expected: CHECK-PASS. If Caddy is installed locally, the validate command exits 0.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/Caddyfile
git commit -s -m "feat(deploy): Caddyfile — tuned timeouts, honest X-Forwarded, stream_close_delay 24h (deploy-B §3.2)

The shared edge config for rutster-edge (T2 modular) and rutster-allinone
(T1 solo). ~6-line site block + global options per spec §3.2 / TLS brief
§3(a):

* read_body 30s / read_header 30s — Twilio webhook budget (15s cap by
  invariant 6 + slack).
* write 0 — 20ms media frames self-flush; never write-timeout.
* idle 24h / stream_close_delay 24h — above max call duration (invariant
  3); the universal 60s-class default kills quiet mid-call WS.
* reverse_proxy + header_up — Caddy sets honest X-Forwarded-Proto/Host/For
  for the engine's reconstruct_public_url (plan-A Task 5); engine-side
  RUTSTER_TRUSTED_PROXIES is the trust gate (fail-closed default).

CI override via RUTSTER_LOCAL_CERTS=true uses Caddy's internal CA (no
ACME in CI per spec §9). The reload-during-live-call smoke (Task 9)
regresses the stream_close_delay mitigation against caddy #6420/#7222."

Task 4: deploy/.env.example — all RUTSTER_* vars documented

The .env.example is the operator documentation surface (plan G's quickstart-docker.md refers operators here). Every RUTSTER_* var is listed with its default, the slice that introduced it, and the fail-fast behavior.

Files:

  • Create: deploy/.env.example

Interfaces: consumes the env-var surface from crates/rutster/src/config.rs (verified: http_bind, media_address_config, max_sessions, drain_deadline, twilio_credentials + plan A's ws_ping_interval, trusted_proxies) + the artifact-level vars RUTSTER_DOMAIN + RUTSTER_ACME_EMAIL + RUTSTER_LOCAL_CERTS (this slice — Caddyfile-level). Plan C's RUTSTER_TLS_CERT/RUTSTER_TLS_KEY are noted as "land with slice C" — see plan C, NOT here.

  • Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/.env.example \
  && grep -q 'RUTSTER_DRAIN_DEADLINE_SECS=600' deploy/.env.example \
  && grep -q 'RUTSTER_DOMAIN=' deploy/.env.example \
  && grep -q 'RUTSTER_ACME_EMAIL=' deploy/.env.example \
  && grep -q 'RUTSTER_TWILIO_ACCOUNT_SID=' deploy/.env.example \
  && grep -q 'RUTSTER_TRUSTED_PROXIES=' deploy/.env.example \
  && grep -q 'RUTSTER_WS_PING_SECS=' deploy/.env.example \
  && grep -q 'RUTSTER_MEDIA_ADVERTISED_IP=' deploy/.env.example \
  && grep -q 'RUTSTER_LOCAL_CERTS' deploy/.env.example \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Write the complete file

Write deploy/.env.example with exactly this content:

# deploy/.env.example — runtime configuration for the rutster T1 (all-in-one)
# + T2 (compose) artifacts. Copy to `.env` + edit before `docker compose up -d`.
#
# Format: KEY=value (one per line); comments (#) are ignored by `docker
# compose --env-file`. Fail-fast pattern: every var is parsed by
# crates/rutster/src/config.rs's pure-`Option<String>` parsers at startup;
# an invalid value fails the boot loudly with the var name in the error,
# never silently runs with a wrong default. See AGENTS.md "config.rs"
# pattern.
#
# Bundled-binary licenses (aggregation-clean vs GPL-3 per ADR-0004):
#   * Caddy      Apache-2.0
#   * valkey     BSD-3
#   * s6-overlay ISC
#   * libopus0   BSD-3 (system package; Debian)
# Future cargo-deny-adjacent image license scan will assert these in CI
# (spec §6.1 — out of scope for slice B; this comment seeds it).
#
# Registry namespace: git.adlee.work/alee/rutster-* (matches the `alee` tea
# login per AGENTS.md Git workflow + the root Cargo.toml's `repository`
# field). If your Gitea registry uses a different owner segment, replace
# `alee` throughout deploy/compose.yaml + .github/workflows/publish-images.yml.

################################################################################
# ARTIFACT-LEVEL — Caddy edge (this slice)
################################################################################

# RUTSTER_DOMAIN — the public hostname the CPaaS dials. Required for T1 (solo
# all-in-one) and T2 (compose). Used by Caddyfile as the site block name.
# Example: pbx.example.com
RUTSTER_DOMAIN=pbx.example.com

# RUTSTER_ACME_EMAIL — the Let's Encrypt account email. Used by Caddy for
# ACME account creation. Required for production (ACME issuance); ignored
# in CI (RUTSTER_LOCAL_CERTS=true bypasses ACME).
RUTSTER_ACME_EMAIL=you@example.com

# RUTSTER_LOCAL_CERTS — set to "true" in CI to use Caddy's internal CA (no
# ACME round-trips, no rate-limit budget burns). Production leaves this
# unset (or "false") so Caddy uses Let's Encrypt. The Caddyfile reads this
# via the {$RUTSTER_LOCAL_CERTS:local_certs_off} interpolation.
# RUTSTER_LOCAL_CERTS=false

################################################################################
# FOB engine — bind + drain (slice-5 config.rs + plan-A)
################################################################################

# RUTSTER_HTTP_BIND — the FOB's HTTP/WS listener. Behind Caddy (T1/T2 prod):
# 0.0.0.0:8080 (compose network) or 127.0.0.1:8080 (single container T1).
# Default if unset: 0.0.0.0:8080 (config.rs http_bind).
RUTSTER_HTTP_BIND=0.0.0.0:8080

# RUTSTER_DRAIN_DEADLINE_SECS — how long shutdown waits for in-flight calls
# before the hard stop. Default 0 = instant shutdown (dev loop); production
# sets minutes. Spec §2.2 pins 600 (paired with compose stop_grace_period
# 660s — grace MUST exceed drain or the orchestrator kills calls the
# engine was gracefully draining).
RUTSTER_DRAIN_DEADLINE_SECS=600

# RUTSTER_MAX_SESSIONS — admission cap. Default 64 (placeholder until the
# ADR-0010 benchmark measures the real per-node ceiling). /readyz flips 503
# when at the cap so the LB pulls the node.
RUTSTER_MAX_SESSIONS=64

################################################################################
# FOB engine — WebRTC media (slice-5 config.rs media_address_config)
################################################################################

# RUTSTER_MEDIA_BIND_IP — the IP WebRTC DTLS-SRTP binds. Default 127.0.0.1
# (loopback — dev loop). Production hosts a public IP.
RUTSTER_MEDIA_BIND_IP=0.0.0.0

# RUTSTER_MEDIA_ADVERTISED_IP — the IP advertised in SDP to WebRTC callers.
# They send media UDP straight to this IP (NAT 1:1 — no STUN, no TURN).
# In T1 (host networking): the host's public IP. In T2 (compose): the
# engine container's published port's host IP.
RUTSTER_MEDIA_ADVERTISED_IP=203.0.113.7

# RUTSTER_MEDIA_PORT_RANGE — UDP port range for WebRTC media. Format lo-hi
# inclusive. Must be open inbound on the host's firewall + published in
# compose (deploy/compose.yaml publishes this range).
RUTSTER_MEDIA_PORT_RANGE=49152-49407

################################################################################
# FOB engine — WS hygiene (plan-A §5.2)
################################################################################

# RUTSTER_WS_PING_SECS — engine-originated app-level WS ping interval on
# the trunk media-stream WS. Default 20s. 0 rejected (would busy-loop the
# ping timer); no "off" spelling by design.
RUTSTER_WS_PING_SECS=20

################################################################################
# FOB engine — trusted-proxy posture (plan-A §5.3)
################################################################################

# RUTSTER_TRUSTED_PROXIES — comma-separated CIDR list of edge peers whose
# X-Forwarded-Proto/Host are believed. Empty or unset = headers IGNORED
# (fail-closed default). Bare IPs accepted as host-length prefixes. In T1/T2
# with Caddy as edge, set to the Caddy container's source IP (compose:
# the compose network's subnet CIDR; T1: 127.0.0.1/32). An internet peer
# must never choose the public URL that Twilio signature validation
# reconstructs (spec §3.1 invariant 5).
RUTSTER_TRUSTED_PROXIES=127.0.0.1/32

################################################################################
# Trunk — Twilio credentials (slice-5 config.rs twilio_credentials)
################################################################################

# All four RUTSTER_TWILIO_* vars are ALL-OR-NONE: the parser rejects partial
# config at startup. Omit all four to run WebRTC-only.
RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token_here
RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081
RUTSTER_TWILIO_WEBHOOK_BASE=https://pbx.example.com

################################################################################
# Brain — tap URL (slice-2/3; loopback-only until step 6)
################################################################################

# RUTSTER_TAP_URL — the WS URL the FOB opens to the brain. Default
# ws://127.0.0.1:8081/echo (slice-2 dev-loop echo brain). Production with
# rutster-brain-realtime: ws://127.0.0.1:8082 (loopback-only —
# resolve_tap_url rejects non-loopback until step 6's wss:// tap). In T2
# (compose), the brain shares the engine's netns
# (network_mode: "service:engine") so this loopback IS the engine's
# loopback.
RUTSTER_TAP_URL=ws://127.0.0.1:8082

################################################################################
# Future vars — land with sibling slices, NOT this slice
################################################################################

# RUTSTER_TLS_CERT / RUTSTER_TLS_KEY — land with slice C (rustls Phase 1,
# BYO-cert in-process TLS). The engine binary stays compiled-with-rustls
# from slice C onward; runtime-gated by these vars. See the slice-C plan
# at docs/superpowers/plans/2026-07-05-deploy-c-rustls-tls.md (path may
# vary — check the plans dir).
# RUTSTER_TLS_CERT=/etc/rutster/tls/fullchain.pem
# RUTSTER_TLS_KEY=/etc/rutster/tls/privkey.pem

# RUTSTER_METRICS_BIND — lands with slice D (/metrics endpoint). Default
# 127.0.0.1:9090 (internal — never routed through Caddy).
# RUTSTER_METRICS_BIND=127.0.0.1:9090

# RUTSTER_VALKEY_URL — lands with slice E (ValkeyEventSink). Unset = today's
# TracingEventSink (logs to stdout). Set to redis://valkey:6379/0 in T2.
# RUTSTER_VALKEY_URL=redis://valkey:6379/0
  • Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/.env.example \
  && grep -q 'RUTSTER_DRAIN_DEADLINE_SECS=600' deploy/.env.example \
  && grep -q 'RUTSTER_DOMAIN=' deploy/.env.example \
  && grep -q 'RUTSTER_ACME_EMAIL=' deploy/.env.example \
  && grep -q 'RUTSTER_TWILIO_ACCOUNT_SID=' deploy/.env.example \
  && grep -q 'RUTSTER_TRUSTED_PROXIES=' deploy/.env.example \
  && grep -q 'RUTSTER_WS_PING_SECS=' deploy/.env.example \
  && grep -q 'RUTSTER_MEDIA_ADVERTISED_IP=' deploy/.env.example \
  && grep -q 'RUTSTER_LOCAL_CERTS' deploy/.env.example \
  && grep -q 'RUTSTER_TLS_CERT' deploy/.env.example \
  && grep -q 'RUTSTER_VALKEY_URL' deploy/.env.example \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/.env.example
git commit -s -m "docs(deploy): .env.example — all RUTSTER_* vars documented (deploy-B §6.2)

The operator documentation surface. Lists every RUTSTER_* var with its
default + the slice that introduced it + the fail-fast behavior. Includes:

* Artifact-level (this slice): RUTSTER_DOMAIN, RUTSTER_ACME_EMAIL,
  RUTSTER_LOCAL_CERTS (CI override).
* FOB engine (slice-5 + plan-A): RUTSTER_HTTP_BIND,
  RUTSTER_DRAIN_DEADLINE_SECS=600 (paired with compose stop_grace_period
  660s per spec §2.2 — grace exceeds drain), RUTSTER_MAX_SESSIONS,
  RUTSTER_MEDIA_BIND_IP/ADVERTISED_IP/PORT_RANGE, RUTSTER_WS_PING_SECS,
  RUTSTER_TRUSTED_PROXIES.
* Trunk (slice-5): the four RUTSTER_TWILIO_* (all-or-none — config.rs
  twilio_credentials rejects partial).
* Brain (slice-2/3): RUTSTER_TAP_URL (loopback-only until step 6).
* Future var stubs: RUTSTER_TLS_CERT/KEY (slice C), RUTSTER_METRICS_BIND
  (slice D), RUTSTER_VALKEY_URL (slice E) — commented out with a pointer
  to the sibling plan.

Bundled-binary licenses noted in the header comment (Caddy Apache-2.0,
valkey BSD-3, s6-overlay ISC, libopus0 BSD-3 — aggregation-clean vs
GPL-3 per ADR-0004); seeds the future cargo-deny-adjacent image license
scan (spec §6.1, out of scope for this slice).

Registry namespace git.adlee.work/alee/rutster-* + the 'if your Gitea
registry uses a different owner segment, replace alee throughout' note."

Task 5: deploy/compose.yaml — T2 reference deployment

The compose file is the T2 reference deployment (spec §2.2). Four services, stop_grace_period: 660s (paired with RUTSTER_DRAIN_DEADLINE_SECS=600), brain shares the engine's netns so the loopback tap posture holds, two named volumes for Caddy /data + valkey persistence.

Files:

  • Create: deploy/compose.yaml

Interfaces:

  • Consumes: image names from spec §6.1 (rutster-edge, rutster-engine, rutster-brain, upstream valkey/valkey), env from deploy/.env.example (Task 4), Caddyfile from Task 3.

  • Produces: docker compose up -d boots a working T2 stack.

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
test -f deploy/compose.yaml \
  && grep -q 'services:' deploy/compose.yaml \
  && grep -q 'image: rutster-edge' deploy/compose.yaml \
  && grep -q 'image: rutster-engine' deploy/compose.yaml \
  && grep -q 'image: rutster-brain' deploy/compose.yaml \
  && grep -q 'image: valkey/valkey' deploy/compose.yaml \
  && grep -q 'network_mode: "service:engine"' deploy/compose.yaml \
  && grep -q 'stop_grace_period: 660s' deploy/compose.yaml \
  && grep -q 'volumes:' deploy/compose.yaml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Write the complete file

Write deploy/compose.yaml with exactly this content:

# deploy/compose.yaml — T2 modular reference deployment (spec §2.2 / ADR-0011).
#
# Four services: caddy (rutster-edge), engine (rutster-engine), brain
# (rutster-brain, network_mode: "service:engine" so the loopback tap posture
# survives unchanged — resolve_tap_url rejects non-loopback until step 6),
# valkey (upstream valkey/valkey).
#
# stop_grace_period 660s paired with RUTSTER_DRAIN_DEADLINE_SECS=600 —
# grace EXCEEDS drain or the orchestrator kills calls the engine was
# gracefully draining (spec §2.2 / §8). 60s slack accounts for a
# legitimately-long in-flight call (599s) + shutdown round-trip.
#
# Bring your own proxy: comment out the `caddy` service; the engine keeps
# its plaintext :8080 listener; tuned-timeout configs for nginx/HAProxy/
# Traefik ship in docs/deploy/reverse-proxies.md (slice G).
#
# Bring-up:
#   cp .env.example .env
#   $EDITOR .env     # set DOMAIN, ACME_EMAIL, MEDIA_ADVERTISED_IP, TWILIO_*
#   docker compose up -d
#   curl -fsS https://$RUTSTER_DOMAIN/healthz && echo OK

services:
  caddy:
    # The custom xcaddy build with curated DNS plugins (cloudflare/
    # route53/porkbun/hetzner/desec — duckdns excluded per spec §3.2).
    # In production, pulls from the git.adlee.work/alee/rutster-edge
    # registry (slice F). For local dev / CI without registry auth, build
    # from the Dockerfile:
    #   build:
    #     context: ..
    #     dockerfile: deploy/Dockerfile
    #     target: rutster-edge
    image: rutster-edge:latest
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - caddy-data:/data
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
    env_file:
      - .env
    depends_on:
      - engine
    networks:
      - rutster-net

  engine:
    # The FOB: axum signaling + media thread + trunk WS + /metrics.
    # Plaintext :8080 behind Caddy (slice C's rustls will terminate
    # in-process TLS — same listener, operator-choice).
    image: rutster-engine:latest
    restart: unless-stopped
    # 660s grace > 600s drain (RUTSTER_DRAIN_DEADLINE_SECS) — invariant.
    stop_grace_period: 660s
    ports:
      # Signaling + trunk WS behind Caddy (Caddy proxies :443 -> engine:8080).
      # Comment out for production hardening (Caddy is the only entry).
      - "8080:8080"
      # WebRTC media UDP direct — never proxied (spec §2.1).
      - "49152-49407:49152-49407/udp"
    volumes:
      - ./Caddyfile:/etc/rutster/Caddyfile:ro   # copied in image already; mount for edit-without-rebuild
    env_file:
      - .env
    networks:
      - rutster-net

  brain:
    # Brain shares engine's network namespace — the FOB reaches the brain's
    # tap port via 127.0.0.1:8082 (loopback-only posture; resolve_tap_url
    # rejects non-loopback URLs until step 6 per spec §2.2). The brain
    # graduates to its own netns + wss:// tap auth with step 6.
    image: rutster-brain:latest
    restart: unless-stopped
    # CRITICAL: quoted — YAML would otherwise parse `service:engine` as a
    # `service: engine` mapping (key:value). The quotes make it a string.
    network_mode: "service:engine"
    env_file:
      - .env
    depends_on:
      - engine

  valkey:
    # Upstream valkey image (BSD-3; aggregation-clean vs GPL-3 per ADR-0004).
    # Dark on day one except EventSink (slice E lands ValkeyEventSink
    # against this service). The operator contract (volumes, ports, upgrade
    # shape) is stable from v1 per spec §2.1 — the spend ledger (ADR-0009)
    # + fleet directory land later without changing this service.
    image: valkey/valkey:8.0
    restart: unless-stopped
    command: ["valkey-server", "--dir", "/data", "--save", "60", "1", "--appendonly", "yes"]
    volumes:
      - valkey-data:/data
    networks:
      - rutster-net

volumes:
  # /data — Caddy's cert/ACME state. Loss risks the Let's Encrypt
  # duplicate-cert lockout (5/week — total inbound outage class).
  # See docs/deploy/certificates.md (slice G) + TLS brief §5 risk 5.
  caddy-data:
  # /var/lib/valkey — stream/state persistence (ADR-0005).
  valkey-data:

networks:
  rutster-net:
    driver: bridge
  • Step 3: Re-run the content-check + docker compose config validate
cd /home/alee/Sources/rutster
test -f deploy/compose.yaml \
  && grep -q 'services:' deploy/compose.yaml \
  && grep -q 'image: rutster-edge' deploy/compose.yaml \
  && grep -q 'image: rutster-engine' deploy/compose.yaml \
  && grep -q 'image: rutster-brain' deploy/compose.yaml \
  && grep -q 'image: valkey/valkey' deploy/compose.yaml \
  && grep -q 'network_mode: "service:engine"' deploy/compose.yaml \
  && grep -q 'stop_grace_period: 660s' deploy/compose.yaml \
  && grep -q 'caddy-data:' deploy/compose.yaml \
  && grep -q 'valkey-data:' deploy/compose.yaml \
  && echo CHECK-PASS || echo CHECK-FAIL

# YAML syntax validation — `docker compose config` parses + renders the
# resolved config (with .env vars interpolated). Requires Docker locally;
# CI validates in the compose-smoke job (Task 9).
if command -v docker >/dev/null 2>&1; then
  cd /home/alee/Sources/rutster/deploy
  cp .env.example .env 2>/dev/null || cp -n .env.example .env 2>/dev/null || true
  docker compose --env-file .env config -q
  rm -f .env   # don't commit the .env (it has placeholder secrets from .env.example)
else
  echo "(docker not installed locally; CI validates in Task 9)"
fi

Expected: CHECK-PASS. If docker compose is available, config -q exits 0.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/compose.yaml
git commit -s -m "feat(deploy): compose.yaml — T2 reference deployment (deploy-B §2.2)

Four services per spec §2.2:

* caddy    — image rutster-edge; ports 80/443; volume caddy-data:/data
             (Caddy ACME state — loss risks LE 5/week duplicate-cert lockout
             per TLS brief §5 risk 5; mounts ./Caddyfile for edit-without-
             rebuild).
* engine   — image rutster-engine; ports 8080 + 49152-49407/udp (media
             UDP direct, never proxied — spec §2.1); stop_grace_period 660s
             paired with RUTSTER_DRAIN_DEADLINE_SECS=600 (grace exceeds drain
             invariant, spec §2.2/§8).
* brain    — image rutster-brain; network_mode: \"service:engine\" so the
             loopback-only tap posture survives unchanged (resolve_tap_url
             rejects non-loopback until step 6 per spec §2.2). Quoted in
             YAML — \`service:engine\` would parse as a mapping otherwise.
* valkey   — upstream valkey/valkey:8.0 (BSD-3); volume valkey-data:/data;
             dark on day one except slice-E ValkeyEventSink consumer.

Bring-your-own-proxy: comment out the caddy service; the engine keeps
plaintext :8080; tuned-timeout snippets land in slice-G reverse-proxies.md."

Task 6: deploy/smoke/*.py + deploy/smoke/*.sh — CI smoke harness

The smoke scripts are invoked by the Gitea Actions smoke job (Tasks 79). Python-stdlib only (no pip install) so each CI job is self-contained. Bash for the orchestration glue.

Files:

  • Create: deploy/smoke/allinone_smoke.py — wss:// WS client driving a sim call through the real edge→FOB path inside rutster-allinone.
  • Create: deploy/smoke/compose_smoke.shdocker compose up -d --wait, health probes, service-status JSON parse. The # TODO slice-C: assert lifecycle event in Valkey stream named hook lives here.
  • Create: deploy/smoke/reload_during_call.py — holds a WS open streaming frames while a concurrent caddy reload fires; asserts zero frame drops.

Interfaces:

  • Consumes: Task 1's rutster-allinone image (built by Task 7's CI image-build job), Task 3's Caddyfile (baked into the all-in-one image), Caddy's internal-CA root cert at /data/caddy/pki/authorities/local/root.crt (Caddy's local_certs global option writes it).

  • Produces: Python scripts returning exit code 0 on success, non-zero on failure (CI fails the job on non-zero).

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
test -f deploy/smoke/allinone_smoke.py \
  && test -f deploy/smoke/compose_smoke.sh \
  && test -f deploy/smoke/reload_during_call.py \
  && grep -q 'local_certs' deploy/smoke/allinone_smoke.py \
  && grep -q 'TODO slice-C' deploy/smoke/compose_smoke.sh \
  && grep -q 'caddy reload' deploy/smoke/reload_during_call.py \
  && grep -q 'stream_close_delay' deploy/smoke/reload_during_call.py \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Write deploy/smoke/allinone_smoke.py
#!/usr/bin/env python3
"""deploy/smoke/allinone_smoke.py — T1 all-in-one TLS smoke (deploy-B §9).

Boots `rutster-allinone` with RUTSTER_LOCAL_CERTS=true (Caddy internal CA —
no ACME in CI per spec §9), boots, extracts Caddy's internal-CA root cert
from /data, opens a wss:// WS to /twilio/media-stream, sends Twilio's
connected + start handshake frames, streams 200 PCM-zeroed frames at 20ms
cadence, and asserts the engine replies with at least one text frame
through the real edge->FOB WS path end-to-end through TLS.

Python stdlib ONLY — no pip install required (the CI runner image ships
Python 3 + ssl + socket + json + time). The WS handshake is hand-rolled
for the same reason (websocket-client/websockets would require network
during CI).

Exit code 0 = smoke passed. Non-zero = failed (CI fails the smoke job).

Reusable from: crates/rutster/tests/trunk_sim_e2e.rs (the #[ignore]'d TODO
body sketch — this script implements the same handshake pattern against a
real containerized binary rather than an in-process
MockTwilioMediaStreamsServer) + crates/rutster-trunk/tests/ws_ping.rs (the
connected/start frame sequence — same JSON shapes verbatim).
"""

import json
import os
import socket
import ssl
import struct
import subprocess
import sys
import tempfile
import time
from base64 import b64encode

IMAGE = os.environ.get("RUTSTER_ALLINONE_IMAGE", "rutster-allinone:smoke")
CONTAINER_NAME = "rutster-allinone-smoke"
HOST_PORT = int(os.environ.get("RUTSTER_ALLINONE_PORT", "18443"))
DATA_VOLUME = "rutster-smoke-caddy-data"

# 20ms of 8kHz u-law silence = 160 bytes; base64-encoded in a Twilio
# Media event envelope (matches the slice-A trunk_sim_e2e.rs TODO pattern
# + crates/rutster-trunk/tests/ws_ping.rs' frame shape).
PCM_FRAME_BYTES = b"\x00" * 160
FRAMES_TO_SEND = 200
CADENCE_MS = 20
RECV_DEADLINE_S = 10.0


def log(msg: str) -> None:
    print(f"[allinone_smoke] {msg}", flush=True)


def fail(msg: str) -> "NoReturn":  # noqa: F821 — Python 3.11+ syntax
    print(f"[allinone_smoke] FAIL: {msg}", file=sys.stderr, flush=True)
    subprocess.run(
        ["docker", "logs", CONTAINER_NAME], capture_output=False, timeout=10
    )
    sys.exit(1)


def run(cmd, check=True, timeout=30.0) -> str:
    """Run a command; return stdout. Fail the smoke on non-zero exit (if check)."""
    log(f"$ {' '.join(cmd)}")
    r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
    if check and r.returncode != 0:
        print(f"  stdout: {r.stdout[-2000:]}", file=sys.stderr)
        print(f"  stderr: {r.stderr[-2000:]}", file=sys.stderr)
        fail(f"command exited {r.returncode}: {' '.join(cmd)}")
    return r.stdout


def bootstrap_container() -> None:
    """Boot rutster-allinone with RUTSTER_LOCAL_CERTS=true."""
    subprocess.run(["docker", "rm", "-f", CONTAINER_NAME],
                   capture_output=True, timeout=10)
    subprocess.run(["docker", "volume", "rm", "-f", DATA_VOLUME],
                   capture_output=True, timeout=10)
    env = [
        "-e", "RUTSTER_DOMAIN=localhost",
        "-e", "RUTSTER_ACME_EMAIL=smoke@example.com",
        # CRITICAL: RUTSTER_LOCAL_CERTS=true makes Caddy use its internal CA
        # — no ACME in CI (spec §9).
        "-e", "RUTSTER_LOCAL_CERTS=true",
        "-e", "RUTSTER_HTTP_BIND=0.0.0.0:8080",
        "-e", "RUTSTER_MEDIA_BIND_IP=127.0.0.1",
        "-e", "RUTSTER_MEDIA_PORT_RANGE=49160-49170",
        "-e", "RUTSTER_DRAIN_DEADLINE_SECS=10",
        "-e", "RUTSTER_TAP_URL=ws://127.0.0.1:8082",
        # Twilio creds unset — WebRTC + media-stream WS smoke only.
        "-p", f"{HOST_PORT}:443",
    ]
    run(["docker", "run", "-d", "--name", CONTAINER_NAME,
         "-v", f"{DATA_VOLUME}:/data",
         *env, IMAGE], timeout=120.0)
    root_cert = wait_for_root_cert()
    log(f"root cert extracted ({len(root_cert)} bytes)")
    wait_for_healthz(root_cert)


def wait_for_root_cert() -> bytes:
    """Poll until Caddy wrote /data/caddy/pki/authorities/local/root.crt,
    then docker cp it out. Caddy creates this on first boot with local_certs."""
    deadline = time.time() + 30.0
    while time.time() < deadline:
        r = subprocess.run(
            ["docker", "exec", CONTAINER_NAME, "test", "-f",
             "/data/caddy/pki/authorities/local/root.crt"],
            capture_output=True, timeout=5,
        )
        if r.returncode == 0:
            r = subprocess.run(
                ["docker", "cp",
                 f"{CONTAINER_NAME}:/data/caddy/pki/authorities/local/root.crt", "-"],
                capture_output=True, timeout=10,
            )
            if r.returncode == 0 and r.stdout:
                return r.stdout
        time.sleep(0.5)
    fail("Caddy internal CA root cert did not appear within 30s")
    return b""  # unreachable


def wait_for_healthz(root_cert_pem: bytes) -> None:
    """Curl /healthz through TLS using the extracted root cert."""
    with tempfile.NamedTemporaryFile(suffix=".pem", delete=False) as f:
        f.write(root_cert_pem)
        ca_file = f.name
    try:
        deadline = time.time() + 30.0
        r = subprocess.run(["true"])  # for the warning-less r init
        while time.time() < deadline:
            r = subprocess.run(
                ["curl", "--cacert", ca_file, "-fsS",
                 f"https://localhost:{HOST_PORT}/healthz"],
                capture_output=True, text=True, timeout=5,
            )
            if r.returncode == 0 and r.stdout.strip() == "ok":
                log("/healthz ok")
                return
            time.sleep(0.5)
        fail(f"/healthz never returned ok within 30s (last stderr: "
             f"{getattr(r, 'stderr', '')[-500:]})")
    finally:
        os.unlink(ca_file)


# --- hand-rolled WS frame encode/decode (RFC 6455 §5) ---

def make_ws_frame(payload: bytes, opcode: int = 0x1) -> bytes:
    """Encode a client->server WS frame. opcode 0x1=text, 0x2=binary.
    Client frames MUST be masked (RFC 6455 §5.3)."""
    mask = os.urandom(4)
    masked = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
    header = bytearray([0x80 | opcode])  # FIN + opcode
    n = len(payload)
    if n < 126:
        header.append(0x80 | n)
    elif n < 65536:
        header.append(0x80 | 126)
        header.extend(struct.pack(">H", n))
    else:
        header.append(0x80 | 127)
        header.extend(struct.pack(">Q", n))
    header.extend(mask)
    return bytes(header) + masked


def read_ws_frame(sock) -> tuple:
    """Read one WS frame from the server. Returns (opcode, payload).
    Server frames are NOT masked (RFC 6455 §5.1).
    Returns (-1, b"") on EOF."""
    h1 = sock.recv(1)
    if not h1:
        return -1, b""
    fin_opcode = h1[0]
    h2 = sock.recv(1)[0]
    masked = bool(h2 & 0x80)
    length = h2 & 0x7F
    if length == 126:
        length = struct.unpack(">H", sock.recv(2))[0]
    elif length == 127:
        length = struct.unpack(">Q", sock.recv(8))[0]
    if masked:
        mask = sock.recv(4)
    payload = b""
    while len(payload) < length:
        chunk = sock.recv(length - len(payload))
        if not chunk:
            break
        payload += chunk
    if masked:
        payload = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
    return fin_opcode & 0x0F, payload


def open_wss():
    """Open the wss:// WS handshake to /twilio/media-stream inside the
    containerized all-in-one."""
    sock = socket.create_connection(("localhost", HOST_PORT), timeout=5)
    ctx = ssl.create_default_context(cafile=None)
    r = subprocess.run(
        ["docker", "cp",
         f"{CONTAINER_NAME}:/data/caddy/pki/authorities/local/root.crt", "-"],
        capture_output=True, timeout=10,
    )
    ctx.load_verify_locations(cadata=r.stdout.decode("ascii"))
    ctx.check_hostname = False  # the cert is for "localhost" — Caddy's local_certs issues it
    ssl_sock = ctx.wrap_socket(sock, server_hostname="localhost")
    log(f"TLS established (cipher={ssl_sock.cipher()})")

    key = os.urandom(16).hex()
    handshake = (
        f"GET /twilio/media-stream HTTP/1.1\r\n"
        f"Host: localhost:{HOST_PORT}\r\n"
        f"Upgrade: websocket\r\n"
        f"Connection: Upgrade\r\n"
        f"Sec-WebSocket-Key: {key}\r\n"
        f"Sec-WebSocket-Version: 13\r\n"
        f"\r\n"
    )
    ssl_sock.sendall(handshake.encode("ascii"))
    buf = b""
    while b"\r\n\r\n" not in buf:
        chunk = ssl_sock.recv(4096)
        if not chunk:
            fail("WS upgrade response closed prematurely")
        buf += chunk
    if b"101 Switching Protocols" not in buf:
        fail(f"WS upgrade did not return 101: {buf[:300]!r}")
    log("WS upgraded (HTTP/1.1 101 Switching Protocols)")
    return ssl_sock


def send_twilio_handshake(sock) -> None:
    """Send the Twilio Media Streams connected + start frames (the same
    JSON sequence crates/rutster-trunk/tests/ws_ping.rs uses against the
    in-process router — here against the real containerized edge->FOB path)."""
    connected = json.dumps({
        "event": "connected",
        "protocol": "twilio-media-stream",
        "version": "1.0.0",
    })
    start = json.dumps({
        "event": "start",
        "start": {"streamSid": "MZsmoke", "callSid": "CAsmoke"},
    })
    sock.sendall(make_ws_frame(connected.encode("utf-8")))
    sock.sendall(make_ws_frame(start.encode("utf-8")))
    log("sent Twilio connected + start handshake frames")


def stream_pcm_frames(sock) -> int:
    """Stream 200 PCM-zeroed frames at 20ms cadence as Twilio Media events.
    Returns the count of text frames received back from the engine."""
    received = 0
    deadline_global = time.time() + (FRAMES_TO_SEND * CADENCE_MS / 1000.0) + RECV_DEADLINE_S
    for _ in range(FRAMES_TO_SEND):
        media_event = json.dumps({
            "event": "media",
            "media": {"payload": b64encode(PCM_FRAME_BYTES).decode("ascii")},
        })
        sock.sendall(make_ws_frame(media_event.encode("utf-8")))
        # Drain non-blocking: read anything available without blocking the
        # 20ms cadence.
        sock.setblocking(False)
        try:
            while True:
                op, _ = read_ws_frame(sock)
                if op == -1:
                    break
                if op == 0x1:  # text frame — engine-originated event
                    received += 1
        except (BlockingIOError, ssl.SSLWantReadError):
            pass
        finally:
            sock.setblocking(True)
        time.sleep(CADENCE_MS / 1000.0)
    sock.settimeout(RECV_DEADLINE_S)
    try:
        while time.time() < deadline_global:
            op, _ = read_ws_frame(sock)
            if op == -1:
                break
            if op == 0x1:
                received += 1
    except (socket.timeout, ssl.SSLWantReadError):
        pass
    return received


def main() -> int:
    log(f"image={IMAGE}, host port={HOST_PORT}")
    bootstrap_container()
    sock = open_wss()
    send_twilio_handshake(sock)
    received = stream_pcm_frames(sock)
    log(f"sent {FRAMES_TO_SEND} frames; received {received} text frames back")
    if received < 1:
        fail("expected at least one engine-originated text frame (mark/outbound)")
    log("PASS: real edge->FOB WS path through Caddy TLS verified")
    return 0


if __name__ == "__main__":
    try:
        sys.exit(main())
    finally:
        # Always clean up the container + volume — even on FAIL.
        subprocess.run(["docker", "rm", "-f", CONTAINER_NAME],
                       capture_output=True, timeout=10)
        subprocess.run(["docker", "volume", "rm", "-f", DATA_VOLUME],
                       capture_output=True, timeout=10)
  • Step 3: Write deploy/smoke/compose_smoke.sh
#!/usr/bin/env bash
# deploy/smoke/compose_smoke.sh — T2 compose smoke (deploy-B §9).
#
# Brings up deploy/compose.yaml with \`docker compose up -d --wait\`, asserts
# every service reports healthy via \`docker compose ps\`, then curls
# /healthz through Caddy's TLS (using the internal CA —
# RUTSTER_LOCAL_CERTS=true is set in the smoke's env). The compose smoke
# is intentionally lighter than the all-in-one smoke (which already proves
# the WS path) — its job is to assert the four-service orchestrator shape
# boots + the network_mode: "service:engine" brain->engine tap posture.
#
# The ValkeyEventSink-lands-in-stream assertion is slice C's concern.
# Named hook below; do NOT implement it in this slice.

set -euo pipefail

cd "$(dirname "$0")/.."  # deploy/

IMAGES_TAG="${RUTSTER_IMAGES_TAG:-smoke}"
export RUTSTER_DOMAIN=localhost
export RUTSTER_ACME_EMAIL=smoke@example.com
export RUTSTER_LOCAL_CERTS=true
export RUTSTER_HTTP_BIND=0.0.0.0:8080
export RUTSTER_MEDIA_BIND_IP=127.0.0.1
export RUTSTER_MEDIA_PORT_RANGE=49160-49170
export RUTSTER_DRAIN_DEADLINE_SECS=10
export RUTSTER_TAP_URL=ws://127.0.0.1:8082

echo "[compose_smoke] overriding compose.yaml image tags -> :${IMAGES_TAG}"
# Sed the image: tags from :latest to :smoke so CI's locally-built images
# are picked up. In-place + restored on exit.
cp compose.yaml compose.yaml.smoke-backup
trap 'mv compose.yaml.smoke-backup compose.yaml' EXIT
sed -i "s|image: rutster-edge:latest|image: rutster-edge:${IMAGES_TAG}|" compose.yaml
sed -i "s|image: rutster-engine:latest|image: rutster-engine:${IMAGES_TAG}|" compose.yaml
sed -i "s|image: rutster-brain:latest|image: rutster-brain:${IMAGES_TAG}|" compose.yaml

echo "[compose_smoke] docker compose up -d --wait"
docker compose up -d --wait --timeout 120

echo "[compose_smoke] docker compose ps"
docker compose ps

# Each service should report "running" state. (Healthy flag is set by the
# engine's /readyz via the Docker healthcheck — none of the shipped images
# define a HEALTHCHECK yet; slice-G or a future hardening slice can add one.
# For now State="running" + /healthz 200 + /readyz 200 is the gate.)
HEALTHY_COUNT=$(docker compose ps --format json | jq '[.[] | select(.State == "running")] | length')
EXPECTED=4
if [ "$HEALTHY_COUNT" -ne "$EXPECTED" ]; then
  echo "[compose_smoke] FAIL: expected $EXPECTED running services, got $HEALTHY_COUNT" >&2
  docker compose logs
  exit 1
fi
echo "[compose_smoke] $EXPECTED services running"

# Caddy /healthz via TLS (internal CA). Port 443 is published by the caddy service.
ROOT_CERT=$(docker compose exec -T caddy cat /data/caddy/pki/authorities/local/root.crt)
echo "$ROOT_CERT" > /tmp/rutster-smoke-root.pem

# Wait for /healthz to return 200 via Caddy TLS.
for _ in $(seq 1 60); do
  if curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/healthz \
        | grep -q '^ok$'; then
    echo "[compose_smoke] /healthz ok"
    break
  fi
  sleep 0.5
done
curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/healthz \
  || { echo "[compose_smoke] FAIL: /healthz never returned ok"; docker compose logs; exit 1; }

# Engine /readyz — fresh boot, MUST be 200 (drain happens on shutdown, not
# boot; cap only after RUTSTER_MAX_SESSIONS sessions exist).
curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/readyz \
  || { echo "[compose_smoke] FAIL: /readyz never returned ok"; docker compose logs; exit 1; }
echo "[compose_smoke] /readyz ok"

# TODO slice-C: assert lifecycle event in Valkey stream.
# Slice C lands ValkeyEventSink (spec §5.6) + the assert-event-lands-in-
# Valkey-stream smoke addition. The hook here documents the contract:
#   Once slice C lands, this smoke runs:
#     docker compose exec valkey valkey-cli XLEN rutster:events
#   and asserts > 0 after a sim call completes (an EventSink::on_event
#   call writes via XADD MAXLEN ~). Until then, the valkey service is dark
#   (no consumers) — counting the stream length would always be 0.
# Leaving the named hook so slice C's plan can grep for it.
echo "[compose_smoke] PASS: T2 compose boots + /healthz + /readyz verified"

# Cleanup (compose down; volumes left for warm cache on re-run).
docker compose down --volumes --timeout 30 || true
exit 0
  • Step 4: Write deploy/smoke/reload_during_call.py
#!/usr/bin/env python3
"""deploy/smoke/reload_during_call.py — Caddy config reload during a live
sim call, asserting zero frame drops (deploy-B §9; spec §3.2; TLS brief
§5 risk 1).

Spec §3.2 mandates a CI e2e case for Caddy config-reload-during-live-call
because the upstream mitigation (`stream_close_delay` in Caddyfile) has an
open bug trail (caddy #6420, #7222) and is NOT trusted untested.

This script reuses the all-in-one smoke's WS helpers (open wss://
to /twilio/media-stream, send connected+start handshake); starts a streaming
loop sending PCM frames at 20ms cadence; concurrently triggers a `caddy
reload` mid-call (the operator's config-edit path); asserts the WS stays
up across the reload + no frames are dropped (the recv count matches the
sent count within tolerance).

Exit code 0 = reload-during-call passed. Non-zero = failed.
"""

import json
import os
import subprocess
import sys
import threading
import time
from base64 import b64encode

# Reuse the all-in-one smoke's WS helpers via import.
sys.path.insert(0, os.path.dirname(__file__))
from allinone_smoke import (  # noqa: E402
    make_ws_frame, read_ws_frame, bootstrap_container,
    open_wss, send_twilio_handshake, fail, log,
    CONTAINER_NAME, IMAGE, HOST_PORT, PCM_FRAME_BYTES,
)

FRAMES_TO_SEND = 400          # 8s of wall clock at 20ms cadence
CADENCE_MS = 20
RELOAD_AT_FRAME = 150         # fire `caddy reload` 3s into the 8s call


def trigger_caddy_reload() -> None:
    """Run `docker exec rutster-allinone-smoke caddy reload` inside the
    container. Caddy reloads its Caddyfile in-memory; stream_close_delay
    24h should keep upgraded WS tunnels alive."""
    log(f"triggering `caddy reload` at frame {RELOAD_AT_FRAME}...")
    r = subprocess.run(
        ["docker", "exec", CONTAINER_NAME,
         "caddy", "reload", "--config", "/etc/caddy/Caddyfile",
         "--adapter", "caddyfile"],
        capture_output=True, text=True, timeout=30.0,
    )
    if r.returncode != 0:
        log(f"caddy reload stderr: {r.stderr[-500:]}")
    # We do NOT fail the smoke on a non-zero caddy reload — the smoke's
    # question is "did the live WS survive?", not "did caddy reload work?"
    # A failed reload is its own bug; the WS-drop is the caddy #6420/#7222
    # concern this smoke is testing.


def stream_frames_with_midcall_reload(sock):
    """Returns (frames_sent, frames_received, ws_dropped_during_reload).
    ws_dropped_during_reload is True if the WS read loop got EOF/exception
    in the 1-second window after the reload fired."""
    sent = 0
    received = 0
    ws_dropped = False
    reload_fired = False
    reload_time = 0.0

    for i in range(FRAMES_TO_SEND):
        if i == RELOAD_AT_FRAME and not reload_fired:
            reload_fired = True
            reload_time = time.time()
            threading.Thread(target=trigger_caddy_reload, daemon=True).start()

        media_event = json.dumps({
            "event": "media",
            "media": {"payload": b64encode(PCM_FRAME_BYTES).decode("ascii")},
        })
        try:
            sock.sendall(make_ws_frame(media_event.encode("utf-8"), opcode=0x1))
            sent += 1
        except (BrokenPipeError, ConnectionResetError) as e:
            log(f"WS send failed at frame {i}: {e}")
            if not reload_fired or (time.time() - reload_time) < 2.0:
                ws_dropped = True
            break

        # Non-blocking drain.
        import socket as _socket
        sock.setblocking(False)
        try:
            while True:
                op, _ = read_ws_frame(sock)
                if op == -1:
                    break
                if op == 0x1:
                    received += 1
        except (BlockingIOError, ):
            pass
        finally:
            sock.setblocking(True)
        time.sleep(CADENCE_MS / 1000.0)

    return sent, received, ws_dropped


def main() -> int:
    log(f"image={IMAGE}, host port={HOST_PORT}")
    bootstrap_container()
    sock = open_wss()
    send_twilio_handshake(sock)
    sent, received, ws_dropped = stream_frames_with_midcall_reload(sock)
    log(f"sent={sent}, received={received}, ws_dropped={ws_dropped}")
    if ws_dropped:
        fail("WS dropped during caddy reload — stream_close_delay mitigation "
             "failed (caddy #6420/#7222) — the configuration MUST be re-evaluated")
    if sent != FRAMES_TO_SEND:
        fail(f"sent only {sent} of {FRAMES_TO_SEND} frames — WS died before reload")
    log("PASS: caddy reload during live call dropped zero frames")
    return 0


if __name__ == "__main__":
    try:
        sys.exit(main())
    finally:
        subprocess.run(
            ["docker", "rm", "-f", CONTAINER_NAME],
            capture_output=True, timeout=10,
        )
  • Step 5: Make the smoke scripts executable + re-run the content-check
cd /home/alee/Sources/rutster
chmod +x deploy/smoke/allinone_smoke.py deploy/smoke/compose_smoke.sh deploy/smoke/reload_during_call.py

test -f deploy/smoke/allinone_smoke.py \
  && test -f deploy/smoke/compose_smoke.sh \
  && test -f deploy/smoke/reload_during_call.py \
  && grep -q 'local_certs' deploy/smoke/allinone_smoke.py \
  && grep -q 'TODO slice-C' deploy/smoke/compose_smoke.sh \
  && grep -q 'caddy reload' deploy/smoke/reload_during_call.py \
  && grep -q 'stream_close_delay' deploy/smoke/reload_during_call.py \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 6: Commit
cd /home/alee/Sources/rutster
git add deploy/smoke/
git commit -s -m "feat(deploy/smoke): all-in-one + compose + reload-during-call smoke harness (deploy-B §9)

Three Python-stdlib-only smoke scripts (no pip install required in CI;
the runner image ships Python 3 + ssl/socket/json/base64/time):

* allinone_smoke.py — boots rutster-allinone with RUTSTER_LOCAL_CERTS=true
  (Caddy internal CA, no ACME per spec §9), extracts the root cert from
  /data, opens wss:// WS to /twilio/media-stream, sends Twilio's
  connected+start handshake (the same JSON sequence the in-tree
  ws_ping.rs/nodelay.rs tests use), streams 200 PCM frames at 20ms cadence,
  asserts the engine echoes/replies at least one text frame. Reuses the
  T8 stub's handshake pattern (crates/rutster/tests/trunk_sim_e2e.rs TODO
  body) against a real containerized binary rather than an in-process
  MockTwilioMediaStreamsServer.
* compose_smoke.sh — T2 four-service boots + health checks via Caddy TLS.
  The # TODO slice-C: assert lifecycle event in Valkey stream named hook
  documents the contract slice C will assert; the valkey service is dark
  on day one (no consumers) so the stream length is always 0 today.
* reload_during_call.py — holds a live WS streaming frames at 20ms while a
  concurrent \`caddy reload\` fires mid-call; asserts zero frames dropped
  + WS survives across the reload. Regresses the stream_close_delay 24h
  mitigation against caddy #6420/#7222 (TLS brief §5 risk 1; spec §9
  reload-during-call smoke).

WS frame encode/decode is hand-rolled (RFC 6455 §5 — ten lines on top of
socket+ssl) so the smoke needs no websocket-client/websockets pip
dependency + no network round-trip in CI."

Task 7: Extend .github/workflows/ci.ymlimage-build job

The image-build job builds all four images via docker buildx build against deploy/Dockerfile, with GHA cache from type=gha. It runs AFTER and GATED ON the existing fmt/clippy/test/deny/ sim-bench matrix (the smoke job after it depends on these built images).

Files:

  • Modify: .github/workflows/ci.yml (add image-build job at the end; do NOT touch existing fmt/clippy/test/deny/sim-bench/twilio-live jobs — verified they exist at the current HEAD + their content is unchanged. The file currently ends at line 146 with the twilio-live job's closing run: cargo test ... --include-ignored step).

Interfaces:

  • Consumes: Task 1's deploy/Dockerfile + .dockerignore (Task 2) + deploy/Caddyfile (Task 3) + deploy/s6-rc.d/ (Task 1 Step 4) + vendored s6-overlay tarballs (Task 1 Step 4).

  • Produces: four Docker images tagged rutster-{engine,brain,edge,allinone}:ci-${{ github.sha }}

    • re-tagged as :smoke for the downstream smoke jobs.
  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
grep -q 'name: image-build' .github/workflows/ci.yml && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Append the image-build job to .github/workflows/ci.yml

Append the following job to the end of .github/workflows/ci.yml (after the twilio-live job's closing block):


  # deploy-B §6.1 image-build — builds all four first-party images via
  # `docker buildx build --target` against deploy/Dockerfile. Runs AFTER the
  # fmt/clippy/test/deny/sim-bench matrix gates: the smoke job downstream
  # depends on these built images.
  #
  # Single builder invocation that builds all four --targets would be ideal
  # (shared cache), but buildx's `--target` is one-per-invocation — so four
  # build steps share the GHA cache via `cache-from/cache-to: type=gha`.
  # Warm caches take image-build from ~6 min to ~90s.
  image-build:
    name: image-build (4 images)
    runs-on: ubuntu-latest
    needs: [fmt, clippy, test, deny, sim-bench]
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      # libopus-dev — the headers the `opus` crate's build.rs expects.
      # (The builder stage of deploy/Dockerfile already installs this
      # inside the build image; this is a belt-and-braces step so a local
      # buildx cache root has the headers if the cache misses.)
      - name: Build-time host deps (belt-and-braces)
        run: sudo apt-get update && sudo apt-get install -y libopus-dev

      - name: Build rutster-engine image
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-engine
          tags: rutster-engine:ci-${{ github.sha }}
          push: false
          load: true
          cache-from: type=gha,scope=rutster-engine
          cache-to: type=gha,mode=max,scope=rutster-engine

      - name: Build rutster-brain image
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-brain
          tags: rutster-brain:ci-${{ github.sha }}
          push: false
          load: true
          cache-from: type=gha,scope=rutster-brain
          cache-to: type=gha,mode=max,scope=rutster-brain

      - name: Build rutster-edge image
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-edge
          tags: rutster-edge:ci-${{ github.sha }}
          push: false
          load: true
          cache-from: type=gha,scope=rutster-edge
          cache-to: type=gha,mode=max,scope=rutster-edge

      - name: Build rutster-allinone image
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-allinone
          tags: rutster-allinone:ci-${{ github.sha }}
          push: false
          load: true
          cache-from: type=gha,scope=rutster-allinone
          cache-to: type=gha,mode=max,scope=rutster-allinone

      - name: Smoke-tag the images for the downstream smoke job
        # The smoke job references images as `:smoke`
        # (deploy/smoke/compose_smoke.sh tags images with the
        # RUTSTER_IMAGES_TAG env). Tag the freshly-built :ci-<sha> images as
        # :smoke too, so the smoke job picks them up.
        run: |
          for img in rutster-engine rutster-brain rutster-edge rutster-allinone; do
            docker tag "${img}:ci-${{ github.sha }}" "${img}:smoke"
          done
          docker images | grep rutster

      - name: caddy validate
        # Validate the Caddyfile inside the freshly-built rutster-edge image.
        # Cheap regression against malformed Caddyfile (the local `caddy
        # validate` in Task 3 is optional; this one is mandatory in CI).
        run: |
          docker run --rm \
            -e RUTSTER_DOMAIN=pbx.example.com \
            -e RUTSTER_ACME_EMAIL=test@example.com \
            -v "$PWD/deploy/Caddyfile:/etc/caddy/Caddyfile:ro" \
            rutster-edge:smoke \
            caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
  • Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: image-build' .github/workflows/ci.yml \
  && grep -q 'needs: \[fmt, clippy, test, deny, sim-bench\]' .github/workflows/ci.yml \
  && grep -q 'docker/build-push-action@v5' .github/workflows/ci.yml \
  && grep -q 'cache-from: type=gha' .github/workflows/ci.yml \
  && grep -q 'target: rutster-engine' .github/workflows/ci.yml \
  && grep -q 'target: rutster-brain' .github/workflows/ci.yml \
  && grep -q 'target: rutster-edge' .github/workflows/ci.yml \
  && grep -q 'target: rutster-allinone' .github/workflows/ci.yml \
  && grep -q 'caddy validate' .github/workflows/ci.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: image-build job — builds the four first-party Docker images (deploy-B §6.1)

Extends .github/workflows/ci.yml with an image-build job, gated on the
existing fmt/clippy/test/deny/sim-bench matrix
(needs: [fmt, clippy, test, deny, sim-bench]). Builds all four images via
docker buildx against deploy/Dockerfile's named --targets, with type=gha
caching scoped per image so warm caches take image-build from ~6min to
~90s.

Tags the freshly-built :ci-<sha> images as :smoke too, for the
downstream smoke job to consume. Runs \`caddy validate\` inside the
freshly-built rutster-edge as a cheap Caddyfile-syntax regression.

No changes to the existing fmt/clippy/test/deny/sim-bench/twilio-live
jobs — those stay the cargo-side source of truth."

Task 8: Extend .github/workflows/ci.ymlsmoke job (all-in-one TLS sim call)

The smoke job runs deploy/smoke/allinone_smoke.py against the freshly-built rutster-allinone image from Task 7. It depends on image-build (which itself depends on the fmt/clippy/test/deny/ sim-bench matrix). The smoke job is where spec §9's "all-in-one smoke — boots, Caddy terminates TLS via its INTERNAL CA (no ACME in CI), full sim call through the real edge→FOB WS path" assertion is enforced.

Files:

  • Modify: .github/workflows/ci.yml (add smoke job after image-build).

Interfaces:

  • Consumes: Task 6's deploy/smoke/allinone_smoke.py + Task 7's rutster-allinone:smoke image.

  • Produces: pass/fail for the smoke job + downstream smoke-reload and smoke-compose jobs (Task 9) gate on this job.

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
grep -q 'name: smoke' .github/workflows/ci.yml && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Append the smoke job to .github/workflows/ci.yml

Append the following job to the end of .github/workflows/ci.yml (after the image-build job's closing caddy validate step from Task 7):


  # deploy-B §9 all-in-one smoke — boots rutster-allinone with
  # RUTSTER_LOCAL_CERTS=true (Caddy internal CA, no ACME in CI), extracts
  # the CA root cert from /data, opens wss:// to /twilio/media-stream,
  # sends Twilio's connected+start handshake frames, streams 200 PCM-zeroed
  # frames at 20ms cadence, asserts the engine replies with at least one
  # text frame through the real edge->FOB WS path. Spec §9 acceptance:
  #   * "boots"
  #   * "Caddy terminates TLS via its internal CA (no ACME in CI)"
  #   * "full sim call through the real edge->FOB WS path"
  # The reload-during-call smoke (Task 9 below) and compose smoke (Task 9)
  # are separate jobs that depend on this one.
  smoke:
    name: smoke (all-in-one TLS sim call)
    runs-on: ubuntu-latest
    needs: [image-build]
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python (stdlib-only; no pip)
        uses: actions/setup-python@v5
        with:
          python-version: "3.x"

      # Pull the freshly-built image from the image-build job. Since
      # jobs run on separate runners + `push: false` in image-build means
      # the image isn't in any registry, the smoke job rebuilds from the
      # GHA cache (warm — image-build populated it). This adds ~30s vs
      # an image-download artifact, but avoids the registry-auth + image-
      # save/load plumbing.
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Rebuild rutster-allinone:smoke (warm cache from image-build)
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-allinone
          tags: rutster-allinone:smoke
          push: false
          load: true
          cache-from: type=gha,scope=rutster-allinone

      - name: Run all-in-one smoke (deploy/smoke/allinone_smoke.py)
        run: |
          python3 deploy/smoke/allinone_smoke.py
        env:
          RUTSTER_ALLINONE_IMAGE: rutster-allinone:smoke
          RUTSTER_ALLINONE_PORT: "18443"
  • Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke' .github/workflows/ci.yml \
  && grep -q 'needs: \[image-build\]' .github/workflows/ci.yml \
  && grep -q 'allinone_smoke.py' .github/workflows/ci.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: smoke job — all-in-one TLS sim call through real edge->FOB WS path (deploy-B §9)

Adds the smoke job to .github/workflows/ci.yml, gated on image-build.
Runs deploy/smoke/allinone_smoke.py against the freshly-built
rutster-allinone:smoke image:

* Boots the all-in-one container with RUTSTER_LOCAL_CERTS=true (Caddy
  internal CA, no ACME in CI per spec §9).
* Extracts Caddy's internal-CA root cert from /data/caddy/pki/
  authorities/local/root.crt via docker cp.
* Opens wss:// WS to /twilio/media-stream using the extracted root
  (Python stdlib ssl + socket — no pip install required, CI runner-
  image ships Python 3 + ssl).
* Sends Twilio's connected+start handshake frames (the same JSON
  sequence crates/rutster-trunk/tests/ws_ping.rs uses).
* Streams 200 PCM-zeroed frames at 20ms cadence as Twilio Media events.
* Asserts the engine replies with at least one text frame — proving
  the real edge->FOB WS path works end-to-end through TLS.

The compose smoke + reload-during-call smoke (Task 9) gate on this job."

Task 9: Extend .github/workflows/ci.ymlsmoke-compose + smoke-reload jobs

Two more smoke jobs, each gated on image-build (Task 7). smoke-compose runs deploy/smoke/compose_smoke.sh against the four fresh images from image-build. smoke-reload runs deploy/smoke/reload_during_call.py (reuses the all-in-one image — both smokes are all-in-one-shaped; these are separate jobs not steps within smoke for clearer failure attribution + parallelism).

Files:

  • Modify: .github/workflows/ci.yml (append two more jobs after smoke).

Interfaces:

  • Consumes: Task 6's compose_smoke.sh + reload_during_call.py + Task 7's images.

  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
grep -q 'name: smoke-compose' .github/workflows/ci.yml \
  && grep -q 'name: smoke-reload' .github/workflows/ci.yml \
  && grep -q 'reload_during_call.py' .github/workflows/ci.yml \
  && grep -q 'compose_smoke.sh' .github/workflows/ci.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Append the two new jobs to .github/workflows/ci.yml

Append the following to the end of .github/workflows/ci.yml (after the smoke job from Task 8):


  # deploy-B §9 compose smoke — T2 four-service compose stack boots + all
  # services healthy + Caddy TLS /healthz + /readyz. Lighter than the
  # all-in-one smoke (which already proved the WS path) — its job is the
  # orchestrator shape + the network_mode: "service:engine" brain->engine tap
  # posture across the four services.
  smoke-compose:
    name: smoke (T2 compose four-service)
    runs-on: ubuntu-latest
    needs: [image-build]
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      # Rebuild all three first-party images from the warm GHA cache
      # populated by image-build. Valkey is upstream (no rebuild needed).
      - name: Rebuild rutster-edge:smoke
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-edge
          tags: rutster-edge:smoke
          push: false
          load: true
          cache-from: type=gha,scope=rutster-edge

      - name: Rebuild rutster-engine:smoke
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-engine
          tags: rutster-engine:smoke
          push: false
          load: true
          cache-from: type=gha,scope=rutster-engine

      - name: Rebuild rutster-brain:smoke
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-brain
          tags: rutster-brain:smoke
          push: false
          load: true
          cache-from: type=gha,scope=rutster-brain

      # jq is pre-installed on ubuntu-latest GHA runners.
      - name: Run compose smoke (deploy/smoke/compose_smoke.sh)
        run: bash deploy/smoke/compose_smoke.sh
        env:
          RUTSTER_IMAGES_TAG: smoke

  # deploy-B §9 + spec §3.2 + TLS brief §5 risk 1: Caddy config-reload
  # during a live simulated call, asserting zero frame drops. The
  # stream_close_delay 24h mitigation in deploy/Caddyfile is the load-
  # bearing fixture; caddy #6420 / #7222 are open upstream bugs — the
  # smoke is "don't trust untested" made CI-regressed.
  smoke-reload:
    name: smoke (caddy reload during live call, zero drops)
    runs-on: ubuntu-latest
    needs: [smoke]   # reuses the rutster-allinone:smoke image, warm cache
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python (stdlib-only; no pip)
        uses: actions/setup-python@v5
        with:
          python-version: "3.x"

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Rebuild rutster-allinone:smoke (warm cache)
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-allinone
          tags: rutster-allinone:smoke
          push: false
          load: true
          cache-from: type=gha,scope=rutster-allinone

      - name: Run reload-during-call smoke (deploy/smoke/reload_during_call.py)
        run: python3 deploy/smoke/reload_during_call.py
        env:
          RUTSTER_ALLINONE_IMAGE: rutster-allinone:smoke
          RUTSTER_ALLINONE_PORT: "18444"
  • Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke-compose' .github/workflows/ci.yml \
  && grep -q 'name: smoke-reload' .github/workflows/ci.yml \
  && grep -q 'reload_during_call.py' .github/workflows/ci.yml \
  && grep -q 'compose_smoke.sh' .github/workflows/ci.yml \
  && grep -q 'needs: \[smoke\]' .github/workflows/ci.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-PASS.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: smoke-compose + smoke-reload jobs — T2 four-service boot + caddy reload-during-call (deploy-B §9)

smoke-compose (gated on image-build):
* Rebuilds rutster-edge/engine/brain:smoke from warm GHA cache.
* Runs deploy/smoke/compose_smoke.sh — \`docker compose up -d --wait\`,
  asserts all four services State=running, curls /healthz + /readyz
  through Caddy TLS (internal CA). Lighter than the all-in-one smoke
  (which already proved the WS path) — its job is the orchestrator
  shape + the network_mode: \"service:engine\" brain->engine tap posture.

smoke-reload (gated on smoke):
* Reuses rutster-allinone:smoke + the allinone_smoke.py WS helpers.
* Runs deploy/smoke/reload_during_call.py — holds a live WS streaming
  frames at 20ms while a concurrent \`caddy reload\` fires mid-call;
  asserts zero frames dropped + WS survives across the reload.
  Regresses the stream_close_delay 24h mitigation against caddy #6420/
  #7222 (TLS brief §5 risk 1; spec §9 reload-during-call smoke).

Both jobs run on ubuntu-latest — separate runners for clearer failure
attribution + parallelism (the smoke gate is the failing CI signal,
not a single serial run)."

Task 10: Slice F — .github/workflows/publish-images.yml tag-push image publish

Slice F (spec §6.3 + §11): tag-push workflow publishing all four images to the git.adlee.work/alee/rutster-* registry namespace. linux/amd64 only. Runs AFTER and GATED ON the existing fmt/clippy/test/deny/sim-bench/image-build pipeline (re-runs the build to assert freshness, then publishes; caching keeps the build cost ~30s when nothing changed).

Files:

  • Create: .github/workflows/publish-images.yml

Interfaces:

  • Consumes: Task 1's deploy/Dockerfile + the same docker buildx build --target invocations as Task 7's image-build job (the publish workflow is structurally a copy of image-build with push: true + tags: git.adlee.work/alee/... instead of push: false + tags: ...:ci-<sha>). Reuses the GHA cache (warm from image-build's cache-to: type=gha,mode=max).

  • Produces: four published Docker images at git.adlee.work/alee/rutster-{allinone,engine,brain,edge}:${{ github.ref_name }}

    • :latest on every v* tag push.
  • Step 1: Run the failing content-check

cd /home/alee/Sources/rutster
test -f .github/workflows/publish-images.yml \
  && grep -q 'on:' .github/workflows/publish-images.yml \
  && grep -q 'push:' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-allinone' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-engine' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-brain' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-edge' .github/workflows/publish-images.yml \
  && grep -q 'linux/amd64' .github/workflows/publish-images.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

Expected: CHECK-FAIL.

  • Step 2: Write the complete workflow file

Write .github/workflows/publish-images.yml with exactly this content:

# .github/workflows/publish-images.yml — slice F (spec §6.3 + §11).
#
# Tag-push workflow: builds + pushes all four first-party images to the
# git.adlee.work/alee/rutster-* registry namespace. linux/amd64 only (spec
# §1.2 / ADR-0011 — arm64 is a named deferral).
#
# Gating: runs AFTER and gated on the existing fmt/clippy/test/deny/sim-bench
# + image-build pipeline. The publish workflow re-runs the fmt/clippy/test/
# deny matrix as a `needs:` chain (cheap when nothing changed) + builds all
# four images from the same Dockerfile CI's image-build job uses. This makes
# the published image byte-identical to the one CI tested — the "registry tag
# points at a tested image" invariant operators rely on.
#
# Registry namespace: git.adlee.work/alee/rutster-* (matches the `alee` tea
# login per AGENTS.md Git workflow + the root Cargo.toml's `repository`
# field). If your Gitea registry uses a different owner segment, replace
# `alee` throughout this file (REGISTRY_NAMESPACE below).

name: Publish Images

on:
  push:
    tags:
      - 'v*'

env:
  REGISTRY: git.adlee.work
  REGISTRY_NAMESPACE: alee
  PLATFORMS: linux/amd64

jobs:
  # The publish workflow is its own pipeline; it re-runs fmt/clippy/test/deny
  # + builds the images + then publishes. The fmt/clippy/test/deny re-runs
  # are cheap on a green repo (cache hits, fast fmt + clippy + test), and
  # they make a critical guarantee: the version published under a `v*` tag
  # has ALL the repo's quality gates green at that exact commit. Without
  # this `needs:` chain, a tag could be pushed at a commit where the matrix
  # failed but image-build happened to succeed on a flake.
  fmt:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: rustfmt
      - run: cargo fmt --check

  clippy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy
      - name: Install libopus (media crate FFI dep)
        run: apt-get update && apt-get install -y libopus-dev
      - uses: Swatinem/rust-cache@v2
      - run: cargo clippy --all --all-targets -- -D warnings

  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        toolchain: [stable, "1.85"]
    steps:
      - uses: actions/checkout@v4
      # Seam gate (mirror of .github/workflows/ci.yml's test job —
      # loop_driver.rs + rtc_session.rs stay byte-identical to slice-3/5).
      - name: Seam gate — loop_driver frozen + rtc_session pinned
        run: |
          EXPECTED_LOOP_DRIVER='744bf314edf7f4925c8bb3bd0f5176dbc88f8113'
          EXPECTED_RTC_SESSION='f47d63b9a2883d37066a93c9daa0e2cf8816bec4'
          GOT_LOOP_DRIVER=$(git hash-object crates/rutster-media/src/loop_driver.rs)
          GOT_RTC_SESSION=$(git hash-object crates/rutster-media/src/rtc_session.rs)
          [ "$GOT_LOOP_DRIVER" = "$EXPECTED_LOOP_DRIVER" ] || exit 1
          [ "$GOT_RTC_SESSION" = "$EXPECTED_RTC_SESSION" ] || exit 1
      - uses: dtolnay/rust-toolchain@master
        with:
          toolchain: ${{ matrix.toolchain }}
      - name: Install libopus (media crate FFI dep)
        run: apt-get update && apt-get install -y libopus-dev
      - uses: Swatinem/rust-cache@v2
      - run: cargo test --all

  deny:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - uses: EmbarkStudios/cargo-deny-action@v2
        with:
          command: check

  sim-bench:
    name: sim-bench (stable)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy
      - name: Install libopus (media crate FFI dep)
        run: apt-get update && apt-get install -y libopus-dev
      - uses: Swatinem/rust-cache@v2
      - name: cargo fmt + clippy on sim-bench feature paths
        run: |
          cargo fmt --all --check
          cargo clippy --all --all-targets --features=sim-bench -- -D warnings
      - name: Run sim-bench threshold sweep
        run: cargo test --all --features=sim-bench -- --test-threads=1

  publish:
    name: Publish 4 images to git.adlee.work/alee/rutster-*
    runs-on: ubuntu-latest
    needs: [fmt, clippy, test, deny, sim-bench]
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build-time host deps (belt-and-braces)
        run: sudo apt-get update && sudo apt-get install -y libopus-dev

      - name: Log in to Gitea registry
        # GITEA_REGISTRY_USERNAME + GITEA_REGISTRY_PASSWORD are repo secrets
        # set by the maintainer (per AGENTS.md Git workflow — the `alee`
        # tea login). The username is `alee` (matches REGISTRY_NAMESPACE).
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ secrets.GITEA_REGISTRY_USERNAME }}
          password: ${{ secrets.GITEA_REGISTRY_PASSWORD }}

      - name: Determine tags
        id: tags
        # Two tags per image: the version tag (github.ref_name, e.g. v0.1.0)
        # + `latest`. Operators can pin to either; `latest` is the
        # convenience tag that always points at the most recent release.
        run: |
          echo "version_tag=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
          echo "namespace=${REGISTRY}/${REGISTRY_NAMESPACE}" >> $GITHUB_OUTPUT

      - name: Build + push rutster-engine
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-engine
          platforms: ${{ env.PLATFORMS }}
          tags: |
            ${{ steps.tags.outputs.namespace }}/rutster-engine:${{ steps.tags.outputs.version_tag }}
            ${{ steps.tags.outputs.namespace }}/rutster-engine:latest
          push: true
          cache-from: type=gha,scope=rutster-engine
          cache-to: type=gha,mode=max,scope=rutster-engine

      - name: Build + push rutster-brain
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-brain
          platforms: ${{ env.PLATFORMS }}
          tags: |
            ${{ steps.tags.outputs.namespace }}/rutster-brain:${{ steps.tags.outputs.version_tag }}
            ${{ steps.tags.outputs.namespace }}/rutster-brain:latest
          push: true
          cache-from: type=gha,scope=rutster-brain
          cache-to: type=gha,mode=max,scope=rutster-brain

      - name: Build + push rutster-edge
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-edge
          platforms: ${{ env.PLATFORMS }}
          tags: |
            ${{ steps.tags.outputs.namespace }}/rutster-edge:${{ steps.tags.outputs.version_tag }}
            ${{ steps.tags.outputs.namespace }}/rutster-edge:latest
          push: true
          cache-from: type=gha,scope=rutster-edge
          cache-to: type=gha,mode=max,scope=rutster-edge

      - name: Build + push rutster-allinone
        uses: docker/build-push-action@v5
        with:
          context: .
          file: deploy/Dockerfile
          target: rutster-allinone
          platforms: ${{ env.PLATFORMS }}
          tags: |
            ${{ steps.tags.outputs.namespace }}/rutster-allinone:${{ steps.tags.outputs.version_tag }}
            ${{ steps.tags.outputs.namespace }}/rutster-allinone:latest
          push: true
          cache-from: type=gha,scope=rutster-allinone
          cache-to: type=gha,mode=max,scope=rutster-allinone
  • Step 3: Re-run the content-check + YAML lint
cd /home/alee/Sources/rutster
test -f .github/workflows/publish-images.yml \
  && grep -q 'on:' .github/workflows/publish-images.yml \
  && grep -q 'push:' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-allinone' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-engine' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-brain' .github/workflows/publish-images.yml \
  && grep -q 'git.adlee.work/alee/rutster-edge' .github/workflows/publish-images.yml \
  && grep -q 'linux/amd64' .github/workflows/publish-images.yml \
  && grep -q 'needs: \[fmt, clippy, test, deny, sim-bench\]' .github/workflows/publish-images.yml \
  && grep -q 'GITEA_REGISTRY_USERNAME' .github/workflows/publish-images.yml \
  && grep -q 'v0.1.0' .github/workflows/publish-images.yml \
  && echo CHECK-PASS || echo CHECK-FAIL

# YAML syntax validation via Python stdlib (no yamllint dep needed).
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/publish-images.yml'))" \
  || { echo "YAML syntax error"; exit 1; }

Expected: CHECK-PASS. The YAML loads cleanly.

  • Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/publish-images.yml
git commit -s -m "ci: publish-images.yml — slice F tag-push image publish (deploy-F §6.3)

Tag-push workflow (\`on: push: tags: ['v*']\`) that publishes all four
first-party images to git.adlee.work/alee/rutster-* (REGISTRY_NAMESPACE
matches the \`alee\` tea login per AGENTS.md Git workflow + the root
Cargo.toml's repository field). linux/amd64 only (spec §1.2 — arm64 is a
named deferral per ADR-0011).

Gating: re-runs fmt/clippy/test/deny/sim-bench as a \`needs:\` chain so a
tag's published image is byte-identical to the one tested by the routine
CI matrix. The fmt/clippy/test/deny re-runs are cheap on a green repo
(cache hits, fast checks), and they make a critical guarantee: the
version under a \`v*\` tag has ALL the repo's quality gates green at
that exact commit. Without this \`needs:\` chain, a tag could be pushed
at a commit where the matrix failed but image-build happened to succeed
on a flake.

Two tags per image: the version tag (github.ref_name, e.g. v0.1.0) +
\`latest\` (the convenience tag operators pin to).

Docker login via GITEA_REGISTRY_USERNAME + GITEA_REGISTRY_PASSWORD
secrets, set by the maintainer. If your Gitea registry uses a different
owner segment, replace \`alee\` throughout (REGISTRY_NAMESPACE + the
namespace/git.adlee.work/alee/ strings in .env.example + compose.yaml)."

Task 11: Final verification sweep

No code. Evidence before assertions — run every gate the CI runs, plus the seam-gate check, before calling the slice done.

  • Step 1: Full gate run (cargo-side, unchanged from existing CI)
cd /home/alee/Sources/rutster
cargo fmt --all --check
cargo clippy --all -- -D warnings
cargo clippy --all --all-targets --features=sim-bench -- -D warnings
cargo test --all
cargo test --all --features=sim-bench -- --test-threads=1
cargo deny check
cargo doc --no-deps

Expected: every command exits 0. (This slice adds no Rust code; these gates stay green from plan A's landing on main. The sweep verifies the slice-B file additions don't accidentally break the cargo build — e.g. a stray Cargo.toml edit.)

  • Step 2: Seam gate verification (unchanged hashes)
cd /home/alee/Sources/rutster
git hash-object crates/rutster-media/src/loop_driver.rs
git hash-object crates/rutster-media/src/rtc_session.rs

Expected output, exactly:

744bf314edf7f4925c8bb3bd0f5176dbc88f8113
f47d63b9a2883d37066a93c9daa0e2cf8816bec4

If either hash differs, STOP — a task touched a seam-gated file; restore with git checkout main -- crates/rutster-media/src/loop_driver.rs crates/rutster-media/src/rtc_session.rs and re-run the gates.

  • Step 3: Artifact inventory (every planned file exists)
cd /home/alee/Sources/rutster
for f in \
  deploy/Dockerfile \
  deploy/.dockerignore \
  deploy/Caddyfile \
  deploy/compose.yaml \
  deploy/.env.example \
  deploy/s6-overlay-noarch.tar.xz \
  deploy/s6-overlay-x86_64.tar.xz \
  deploy/s6-overlay.sha256sum \
  deploy/s6-rc.d/caddy/run \
  deploy/s6-rc.d/engine/run \
  deploy/s6-rc.d/brain/run \
  deploy/s6-rc.d/valkey-database/run \
  deploy/smoke/allinone_smoke.py \
  deploy/smoke/compose_smoke.sh \
  deploy/smoke/reload_during_call.py \
  .github/workflows/publish-images.yml \
  ; do
  test -e "$f" || echo "MISSING: $f"
done
echo "inventory check complete"

Expected: zero MISSING: lines.

  • Step 4: CI workflow sanity (both workflow files parse cleanly + image-build/smoke jobs are present)
cd /home/alee/Sources/rutster
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/publish-images.yml'))"
grep -E 'name: (image-build|smoke|smoke-compose|smoke-reload)' .github/workflows/ci.yml
grep -E 'name: (Publish|fmt|clippy|test|deny|sim-bench|publish)' .github/workflows/publish-images.yml

Expected: both YAML files load; ci.yml lists all four new jobs (image-build, smoke, smoke-compose, smoke-reload); publish-images.yml lists its publish job + the gating matrix.

  • Step 5: TODO slice-C hook verification (the named hook stays a comment, not implemented)
cd /home/alee/Sources/rutster
grep -n 'TODO slice-C' deploy/smoke/compose_smoke.sh

Expected: exactly one match, naming the ValkeyEventSink assertion as deferred to slice C. The slice-E (ValkeyEventSink) plan will grep this same line.

  • Step 6: DCO verification (every commit on this branch signed off)
cd /home/alee/Sources/rutster
git log main..HEAD --format='%h %s' \
  | head -20 \
  && echo "---DCO count vs commit count---" \
  && COMMITS=$(git log main..HEAD --format='%H' | wc -l) \
  && DCOS=$(git log main..HEAD --format='%(trailers:key=Signed-off-by)' | grep -c 'Signed-off-by') \
  && echo "commits: $COMMITS, signoffs: $DCOS"

Expected: the signoffs count equals the commit count (every git commit -s produced a Signed-off-by trailer per AGENTS.md DCO requirement).

  • Step 7: PR opening
cd /home/alee/Sources/rutster
git push -u origin deploy-b/packaging-ci
tea pulls create \
  --head deploy-b/packaging-ci \
  --base main \
  --title "deploy slice B + F: deploy/ artifacts + image-build + container smoke CI + tag-push publish" \
  --description "## What lands

- deploy/Dockerfile (multi-stage, four named --target stages: rutster-engine/brain/edge/allinone)
- deploy/compose.yaml (T2 reference; stop_grace_period 660s; brain network_mode: \"service:engine\")
- deploy/Caddyfile (tuned timeouts, honest X-Forwarded, stream_close_delay 24h)
- deploy/.env.example (all RUTSTER_* vars documented)
- deploy/smoke/{allinone_smoke.py,compose_smoke.sh,reload_during_call.py}
- .github/workflows/ci.yml extended with image-build + smoke + smoke-compose + smoke-reload jobs
- .github/workflows/publish-images.yml (slice F tag-push image publish)

## Dependencies

Depends on plan A (deploy-a/engine-hygiene) merged first — TCP_NODELAY,
WS pings, and the trunk config fixes make the containerized PSTN path honest.

## Reviews

- Spec: docs/superpowers/specs/2026-07-05-deployment-topology-design.md §2.1, §2.2, §3.2, §6, §9, §11
- TLS brief: docs/superpowers/specs/2026-07-05-tls-edge-decision-brief.md §3(a) + §5 risk 1
- Slice-C TODO hook in deploy/smoke/compose_smoke.sh — the ValkeyEventSink smoke assertion is
  the named slice-C concern; this slice ships the valkey service shape but no event-lands check

## Merge instructions (if non-default)

Default squash-merge. The deploy-b/packaging-ci branch is standalone (no
stacked branches depend on its commit SHAs), so squash keeps main linear."

Expected: the tea PR URL is printed; the PR is open + targeting main.


Final acceptance checklist

After all 11 tasks land:

  • All planned artifacts exist on disk (Task 11 Step 3 — no MISSING: lines).
  • cargo fmt --check, cargo clippy -D warnings (default + --features=sim-bench), cargo test --all, sim-bench sweep (--test-threads=1), cargo deny check, cargo doc --no-deps all clean. (This slice adds no Rust; the gates stay green from plan A. The sweep verifies nothing in the cargo graph was disturbed.)
  • Seam gate: loop_driver.rs + rtc_session.rs blob hashes unchanged (Task 11 Step 2).
  • deploy/Dockerfile builds all four named --targets (rutster-engine, rutster-brain, rutster-edge, rutster-allinone). Builder pins rust:1.85-slim-bookworm (matches rust-toolchain.toml). libopus-dev in builder, libopus0 in runtime (matches dev — spec §6.1). Brain image gets libopus0 too (spec ambiguity #2 — the linker requires it). No audiopus_sys bundled-cmake fallback.
  • deploy/Caddyfile has stream_close_delay 24h, idle 24h, write 0, honest X-Forwarded-Proto/Host/For (engine-side RUTSTER_TRUSTED_PROXIES is the trust gate; fail-closed default). The RUTSTER_LOCAL_CERTS=true env override makes Caddy use its internal CA (no ACME in CI per spec §9).
  • deploy/compose.yaml ships four services (caddy=rutster-edge, engine=rutster-engine, brain=rutster-brain with network_mode: "service:engine", valkey=upstream valkey/valkey), stop_grace_period: 660s, two named volumes (caddy-data, valkey-data).
  • deploy/.env.example lists every RUTSTER_* var with a default, the slice that owns it, and the fail-fast behavior. RUTSTER_DRAIN_DEADLINE_SECS=600 present.
  • .github/workflows/ci.yml has image-build + smoke + smoke-compose + smoke-reload jobs; the existing fmt/clippy/test/deny/sim-bench/twilio-live jobs are unchanged.
  • .github/workflows/publish-images.yml (slice F) publishes all four images to git.adlee.work/alee/rutster-* on v* tag push, gated on [fmt, clippy, test, deny, sim-bench]. Linux/amd64 only.
  • Caddy-reload-during-live-call smoke (Task 9) asserts zero frames dropped across a caddy reload mid-call — the stream_close_delay 24h mitigation made CI-regressed against caddy #6420/#7222 (TLS brief §5 risk 1; spec §9 reload-during-call smoke).
  • All-in-one TLS smoke (Task 8) boots rutster-allinone with RUTSTER_LOCAL_CERTS=true, extracts the internal-CA root cert from /data, opens a wss:// WS to /twilio/media-stream, sends Twilio's connected+start handshake, streams 200 PCM frames at 20 ms cadence, asserts at least one engine-originated text frame — proving the real edge→FOB WS path works end-to-end through TLS.
  • Compose smoke (Task 9) boots all four services via docker compose up -d --wait, asserts all four running + /healthz + /readyz via Caddy TLS.
  • TODO slice-C named hook (Task 6 Step 3) in deploy/smoke/compose_smoke.sh — the ValkeyEventSink smoke assertion is deferred to slice C (one grep -n 'TODO slice-C' match).
  • 11 commits, each signed offgit log --format='%h %s' main..HEAD shows the slice-B series; git log main..HEAD --format='%(trailers:key=Signed-off-by)' | grep -c 'Signed-off-by' equals the commit count (Task 11 Step 6).
  • PR opened via tea per house workflow (Task 11 Step 7); the PR description names the plan-A dependency root + the slice-C TODO hook + the squash-merge default.

Out-of-scope reminder (spec §1.2 — do NOT add in this slice, named-deferral boundary)

  • rustls Phase 1 in-process TLS (BYO-cert) — slice C (.github/workflows/plans/2026-07-05-deploy-c-rustls-tls.md, path may vary). The RUTSTER_TLS_CERT/RUTSTER_TLS_KEY env vars are noted in .env.example as commented future stubs.
  • /metrics endpoint — slice D. RUTSTER_METRICS_BIND is noted as a future var.
  • ValkeyEventSink — slice E. RUTSTER_VALKEY_URL is noted as a future var; the compose-smoke TODO slice-C hook documents the contract the slice-C/E work fills.
  • arm64 images — named deferral per ADR-0011 §1.2; the publish workflow pins linux/amd64.
  • T3 fleet artifacts (presence heartbeats, directory redirects, placement code) — ADR chapter only this epoch.
  • Object-storage CDR pipelineValkeyEventSink is evidence preservation, not the billing ledger (ADR-0005 source-of-truth rule); CDR→object storage stays deferred.
  • In-binary ACME (rustls Phase 2) — behind the four named triggers in ADR-0011.
  • Cargo-deny-adjacent image license scan — bundled-binary licenses are aggregation-clean vs GPL-3 (noted in .env.example header); the automated assertion is a future CI muscle.

If an agent proposes adding any of these in slice B, the right answer is "no — see spec §1.2 / ADR-0011."