Plan A: engine hygiene (TCP_NODELAY via axum 0.7.9 Serve::tcp_nodelay, trunk WS pings, webhook-base TwiML derivation, trusted-proxy posture). Plan B: packaging + CI (four images, s6 all-in-one, compose, Caddyfile, smoke suite incl. reload-during-call, slice-F publish workflow). Plan C: in-binary features (rustls Phase 1 BYO-cert, /metrics, ValkeyEventSink + smoke hook fill). Plan G: docs/deploy tree + ADR-0011 drafting. Authored by parallel plan-writers (this session + omo rescue after a session-limit interruption); verified: placeholder scan, cited-path existence, cross-plan image/env-var consistency, seam-gate compliance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012QndwfhjyTiZcUYp87dwW8 Signed-off-by: Aaron D. Lee <himself@adlee.work>
135 KiB
Deploy slice B + F — deploy/ artifacts (Dockerfile/compose/Caddyfile/.env) + image-build + container smoke CI + tag-push publish — Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use
superpowers:subagent-driven-development(recommended) orsuperpowers:executing-plansto implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Dependency: This plan depends on plan A (2026-07-05-deploy-a-engine-hygiene.md) landing
on main first. Plan A's three hygiene items are what make the containerized PSTN path honest:
TCP_NODELAY (serve_with_nodelay) so the 20 ms WS cadence isn't Nagle-coalesced behind the
edge, app-level WS pings so neither Twilio's nor Telnyx's idle timers can kill a quiet
mid-call WS, and the trunk config fixes (RUTSTER_TWILIO_WEBHOOK_BASE → derived TwiML Stream
URL + config::twilio_credentials startup validation + RUTSTER_TRUSTED_PROXIES
trusted-X-Forwarded-Proto/Host reconstruction) so a container that takes a real Twilio call is
actually answering the public URL the CPaaS dialed, not ws://127.0.0.1:8080 from slice-5's
stub. Slice C (rustls Phase 1) also branches after A; B and C are order-independent. Do not
start Task 1 of this plan until git log main --oneline | grep -c 'deploy-A' >= 1.
Goal: Land the deploy/ artifact tree (multi-stage Dockerfile producing four first-party
images, compose.yaml, Caddyfile, .env.example) and the containerized CI muscle that
proves them — image-build job, all-in-one TLS smoke through Caddy's internal CA, compose-health
smoke, and Caddy config-reload-during-live-call — then add the slice-F tag-push publish workflow
to the git.adlee.work/alee/ registry namespace (spec §2.1, §2.2, §3.2, §6, §9, §11).
Architecture: One multi-stage deploy/Dockerfile with named build targets (rutster-engine,
rutster-brain, rutster-edge, rutster-allinone) so docker buildx build --target … produces
each image from the same Dockerfile — eliminating the per-image Dockerfile drift that broke
Asterisk-era integrators. The builder stage compiles against rust:1.85-slim-bookworm +
libopus-dev (matching rust-toolchain.toml + dev loop exactly) and is a single target shared
across engine/brain (both derive from the same cargo build --bins); the all-in-one image adds
s6-overlay v3 + the upstream valkey/valkey server binary + the custom Caddy from a parallel
xcaddy builder. CI is added to the existing .github/workflows/ci.yml (Gitea Actions
consuming GitHub-workflow YAML) as two new jobs (image-build + smoke) plus a third workflow
file (.github/workflows/publish-images.yml) for the slice-F tag-push publish. The smoke job
reuses the WS-handshake protocol sequence (the {"event":"connected"…} + {"event":"start"…}
frames) proven in crates/rutster/tests/trunk_sim_e2e.rs's TODO body and the in-tree
ws_ping.rs/nodelay.rs integration tests — but driving it through the real edge→FOB path
inside a booted container, against Caddy's internal CA (no ACME in CI). The
ValkeyEventSink-lands-in-stream assertion is explicitly a slice-C concern; this plan leaves a
named TODO hook comment in the compose-smoke step and does NOT implement it.
Tech Stack: Docker (BuildKit multi-stage), rust:1.85-slim-bookworm builder (pinned to
rust-toolchain.toml), debian:bookworm-slim runtime (matches dev: apt-get install libopus0
in runtime, libopus-dev in builder — spec §6.1 "distro libopus0 over static-bundled"), s6-overlay
v3.2.0.0 (ISC), xcaddy with the curated DNS-plugin set (cloudflare/route53/porkbun/hetzner/desec
— duckdns excluded, no license file per spec §3.2), upstream valkey/valkey:latest (BSD-3),
Caddy 2.8 builder image. CI: Gitea Actions (GitHub Actions YAML syntax), docker CLI, Python 3
for the smoke-test WS client (stdlib only — no pip install step in CI). Zero new Rust crates.
Zero changes to cargo graphs.
Global Constraints
House rules (every task's requirements implicitly include this section):
- License: GPL-3.0-or-later on every crate manifest (ADR-0004). No Rust crates touched in this slice; the rule stands for any task that would.
- DCO: every commit signed off —
git commit -s(AGENTS.md Git workflow). - SEAM GATE (UNCHANGED):
crates/rutster-media/src/loop_driver.rsandcrates/rutster-media/src/rtc_session.rsstay byte-identical — CI pins their blob hashes (744bf314edf7f4925c8bb3bd0f5176dbc88f8113andf47d63b9a2883d37066a93c9daa0e2cf8816bec4respectively, in.github/workflows/ci.ymllines 60–61). NO task in this plan touches them. This slice is artifacts + CI, not the RTP path. Verify unchanged hashes in the final sweep (Task 11 Step 2). - Hot-path policy: never
?-propagate on the 20 ms tick; nounwrap()/expect()outside tests/startup. (No Rust code added; every smoke-test Python script honors it: WS read failuresassertloudly with a counter snapshot — they never silently loop.) - Code style:
cargo fmt --all --check+cargo clippy --all --all-targets -- -D warningsmust stay green. (This slice adds no Rust; CI's clippy job stays the source of truth.) - MSRV: CI matrix runs stable + 1.85 (per
rust-toolchain.toml). The Dockerfile builder stage MUST pin1.85exactly — notstable— so image builds reproduce what the MSRV matrix gates. Nocargo updatewholesale (rcgen 0.14.7 lock rationale per rootCargo.toml). - Workspace dep pinning: new deps go in the ROOT
Cargo.toml[workspace.dependencies]; members reference withdep.workspace = true. (None added here — all artifact deps are apt packages or upstream Docker images, not crates.) - cargo-deny license gate:
cargo deny checkmust stay green. The bundled-binary licenses in the images (Caddy Apache-2.0, valkey BSD-3, s6-overlay ISC, Debian GPL/MIT stack) are aggregation-clean vs GPL-3 — Rust's cargo-deny operates on the Rust crate graph only and is unaffected. The image license scan is a future cargo-deny-adjacent CI muscle (spec §6.1), not this slice's concern; this slice notes the license posture indeploy/.env.example's header comment. - sim-bench CI: the
sim-benchjob runscargo test --all --features=sim-bench -- --test-threads=1—--test-threads=1is load-bearing (shared tick-lag gauge). Unaffected by this slice; the new smoke jobs are independent Gitea Actions jobs, not additionalcargo testinvocations. - Branch/PR: branch
deploy-b/packaging-ci; PR tomainviatea. Slices A landed first (this plan's dependency root); slice C branches after A merges and is order-independent with B. The slice-F publish workflow lives in the same PR (the file is inert until a tag is pushed — no risk of running before artifacts exist).
Slice-B-specific constraints:
- The four image names are pinned to spec §6.1:
rutster-allinone,rutster-engine,rutster-brain,rutster-edge. The CI image-build job asserts each--targetproduces a tagged image with exactly that name. Do not invent variants (rutster-fob,rutster-core, etc.) — the docs tree (plan G) hard-codes these four names. - The publish registry namespace is
git.adlee.work/alee/rutster-*(matches thealeetea login per AGENTS.md Git workflow; the rootCargo.toml'srepository = "https://git.adlee.work/alee/rutster"already commits to this). If your Gitea registry uses a different owner segment, replacealeethroughout the publish workflow +deploy/.env.example's registry-namespace comment. - linux/amd64 only. arm64 is a named deferral (spec §1.2, ADR-0011); the publish workflow
pins
platforms: linux/amd64. - Caddy's
local_certs(internal CA) is the CI TLS path. No ACME in CI (no public DNS, no rate-limit budget, hits Let's Encrypt's duplicate-cert path the instant a job re-runs). Production Caddyfiles default to ACME; CI overrides viaRUTSTER_LOCAL_CERTS=trueenv in the smoke job. Therutster-edgeimage is the same in CI and prod — only the runtime env differs. - The smoke WS client is Python stdlib only. No
pip installstep in CI (Gitea runners ship Python 3 + stdlib;websocket-client/websocketswould require apip installnetwork round-trip per job — a friction the slice-A sim-bench posture avoids). The WS handshake is ten lines of code on top ofsocket+ssl. - The
ValkeyEventSinksmoke assertion belongs to plan C. Leave a# TODO slice-C: assert lifecycle event in Valkey streamnamed hook comment in the compose-smoke step. Do NOT implement the assertion; the Valkey service ships in compose (so the docs + the smoke shape the contract) but no event-lands check runs in this slice. - Image build cache: the
image-buildjob usesdocker/build-push-action@v5withcache-from: type=gha+cache-to: type=gha,mode=max. The cache scope includes the Rust target dir; warm caches take image-build from ~6 min to ~90 s. Documented in the CI job.
Spec ambiguities resolved (load-bearing — flagged so the implementing engineer doesn't
re-derive them)
- "the T8 e2e stub" (referenced in the assignment): there is no
t8*file in the tree. The closest match iscrates/rutster/tests/trunk_sim_e2e.rs— a#[ignore]'d TODO test whose body sketches a full PSTN e2e againstMockTwilioMediaStreamsServer+MediaCmd::RegisterTrunkthrough the binary'sMediaThread. That file's TODO text is exactly the slice-B all-in-one smoke, except this slice drives it against the real containerized edge→FOB WS path through Caddy's internal CA, rather than aMockTwilioMediaStreamsServer. The WS-handshake JSON sequence (connected+startframes) is borrowed verbatim fromcrates/rutster-trunk/tests/ws_ping.rsandcrates/rutster-sim/src/nodelay.rs(both slice-A additions). Thetrunk_sim_e2e.rsfile itself stays untouched — its#[ignore]TODO is a unit-test concern (in-processMediaThreaddriving); the CI smoke is a black-box container concern (whole binary booted, whole Caddy in front, external Python client). They are different test layers. rutster-brainruntime image libopus0: spec §6.1 table listsdebian-slimfor the brain (nolibopus0). Butcrates/rutster-brain-realtime/Cargo.tomldepends onrutster-media, which depends on theopuscrate (system FFI against libopus.so.0). Even though the brain binary never calls an opus symbol, the libopus shared library is recorded in the binary ELF asNEEDED: libopus.so.0because the linker resolves every Rust dep's[dependencies]— a missing.so.0at startup would make the brain crash witherror while loading shared libraries: libopus.so.0. Resolution: installlibopus0in the brain runtime image too (one extraapt-get install -y libopus0, ~500 KB additional). This is a defensive deviation from spec §6.1's brain row; flagged in the Dockerfile inline comment with the linker rationale so a future brain-crate refactor that drops therutster-mediadep can drop this package too.RUTSTER_DOMAIN+RUTSTER_ACME_EMAILenv-var names: plan G'squickstart-docker.mduses these two names (they are artifact-level names owned by slice B's Caddyfile — the spec does not fix them). Adopted as the canonical names; the Caddyfile reads them via env interpolation. The.env.exampledocuments both. Therutster-edgeCaddy binary has zero knowledge ofRUTSTER_*semantics — Caddy reads{$DOMAIN}+{$ACME_EMAIL}from its environment per Caddyfile env-substitution syntax. Wait — the Caddyfile uses{$RUTSTER_DOMAIN}and{$RUTSTER_ACME_EMAIL}to match the rutster convention, even though Caddy is consuming them. This keeps the operator'sRUTSTER_*namespace uniform across both processes.stop_grace_periodexactly 660 s (not 600 s): the spec pins 660 s + drain 600 s (grace-exceeds-drain invariant — spec §8). 60 s is the slack so the orchestrator doesn't cut a drain that's legitimately waiting on a 599-second in-flight call. Pinned exactly; do not "tune."- Caddy reload-during-live-call smoke: the spec §3.2 sentence "verified protocol-unaware
WS tunneling (no frame buffering)" + §9 bullet 4 ("Caddy config reload during a live
simulated call, asserting zero drops") load the heaviest claim in this slice. The smoke
asserts: during a live WS streaming media frames at 20 ms cadence through Caddy to the FOB,
a
caddy reload(the operator's config-edit path) drops zero frames. The mitigation under test isstream_close_delay 24h(Caddyfile) — above max call duration. The upstream bug trail (caddy #6420, #7222) is documented in the TLS brief §5 risk 1 — the smoke is the "don't trust untested" mitigation made CI-regressed.
File Structure
New files
| Path | Responsibility |
|---|---|
deploy/Dockerfile |
Multi-stage: one builder (rust:1.85-slim-bookworm + libopus-dev) producing rutster + rutster-brain-realtime binaries; one xcaddy builder producing caddy with the curated DNS-plugin set; one valkey binary source stage; four runtime targets: rutster-engine, rutster-brain, rutster-edge, rutster-allinone (s6-overlay v3 supervising caddy + FOB + brain + valkey-server). |
deploy/.dockerignore |
Build-context pruning: excludes target/, .git/, node_modules/, *.md (other than README), .env*, crates/*/target/, docs/, .github/, .opencode/. Halves image-build context size. |
deploy/Caddyfile |
~6-line site block + global options: tuned timeouts (read_body 30s, read_header 30s, write 0, idle 24h), honest X-Forwarded-Proto/X-Forwarded-Host (no X-Forwarded-For-trust of client-controlled values — only the honest hop), stream_close_delay 24h (above max call duration; spec §2.1 / TLS brief §3(a)), reverse_proxy to 127.0.0.1:8080 (loopback-only FOB). |
deploy/compose.yaml |
Four services — caddy (image rutster-edge), engine (image rutster-engine), brain (image rutster-brain, network_mode: "service:engine" so the loopback-only tap posture survives), valkey (upstream valkey/valkey). stop_grace_period: 660s. Volumes for caddy /data + valkey /data. |
deploy/.env.example |
All RUTSTER_* vars incl. RUTSTER_DRAIN_DEADLINE_SECS=600, RUTSTER_DOMAIN + RUTSTER_ACME_EMAIL, the four RUTSTER_TWILIO_*, RUTSTER_MEDIA_ADVERTISED_IP, RUTSTER_MEDIA_PORT_RANGE, RUTSTER_WS_PING_SECS, RUTSTER_TRUSTED_PROXIES. Each line commented with the slice that owns it + the failing-fast behavior. |
deploy/s6-rc.d/{caddy,engine,brain,valkey-database}/run + dependencies.d |
s6-overlay v3 service definitions — four run exec scripts + their dependency edges (engine before brain; valkey before engine; caddy before engine, etc.). |
deploy/s6-overlay-noarch.tar.xz + deploy/s6-overlay-x86_64.tar.xz |
Vendored s6-overlay v3.2.0.0 binaries (ISC). Pinned at vendoring time per AGENTS.md reproducibility posture; SHA256 recorded alongside. |
.github/workflows/publish-images.yml |
Slice F: tag-push workflow. Builds + pushes all four images to git.adlee.work/alee/rutster-* on v* tag push. linux/amd64 only. Gated on the existing ci.yml pipeline via workflow_run trigger (publish re-runs the fmt/clippy/test/deny/sim-bench + image-build as a gate, then publishes). |
deploy/smoke/allinone_smoke.py |
Python-stdlib WS client (no pip deps): boots rutster-allinone, extracts Caddy's internal-CA root cert from /data, opens a wss:// WS to /twilio/media-stream, sends Twilio's connected + start handshake frames, streams 200 PCM frames at 20 ms cadence, asserts received frames count within tolerance. No ValkeyEventSink assertion — TODO slice-C hook is in this file as a comment. |
deploy/smoke/compose_smoke.sh |
Bash: docker compose up -d --wait, then curl /healthz against each service, then docker compose ps JSON parse for healthy statuses. The # TODO slice-C: assert lifecycle event in Valkey stream named hook is here. |
deploy/smoke/reload_during_call.py |
Python-stdlib WS client that holds the WS open streaming frames while a concurrent caddy reload fires mid-call; asserts zero frames dropped across the reload window. |
Modified files
| Path | What changes |
|---|---|
.github/workflows/ci.yml |
Add three new jobs: image-build (after existing test + deny + sim-bench), smoke (after image-build), smoke-compose + smoke-reload (after smoke). The existing fmt / clippy / test / deny / sim-bench / twilio-live jobs are unchanged. |
SEAM-INVARIANT files (DO NOT TOUCH)
crates/rutster-media/src/loop_driver.rs— byte-identical, all tasks.crates/rutster-media/src/rtc_session.rs— byte-identical, all tasks.
Interfaces
Consumes (no new code; reuses what slice A + slice-5 built):
rutster::serve::serve_with_nodelay(plan A Task 1) — the production serve path the all-in-one binary runs.rutster_trunk::twilio_media_streams::TwilioMediaStreamsServer::router(register_tx, ping_interval)(plan A Task 2) — the trunk WS route mounted on the main axum listener.rutster::config::twilio_credentials(...)(plan A Task 4) — startup validation of the fourRUTSTER_TWILIO_*vars.rutster::session_map::AppState::with_trunk_webhook_base(_).with_trusted_proxies(_)(plan A Tasks 4+5) — theAppStatebuilder chain.- Plan A's slice env additions:
RUTSTER_WS_PING_SECS(default 20),RUTSTER_TRUSTED_PROXIES(empty default). - Existing config.rs env parsers (slice-5):
RUTSTER_HTTP_BIND,RUTSTER_MEDIA_BIND_IP,RUTSTER_MEDIA_ADVERTISED_IP,RUTSTER_MEDIA_PORT_RANGE,RUTSTER_MAX_SESSIONS,RUTSTER_DRAIN_DEADLINE_SECS, the fourRUTSTER_TWILIO_*. - Existing routes:
/healthz(200 = liveness),/readyz(200 = can-accept-call, 503 = draining / at admission cap),/v1/trunk/webhook(POST → TwiML),/twilio/media-stream(WS upgrade). - The WebRTC static client at
/(slice-1 — booted binary serves it via axum.fallback_service).
Produces:
- Four first-party Docker images, built from one
deploy/Dockerfilevia named--targets:rutster-engine:latest— FOB only (binaryrutster+libopus0runtime).rutster-brain:latest— brain only (binaryrutster-brain-realtime+libopus0runtime — see spec-ambiguity #2 above for the presence of libopus0 in this image).rutster-edge:latest— custom xcaddy build with curated DNS plugins (cloudflare/route53/porkbun/hetzner/desec — duckdns excluded per spec §3.2 + TLS brief §3(a)).rutster-allinone:latest— s6-overlay v3 supervisingcaddy+rutster(FOB) +rutster-brain-realtime+valkey-server(T1 per spec §2.1).
- A composed
docker compose up -dthat runs the T2 reference deployment (spec §2.2) — thecaddyservice isrutster-edge,engineisrutster-engine,brainisrutster-brain(sharing the engine's netns vianetwork_mode: "service:engine"),valkeyis upstreamvalkey/valkey. - Three CI jobs in
.github/workflows/ci.yml:image-build,smoke(all-in-one TLS + reload-during-call),smoke-compose— gated on the existing fmt/clippy/test/deny/sim-bench matrix. - One new workflow
.github/workflows/publish-images.yml(slice F): tag-push publish of all four images togit.adlee.work/alee/rutster-*,linux/amd64only, gated on the image-build job.
Task ordering
- Task 1 —
deploy/Dockerfile(multi-stage, four targets) + vendor s6-overlay tarballs + create s6-rc service definitions. Foundational: all CI smoke jobs build images from this file. No deps on later tasks. - Task 2 —
deploy/.dockerignore. Tiny; lands alongside Task 1 (separate file so the diff is reviewable). - Task 3 —
deploy/Caddyfile. Required by bothrutster-edge(rutster-edge bakes it in via the Dockerfile) andrutster-allinone(s6 Caddy service reads it). Lands before Task 5 (compose) references it. - Task 4 —
deploy/.env.example. Operator documentation; the composeenv_filereference points at it. - Task 5 —
deploy/compose.yaml. Depends on Tasks 1 + 3 (image names + Caddyfile path). - Task 6 —
deploy/smoke/allinone_smoke.py+compose_smoke.sh+reload_during_call.py. These scripts are invoked by the CI jobs (Tasks 7–9) — land them first so the CI YAML can reference stable paths. - Task 7 — Extend
.github/workflows/ci.yml:image-buildjob. Depends on Task 1. - Task 8 — Extend
.github/workflows/ci.yml:smokejob (all-in-one TLS smoke). Depends on Tasks 6 + 7. - Task 9 — Extend
.github/workflows/ci.yml:smoke-compose+smoke-reloadjobs (compose smoke + Caddy-reload-during-call). Depends on Task 8. - Task 10 — Slice F:
.github/workflows/publish-images.yml. Depends on Task 7 (reuses the samedocker buildx buildinvocations + GHA cache config). - Task 11 — Final verification sweep.
Task 1: deploy/Dockerfile — multi-stage, four image targets + s6-overlay + s6-rc
One Dockerfile, four named targets. docker buildx build --target rutster-engine -t rutster-engine:ci . builds just the FOB; --target rutster-allinone builds the s6 bundle.
The same builder stage compiles both Rust binaries (the all-in-one needs both; the
single-purpose images need one each — so the builder builds both, and each runtime stage
COPY --from=builder only the binary it needs).
Files:
- Create:
deploy/Dockerfile - Vendor:
deploy/s6-overlay-noarch.tar.xz,deploy/s6-overlay-x86_64.tar.xz(pulled from upstream s6-overlay v3.2.0.0 release — vendored, not regenerated per build, per AGENTS.md reproducibility posture) - Create:
deploy/s6-overlay.sha256sum(records the SHA256 of both tarballs at vendoring time) - Create:
deploy/s6-rc.d/caddy/run+deploy/s6-rc.d/caddy/dependencies.d/engine - Create:
deploy/s6-rc.d/engine/run+deploy/s6-rc.d/engine/dependencies.d/valkey-database - Create:
deploy/s6-rc.d/brain/run+deploy/s6-rc.d/brain/dependencies.d/engine - Create:
deploy/s6-rc.d/valkey-database/run+deploy/s6-rc.d/valkey-database/dependencies.d/engine - Create:
deploy/s6-rc.d/user/contents.d/{caddy,engine,brain,valkey-database}(empty files) - Test: bash content-grep (Step 5) + optional
docker build --check(Step 3 — local only; CI in Task 7 is the authoritative build).
Interfaces:
-
Consumes:
rust-toolchain.toml(pinned1.85— verified:channel = "1.85"), the workspace rootCargo.toml(workspace member list — verified:crates/rutster+crates/rutster-brain-realtimeare both members),crates/rutster-media/Cargo.toml(confirmsopuscrate hard-deps on libopus FFI — verified, line 12:opus = { workspace = true }), the README's "Quickstart" libopus dev-header note (verified:libopus-dev / opus-devel / brew install opus). Task 3'sdeploy/Caddyfileis COPYed into both edge + all-in-one images — Task 1 lands the Dockerfile that references it; Task 3 lands the Caddyfile content. Both must land before Task 7'simage-buildjob runs in CI. -
Produces: four
--target-named build stages. -
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/Dockerfile \
&& grep -q 'FROM rust:1.85-slim-bookworm' deploy/Dockerfile \
&& grep -q 'AS builder' deploy/Dockerfile \
&& grep -q 'libopus-dev' deploy/Dockerfile \
&& grep -q 'libopus0' deploy/Dockerfile \
&& grep -q 'FROM caddy:2.8-builder' deploy/Dockerfile \
&& grep -q 'xcaddy build' deploy/Dockerfile \
&& grep -q 'FROM valkey/valkey' deploy/Dockerfile \
&& grep -q 's6-overlay' deploy/Dockerfile \
&& grep -q -- 'AS rutster-engine' deploy/Dockerfile \
&& grep -q -- 'AS rutster-brain' deploy/Dockerfile \
&& grep -q -- 'AS rutster-edge' deploy/Dockerfile \
&& grep -q -- 'AS rutster-allinone' deploy/Dockerfile \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL (file does not exist).
- Step 2: Write the complete Dockerfile
Write deploy/Dockerfile with exactly this content:
# syntax=docker/dockerfile:1.7
# deploy/Dockerfile — multi-stage, four first-party images (spec §6.1).
#
# One Dockerfile, four named --target stages:
# * rutster-engine — FOB only (binary `rutster`)
# * rutster-brain — brain only (binary `rutster-brain-realtime`)
# * rutster-edge — custom xcaddy build with curated DNS plugins
# * rutster-allinone — s6-overlay v3 supervising caddy + FOB + brain + valkey-server
#
# Build all four locally:
# docker buildx build --target rutster-engine -t rutster-engine:ci -f deploy/Dockerfile .
# docker buildx build --target rutster-brain -t rutster-brain:ci -f deploy/Dockerfile .
# docker buildx build --target rutster-edge -t rutster-edge:ci -f deploy/Dockerfile .
# docker buildx build --target rutster-allinone -t rutster-allinone:ci -f deploy/Dockerfile .
#
# Toolchain pin: rust-toolchain.toml pins `channel = "1.85"`. The builder
# image is `rust:1.85-slim-bookworm` (NOT `rust:stable` — image builds must
# reproduce exactly what the MSRV matrix in CI gates; `stable` would float).
#
# libopus strategy (spec §6.1 "distro libopus0 over static-bundled"):
# * Builder stage: `apt-get install libopus-dev` — the -dev headers needed
# to compile the `opus` crate (FFI against libopus; verified:
# crates/rutster-media/Cargo.toml line 12: `opus = { workspace = true }`).
# * Runtime stages (engine + brain + all-in-one): `apt-get install libopus0`
# — the shared library binary. Brain image gets libopus0 too (see spec
# ambiguity #2 in this slice's plan): rutster-brain-realtime's
# Cargo.toml depends on rutster-media, so the linker records
# `NEEDED: libopus.so.0` in the brain ELF even though the brain never
# calls an opus symbol. Missing .so.0 would crash the brain at startup.
# * Matches the dev loop exactly. Do NOT use the audiopus_sys bundled-cmake
# fallback — system libopus is available in Debian.
########################################
# Stage 1: Rust builder — compiles both binaries once.
########################################
FROM rust:1.85-slim-bookworm AS builder
# libopus-dev: the headers the `opus` crate's build.rs expects.
RUN apt-get update && apt-get install -y --no-install-recommends \
libopus-dev \
ca-certificates \
curl \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /usr/src/rutster
COPY . .
# Build both binaries in one `cargo build --bins` invocation so the
# dependency graph is compiled once (warm ccache/rustc cache covers both).
# --release is mandatory: debug builds of str0m + opus run 5-10x slower
# than the 20ms tick budget; a debug-image container would burn the tick-lag
# threshold on first call.
RUN cargo build --release --bins \
&& cp /usr/src/rutster/target/release/rutster /usr/local/bin/rutster \
&& cp /usr/src/rutster/target/release/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime
########################################
# Stage 2: Caddy builder — xcaddy with the curated DNS plugin set.
########################################
# caddy:2.8-builder ships Go + xcaddy. The plugin set is exactly spec §3.2
# / TLS brief §3(a): cloudflare, route53, porkbun, hetzner, desec.
# duckdns EXCLUDED — no license file (spec §3.2).
# Plugin licenses: cloudflare=Apache-2.0; route53/porkbun/hetzner/desec=MIT.
# Aggregation-clean vs GPL-3 (Caddy core itself is Apache-2.0).
FROM caddy:2.8-builder AS caddy-builder
RUN xcaddy build \
--with github.com/caddy-dns/cloudflare \
--with github.com/caddy-dns/route53 \
--with github.com/caddy-dns/porkbun \
--with github.com/caddy-dns/hetzner \
--with github.com/caddy-dns/desec \
&& mv /build/caddy /usr/local/bin/caddy
########################################
# Stage 3: Valkey binary source (upstream image, no rebuild).
########################################
# COPY --from pattern: pull just the valkey-server binary out of the
# upstream BSD-3 image; avoids rebuilding valkey from source (saves ~5 min
# per build + ~150 MB of build deps). The binary is statically-linked
# enough for Debian bookworm (verified by upstream's Debian-based image).
FROM valkey/valkey:8.0 AS valkey-src
########################################
# Target: rutster-engine — the FOB only.
########################################
FROM debian:bookworm-slim AS rutster-engine
# libopus0: the shared library (see libopus strategy comment at top).
RUN apt-get update && apt-get install -y --no-install-recommends \
libopus0 \
ca-certificates \
curl \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/bin/rutster /usr/local/bin/rutster
COPY deploy/Caddyfile /etc/rutster/Caddyfile
# RUTSTER_HTTP_BIND defaults to 0.0.0.0:8080 (config.rs). Behind Caddy, the
# engine binds the compose-network address; behind --network host (T1
# solo), it binds 127.0.0.1:8080.
EXPOSE 8080
# Media UDP range — published via RUTSTER_MEDIA_PORT_RANGE.
EXPOSE 49152-49407/udp
ENTRYPOINT ["/usr/local/bin/rutster"]
########################################
# Target: rutster-brain — brain only.
########################################
FROM debian:bookworm-slim AS rutster-brain
# libopus0: see spec ambiguity #2 in this slice's plan —
# rutster-brain-realtime's Cargo.toml depends on rutster-media which
# depends on the `opus` crate; even though the brain never calls an opus
# symbol, the ELF records `NEEDED: libopus.so.0` (linker dead-strips
# unused symbols but the .so is still required at dlopen-time before
# dead-strip can prove they're unused). A future brain-crate refactor
# that drops the rutster-media dep can drop this package too.
RUN apt-get update && apt-get install -y --no-install-recommends \
libopus0 \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/bin/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime
# Brain binds 127.0.0.1:8082 (loopback-only tap posture until step 6).
# In compose (T2), the brain shares the engine's netns via
# `network_mode: "service:engine"` so this loopback IS the engine's
# loopback — the FOB can reach the brain's tap port without exposing it.
EXPOSE 8082
ENTRYPOINT ["/usr/local/bin/rutster-brain-realtime"]
########################################
# Target: rutster-edge — custom Caddy build.
########################################
# scratch/alpine choice (spec §6.1): alpine gives a shell for debugging
# (`docker exec -it rutster-edge sh`); scratch is purer but undebuggable.
# Alpine's musl is fine for the static Go binary xcaddy produces.
FROM alpine:3.20 AS rutster-edge
# ca-certificates: ACME TLS validation. curl: smoke-test `caddy reload`
# invocations + `curl /healthz` from inside the container.
RUN apk add --no-cache ca-certificates curl
COPY --from=caddy-builder /usr/local/bin/caddy /usr/local/bin/caddy
COPY deploy/Caddyfile /etc/caddy/Caddyfile
# Caddy needs /data for ACME state — losing it risks the Let's Encrypt
# duplicate-cert lockout (spec §2.1, TLS brief §5 risk 5). Mount a volume
# at /data in production. In CI (local_certs) /data just holds the internal
# CA — recreating is fine, but the volume mount keeps prod + CI on the
# same code path.
VOLUME /data
EXPOSE 80 443
ENTRYPOINT ["/usr/local/bin/caddy"]
CMD ["run", "--config", "/etc/caddy/Caddyfile", "--adapter", "caddyfile"]
########################################
# Target: rutster-allinone — s6-overlay v3 supervising four processes.
########################################
FROM debian:bookworm-slim AS rutster-allinone
# libopus0 (FOB needs it; brain too — see rutster-brain target above).
# curl: smoke-test invocations + `caddy reload` exec.
# ca-certificates: ACME.
# xz: s6-overlay binary tarball extraction (extract-then-rm).
RUN apt-get update && apt-get install -y --no-install-recommends \
libopus0 \
ca-certificates \
curl \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
# s6-overlay v3.2.0.0 — ISC license (aggregation-clean vs GPL-3).
# Two tarballs: noarch (the service definitions + s6-rc bundle) + x86_64
# (the binaries — linux/amd64 only per spec §1.2). arm64 is a named
# deferral (ADR-0011 §1.2); when it lands, this Dockerfile grows an
# `ARG TARGETARCH` + `COPY` per-arch.
COPY deploy/s6-overlay-noarch.tar.xz /tmp/s6-overlay-noarch.tar.xz
COPY deploy/s6-overlay-x86_64.tar.xz /tmp/s6-overlay-x86_64.tar.xz
RUN tar -C / -Jxpf /tmp/s6-overlay-noarch.tar.xz \
&& tar -C / -Jxpf /tmp/s6-overlay-x86_64.tar.xz \
&& rm /tmp/s6-overlay-noarch.tar.xz /tmp/s6-overlay-x86_64.tar.xz
# Binaries from earlier stages.
COPY --from=builder /usr/local/bin/rutster /usr/local/bin/rutster
COPY --from=builder /usr/local/bin/rutster-brain-realtime /usr/local/bin/rutster-brain-realtime
COPY --from=caddy-builder /usr/local/bin/caddy /usr/local/bin/caddy
COPY --from=valkey-src /usr/local/bin/valkey-server /usr/local/bin/valkey-server
# Caddyfile + s6 service definitions (long-lined below).
COPY deploy/Caddyfile /etc/caddy/Caddyfile
COPY deploy/s6-rc.d/ /etc/s6-overlay/s6-rc.d/
# Volumes: Caddy /data (ACME state — loss risks LE lockout per TLS brief
# §5 risk 5) + Valkey /var/lib/valkey (stream/state persistence per
# ADR-0005).
VOLUME /data /var/lib/valkey
# Caddy :80/:443 public; FOB :8080 behind Caddy; brain :8082 loopback;
# valkey :6379 loopback; media UDP direct.
EXPOSE 80 443 8080 8082 6379 49152-49407/udp
# s6-overlay v3 entrypoint — the binary `s6-overlay-init` (symlinked from
# /s6-init in the noarch tarball) runs the service bundle in
# /etc/s6-overlay/s6-rc.d/ via s6-rc.
ENTRYPOINT ["/s6-init"]
- Step 3: Verify by local Docker build (optional — authoritative build is CI in Task 7)
If Docker is available locally:
cd /home/alee/Sources/rutster
# Buildkit dry-check (parse the Dockerfile without building):
docker buildx build --target rutster-engine -f deploy/Dockerfile --check .
# Full local build + smoke (note: this requires Task 3's Caddyfile + Step 4's
# s6-overlay tarballs + s6-rc.d definitions to be in place first):
docker buildx build --target rutster-engine -t rutster-engine:local -f deploy/Dockerfile .
docker run --rm rutster-engine:local --help 2>&1 | head -5 # binary banner
Expected: --check exits 0 (no syntax errors); the binary banner prints. If Docker is not
available locally, the authoritative build runs in CI (Task 7).
Note on deploy/s6-overlay-*.tar.xz + deploy/s6-rc.d/: the all-in-one target references
two tarball binaries + a service-definition directory. These are fetched artifacts + created
service files — Step 4 below fetches the tarballs into deploy/ and creates the s6-rc.d
directory so the Dockerfile build context is self-contained. The tarballs are vendored (not
regenerated per build) because s6-overlay release cadence is rare + reproducibility favors a
pinned vendored blob.
- Step 4: Vendor the s6-overlay tarballs + create the s6-rc service definitions
cd /home/alee/Sources/rutster/deploy
# Pin v3.2.0.0 (ISC; aggregation-clean vs GPL-3):
curl -fsSLO https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-noarch.tar.xz
curl -fsSLO https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-x86_64.tar.xz
# Record the SHA256 of both tarballs at vendoring time. The operator
# verifies these against the release page on first build; the
# `deploy/s6-overlay.sha256sum` file is committed so a future s6-overlay
# version bump is a loud, reviewable diff.
sha256sum s6-overlay-noarch.tar.xz s6-overlay-x86_64.tar.xz > s6-overlay.sha256sum
Create deploy/s6-rc.d/caddy/run (mark executable: chmod +x deploy/s6-rc.d/caddy/run):
#!/command/execlineb -P
# s6-rc service: caddy — the edge (ACME + TLS termination + WS proxy).
# Depends on: engine (s6-rc starts the engine first per caddy/dependencies.d/engine).
# User: root (caddy needs :80/:443).
foreground { mkdir -p /data }
exec /usr/local/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
Create deploy/s6-rc.d/caddy/dependencies.d/engine (an empty file — s6-rc reads the basename
as the dependency name; the engine must be up before Caddy proxies to it):
# empty file — s6-rc reads the basename as the dependency name.
Create deploy/s6-rc.d/engine/run (mark executable):
#!/command/execlineb -P
# s6-rc service: rutster — the FOB. Runs the rutster binary (axum + media
# thread + trunk WS + /metrics). Reads its env from the container env
# directly (s6-overlay v3 inherits container env via s6-overlay-envdir —
# this slice relies on the container env for simplicity; the per-service
# env files are a future hardening step).
exec /usr/local/bin/rutster
Create deploy/s6-rc.d/engine/dependencies.d/valkey-database:
# empty — engine doesn't WAIT on valkey (it must boot even if valkey is down
# per ADR-0005's fail-safe posture: EventSink counts-and-drops, never blocks).
# The dependency just orders boot for consistent log sequencing.
Create deploy/s6-rc.d/brain/run (mark executable):
#!/command/execlineb -P
# s6-rc service: rutster-brain-realtime. Shares the engine's network
# namespace (T1 solo variant — the all-in-one is a single container so
# loopback IS shared). The brain binds 127.0.0.1:8082 — the FOB reaches it
# via RUTSTER_TAP_URL=ws://127.0.0.1:8082.
exec /usr/local/bin/rutster-brain-realtime
Create deploy/s6-rc.d/brain/dependencies.d/engine:
# empty — brain waits for the engine before attempting the tap WS.
Create deploy/s6-rc.d/valkey-database/run (mark executable):
#!/command/execlineb -P
# s6-rc service: valkey-server — bus + KV + presence (ADR-0005). Dark on
# day one except EventSink consumer; the operator contract (volumes, ports,
# upgrade shape) is stable so the spend ledger + fleet directory land later
# without changing the deployment shape (spec §2.1).
foreground { mkdir -p /var/lib/valkey }
exec /usr/local/bin/valkey-server --dir /var/lib/valkey --save 60 1 --appendonly yes
Create deploy/s6-rc.d/valkey-database/dependencies.d/engine:
# empty — orders valkey after engine in boot sequence.
Create the four empty service-bundle membership files in
deploy/s6-rc.d/user/contents.d/:
cd /home/alee/Sources/rutster/deploy/s6-rc.d/user/contents.d
touch caddy engine brain valkey-database
cd /home/alee/Sources/rutster/deploy
chmod +x s6-rc.d/caddy/run s6-rc.d/engine/run s6-rc.d/brain/run s6-rc.d/valkey-database/run
- Step 5: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/Dockerfile \
&& grep -q 'FROM rust:1.85-slim-bookworm' deploy/Dockerfile \
&& grep -q 'AS builder' deploy/Dockerfile \
&& grep -q 'libopus-dev' deploy/Dockerfile \
&& grep -q 'libopus0' deploy/Dockerfile \
&& grep -q 'FROM caddy:2.8-builder' deploy/Dockerfile \
&& grep -q 'xcaddy build' deploy/Dockerfile \
&& grep -q 'FROM valkey/valkey' deploy/Dockerfile \
&& grep -q 's6-overlay' deploy/Dockerfile \
&& grep -q -- 'AS rutster-engine' deploy/Dockerfile \
&& grep -q -- 'AS rutster-brain' deploy/Dockerfile \
&& grep -q -- 'AS rutster-edge' deploy/Dockerfile \
&& grep -q -- 'AS rutster-allinone' deploy/Dockerfile \
&& test -f deploy/s6-overlay-noarch.tar.xz \
&& test -f deploy/s6-overlay-x86_64.tar.xz \
&& test -x deploy/s6-rc.d/caddy/run \
&& test -x deploy/s6-rc.d/engine/run \
&& test -x deploy/s6-rc.d/brain/run \
&& test -x deploy/s6-rc.d/valkey-database/run \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 6: Commit
cd /home/alee/Sources/rutster
git add deploy/Dockerfile deploy/s6-overlay-noarch.tar.xz deploy/s6-overlay-x86_64.tar.xz deploy/s6-overlay.sha256sum deploy/s6-rc.d/
# Note: deploy/Caddyfile is created in Task 3; if Task 1 is committed without it,
# Step 3's `docker --check` will fail because COPY deploy/Caddyfile references a
# missing file. Two options: (a) commit the Caddyfile in this commit too (Task 3's
# content lands here as a stub + Task 3 just refines it), OR (b) land the Caddyfile
# in Task 3 and verify Task 1's Dockerfile syntax via `docker buildx build --check`
# which does NOT require the COPY source to exist. Option (b) is the executing-plans
# path: Task 1 commits the Dockerfile even though `docker build` would fail until
# Task 3 lands. CI's image-build job (Task 7) runs AFTER both Tasks 1 + 3 are merged.
git commit -s -m "feat(deploy): multi-stage Dockerfile producing four first-party images (deploy-B §6.1)
One Dockerfile, four named --target stages: rutster-engine (FOB),
rutster-brain (brain), rutster-edge (custom xcaddy + curated DNS plugins
cloudflare/route53/porkbun/hetzner/desec, duckdns excluded per spec §3.2),
rutster-allinone (s6-overlay v3 supervising caddy + FOB + brain +
valkey-server).
Builder pins rust:1.85-slim-bookworm (matches rust-toolchain.toml).
libopus-dev in builder, libopus0 in runtime (matches dev — spec §6.1
distro-over-static-bundled). Brain image gets libopus0 too: although the
brain never calls an opus symbol, the linker records NEEDED:libopus.so.0
because rutster-brain-realtime depends on rutster-media (spec ambiguity
resolved in this slice's plan README).
All-in-one vendors s6-overlay v3.2.0.0 (ISC) + uses upstream valkey/valkey
binary (COPY --from pattern, BSD-3). Bundled-binary licenses
aggregation-clean vs GPL-3 (Caddy Apache-2.0). No Rust crates touched;
cargo-deny unaffected.
s6-rc service definitions land in this commit (caddy/engine/brain/
valkey-database run scripts + dependency edges). The Caddyfile lands in
Task 3 (same PR); the image-build CI job (Task 7) runs after both."
Task 2: deploy/.dockerignore — build-context pruning
Without .dockerignore, the Docker build context ships target/ (~5 GB on a warm dev
machine), .git/ (history bloat), docs/, etc. — docker buildx build then spends minutes
transferring context before the first RUN runs. This file halves context size + saves
CI minutes per build.
Files:
- Create:
deploy/.dockerignore
Interfaces: none — file content is a list of globs.
- Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/.dockerignore && echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Write the complete file
Write deploy/.dockerignore with exactly this content:
# deploy/.dockerignore — prune the build context for `docker buildx build`
# invocations against deploy/Dockerfile. Without this, the context would
# ship target/ (~5 GB on warm dev), .git/, docs/, .github/, etc. — minutes
# of pointless context-transfer before the first RUN step.
#
# IMPORTANT: patterns are relative to the build context root (the path
# passed to `docker buildx build`, which is the repo root for this Dockerfile
# since we use `-f deploy/Dockerfile .`). So `target/` matches
# /target/ at the repo root AND `**/target/` catches /crates/*/target/.
# Rust build artifacts.
target/
**/target/
# Git history + CI configuration.
.git/
.github/
# Editor + agent working state.
.vscode/
.idea/
.opencode/
*.swp
*.swo
.DS_Store
# Environment files (never ship secrets — .env.example is the ONLY .env*
# file that should reach the build context, and it has no secrets by design).
.env
.env.local
.env.*.local
# Documentation (the Dockerfile doesn't read any .md file but README).
docs/
*.md
!README.md
# Plans + specs (not needed at image build time — operator reads from the
# repo, not the container).
docs/superpowers/
# Logs + OS junk.
*.log
tmp/
*.tmp
# Tmp artifacts from cargo-deny / cargo-cache.
deny.toml.lock
- Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/.dockerignore \
&& grep -q '^target/' deploy/.dockerignore \
&& grep -q '^.git/' deploy/.dockerignore \
&& grep -q '^\*\*/target/' deploy/.dockerignore \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/.dockerignore
git commit -s -m "chore(deploy): .dockerignore — prune build context (deploy-B §6.1)
Halves the build-context size: excludes target/ + **/target/ (5 GB on a
warm dev machine, ~150 MB cache hit on CI), .git/, docs/, .github/,
.env* (never ship secrets — .env.example is the only .env* the context
ships and it has no secrets by design)."
Task 3: deploy/Caddyfile — tuned timeouts, honest X-Forwarded, stream_close_delay 24h
The Caddyfile is read by both rutster-edge (T2 modular default edge) and rutster-allinone
(T1 bundled edge). Same file, same build — spec §3.2 mandates "same build, same Caddyfile" so
operator knowledge transfers between T1 and T2.
Files:
- Create:
deploy/Caddyfile(already referenced by Task 1's Dockerfile COPY steps; lands here as its own atomic commit so the diff is reviewable + the Caddyfile can be edited without re-touching the Dockerfile).
Interfaces:
-
Consumes: env vars
RUTSTER_DOMAIN(the public hostname —pbx.example.com),RUTSTER_ACME_EMAIL(the LE account email),RUTSTER_LOCAL_CERTS(CI override —"true"makes Caddy use its internal CA, no ACME). -
Produces: the
:443TLS listener, the:80ACME HTTP-01 fallback, the reverse_proxy to127.0.0.1:8080, the WS upgrade pass-through (Caddy auto-detectsUpgrade: websocket— no per-route config needed), theX-Forwarded-Proto/X-Forwarded-Hosthonesty. -
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/Caddyfile \
&& grep -q 'stream_close_delay' deploy/Caddyfile \
&& grep -q 'X-Forwarded-Proto' deploy/Caddyfile \
&& grep -q 'reverse_proxy' deploy/Caddyfile \
&& grep -q '24h' deploy/Caddyfile \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL (file does not exist).
- Step 2: Write the complete Caddyfile
Write deploy/Caddyfile with exactly this content:
# deploy/Caddyfile — the edge config for rutster-edge (T2) + rutster-allinone (T1).
# Same file, same build (spec §3.2 + ADR-0011). Read by Caddy v2.8 (the
# custom xcaddy build from deploy/Dockerfile).
#
# Env vars (Caddyfile interpolation syntax {$VAR}):
# RUTSTER_DOMAIN — the public hostname (e.g. pbx.example.com). Required.
# RUTSTER_ACME_EMAIL — the Let's Encrypt account email. Required for ACME.
# RUTSTER_LOCAL_CERTS — set to "true" in CI to use Caddy's internal CA
# (no ACME round-trips, no rate-limit budget burns).
# Overrides the ACME block below via the `local_certs`
# global option.
#
# Timeouts (TLS brief §3(a); spec §2.1, §2.2, §3.1 invariant 3):
# * read_body 30s — Twilio webhooks fit comfortably in 30s (15s cap
# by invariant 6).
# * read_header 30s — same.
# * write 0 — disable; 20ms media frames must flush immediately.
# * idle 24h — the universal 60s-class default kills quiet WS;
# spec §2.1 invariant 3 sets max call duration 24h.
# * stream_close_delay 24h — above max call duration; the load-bearing
# mitigation for caddy #6420/#7222 (TLS brief §5
# risk 1; spec §9 reload-during-call smoke).
#
# X-Forwarded-* honesty (spec §3.1 invariant 5; plan-A Task 5
# reconstruct_public_url): Caddy sets these to the real client values via
# the `header_up` directives below. Engine-side RUTSTER_TRUSTED_PROXIES is
# the trust gate (fail-closed default).
{
# CI override: set RUTSTER_LOCAL_CERTS=true to use Caddy's internal CA
# (no ACME in CI; spec §9). Production leaves this unset.
{$RUTSTER_LOCAL_CERTS:local_certs_off} local_certs
# Send logs to stdout (s6-overlay / docker logs collects from there).
log {
output stdout
format console
}
}
# Default ACME email — overridden by RUTSTER_LOCAL_CERTS=true in CI.
{
email {$RUTSTER_ACME_EMAIL:you@example.com}
}
# Main site block — the public hostname.
{$RUTSTER_DOMAIN:localhost} {
encode zstd gzip
# The reverse_proxy to the FOB. 127.0.0.1:8080 is correct in T1
# (single container, loopback) AND T2 (compose: the engine service's
# `rutster-engine` DNS name resolves on the compose network; Caddy
# reaches it via that name — replace 127.0.0.1:8080 with
# http://engine:8080 in advance; here we use the loopback literal so
# the same Caddyfile works in both T1 and T2 spelled identically —
# for T2 deployments, the operator overrides the upstream via a env-
# substituted site-address block; the shipped default targets loopback
# for the bundled all-in-one single-container case which is the harder
# constraint).
#
# `header_up Host {host}` + X-Forwarded-* — sets the honest hop values
# (the engine's reconstruct_public_url reads these; spec §3.1 invariant 5).
# `header_up X-Forwarded-For {remote_host}` — included for completeness;
# the engine does NOT use X-Forwarded-For for signature validation, only
# X-Forwarded-Proto/Host. Forwarding the For header is standard proxy
# hygiene — engine-side RUTSTER_TRUSTED_PROXIES is the gate.
reverse_proxy 127.0.0.1:8080 {
header_up Host {host}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-Proto {scheme}
header_up X-Forwarded-Host {host}
header_up X-Forwarded-For {remote_host}
# Tuned timeouts — the load-bearing ones for calls lasting hours.
transport http {
read_buffer 64KiB
write_buffer 64KiB
dial_timeout 5s
response_header_timeout 30s
}
# stream_close_delay above max call duration (24h) — the caddy
# #6420/#7222 mitigation (TLS brief §5 risk 1; spec §9 smoke).
# An operator Caddyfile reload mid-call would normally kill
# upgraded WS tunnels; the delay keeps them alive until the
# streams close naturally (the call ends) — verified by the
# reload-during-live-call CI smoke (Task 9).
stream_close_delay 24h
}
# Timeouts for the public-facing side.
servers {
read_body 30s
read_header 30s
write 0 # never write-timeout; 20ms frames self-flush
idle 24h # above max call duration — the universal 60s-class default kills quiet WS
}
}
- Step 3: Re-run the content-check + Caddy config-validate
cd /home/alee/Sources/rutster
test -f deploy/Caddyfile \
&& grep -q 'stream_close_delay' deploy/Caddyfile \
&& grep -q 'X-Forwarded-Proto' deploy/Caddyfile \
&& grep -q 'reverse_proxy' deploy/Caddyfile \
&& grep -q '24h' deploy/Caddyfile \
&& grep -q 'idle 24h' deploy/Caddyfile \
&& grep -q 'RUTSTER_DOMAIN' deploy/Caddyfile \
&& grep -q 'RUTSTER_LOCAL_CERTS' deploy/Caddyfile \
&& echo CHECK-PASS || echo CHECK-FAIL
# Optional local Caddy validate (if `caddy` binary available — CI does
# this inside the rutster-edge container in Task 7):
if command -v caddy >/dev/null 2>&1; then
RUTSTER_DOMAIN=pbx.example.com RUTSTER_ACME_EMAIL=test@example.com \
caddy validate --config deploy/Caddyfile --adapter caddyfile
else
echo "(caddy not installed locally; CI validates in Task 7)"
fi
Expected: CHECK-PASS. If Caddy is installed locally, the validate command exits 0.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/Caddyfile
git commit -s -m "feat(deploy): Caddyfile — tuned timeouts, honest X-Forwarded, stream_close_delay 24h (deploy-B §3.2)
The shared edge config for rutster-edge (T2 modular) and rutster-allinone
(T1 solo). ~6-line site block + global options per spec §3.2 / TLS brief
§3(a):
* read_body 30s / read_header 30s — Twilio webhook budget (15s cap by
invariant 6 + slack).
* write 0 — 20ms media frames self-flush; never write-timeout.
* idle 24h / stream_close_delay 24h — above max call duration (invariant
3); the universal 60s-class default kills quiet mid-call WS.
* reverse_proxy + header_up — Caddy sets honest X-Forwarded-Proto/Host/For
for the engine's reconstruct_public_url (plan-A Task 5); engine-side
RUTSTER_TRUSTED_PROXIES is the trust gate (fail-closed default).
CI override via RUTSTER_LOCAL_CERTS=true uses Caddy's internal CA (no
ACME in CI per spec §9). The reload-during-live-call smoke (Task 9)
regresses the stream_close_delay mitigation against caddy #6420/#7222."
Task 4: deploy/.env.example — all RUTSTER_* vars documented
The .env.example is the operator documentation surface (plan G's quickstart-docker.md
refers operators here). Every RUTSTER_* var is listed with its default, the slice that
introduced it, and the fail-fast behavior.
Files:
- Create:
deploy/.env.example
Interfaces: consumes the env-var surface from crates/rutster/src/config.rs (verified:
http_bind, media_address_config, max_sessions, drain_deadline, twilio_credentials +
plan A's ws_ping_interval, trusted_proxies) + the artifact-level vars RUTSTER_DOMAIN +
RUTSTER_ACME_EMAIL + RUTSTER_LOCAL_CERTS (this slice — Caddyfile-level). Plan C's
RUTSTER_TLS_CERT/RUTSTER_TLS_KEY are noted as "land with slice C" — see plan C, NOT here.
- Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/.env.example \
&& grep -q 'RUTSTER_DRAIN_DEADLINE_SECS=600' deploy/.env.example \
&& grep -q 'RUTSTER_DOMAIN=' deploy/.env.example \
&& grep -q 'RUTSTER_ACME_EMAIL=' deploy/.env.example \
&& grep -q 'RUTSTER_TWILIO_ACCOUNT_SID=' deploy/.env.example \
&& grep -q 'RUTSTER_TRUSTED_PROXIES=' deploy/.env.example \
&& grep -q 'RUTSTER_WS_PING_SECS=' deploy/.env.example \
&& grep -q 'RUTSTER_MEDIA_ADVERTISED_IP=' deploy/.env.example \
&& grep -q 'RUTSTER_LOCAL_CERTS' deploy/.env.example \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Write the complete file
Write deploy/.env.example with exactly this content:
# deploy/.env.example — runtime configuration for the rutster T1 (all-in-one)
# + T2 (compose) artifacts. Copy to `.env` + edit before `docker compose up -d`.
#
# Format: KEY=value (one per line); comments (#) are ignored by `docker
# compose --env-file`. Fail-fast pattern: every var is parsed by
# crates/rutster/src/config.rs's pure-`Option<String>` parsers at startup;
# an invalid value fails the boot loudly with the var name in the error,
# never silently runs with a wrong default. See AGENTS.md "config.rs"
# pattern.
#
# Bundled-binary licenses (aggregation-clean vs GPL-3 per ADR-0004):
# * Caddy Apache-2.0
# * valkey BSD-3
# * s6-overlay ISC
# * libopus0 BSD-3 (system package; Debian)
# Future cargo-deny-adjacent image license scan will assert these in CI
# (spec §6.1 — out of scope for slice B; this comment seeds it).
#
# Registry namespace: git.adlee.work/alee/rutster-* (matches the `alee` tea
# login per AGENTS.md Git workflow + the root Cargo.toml's `repository`
# field). If your Gitea registry uses a different owner segment, replace
# `alee` throughout deploy/compose.yaml + .github/workflows/publish-images.yml.
################################################################################
# ARTIFACT-LEVEL — Caddy edge (this slice)
################################################################################
# RUTSTER_DOMAIN — the public hostname the CPaaS dials. Required for T1 (solo
# all-in-one) and T2 (compose). Used by Caddyfile as the site block name.
# Example: pbx.example.com
RUTSTER_DOMAIN=pbx.example.com
# RUTSTER_ACME_EMAIL — the Let's Encrypt account email. Used by Caddy for
# ACME account creation. Required for production (ACME issuance); ignored
# in CI (RUTSTER_LOCAL_CERTS=true bypasses ACME).
RUTSTER_ACME_EMAIL=you@example.com
# RUTSTER_LOCAL_CERTS — set to "true" in CI to use Caddy's internal CA (no
# ACME round-trips, no rate-limit budget burns). Production leaves this
# unset (or "false") so Caddy uses Let's Encrypt. The Caddyfile reads this
# via the {$RUTSTER_LOCAL_CERTS:local_certs_off} interpolation.
# RUTSTER_LOCAL_CERTS=false
################################################################################
# FOB engine — bind + drain (slice-5 config.rs + plan-A)
################################################################################
# RUTSTER_HTTP_BIND — the FOB's HTTP/WS listener. Behind Caddy (T1/T2 prod):
# 0.0.0.0:8080 (compose network) or 127.0.0.1:8080 (single container T1).
# Default if unset: 0.0.0.0:8080 (config.rs http_bind).
RUTSTER_HTTP_BIND=0.0.0.0:8080
# RUTSTER_DRAIN_DEADLINE_SECS — how long shutdown waits for in-flight calls
# before the hard stop. Default 0 = instant shutdown (dev loop); production
# sets minutes. Spec §2.2 pins 600 (paired with compose stop_grace_period
# 660s — grace MUST exceed drain or the orchestrator kills calls the
# engine was gracefully draining).
RUTSTER_DRAIN_DEADLINE_SECS=600
# RUTSTER_MAX_SESSIONS — admission cap. Default 64 (placeholder until the
# ADR-0010 benchmark measures the real per-node ceiling). /readyz flips 503
# when at the cap so the LB pulls the node.
RUTSTER_MAX_SESSIONS=64
################################################################################
# FOB engine — WebRTC media (slice-5 config.rs media_address_config)
################################################################################
# RUTSTER_MEDIA_BIND_IP — the IP WebRTC DTLS-SRTP binds. Default 127.0.0.1
# (loopback — dev loop). Production hosts a public IP.
RUTSTER_MEDIA_BIND_IP=0.0.0.0
# RUTSTER_MEDIA_ADVERTISED_IP — the IP advertised in SDP to WebRTC callers.
# They send media UDP straight to this IP (NAT 1:1 — no STUN, no TURN).
# In T1 (host networking): the host's public IP. In T2 (compose): the
# engine container's published port's host IP.
RUTSTER_MEDIA_ADVERTISED_IP=203.0.113.7
# RUTSTER_MEDIA_PORT_RANGE — UDP port range for WebRTC media. Format lo-hi
# inclusive. Must be open inbound on the host's firewall + published in
# compose (deploy/compose.yaml publishes this range).
RUTSTER_MEDIA_PORT_RANGE=49152-49407
################################################################################
# FOB engine — WS hygiene (plan-A §5.2)
################################################################################
# RUTSTER_WS_PING_SECS — engine-originated app-level WS ping interval on
# the trunk media-stream WS. Default 20s. 0 rejected (would busy-loop the
# ping timer); no "off" spelling by design.
RUTSTER_WS_PING_SECS=20
################################################################################
# FOB engine — trusted-proxy posture (plan-A §5.3)
################################################################################
# RUTSTER_TRUSTED_PROXIES — comma-separated CIDR list of edge peers whose
# X-Forwarded-Proto/Host are believed. Empty or unset = headers IGNORED
# (fail-closed default). Bare IPs accepted as host-length prefixes. In T1/T2
# with Caddy as edge, set to the Caddy container's source IP (compose:
# the compose network's subnet CIDR; T1: 127.0.0.1/32). An internet peer
# must never choose the public URL that Twilio signature validation
# reconstructs (spec §3.1 invariant 5).
RUTSTER_TRUSTED_PROXIES=127.0.0.1/32
################################################################################
# Trunk — Twilio credentials (slice-5 config.rs twilio_credentials)
################################################################################
# All four RUTSTER_TWILIO_* vars are ALL-OR-NONE: the parser rejects partial
# config at startup. Omit all four to run WebRTC-only.
RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token_here
RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081
RUTSTER_TWILIO_WEBHOOK_BASE=https://pbx.example.com
################################################################################
# Brain — tap URL (slice-2/3; loopback-only until step 6)
################################################################################
# RUTSTER_TAP_URL — the WS URL the FOB opens to the brain. Default
# ws://127.0.0.1:8081/echo (slice-2 dev-loop echo brain). Production with
# rutster-brain-realtime: ws://127.0.0.1:8082 (loopback-only —
# resolve_tap_url rejects non-loopback until step 6's wss:// tap). In T2
# (compose), the brain shares the engine's netns
# (network_mode: "service:engine") so this loopback IS the engine's
# loopback.
RUTSTER_TAP_URL=ws://127.0.0.1:8082
################################################################################
# Future vars — land with sibling slices, NOT this slice
################################################################################
# RUTSTER_TLS_CERT / RUTSTER_TLS_KEY — land with slice C (rustls Phase 1,
# BYO-cert in-process TLS). The engine binary stays compiled-with-rustls
# from slice C onward; runtime-gated by these vars. See the slice-C plan
# at docs/superpowers/plans/2026-07-05-deploy-c-rustls-tls.md (path may
# vary — check the plans dir).
# RUTSTER_TLS_CERT=/etc/rutster/tls/fullchain.pem
# RUTSTER_TLS_KEY=/etc/rutster/tls/privkey.pem
# RUTSTER_METRICS_BIND — lands with slice D (/metrics endpoint). Default
# 127.0.0.1:9090 (internal — never routed through Caddy).
# RUTSTER_METRICS_BIND=127.0.0.1:9090
# RUTSTER_VALKEY_URL — lands with slice E (ValkeyEventSink). Unset = today's
# TracingEventSink (logs to stdout). Set to redis://valkey:6379/0 in T2.
# RUTSTER_VALKEY_URL=redis://valkey:6379/0
- Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
test -f deploy/.env.example \
&& grep -q 'RUTSTER_DRAIN_DEADLINE_SECS=600' deploy/.env.example \
&& grep -q 'RUTSTER_DOMAIN=' deploy/.env.example \
&& grep -q 'RUTSTER_ACME_EMAIL=' deploy/.env.example \
&& grep -q 'RUTSTER_TWILIO_ACCOUNT_SID=' deploy/.env.example \
&& grep -q 'RUTSTER_TRUSTED_PROXIES=' deploy/.env.example \
&& grep -q 'RUTSTER_WS_PING_SECS=' deploy/.env.example \
&& grep -q 'RUTSTER_MEDIA_ADVERTISED_IP=' deploy/.env.example \
&& grep -q 'RUTSTER_LOCAL_CERTS' deploy/.env.example \
&& grep -q 'RUTSTER_TLS_CERT' deploy/.env.example \
&& grep -q 'RUTSTER_VALKEY_URL' deploy/.env.example \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/.env.example
git commit -s -m "docs(deploy): .env.example — all RUTSTER_* vars documented (deploy-B §6.2)
The operator documentation surface. Lists every RUTSTER_* var with its
default + the slice that introduced it + the fail-fast behavior. Includes:
* Artifact-level (this slice): RUTSTER_DOMAIN, RUTSTER_ACME_EMAIL,
RUTSTER_LOCAL_CERTS (CI override).
* FOB engine (slice-5 + plan-A): RUTSTER_HTTP_BIND,
RUTSTER_DRAIN_DEADLINE_SECS=600 (paired with compose stop_grace_period
660s per spec §2.2 — grace exceeds drain), RUTSTER_MAX_SESSIONS,
RUTSTER_MEDIA_BIND_IP/ADVERTISED_IP/PORT_RANGE, RUTSTER_WS_PING_SECS,
RUTSTER_TRUSTED_PROXIES.
* Trunk (slice-5): the four RUTSTER_TWILIO_* (all-or-none — config.rs
twilio_credentials rejects partial).
* Brain (slice-2/3): RUTSTER_TAP_URL (loopback-only until step 6).
* Future var stubs: RUTSTER_TLS_CERT/KEY (slice C), RUTSTER_METRICS_BIND
(slice D), RUTSTER_VALKEY_URL (slice E) — commented out with a pointer
to the sibling plan.
Bundled-binary licenses noted in the header comment (Caddy Apache-2.0,
valkey BSD-3, s6-overlay ISC, libopus0 BSD-3 — aggregation-clean vs
GPL-3 per ADR-0004); seeds the future cargo-deny-adjacent image license
scan (spec §6.1, out of scope for this slice).
Registry namespace git.adlee.work/alee/rutster-* + the 'if your Gitea
registry uses a different owner segment, replace alee throughout' note."
Task 5: deploy/compose.yaml — T2 reference deployment
The compose file is the T2 reference deployment (spec §2.2). Four services, stop_grace_period: 660s (paired with RUTSTER_DRAIN_DEADLINE_SECS=600), brain shares the engine's netns so the
loopback tap posture holds, two named volumes for Caddy /data + valkey persistence.
Files:
- Create:
deploy/compose.yaml
Interfaces:
-
Consumes: image names from spec §6.1 (
rutster-edge,rutster-engine,rutster-brain, upstreamvalkey/valkey), env fromdeploy/.env.example(Task 4), Caddyfile from Task 3. -
Produces:
docker compose up -dboots a working T2 stack. -
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/compose.yaml \
&& grep -q 'services:' deploy/compose.yaml \
&& grep -q 'image: rutster-edge' deploy/compose.yaml \
&& grep -q 'image: rutster-engine' deploy/compose.yaml \
&& grep -q 'image: rutster-brain' deploy/compose.yaml \
&& grep -q 'image: valkey/valkey' deploy/compose.yaml \
&& grep -q 'network_mode: "service:engine"' deploy/compose.yaml \
&& grep -q 'stop_grace_period: 660s' deploy/compose.yaml \
&& grep -q 'volumes:' deploy/compose.yaml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Write the complete file
Write deploy/compose.yaml with exactly this content:
# deploy/compose.yaml — T2 modular reference deployment (spec §2.2 / ADR-0011).
#
# Four services: caddy (rutster-edge), engine (rutster-engine), brain
# (rutster-brain, network_mode: "service:engine" so the loopback tap posture
# survives unchanged — resolve_tap_url rejects non-loopback until step 6),
# valkey (upstream valkey/valkey).
#
# stop_grace_period 660s paired with RUTSTER_DRAIN_DEADLINE_SECS=600 —
# grace EXCEEDS drain or the orchestrator kills calls the engine was
# gracefully draining (spec §2.2 / §8). 60s slack accounts for a
# legitimately-long in-flight call (599s) + shutdown round-trip.
#
# Bring your own proxy: comment out the `caddy` service; the engine keeps
# its plaintext :8080 listener; tuned-timeout configs for nginx/HAProxy/
# Traefik ship in docs/deploy/reverse-proxies.md (slice G).
#
# Bring-up:
# cp .env.example .env
# $EDITOR .env # set DOMAIN, ACME_EMAIL, MEDIA_ADVERTISED_IP, TWILIO_*
# docker compose up -d
# curl -fsS https://$RUTSTER_DOMAIN/healthz && echo OK
services:
caddy:
# The custom xcaddy build with curated DNS plugins (cloudflare/
# route53/porkbun/hetzner/desec — duckdns excluded per spec §3.2).
# In production, pulls from the git.adlee.work/alee/rutster-edge
# registry (slice F). For local dev / CI without registry auth, build
# from the Dockerfile:
# build:
# context: ..
# dockerfile: deploy/Dockerfile
# target: rutster-edge
image: rutster-edge:latest
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- caddy-data:/data
- ./Caddyfile:/etc/caddy/Caddyfile:ro
env_file:
- .env
depends_on:
- engine
networks:
- rutster-net
engine:
# The FOB: axum signaling + media thread + trunk WS + /metrics.
# Plaintext :8080 behind Caddy (slice C's rustls will terminate
# in-process TLS — same listener, operator-choice).
image: rutster-engine:latest
restart: unless-stopped
# 660s grace > 600s drain (RUTSTER_DRAIN_DEADLINE_SECS) — invariant.
stop_grace_period: 660s
ports:
# Signaling + trunk WS behind Caddy (Caddy proxies :443 -> engine:8080).
# Comment out for production hardening (Caddy is the only entry).
- "8080:8080"
# WebRTC media UDP direct — never proxied (spec §2.1).
- "49152-49407:49152-49407/udp"
volumes:
- ./Caddyfile:/etc/rutster/Caddyfile:ro # copied in image already; mount for edit-without-rebuild
env_file:
- .env
networks:
- rutster-net
brain:
# Brain shares engine's network namespace — the FOB reaches the brain's
# tap port via 127.0.0.1:8082 (loopback-only posture; resolve_tap_url
# rejects non-loopback URLs until step 6 per spec §2.2). The brain
# graduates to its own netns + wss:// tap auth with step 6.
image: rutster-brain:latest
restart: unless-stopped
# CRITICAL: quoted — YAML would otherwise parse `service:engine` as a
# `service: engine` mapping (key:value). The quotes make it a string.
network_mode: "service:engine"
env_file:
- .env
depends_on:
- engine
valkey:
# Upstream valkey image (BSD-3; aggregation-clean vs GPL-3 per ADR-0004).
# Dark on day one except EventSink (slice E lands ValkeyEventSink
# against this service). The operator contract (volumes, ports, upgrade
# shape) is stable from v1 per spec §2.1 — the spend ledger (ADR-0009)
# + fleet directory land later without changing this service.
image: valkey/valkey:8.0
restart: unless-stopped
command: ["valkey-server", "--dir", "/data", "--save", "60", "1", "--appendonly", "yes"]
volumes:
- valkey-data:/data
networks:
- rutster-net
volumes:
# /data — Caddy's cert/ACME state. Loss risks the Let's Encrypt
# duplicate-cert lockout (5/week — total inbound outage class).
# See docs/deploy/certificates.md (slice G) + TLS brief §5 risk 5.
caddy-data:
# /var/lib/valkey — stream/state persistence (ADR-0005).
valkey-data:
networks:
rutster-net:
driver: bridge
- Step 3: Re-run the content-check +
docker compose configvalidate
cd /home/alee/Sources/rutster
test -f deploy/compose.yaml \
&& grep -q 'services:' deploy/compose.yaml \
&& grep -q 'image: rutster-edge' deploy/compose.yaml \
&& grep -q 'image: rutster-engine' deploy/compose.yaml \
&& grep -q 'image: rutster-brain' deploy/compose.yaml \
&& grep -q 'image: valkey/valkey' deploy/compose.yaml \
&& grep -q 'network_mode: "service:engine"' deploy/compose.yaml \
&& grep -q 'stop_grace_period: 660s' deploy/compose.yaml \
&& grep -q 'caddy-data:' deploy/compose.yaml \
&& grep -q 'valkey-data:' deploy/compose.yaml \
&& echo CHECK-PASS || echo CHECK-FAIL
# YAML syntax validation — `docker compose config` parses + renders the
# resolved config (with .env vars interpolated). Requires Docker locally;
# CI validates in the compose-smoke job (Task 9).
if command -v docker >/dev/null 2>&1; then
cd /home/alee/Sources/rutster/deploy
cp .env.example .env 2>/dev/null || cp -n .env.example .env 2>/dev/null || true
docker compose --env-file .env config -q
rm -f .env # don't commit the .env (it has placeholder secrets from .env.example)
else
echo "(docker not installed locally; CI validates in Task 9)"
fi
Expected: CHECK-PASS. If docker compose is available, config -q exits 0.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add deploy/compose.yaml
git commit -s -m "feat(deploy): compose.yaml — T2 reference deployment (deploy-B §2.2)
Four services per spec §2.2:
* caddy — image rutster-edge; ports 80/443; volume caddy-data:/data
(Caddy ACME state — loss risks LE 5/week duplicate-cert lockout
per TLS brief §5 risk 5; mounts ./Caddyfile for edit-without-
rebuild).
* engine — image rutster-engine; ports 8080 + 49152-49407/udp (media
UDP direct, never proxied — spec §2.1); stop_grace_period 660s
paired with RUTSTER_DRAIN_DEADLINE_SECS=600 (grace exceeds drain
invariant, spec §2.2/§8).
* brain — image rutster-brain; network_mode: \"service:engine\" so the
loopback-only tap posture survives unchanged (resolve_tap_url
rejects non-loopback until step 6 per spec §2.2). Quoted in
YAML — \`service:engine\` would parse as a mapping otherwise.
* valkey — upstream valkey/valkey:8.0 (BSD-3); volume valkey-data:/data;
dark on day one except slice-E ValkeyEventSink consumer.
Bring-your-own-proxy: comment out the caddy service; the engine keeps
plaintext :8080; tuned-timeout snippets land in slice-G reverse-proxies.md."
Task 6: deploy/smoke/*.py + deploy/smoke/*.sh — CI smoke harness
The smoke scripts are invoked by the Gitea Actions smoke job (Tasks 7–9). Python-stdlib only
(no pip install) so each CI job is self-contained. Bash for the orchestration glue.
Files:
- Create:
deploy/smoke/allinone_smoke.py— wss:// WS client driving a sim call through the real edge→FOB path insiderutster-allinone. - Create:
deploy/smoke/compose_smoke.sh—docker compose up -d --wait, health probes, service-status JSON parse. The# TODO slice-C: assert lifecycle event in Valkey streamnamed hook lives here. - Create:
deploy/smoke/reload_during_call.py— holds a WS open streaming frames while a concurrentcaddy reloadfires; asserts zero frame drops.
Interfaces:
-
Consumes: Task 1's
rutster-allinoneimage (built by Task 7's CI image-build job), Task 3's Caddyfile (baked into the all-in-one image), Caddy's internal-CA root cert at/data/caddy/pki/authorities/local/root.crt(Caddy'slocal_certsglobal option writes it). -
Produces: Python scripts returning exit code 0 on success, non-zero on failure (CI fails the job on non-zero).
-
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f deploy/smoke/allinone_smoke.py \
&& test -f deploy/smoke/compose_smoke.sh \
&& test -f deploy/smoke/reload_during_call.py \
&& grep -q 'local_certs' deploy/smoke/allinone_smoke.py \
&& grep -q 'TODO slice-C' deploy/smoke/compose_smoke.sh \
&& grep -q 'caddy reload' deploy/smoke/reload_during_call.py \
&& grep -q 'stream_close_delay' deploy/smoke/reload_during_call.py \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Write
deploy/smoke/allinone_smoke.py
#!/usr/bin/env python3
"""deploy/smoke/allinone_smoke.py — T1 all-in-one TLS smoke (deploy-B §9).
Boots `rutster-allinone` with RUTSTER_LOCAL_CERTS=true (Caddy internal CA —
no ACME in CI per spec §9), boots, extracts Caddy's internal-CA root cert
from /data, opens a wss:// WS to /twilio/media-stream, sends Twilio's
connected + start handshake frames, streams 200 PCM-zeroed frames at 20ms
cadence, and asserts the engine replies with at least one text frame
through the real edge->FOB WS path end-to-end through TLS.
Python stdlib ONLY — no pip install required (the CI runner image ships
Python 3 + ssl + socket + json + time). The WS handshake is hand-rolled
for the same reason (websocket-client/websockets would require network
during CI).
Exit code 0 = smoke passed. Non-zero = failed (CI fails the smoke job).
Reusable from: crates/rutster/tests/trunk_sim_e2e.rs (the #[ignore]'d TODO
body sketch — this script implements the same handshake pattern against a
real containerized binary rather than an in-process
MockTwilioMediaStreamsServer) + crates/rutster-trunk/tests/ws_ping.rs (the
connected/start frame sequence — same JSON shapes verbatim).
"""
import json
import os
import socket
import ssl
import struct
import subprocess
import sys
import tempfile
import time
from base64 import b64encode
IMAGE = os.environ.get("RUTSTER_ALLINONE_IMAGE", "rutster-allinone:smoke")
CONTAINER_NAME = "rutster-allinone-smoke"
HOST_PORT = int(os.environ.get("RUTSTER_ALLINONE_PORT", "18443"))
DATA_VOLUME = "rutster-smoke-caddy-data"
# 20ms of 8kHz u-law silence = 160 bytes; base64-encoded in a Twilio
# Media event envelope (matches the slice-A trunk_sim_e2e.rs TODO pattern
# + crates/rutster-trunk/tests/ws_ping.rs' frame shape).
PCM_FRAME_BYTES = b"\x00" * 160
FRAMES_TO_SEND = 200
CADENCE_MS = 20
RECV_DEADLINE_S = 10.0
def log(msg: str) -> None:
print(f"[allinone_smoke] {msg}", flush=True)
def fail(msg: str) -> "NoReturn": # noqa: F821 — Python 3.11+ syntax
print(f"[allinone_smoke] FAIL: {msg}", file=sys.stderr, flush=True)
subprocess.run(
["docker", "logs", CONTAINER_NAME], capture_output=False, timeout=10
)
sys.exit(1)
def run(cmd, check=True, timeout=30.0) -> str:
"""Run a command; return stdout. Fail the smoke on non-zero exit (if check)."""
log(f"$ {' '.join(cmd)}")
r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
if check and r.returncode != 0:
print(f" stdout: {r.stdout[-2000:]}", file=sys.stderr)
print(f" stderr: {r.stderr[-2000:]}", file=sys.stderr)
fail(f"command exited {r.returncode}: {' '.join(cmd)}")
return r.stdout
def bootstrap_container() -> None:
"""Boot rutster-allinone with RUTSTER_LOCAL_CERTS=true."""
subprocess.run(["docker", "rm", "-f", CONTAINER_NAME],
capture_output=True, timeout=10)
subprocess.run(["docker", "volume", "rm", "-f", DATA_VOLUME],
capture_output=True, timeout=10)
env = [
"-e", "RUTSTER_DOMAIN=localhost",
"-e", "RUTSTER_ACME_EMAIL=smoke@example.com",
# CRITICAL: RUTSTER_LOCAL_CERTS=true makes Caddy use its internal CA
# — no ACME in CI (spec §9).
"-e", "RUTSTER_LOCAL_CERTS=true",
"-e", "RUTSTER_HTTP_BIND=0.0.0.0:8080",
"-e", "RUTSTER_MEDIA_BIND_IP=127.0.0.1",
"-e", "RUTSTER_MEDIA_PORT_RANGE=49160-49170",
"-e", "RUTSTER_DRAIN_DEADLINE_SECS=10",
"-e", "RUTSTER_TAP_URL=ws://127.0.0.1:8082",
# Twilio creds unset — WebRTC + media-stream WS smoke only.
"-p", f"{HOST_PORT}:443",
]
run(["docker", "run", "-d", "--name", CONTAINER_NAME,
"-v", f"{DATA_VOLUME}:/data",
*env, IMAGE], timeout=120.0)
root_cert = wait_for_root_cert()
log(f"root cert extracted ({len(root_cert)} bytes)")
wait_for_healthz(root_cert)
def wait_for_root_cert() -> bytes:
"""Poll until Caddy wrote /data/caddy/pki/authorities/local/root.crt,
then docker cp it out. Caddy creates this on first boot with local_certs."""
deadline = time.time() + 30.0
while time.time() < deadline:
r = subprocess.run(
["docker", "exec", CONTAINER_NAME, "test", "-f",
"/data/caddy/pki/authorities/local/root.crt"],
capture_output=True, timeout=5,
)
if r.returncode == 0:
r = subprocess.run(
["docker", "cp",
f"{CONTAINER_NAME}:/data/caddy/pki/authorities/local/root.crt", "-"],
capture_output=True, timeout=10,
)
if r.returncode == 0 and r.stdout:
return r.stdout
time.sleep(0.5)
fail("Caddy internal CA root cert did not appear within 30s")
return b"" # unreachable
def wait_for_healthz(root_cert_pem: bytes) -> None:
"""Curl /healthz through TLS using the extracted root cert."""
with tempfile.NamedTemporaryFile(suffix=".pem", delete=False) as f:
f.write(root_cert_pem)
ca_file = f.name
try:
deadline = time.time() + 30.0
r = subprocess.run(["true"]) # for the warning-less r init
while time.time() < deadline:
r = subprocess.run(
["curl", "--cacert", ca_file, "-fsS",
f"https://localhost:{HOST_PORT}/healthz"],
capture_output=True, text=True, timeout=5,
)
if r.returncode == 0 and r.stdout.strip() == "ok":
log("/healthz ok")
return
time.sleep(0.5)
fail(f"/healthz never returned ok within 30s (last stderr: "
f"{getattr(r, 'stderr', '')[-500:]})")
finally:
os.unlink(ca_file)
# --- hand-rolled WS frame encode/decode (RFC 6455 §5) ---
def make_ws_frame(payload: bytes, opcode: int = 0x1) -> bytes:
"""Encode a client->server WS frame. opcode 0x1=text, 0x2=binary.
Client frames MUST be masked (RFC 6455 §5.3)."""
mask = os.urandom(4)
masked = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
header = bytearray([0x80 | opcode]) # FIN + opcode
n = len(payload)
if n < 126:
header.append(0x80 | n)
elif n < 65536:
header.append(0x80 | 126)
header.extend(struct.pack(">H", n))
else:
header.append(0x80 | 127)
header.extend(struct.pack(">Q", n))
header.extend(mask)
return bytes(header) + masked
def read_ws_frame(sock) -> tuple:
"""Read one WS frame from the server. Returns (opcode, payload).
Server frames are NOT masked (RFC 6455 §5.1).
Returns (-1, b"") on EOF."""
h1 = sock.recv(1)
if not h1:
return -1, b""
fin_opcode = h1[0]
h2 = sock.recv(1)[0]
masked = bool(h2 & 0x80)
length = h2 & 0x7F
if length == 126:
length = struct.unpack(">H", sock.recv(2))[0]
elif length == 127:
length = struct.unpack(">Q", sock.recv(8))[0]
if masked:
mask = sock.recv(4)
payload = b""
while len(payload) < length:
chunk = sock.recv(length - len(payload))
if not chunk:
break
payload += chunk
if masked:
payload = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
return fin_opcode & 0x0F, payload
def open_wss():
"""Open the wss:// WS handshake to /twilio/media-stream inside the
containerized all-in-one."""
sock = socket.create_connection(("localhost", HOST_PORT), timeout=5)
ctx = ssl.create_default_context(cafile=None)
r = subprocess.run(
["docker", "cp",
f"{CONTAINER_NAME}:/data/caddy/pki/authorities/local/root.crt", "-"],
capture_output=True, timeout=10,
)
ctx.load_verify_locations(cadata=r.stdout.decode("ascii"))
ctx.check_hostname = False # the cert is for "localhost" — Caddy's local_certs issues it
ssl_sock = ctx.wrap_socket(sock, server_hostname="localhost")
log(f"TLS established (cipher={ssl_sock.cipher()})")
key = os.urandom(16).hex()
handshake = (
f"GET /twilio/media-stream HTTP/1.1\r\n"
f"Host: localhost:{HOST_PORT}\r\n"
f"Upgrade: websocket\r\n"
f"Connection: Upgrade\r\n"
f"Sec-WebSocket-Key: {key}\r\n"
f"Sec-WebSocket-Version: 13\r\n"
f"\r\n"
)
ssl_sock.sendall(handshake.encode("ascii"))
buf = b""
while b"\r\n\r\n" not in buf:
chunk = ssl_sock.recv(4096)
if not chunk:
fail("WS upgrade response closed prematurely")
buf += chunk
if b"101 Switching Protocols" not in buf:
fail(f"WS upgrade did not return 101: {buf[:300]!r}")
log("WS upgraded (HTTP/1.1 101 Switching Protocols)")
return ssl_sock
def send_twilio_handshake(sock) -> None:
"""Send the Twilio Media Streams connected + start frames (the same
JSON sequence crates/rutster-trunk/tests/ws_ping.rs uses against the
in-process router — here against the real containerized edge->FOB path)."""
connected = json.dumps({
"event": "connected",
"protocol": "twilio-media-stream",
"version": "1.0.0",
})
start = json.dumps({
"event": "start",
"start": {"streamSid": "MZsmoke", "callSid": "CAsmoke"},
})
sock.sendall(make_ws_frame(connected.encode("utf-8")))
sock.sendall(make_ws_frame(start.encode("utf-8")))
log("sent Twilio connected + start handshake frames")
def stream_pcm_frames(sock) -> int:
"""Stream 200 PCM-zeroed frames at 20ms cadence as Twilio Media events.
Returns the count of text frames received back from the engine."""
received = 0
deadline_global = time.time() + (FRAMES_TO_SEND * CADENCE_MS / 1000.0) + RECV_DEADLINE_S
for _ in range(FRAMES_TO_SEND):
media_event = json.dumps({
"event": "media",
"media": {"payload": b64encode(PCM_FRAME_BYTES).decode("ascii")},
})
sock.sendall(make_ws_frame(media_event.encode("utf-8")))
# Drain non-blocking: read anything available without blocking the
# 20ms cadence.
sock.setblocking(False)
try:
while True:
op, _ = read_ws_frame(sock)
if op == -1:
break
if op == 0x1: # text frame — engine-originated event
received += 1
except (BlockingIOError, ssl.SSLWantReadError):
pass
finally:
sock.setblocking(True)
time.sleep(CADENCE_MS / 1000.0)
sock.settimeout(RECV_DEADLINE_S)
try:
while time.time() < deadline_global:
op, _ = read_ws_frame(sock)
if op == -1:
break
if op == 0x1:
received += 1
except (socket.timeout, ssl.SSLWantReadError):
pass
return received
def main() -> int:
log(f"image={IMAGE}, host port={HOST_PORT}")
bootstrap_container()
sock = open_wss()
send_twilio_handshake(sock)
received = stream_pcm_frames(sock)
log(f"sent {FRAMES_TO_SEND} frames; received {received} text frames back")
if received < 1:
fail("expected at least one engine-originated text frame (mark/outbound)")
log("PASS: real edge->FOB WS path through Caddy TLS verified")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
finally:
# Always clean up the container + volume — even on FAIL.
subprocess.run(["docker", "rm", "-f", CONTAINER_NAME],
capture_output=True, timeout=10)
subprocess.run(["docker", "volume", "rm", "-f", DATA_VOLUME],
capture_output=True, timeout=10)
- Step 3: Write
deploy/smoke/compose_smoke.sh
#!/usr/bin/env bash
# deploy/smoke/compose_smoke.sh — T2 compose smoke (deploy-B §9).
#
# Brings up deploy/compose.yaml with \`docker compose up -d --wait\`, asserts
# every service reports healthy via \`docker compose ps\`, then curls
# /healthz through Caddy's TLS (using the internal CA —
# RUTSTER_LOCAL_CERTS=true is set in the smoke's env). The compose smoke
# is intentionally lighter than the all-in-one smoke (which already proves
# the WS path) — its job is to assert the four-service orchestrator shape
# boots + the network_mode: "service:engine" brain->engine tap posture.
#
# The ValkeyEventSink-lands-in-stream assertion is slice C's concern.
# Named hook below; do NOT implement it in this slice.
set -euo pipefail
cd "$(dirname "$0")/.." # deploy/
IMAGES_TAG="${RUTSTER_IMAGES_TAG:-smoke}"
export RUTSTER_DOMAIN=localhost
export RUTSTER_ACME_EMAIL=smoke@example.com
export RUTSTER_LOCAL_CERTS=true
export RUTSTER_HTTP_BIND=0.0.0.0:8080
export RUTSTER_MEDIA_BIND_IP=127.0.0.1
export RUTSTER_MEDIA_PORT_RANGE=49160-49170
export RUTSTER_DRAIN_DEADLINE_SECS=10
export RUTSTER_TAP_URL=ws://127.0.0.1:8082
echo "[compose_smoke] overriding compose.yaml image tags -> :${IMAGES_TAG}"
# Sed the image: tags from :latest to :smoke so CI's locally-built images
# are picked up. In-place + restored on exit.
cp compose.yaml compose.yaml.smoke-backup
trap 'mv compose.yaml.smoke-backup compose.yaml' EXIT
sed -i "s|image: rutster-edge:latest|image: rutster-edge:${IMAGES_TAG}|" compose.yaml
sed -i "s|image: rutster-engine:latest|image: rutster-engine:${IMAGES_TAG}|" compose.yaml
sed -i "s|image: rutster-brain:latest|image: rutster-brain:${IMAGES_TAG}|" compose.yaml
echo "[compose_smoke] docker compose up -d --wait"
docker compose up -d --wait --timeout 120
echo "[compose_smoke] docker compose ps"
docker compose ps
# Each service should report "running" state. (Healthy flag is set by the
# engine's /readyz via the Docker healthcheck — none of the shipped images
# define a HEALTHCHECK yet; slice-G or a future hardening slice can add one.
# For now State="running" + /healthz 200 + /readyz 200 is the gate.)
HEALTHY_COUNT=$(docker compose ps --format json | jq '[.[] | select(.State == "running")] | length')
EXPECTED=4
if [ "$HEALTHY_COUNT" -ne "$EXPECTED" ]; then
echo "[compose_smoke] FAIL: expected $EXPECTED running services, got $HEALTHY_COUNT" >&2
docker compose logs
exit 1
fi
echo "[compose_smoke] $EXPECTED services running"
# Caddy /healthz via TLS (internal CA). Port 443 is published by the caddy service.
ROOT_CERT=$(docker compose exec -T caddy cat /data/caddy/pki/authorities/local/root.crt)
echo "$ROOT_CERT" > /tmp/rutster-smoke-root.pem
# Wait for /healthz to return 200 via Caddy TLS.
for _ in $(seq 1 60); do
if curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/healthz \
| grep -q '^ok$'; then
echo "[compose_smoke] /healthz ok"
break
fi
sleep 0.5
done
curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/healthz \
|| { echo "[compose_smoke] FAIL: /healthz never returned ok"; docker compose logs; exit 1; }
# Engine /readyz — fresh boot, MUST be 200 (drain happens on shutdown, not
# boot; cap only after RUTSTER_MAX_SESSIONS sessions exist).
curl --cacert /tmp/rutster-smoke-root.pem -fsS https://localhost/readyz \
|| { echo "[compose_smoke] FAIL: /readyz never returned ok"; docker compose logs; exit 1; }
echo "[compose_smoke] /readyz ok"
# TODO slice-C: assert lifecycle event in Valkey stream.
# Slice C lands ValkeyEventSink (spec §5.6) + the assert-event-lands-in-
# Valkey-stream smoke addition. The hook here documents the contract:
# Once slice C lands, this smoke runs:
# docker compose exec valkey valkey-cli XLEN rutster:events
# and asserts > 0 after a sim call completes (an EventSink::on_event
# call writes via XADD MAXLEN ~). Until then, the valkey service is dark
# (no consumers) — counting the stream length would always be 0.
# Leaving the named hook so slice C's plan can grep for it.
echo "[compose_smoke] PASS: T2 compose boots + /healthz + /readyz verified"
# Cleanup (compose down; volumes left for warm cache on re-run).
docker compose down --volumes --timeout 30 || true
exit 0
- Step 4: Write
deploy/smoke/reload_during_call.py
#!/usr/bin/env python3
"""deploy/smoke/reload_during_call.py — Caddy config reload during a live
sim call, asserting zero frame drops (deploy-B §9; spec §3.2; TLS brief
§5 risk 1).
Spec §3.2 mandates a CI e2e case for Caddy config-reload-during-live-call
because the upstream mitigation (`stream_close_delay` in Caddyfile) has an
open bug trail (caddy #6420, #7222) and is NOT trusted untested.
This script reuses the all-in-one smoke's WS helpers (open wss://
to /twilio/media-stream, send connected+start handshake); starts a streaming
loop sending PCM frames at 20ms cadence; concurrently triggers a `caddy
reload` mid-call (the operator's config-edit path); asserts the WS stays
up across the reload + no frames are dropped (the recv count matches the
sent count within tolerance).
Exit code 0 = reload-during-call passed. Non-zero = failed.
"""
import json
import os
import subprocess
import sys
import threading
import time
from base64 import b64encode
# Reuse the all-in-one smoke's WS helpers via import.
sys.path.insert(0, os.path.dirname(__file__))
from allinone_smoke import ( # noqa: E402
make_ws_frame, read_ws_frame, bootstrap_container,
open_wss, send_twilio_handshake, fail, log,
CONTAINER_NAME, IMAGE, HOST_PORT, PCM_FRAME_BYTES,
)
FRAMES_TO_SEND = 400 # 8s of wall clock at 20ms cadence
CADENCE_MS = 20
RELOAD_AT_FRAME = 150 # fire `caddy reload` 3s into the 8s call
def trigger_caddy_reload() -> None:
"""Run `docker exec rutster-allinone-smoke caddy reload` inside the
container. Caddy reloads its Caddyfile in-memory; stream_close_delay
24h should keep upgraded WS tunnels alive."""
log(f"triggering `caddy reload` at frame {RELOAD_AT_FRAME}...")
r = subprocess.run(
["docker", "exec", CONTAINER_NAME,
"caddy", "reload", "--config", "/etc/caddy/Caddyfile",
"--adapter", "caddyfile"],
capture_output=True, text=True, timeout=30.0,
)
if r.returncode != 0:
log(f"caddy reload stderr: {r.stderr[-500:]}")
# We do NOT fail the smoke on a non-zero caddy reload — the smoke's
# question is "did the live WS survive?", not "did caddy reload work?"
# A failed reload is its own bug; the WS-drop is the caddy #6420/#7222
# concern this smoke is testing.
def stream_frames_with_midcall_reload(sock):
"""Returns (frames_sent, frames_received, ws_dropped_during_reload).
ws_dropped_during_reload is True if the WS read loop got EOF/exception
in the 1-second window after the reload fired."""
sent = 0
received = 0
ws_dropped = False
reload_fired = False
reload_time = 0.0
for i in range(FRAMES_TO_SEND):
if i == RELOAD_AT_FRAME and not reload_fired:
reload_fired = True
reload_time = time.time()
threading.Thread(target=trigger_caddy_reload, daemon=True).start()
media_event = json.dumps({
"event": "media",
"media": {"payload": b64encode(PCM_FRAME_BYTES).decode("ascii")},
})
try:
sock.sendall(make_ws_frame(media_event.encode("utf-8"), opcode=0x1))
sent += 1
except (BrokenPipeError, ConnectionResetError) as e:
log(f"WS send failed at frame {i}: {e}")
if not reload_fired or (time.time() - reload_time) < 2.0:
ws_dropped = True
break
# Non-blocking drain.
import socket as _socket
sock.setblocking(False)
try:
while True:
op, _ = read_ws_frame(sock)
if op == -1:
break
if op == 0x1:
received += 1
except (BlockingIOError, ):
pass
finally:
sock.setblocking(True)
time.sleep(CADENCE_MS / 1000.0)
return sent, received, ws_dropped
def main() -> int:
log(f"image={IMAGE}, host port={HOST_PORT}")
bootstrap_container()
sock = open_wss()
send_twilio_handshake(sock)
sent, received, ws_dropped = stream_frames_with_midcall_reload(sock)
log(f"sent={sent}, received={received}, ws_dropped={ws_dropped}")
if ws_dropped:
fail("WS dropped during caddy reload — stream_close_delay mitigation "
"failed (caddy #6420/#7222) — the configuration MUST be re-evaluated")
if sent != FRAMES_TO_SEND:
fail(f"sent only {sent} of {FRAMES_TO_SEND} frames — WS died before reload")
log("PASS: caddy reload during live call dropped zero frames")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
finally:
subprocess.run(
["docker", "rm", "-f", CONTAINER_NAME],
capture_output=True, timeout=10,
)
- Step 5: Make the smoke scripts executable + re-run the content-check
cd /home/alee/Sources/rutster
chmod +x deploy/smoke/allinone_smoke.py deploy/smoke/compose_smoke.sh deploy/smoke/reload_during_call.py
test -f deploy/smoke/allinone_smoke.py \
&& test -f deploy/smoke/compose_smoke.sh \
&& test -f deploy/smoke/reload_during_call.py \
&& grep -q 'local_certs' deploy/smoke/allinone_smoke.py \
&& grep -q 'TODO slice-C' deploy/smoke/compose_smoke.sh \
&& grep -q 'caddy reload' deploy/smoke/reload_during_call.py \
&& grep -q 'stream_close_delay' deploy/smoke/reload_during_call.py \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 6: Commit
cd /home/alee/Sources/rutster
git add deploy/smoke/
git commit -s -m "feat(deploy/smoke): all-in-one + compose + reload-during-call smoke harness (deploy-B §9)
Three Python-stdlib-only smoke scripts (no pip install required in CI;
the runner image ships Python 3 + ssl/socket/json/base64/time):
* allinone_smoke.py — boots rutster-allinone with RUTSTER_LOCAL_CERTS=true
(Caddy internal CA, no ACME per spec §9), extracts the root cert from
/data, opens wss:// WS to /twilio/media-stream, sends Twilio's
connected+start handshake (the same JSON sequence the in-tree
ws_ping.rs/nodelay.rs tests use), streams 200 PCM frames at 20ms cadence,
asserts the engine echoes/replies at least one text frame. Reuses the
T8 stub's handshake pattern (crates/rutster/tests/trunk_sim_e2e.rs TODO
body) against a real containerized binary rather than an in-process
MockTwilioMediaStreamsServer.
* compose_smoke.sh — T2 four-service boots + health checks via Caddy TLS.
The # TODO slice-C: assert lifecycle event in Valkey stream named hook
documents the contract slice C will assert; the valkey service is dark
on day one (no consumers) so the stream length is always 0 today.
* reload_during_call.py — holds a live WS streaming frames at 20ms while a
concurrent \`caddy reload\` fires mid-call; asserts zero frames dropped
+ WS survives across the reload. Regresses the stream_close_delay 24h
mitigation against caddy #6420/#7222 (TLS brief §5 risk 1; spec §9
reload-during-call smoke).
WS frame encode/decode is hand-rolled (RFC 6455 §5 — ten lines on top of
socket+ssl) so the smoke needs no websocket-client/websockets pip
dependency + no network round-trip in CI."
Task 7: Extend .github/workflows/ci.yml — image-build job
The image-build job builds all four images via docker buildx build against deploy/Dockerfile,
with GHA cache from type=gha. It runs AFTER and GATED ON the existing fmt/clippy/test/deny/
sim-bench matrix (the smoke job after it depends on these built images).
Files:
- Modify:
.github/workflows/ci.yml(addimage-buildjob at the end; do NOT touch existingfmt/clippy/test/deny/sim-bench/twilio-livejobs — verified they exist at the current HEAD + their content is unchanged. The file currently ends at line 146 with thetwilio-livejob's closingrun: cargo test ... --include-ignoredstep).
Interfaces:
-
Consumes: Task 1's
deploy/Dockerfile+.dockerignore(Task 2) +deploy/Caddyfile(Task 3) +deploy/s6-rc.d/(Task 1 Step 4) + vendored s6-overlay tarballs (Task 1 Step 4). -
Produces: four Docker images tagged
rutster-{engine,brain,edge,allinone}:ci-${{ github.sha }}- re-tagged as
:smokefor the downstream smoke jobs.
- re-tagged as
-
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
grep -q 'name: image-build' .github/workflows/ci.yml && echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Append the
image-buildjob to.github/workflows/ci.yml
Append the following job to the end of .github/workflows/ci.yml (after the twilio-live
job's closing block):
# deploy-B §6.1 image-build — builds all four first-party images via
# `docker buildx build --target` against deploy/Dockerfile. Runs AFTER the
# fmt/clippy/test/deny/sim-bench matrix gates: the smoke job downstream
# depends on these built images.
#
# Single builder invocation that builds all four --targets would be ideal
# (shared cache), but buildx's `--target` is one-per-invocation — so four
# build steps share the GHA cache via `cache-from/cache-to: type=gha`.
# Warm caches take image-build from ~6 min to ~90s.
image-build:
name: image-build (4 images)
runs-on: ubuntu-latest
needs: [fmt, clippy, test, deny, sim-bench]
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# libopus-dev — the headers the `opus` crate's build.rs expects.
# (The builder stage of deploy/Dockerfile already installs this
# inside the build image; this is a belt-and-braces step so a local
# buildx cache root has the headers if the cache misses.)
- name: Build-time host deps (belt-and-braces)
run: sudo apt-get update && sudo apt-get install -y libopus-dev
- name: Build rutster-engine image
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-engine
tags: rutster-engine:ci-${{ github.sha }}
push: false
load: true
cache-from: type=gha,scope=rutster-engine
cache-to: type=gha,mode=max,scope=rutster-engine
- name: Build rutster-brain image
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-brain
tags: rutster-brain:ci-${{ github.sha }}
push: false
load: true
cache-from: type=gha,scope=rutster-brain
cache-to: type=gha,mode=max,scope=rutster-brain
- name: Build rutster-edge image
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-edge
tags: rutster-edge:ci-${{ github.sha }}
push: false
load: true
cache-from: type=gha,scope=rutster-edge
cache-to: type=gha,mode=max,scope=rutster-edge
- name: Build rutster-allinone image
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-allinone
tags: rutster-allinone:ci-${{ github.sha }}
push: false
load: true
cache-from: type=gha,scope=rutster-allinone
cache-to: type=gha,mode=max,scope=rutster-allinone
- name: Smoke-tag the images for the downstream smoke job
# The smoke job references images as `:smoke`
# (deploy/smoke/compose_smoke.sh tags images with the
# RUTSTER_IMAGES_TAG env). Tag the freshly-built :ci-<sha> images as
# :smoke too, so the smoke job picks them up.
run: |
for img in rutster-engine rutster-brain rutster-edge rutster-allinone; do
docker tag "${img}:ci-${{ github.sha }}" "${img}:smoke"
done
docker images | grep rutster
- name: caddy validate
# Validate the Caddyfile inside the freshly-built rutster-edge image.
# Cheap regression against malformed Caddyfile (the local `caddy
# validate` in Task 3 is optional; this one is mandatory in CI).
run: |
docker run --rm \
-e RUTSTER_DOMAIN=pbx.example.com \
-e RUTSTER_ACME_EMAIL=test@example.com \
-v "$PWD/deploy/Caddyfile:/etc/caddy/Caddyfile:ro" \
rutster-edge:smoke \
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
- Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: image-build' .github/workflows/ci.yml \
&& grep -q 'needs: \[fmt, clippy, test, deny, sim-bench\]' .github/workflows/ci.yml \
&& grep -q 'docker/build-push-action@v5' .github/workflows/ci.yml \
&& grep -q 'cache-from: type=gha' .github/workflows/ci.yml \
&& grep -q 'target: rutster-engine' .github/workflows/ci.yml \
&& grep -q 'target: rutster-brain' .github/workflows/ci.yml \
&& grep -q 'target: rutster-edge' .github/workflows/ci.yml \
&& grep -q 'target: rutster-allinone' .github/workflows/ci.yml \
&& grep -q 'caddy validate' .github/workflows/ci.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: image-build job — builds the four first-party Docker images (deploy-B §6.1)
Extends .github/workflows/ci.yml with an image-build job, gated on the
existing fmt/clippy/test/deny/sim-bench matrix
(needs: [fmt, clippy, test, deny, sim-bench]). Builds all four images via
docker buildx against deploy/Dockerfile's named --targets, with type=gha
caching scoped per image so warm caches take image-build from ~6min to
~90s.
Tags the freshly-built :ci-<sha> images as :smoke too, for the
downstream smoke job to consume. Runs \`caddy validate\` inside the
freshly-built rutster-edge as a cheap Caddyfile-syntax regression.
No changes to the existing fmt/clippy/test/deny/sim-bench/twilio-live
jobs — those stay the cargo-side source of truth."
Task 8: Extend .github/workflows/ci.yml — smoke job (all-in-one TLS sim call)
The smoke job runs deploy/smoke/allinone_smoke.py against the freshly-built rutster-allinone
image from Task 7. It depends on image-build (which itself depends on the fmt/clippy/test/deny/
sim-bench matrix). The smoke job is where spec §9's "all-in-one smoke — boots, Caddy terminates
TLS via its INTERNAL CA (no ACME in CI), full sim call through the real edge→FOB WS path"
assertion is enforced.
Files:
- Modify:
.github/workflows/ci.yml(addsmokejob afterimage-build).
Interfaces:
-
Consumes: Task 6's
deploy/smoke/allinone_smoke.py+ Task 7'srutster-allinone:smokeimage. -
Produces: pass/fail for the smoke job + downstream
smoke-reloadandsmoke-composejobs (Task 9) gate on this job. -
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke' .github/workflows/ci.yml && echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Append the
smokejob to.github/workflows/ci.yml
Append the following job to the end of .github/workflows/ci.yml (after the image-build
job's closing caddy validate step from Task 7):
# deploy-B §9 all-in-one smoke — boots rutster-allinone with
# RUTSTER_LOCAL_CERTS=true (Caddy internal CA, no ACME in CI), extracts
# the CA root cert from /data, opens wss:// to /twilio/media-stream,
# sends Twilio's connected+start handshake frames, streams 200 PCM-zeroed
# frames at 20ms cadence, asserts the engine replies with at least one
# text frame through the real edge->FOB WS path. Spec §9 acceptance:
# * "boots"
# * "Caddy terminates TLS via its internal CA (no ACME in CI)"
# * "full sim call through the real edge->FOB WS path"
# The reload-during-call smoke (Task 9 below) and compose smoke (Task 9)
# are separate jobs that depend on this one.
smoke:
name: smoke (all-in-one TLS sim call)
runs-on: ubuntu-latest
needs: [image-build]
steps:
- uses: actions/checkout@v4
- name: Set up Python (stdlib-only; no pip)
uses: actions/setup-python@v5
with:
python-version: "3.x"
# Pull the freshly-built image from the image-build job. Since
# jobs run on separate runners + `push: false` in image-build means
# the image isn't in any registry, the smoke job rebuilds from the
# GHA cache (warm — image-build populated it). This adds ~30s vs
# an image-download artifact, but avoids the registry-auth + image-
# save/load plumbing.
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Rebuild rutster-allinone:smoke (warm cache from image-build)
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-allinone
tags: rutster-allinone:smoke
push: false
load: true
cache-from: type=gha,scope=rutster-allinone
- name: Run all-in-one smoke (deploy/smoke/allinone_smoke.py)
run: |
python3 deploy/smoke/allinone_smoke.py
env:
RUTSTER_ALLINONE_IMAGE: rutster-allinone:smoke
RUTSTER_ALLINONE_PORT: "18443"
- Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke' .github/workflows/ci.yml \
&& grep -q 'needs: \[image-build\]' .github/workflows/ci.yml \
&& grep -q 'allinone_smoke.py' .github/workflows/ci.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: smoke job — all-in-one TLS sim call through real edge->FOB WS path (deploy-B §9)
Adds the smoke job to .github/workflows/ci.yml, gated on image-build.
Runs deploy/smoke/allinone_smoke.py against the freshly-built
rutster-allinone:smoke image:
* Boots the all-in-one container with RUTSTER_LOCAL_CERTS=true (Caddy
internal CA, no ACME in CI per spec §9).
* Extracts Caddy's internal-CA root cert from /data/caddy/pki/
authorities/local/root.crt via docker cp.
* Opens wss:// WS to /twilio/media-stream using the extracted root
(Python stdlib ssl + socket — no pip install required, CI runner-
image ships Python 3 + ssl).
* Sends Twilio's connected+start handshake frames (the same JSON
sequence crates/rutster-trunk/tests/ws_ping.rs uses).
* Streams 200 PCM-zeroed frames at 20ms cadence as Twilio Media events.
* Asserts the engine replies with at least one text frame — proving
the real edge->FOB WS path works end-to-end through TLS.
The compose smoke + reload-during-call smoke (Task 9) gate on this job."
Task 9: Extend .github/workflows/ci.yml — smoke-compose + smoke-reload jobs
Two more smoke jobs, each gated on image-build (Task 7). smoke-compose runs
deploy/smoke/compose_smoke.sh against the four fresh images from image-build.
smoke-reload runs deploy/smoke/reload_during_call.py (reuses the all-in-one image — both
smokes are all-in-one-shaped; these are separate jobs not steps within smoke for clearer
failure attribution + parallelism).
Files:
- Modify:
.github/workflows/ci.yml(append two more jobs aftersmoke).
Interfaces:
-
Consumes: Task 6's
compose_smoke.sh+reload_during_call.py+ Task 7's images. -
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke-compose' .github/workflows/ci.yml \
&& grep -q 'name: smoke-reload' .github/workflows/ci.yml \
&& grep -q 'reload_during_call.py' .github/workflows/ci.yml \
&& grep -q 'compose_smoke.sh' .github/workflows/ci.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Append the two new jobs to
.github/workflows/ci.yml
Append the following to the end of .github/workflows/ci.yml (after the smoke job from
Task 8):
# deploy-B §9 compose smoke — T2 four-service compose stack boots + all
# services healthy + Caddy TLS /healthz + /readyz. Lighter than the
# all-in-one smoke (which already proved the WS path) — its job is the
# orchestrator shape + the network_mode: "service:engine" brain->engine tap
# posture across the four services.
smoke-compose:
name: smoke (T2 compose four-service)
runs-on: ubuntu-latest
needs: [image-build]
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Rebuild all three first-party images from the warm GHA cache
# populated by image-build. Valkey is upstream (no rebuild needed).
- name: Rebuild rutster-edge:smoke
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-edge
tags: rutster-edge:smoke
push: false
load: true
cache-from: type=gha,scope=rutster-edge
- name: Rebuild rutster-engine:smoke
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-engine
tags: rutster-engine:smoke
push: false
load: true
cache-from: type=gha,scope=rutster-engine
- name: Rebuild rutster-brain:smoke
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-brain
tags: rutster-brain:smoke
push: false
load: true
cache-from: type=gha,scope=rutster-brain
# jq is pre-installed on ubuntu-latest GHA runners.
- name: Run compose smoke (deploy/smoke/compose_smoke.sh)
run: bash deploy/smoke/compose_smoke.sh
env:
RUTSTER_IMAGES_TAG: smoke
# deploy-B §9 + spec §3.2 + TLS brief §5 risk 1: Caddy config-reload
# during a live simulated call, asserting zero frame drops. The
# stream_close_delay 24h mitigation in deploy/Caddyfile is the load-
# bearing fixture; caddy #6420 / #7222 are open upstream bugs — the
# smoke is "don't trust untested" made CI-regressed.
smoke-reload:
name: smoke (caddy reload during live call, zero drops)
runs-on: ubuntu-latest
needs: [smoke] # reuses the rutster-allinone:smoke image, warm cache
steps:
- uses: actions/checkout@v4
- name: Set up Python (stdlib-only; no pip)
uses: actions/setup-python@v5
with:
python-version: "3.x"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Rebuild rutster-allinone:smoke (warm cache)
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-allinone
tags: rutster-allinone:smoke
push: false
load: true
cache-from: type=gha,scope=rutster-allinone
- name: Run reload-during-call smoke (deploy/smoke/reload_during_call.py)
run: python3 deploy/smoke/reload_during_call.py
env:
RUTSTER_ALLINONE_IMAGE: rutster-allinone:smoke
RUTSTER_ALLINONE_PORT: "18444"
- Step 3: Re-run the content-check
cd /home/alee/Sources/rutster
grep -q 'name: smoke-compose' .github/workflows/ci.yml \
&& grep -q 'name: smoke-reload' .github/workflows/ci.yml \
&& grep -q 'reload_during_call.py' .github/workflows/ci.yml \
&& grep -q 'compose_smoke.sh' .github/workflows/ci.yml \
&& grep -q 'needs: \[smoke\]' .github/workflows/ci.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-PASS.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/ci.yml
git commit -s -m "ci: smoke-compose + smoke-reload jobs — T2 four-service boot + caddy reload-during-call (deploy-B §9)
smoke-compose (gated on image-build):
* Rebuilds rutster-edge/engine/brain:smoke from warm GHA cache.
* Runs deploy/smoke/compose_smoke.sh — \`docker compose up -d --wait\`,
asserts all four services State=running, curls /healthz + /readyz
through Caddy TLS (internal CA). Lighter than the all-in-one smoke
(which already proved the WS path) — its job is the orchestrator
shape + the network_mode: \"service:engine\" brain->engine tap posture.
smoke-reload (gated on smoke):
* Reuses rutster-allinone:smoke + the allinone_smoke.py WS helpers.
* Runs deploy/smoke/reload_during_call.py — holds a live WS streaming
frames at 20ms while a concurrent \`caddy reload\` fires mid-call;
asserts zero frames dropped + WS survives across the reload.
Regresses the stream_close_delay 24h mitigation against caddy #6420/
#7222 (TLS brief §5 risk 1; spec §9 reload-during-call smoke).
Both jobs run on ubuntu-latest — separate runners for clearer failure
attribution + parallelism (the smoke gate is the failing CI signal,
not a single serial run)."
Task 10: Slice F — .github/workflows/publish-images.yml tag-push image publish
Slice F (spec §6.3 + §11): tag-push workflow publishing all four images to the
git.adlee.work/alee/rutster-* registry namespace. linux/amd64 only. Runs AFTER and GATED ON
the existing fmt/clippy/test/deny/sim-bench/image-build pipeline (re-runs the build to assert
freshness, then publishes; caching keeps the build cost ~30s when nothing changed).
Files:
- Create:
.github/workflows/publish-images.yml
Interfaces:
-
Consumes: Task 1's
deploy/Dockerfile+ the samedocker buildx build --targetinvocations as Task 7'simage-buildjob (the publish workflow is structurally a copy of image-build withpush: true+tags: git.adlee.work/alee/...instead ofpush: false + tags: ...:ci-<sha>). Reuses the GHA cache (warm from image-build'scache-to: type=gha,mode=max). -
Produces: four published Docker images at
git.adlee.work/alee/rutster-{allinone,engine,brain,edge}:${{ github.ref_name }}:lateston everyv*tag push.
-
Step 1: Run the failing content-check
cd /home/alee/Sources/rutster
test -f .github/workflows/publish-images.yml \
&& grep -q 'on:' .github/workflows/publish-images.yml \
&& grep -q 'push:' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-allinone' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-engine' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-brain' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-edge' .github/workflows/publish-images.yml \
&& grep -q 'linux/amd64' .github/workflows/publish-images.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
Expected: CHECK-FAIL.
- Step 2: Write the complete workflow file
Write .github/workflows/publish-images.yml with exactly this content:
# .github/workflows/publish-images.yml — slice F (spec §6.3 + §11).
#
# Tag-push workflow: builds + pushes all four first-party images to the
# git.adlee.work/alee/rutster-* registry namespace. linux/amd64 only (spec
# §1.2 / ADR-0011 — arm64 is a named deferral).
#
# Gating: runs AFTER and gated on the existing fmt/clippy/test/deny/sim-bench
# + image-build pipeline. The publish workflow re-runs the fmt/clippy/test/
# deny matrix as a `needs:` chain (cheap when nothing changed) + builds all
# four images from the same Dockerfile CI's image-build job uses. This makes
# the published image byte-identical to the one CI tested — the "registry tag
# points at a tested image" invariant operators rely on.
#
# Registry namespace: git.adlee.work/alee/rutster-* (matches the `alee` tea
# login per AGENTS.md Git workflow + the root Cargo.toml's `repository`
# field). If your Gitea registry uses a different owner segment, replace
# `alee` throughout this file (REGISTRY_NAMESPACE below).
name: Publish Images
on:
push:
tags:
- 'v*'
env:
REGISTRY: git.adlee.work
REGISTRY_NAMESPACE: alee
PLATFORMS: linux/amd64
jobs:
# The publish workflow is its own pipeline; it re-runs fmt/clippy/test/deny
# + builds the images + then publishes. The fmt/clippy/test/deny re-runs
# are cheap on a green repo (cache hits, fast fmt + clippy + test), and
# they make a critical guarantee: the version published under a `v*` tag
# has ALL the repo's quality gates green at that exact commit. Without
# this `needs:` chain, a tag could be pushed at a commit where the matrix
# failed but image-build happened to succeed on a flake.
fmt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
- run: cargo fmt --check
clippy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install libopus (media crate FFI dep)
run: apt-get update && apt-get install -y libopus-dev
- uses: Swatinem/rust-cache@v2
- run: cargo clippy --all --all-targets -- -D warnings
test:
runs-on: ubuntu-latest
strategy:
matrix:
toolchain: [stable, "1.85"]
steps:
- uses: actions/checkout@v4
# Seam gate (mirror of .github/workflows/ci.yml's test job —
# loop_driver.rs + rtc_session.rs stay byte-identical to slice-3/5).
- name: Seam gate — loop_driver frozen + rtc_session pinned
run: |
EXPECTED_LOOP_DRIVER='744bf314edf7f4925c8bb3bd0f5176dbc88f8113'
EXPECTED_RTC_SESSION='f47d63b9a2883d37066a93c9daa0e2cf8816bec4'
GOT_LOOP_DRIVER=$(git hash-object crates/rutster-media/src/loop_driver.rs)
GOT_RTC_SESSION=$(git hash-object crates/rutster-media/src/rtc_session.rs)
[ "$GOT_LOOP_DRIVER" = "$EXPECTED_LOOP_DRIVER" ] || exit 1
[ "$GOT_RTC_SESSION" = "$EXPECTED_RTC_SESSION" ] || exit 1
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ matrix.toolchain }}
- name: Install libopus (media crate FFI dep)
run: apt-get update && apt-get install -y libopus-dev
- uses: Swatinem/rust-cache@v2
- run: cargo test --all
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check
sim-bench:
name: sim-bench (stable)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install libopus (media crate FFI dep)
run: apt-get update && apt-get install -y libopus-dev
- uses: Swatinem/rust-cache@v2
- name: cargo fmt + clippy on sim-bench feature paths
run: |
cargo fmt --all --check
cargo clippy --all --all-targets --features=sim-bench -- -D warnings
- name: Run sim-bench threshold sweep
run: cargo test --all --features=sim-bench -- --test-threads=1
publish:
name: Publish 4 images to git.adlee.work/alee/rutster-*
runs-on: ubuntu-latest
needs: [fmt, clippy, test, deny, sim-bench]
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build-time host deps (belt-and-braces)
run: sudo apt-get update && sudo apt-get install -y libopus-dev
- name: Log in to Gitea registry
# GITEA_REGISTRY_USERNAME + GITEA_REGISTRY_PASSWORD are repo secrets
# set by the maintainer (per AGENTS.md Git workflow — the `alee`
# tea login). The username is `alee` (matches REGISTRY_NAMESPACE).
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.GITEA_REGISTRY_USERNAME }}
password: ${{ secrets.GITEA_REGISTRY_PASSWORD }}
- name: Determine tags
id: tags
# Two tags per image: the version tag (github.ref_name, e.g. v0.1.0)
# + `latest`. Operators can pin to either; `latest` is the
# convenience tag that always points at the most recent release.
run: |
echo "version_tag=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
echo "namespace=${REGISTRY}/${REGISTRY_NAMESPACE}" >> $GITHUB_OUTPUT
- name: Build + push rutster-engine
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-engine
platforms: ${{ env.PLATFORMS }}
tags: |
${{ steps.tags.outputs.namespace }}/rutster-engine:${{ steps.tags.outputs.version_tag }}
${{ steps.tags.outputs.namespace }}/rutster-engine:latest
push: true
cache-from: type=gha,scope=rutster-engine
cache-to: type=gha,mode=max,scope=rutster-engine
- name: Build + push rutster-brain
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-brain
platforms: ${{ env.PLATFORMS }}
tags: |
${{ steps.tags.outputs.namespace }}/rutster-brain:${{ steps.tags.outputs.version_tag }}
${{ steps.tags.outputs.namespace }}/rutster-brain:latest
push: true
cache-from: type=gha,scope=rutster-brain
cache-to: type=gha,mode=max,scope=rutster-brain
- name: Build + push rutster-edge
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-edge
platforms: ${{ env.PLATFORMS }}
tags: |
${{ steps.tags.outputs.namespace }}/rutster-edge:${{ steps.tags.outputs.version_tag }}
${{ steps.tags.outputs.namespace }}/rutster-edge:latest
push: true
cache-from: type=gha,scope=rutster-edge
cache-to: type=gha,mode=max,scope=rutster-edge
- name: Build + push rutster-allinone
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile
target: rutster-allinone
platforms: ${{ env.PLATFORMS }}
tags: |
${{ steps.tags.outputs.namespace }}/rutster-allinone:${{ steps.tags.outputs.version_tag }}
${{ steps.tags.outputs.namespace }}/rutster-allinone:latest
push: true
cache-from: type=gha,scope=rutster-allinone
cache-to: type=gha,mode=max,scope=rutster-allinone
- Step 3: Re-run the content-check + YAML lint
cd /home/alee/Sources/rutster
test -f .github/workflows/publish-images.yml \
&& grep -q 'on:' .github/workflows/publish-images.yml \
&& grep -q 'push:' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-allinone' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-engine' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-brain' .github/workflows/publish-images.yml \
&& grep -q 'git.adlee.work/alee/rutster-edge' .github/workflows/publish-images.yml \
&& grep -q 'linux/amd64' .github/workflows/publish-images.yml \
&& grep -q 'needs: \[fmt, clippy, test, deny, sim-bench\]' .github/workflows/publish-images.yml \
&& grep -q 'GITEA_REGISTRY_USERNAME' .github/workflows/publish-images.yml \
&& grep -q 'v0.1.0' .github/workflows/publish-images.yml \
&& echo CHECK-PASS || echo CHECK-FAIL
# YAML syntax validation via Python stdlib (no yamllint dep needed).
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/publish-images.yml'))" \
|| { echo "YAML syntax error"; exit 1; }
Expected: CHECK-PASS. The YAML loads cleanly.
- Step 4: Commit
cd /home/alee/Sources/rutster
git add .github/workflows/publish-images.yml
git commit -s -m "ci: publish-images.yml — slice F tag-push image publish (deploy-F §6.3)
Tag-push workflow (\`on: push: tags: ['v*']\`) that publishes all four
first-party images to git.adlee.work/alee/rutster-* (REGISTRY_NAMESPACE
matches the \`alee\` tea login per AGENTS.md Git workflow + the root
Cargo.toml's repository field). linux/amd64 only (spec §1.2 — arm64 is a
named deferral per ADR-0011).
Gating: re-runs fmt/clippy/test/deny/sim-bench as a \`needs:\` chain so a
tag's published image is byte-identical to the one tested by the routine
CI matrix. The fmt/clippy/test/deny re-runs are cheap on a green repo
(cache hits, fast checks), and they make a critical guarantee: the
version under a \`v*\` tag has ALL the repo's quality gates green at
that exact commit. Without this \`needs:\` chain, a tag could be pushed
at a commit where the matrix failed but image-build happened to succeed
on a flake.
Two tags per image: the version tag (github.ref_name, e.g. v0.1.0) +
\`latest\` (the convenience tag operators pin to).
Docker login via GITEA_REGISTRY_USERNAME + GITEA_REGISTRY_PASSWORD
secrets, set by the maintainer. If your Gitea registry uses a different
owner segment, replace \`alee\` throughout (REGISTRY_NAMESPACE + the
namespace/git.adlee.work/alee/ strings in .env.example + compose.yaml)."
Task 11: Final verification sweep
No code. Evidence before assertions — run every gate the CI runs, plus the seam-gate check, before calling the slice done.
- Step 1: Full gate run (cargo-side, unchanged from existing CI)
cd /home/alee/Sources/rutster
cargo fmt --all --check
cargo clippy --all -- -D warnings
cargo clippy --all --all-targets --features=sim-bench -- -D warnings
cargo test --all
cargo test --all --features=sim-bench -- --test-threads=1
cargo deny check
cargo doc --no-deps
Expected: every command exits 0. (This slice adds no Rust code; these gates stay green from
plan A's landing on main. The sweep verifies the slice-B file additions don't accidentally
break the cargo build — e.g. a stray Cargo.toml edit.)
- Step 2: Seam gate verification (unchanged hashes)
cd /home/alee/Sources/rutster
git hash-object crates/rutster-media/src/loop_driver.rs
git hash-object crates/rutster-media/src/rtc_session.rs
Expected output, exactly:
744bf314edf7f4925c8bb3bd0f5176dbc88f8113
f47d63b9a2883d37066a93c9daa0e2cf8816bec4
If either hash differs, STOP — a task touched a seam-gated file; restore with
git checkout main -- crates/rutster-media/src/loop_driver.rs crates/rutster-media/src/rtc_session.rs
and re-run the gates.
- Step 3: Artifact inventory (every planned file exists)
cd /home/alee/Sources/rutster
for f in \
deploy/Dockerfile \
deploy/.dockerignore \
deploy/Caddyfile \
deploy/compose.yaml \
deploy/.env.example \
deploy/s6-overlay-noarch.tar.xz \
deploy/s6-overlay-x86_64.tar.xz \
deploy/s6-overlay.sha256sum \
deploy/s6-rc.d/caddy/run \
deploy/s6-rc.d/engine/run \
deploy/s6-rc.d/brain/run \
deploy/s6-rc.d/valkey-database/run \
deploy/smoke/allinone_smoke.py \
deploy/smoke/compose_smoke.sh \
deploy/smoke/reload_during_call.py \
.github/workflows/publish-images.yml \
; do
test -e "$f" || echo "MISSING: $f"
done
echo "inventory check complete"
Expected: zero MISSING: lines.
- Step 4: CI workflow sanity (both workflow files parse cleanly + image-build/smoke jobs are present)
cd /home/alee/Sources/rutster
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/publish-images.yml'))"
grep -E 'name: (image-build|smoke|smoke-compose|smoke-reload)' .github/workflows/ci.yml
grep -E 'name: (Publish|fmt|clippy|test|deny|sim-bench|publish)' .github/workflows/publish-images.yml
Expected: both YAML files load; ci.yml lists all four new jobs (image-build, smoke,
smoke-compose, smoke-reload); publish-images.yml lists its publish job + the gating matrix.
- Step 5: TODO slice-C hook verification (the named hook stays a comment, not implemented)
cd /home/alee/Sources/rutster
grep -n 'TODO slice-C' deploy/smoke/compose_smoke.sh
Expected: exactly one match, naming the ValkeyEventSink assertion as deferred to slice C. The slice-E (ValkeyEventSink) plan will grep this same line.
- Step 6: DCO verification (every commit on this branch signed off)
cd /home/alee/Sources/rutster
git log main..HEAD --format='%h %s' \
| head -20 \
&& echo "---DCO count vs commit count---" \
&& COMMITS=$(git log main..HEAD --format='%H' | wc -l) \
&& DCOS=$(git log main..HEAD --format='%(trailers:key=Signed-off-by)' | grep -c 'Signed-off-by') \
&& echo "commits: $COMMITS, signoffs: $DCOS"
Expected: the signoffs count equals the commit count (every git commit -s produced a
Signed-off-by trailer per AGENTS.md DCO requirement).
- Step 7: PR opening
cd /home/alee/Sources/rutster
git push -u origin deploy-b/packaging-ci
tea pulls create \
--head deploy-b/packaging-ci \
--base main \
--title "deploy slice B + F: deploy/ artifacts + image-build + container smoke CI + tag-push publish" \
--description "## What lands
- deploy/Dockerfile (multi-stage, four named --target stages: rutster-engine/brain/edge/allinone)
- deploy/compose.yaml (T2 reference; stop_grace_period 660s; brain network_mode: \"service:engine\")
- deploy/Caddyfile (tuned timeouts, honest X-Forwarded, stream_close_delay 24h)
- deploy/.env.example (all RUTSTER_* vars documented)
- deploy/smoke/{allinone_smoke.py,compose_smoke.sh,reload_during_call.py}
- .github/workflows/ci.yml extended with image-build + smoke + smoke-compose + smoke-reload jobs
- .github/workflows/publish-images.yml (slice F tag-push image publish)
## Dependencies
Depends on plan A (deploy-a/engine-hygiene) merged first — TCP_NODELAY,
WS pings, and the trunk config fixes make the containerized PSTN path honest.
## Reviews
- Spec: docs/superpowers/specs/2026-07-05-deployment-topology-design.md §2.1, §2.2, §3.2, §6, §9, §11
- TLS brief: docs/superpowers/specs/2026-07-05-tls-edge-decision-brief.md §3(a) + §5 risk 1
- Slice-C TODO hook in deploy/smoke/compose_smoke.sh — the ValkeyEventSink smoke assertion is
the named slice-C concern; this slice ships the valkey service shape but no event-lands check
## Merge instructions (if non-default)
Default squash-merge. The deploy-b/packaging-ci branch is standalone (no
stacked branches depend on its commit SHAs), so squash keeps main linear."
Expected: the tea PR URL is printed; the PR is open + targeting main.
Final acceptance checklist
After all 11 tasks land:
- All planned artifacts exist on disk (Task 11 Step 3 — no
MISSING:lines). cargo fmt --check,cargo clippy -D warnings(default +--features=sim-bench),cargo test --all, sim-bench sweep (--test-threads=1),cargo deny check,cargo doc --no-depsall clean. (This slice adds no Rust; the gates stay green from plan A. The sweep verifies nothing in the cargo graph was disturbed.)- Seam gate:
loop_driver.rs+rtc_session.rsblob hashes unchanged (Task 11 Step 2). deploy/Dockerfilebuilds all four named--targets (rutster-engine,rutster-brain,rutster-edge,rutster-allinone). Builder pinsrust:1.85-slim-bookworm(matchesrust-toolchain.toml). libopus-dev in builder, libopus0 in runtime (matches dev — spec §6.1). Brain image gets libopus0 too (spec ambiguity #2 — the linker requires it). Noaudiopus_sysbundled-cmake fallback.deploy/Caddyfilehasstream_close_delay 24h,idle 24h,write 0, honestX-Forwarded-Proto/Host/For(engine-sideRUTSTER_TRUSTED_PROXIESis the trust gate; fail-closed default). TheRUTSTER_LOCAL_CERTS=trueenv override makes Caddy use its internal CA (no ACME in CI per spec §9).deploy/compose.yamlships four services (caddy=rutster-edge,engine=rutster-engine,brain=rutster-brainwithnetwork_mode: "service:engine",valkey=upstreamvalkey/valkey),stop_grace_period: 660s, two named volumes (caddy-data,valkey-data).deploy/.env.examplelists everyRUTSTER_*var with a default, the slice that owns it, and the fail-fast behavior.RUTSTER_DRAIN_DEADLINE_SECS=600present..github/workflows/ci.ymlhasimage-build+smoke+smoke-compose+smoke-reloadjobs; the existingfmt/clippy/test/deny/sim-bench/twilio-livejobs are unchanged..github/workflows/publish-images.yml(slice F) publishes all four images togit.adlee.work/alee/rutster-*onv*tag push, gated on[fmt, clippy, test, deny, sim-bench]. Linux/amd64 only.- Caddy-reload-during-live-call smoke (Task 9) asserts zero frames dropped across a
caddy reloadmid-call — thestream_close_delay 24hmitigation made CI-regressed against caddy #6420/#7222 (TLS brief §5 risk 1; spec §9 reload-during-call smoke). - All-in-one TLS smoke (Task 8) boots
rutster-allinonewithRUTSTER_LOCAL_CERTS=true, extracts the internal-CA root cert from/data, opens a wss:// WS to/twilio/media-stream, sends Twilio's connected+start handshake, streams 200 PCM frames at 20 ms cadence, asserts at least one engine-originated text frame — proving the real edge→FOB WS path works end-to-end through TLS. - Compose smoke (Task 9) boots all four services via
docker compose up -d --wait, asserts all four running +/healthz+/readyzvia Caddy TLS. - TODO slice-C named hook (Task 6 Step 3) in
deploy/smoke/compose_smoke.sh— theValkeyEventSinksmoke assertion is deferred to slice C (onegrep -n 'TODO slice-C'match). - 11 commits, each signed off —
git log --format='%h %s' main..HEADshows the slice-B series;git log main..HEAD --format='%(trailers:key=Signed-off-by)' | grep -c 'Signed-off-by'equals the commit count (Task 11 Step 6). - PR opened via
teaper house workflow (Task 11 Step 7); the PR description names the plan-A dependency root + the slice-C TODO hook + the squash-merge default.
Out-of-scope reminder (spec §1.2 — do NOT add in this slice, named-deferral boundary)
- rustls Phase 1 in-process TLS (BYO-cert) — slice C (
.github/workflows/plans/2026-07-05-deploy-c-rustls-tls.md, path may vary). TheRUTSTER_TLS_CERT/RUTSTER_TLS_KEYenv vars are noted in.env.exampleas commented future stubs. /metricsendpoint — slice D.RUTSTER_METRICS_BINDis noted as a future var.ValkeyEventSink— slice E.RUTSTER_VALKEY_URLis noted as a future var; the compose-smoke TODO slice-C hook documents the contract the slice-C/E work fills.- arm64 images — named deferral per ADR-0011 §1.2; the publish workflow pins
linux/amd64. - T3 fleet artifacts (presence heartbeats, directory redirects, placement code) — ADR chapter only this epoch.
- Object-storage CDR pipeline —
ValkeyEventSinkis evidence preservation, not the billing ledger (ADR-0005 source-of-truth rule); CDR→object storage stays deferred. - In-binary ACME (rustls Phase 2) — behind the four named triggers in ADR-0011.
- Cargo-deny-adjacent image license scan — bundled-binary licenses are aggregation-clean
vs GPL-3 (noted in
.env.exampleheader); the automated assertion is a future CI muscle.
If an agent proposes adding any of these in slice B, the right answer is "no — see spec §1.2 / ADR-0011."