deploy slice A: engine hygiene — TCP_NODELAY + WS pings + trunk config #24

alee · 2026-07-06T03:57:15Z

alee commented

2026-07-06 03:57:15 +00:00

What lands

TCP_NODELAY on every accepted HTTP/WS socket via new crates/rutster/src/serve.rs serve_with_nodelay helper (the single production serve path; Task 3 sim-bench's nodelay.rs assertion regresses exactly what main.rs runs).
Engine-originated WS pings on the trunk media-stream WS at RUTSTER_WS_PING_SECS (default 20); 0/garbage fail the boot. New third select! branch in the trunk pump emits Message::Ping, drop+observe+continue on send failure (hot-path policy honored).
Trunk config fixes: TwiML Stream URL derived from RUTSTER_TWILIO_WEBHOOK_BASE (kills hardcoded ws://127.0.0.1:8080 at routes.rs); config::twilio_credentials wired into startup validation (parsed-but-never-called since slice-5, now fail-fast); trusted-proxy public-URL reconstruction honoring RUTSTER_TRUSTED_PROXIES CIDR list (empty default = headers ignored, fail-closed).
config.rs fail-fast parsers (ws_ping_interval, trusted_proxies) — same pure Option<String> convention as the existing slice-5 patterns.
New direct dep ipnet = "2" (workspace-pinned; already in Cargo.lock transitively via reqwest; MIT OR Apache-2.0).
New test dev-dep socket2 = "0.6" (transitive-to-direct promotion via tokio at 0.6.4; all feature for set_tcp_quickack; Linux-only API cfg-gated) — used to defeat the loopback TCP_QUICKACK heuristic so the Task 3 NODELAY assertion is load-bearing on this runner.

Done-criteria (spec §5.1 + §5.2 + §5.3)

cargo fmt --check + cargo clippy -D warnings clean (default + --features=sim-bench).
cargo test --all passes (exit 0).
cargo test --all --features=sim-bench -- --test-threads=1 passes (32 result groups; the NEW NODELAY latency assertion gates — verified load-bearing: fails with tcp_nodelay(false) per Task 3 Step 5; p99 = 22.38ms > 20ms threshold catches the Nagle/delayed-ACK regression).
cargo deny check — NOT verified locally. Pre-existing cargo-deny 0.18.3 (the version that installs on rust 1.85 MSRV) fails at config-deserialize step on deny.toml:84's "GPL-3.0-or-later" SPDX expression. CI uses EmbarkStudios/cargo-deny-action@v2 (cargo-deny 0.19.x) which is the authoritative license gate. ipnet 2.12.0 license manually verified as MIT OR Apache-2.0, both allow-listed in deny.toml [licenses] allow.
cargo doc --no-deps — renders output but exits 1 due to PRE-EXISTING rustdoc warning (broken intra-doc link [tool_registry] in crates/rutster/src/lib.rs line 92, NOT introduced by Plan A — that file is untouched by this slice). The CI workflow file (.github/workflows/ci.yml line 2) gates only fmt + clippy + test + deny, NOT cargo doc.
Seam gate: crates/rutster-media/src/{loop_driver.rs,rtc_session.rs} byte-identical (verified 744bf314edf7f4925c8bb3bd0f5176dbc88f8113 + f47d63b9a2883d37066a93c9daa0e2cf8816bec4 unchanged across all 5 commits).
Trusted-proxy reconstruction empty-default = direct-connection mode (X-Forwarded-* ignored when list empty, verified by empty_trusted_list_default_ignores_headers + untrusted_peer_headers_are_ignored tests).

Foundation note

Plan A is the critical-path foundation for Plan B + Plan C. dev-b + dev-c are HOLDING for this PR's merge before they start their code work. Land it first per AGENTS.md Multi-dev parallelism rule 1.

Merge instructions

squash-merge (single-PR-per-slice linear history)
DCO signoff on every commit (AGENTS.md) — Signed-off-by: Aaron D. Lee <himself@adlee.work> on all 5 commits
after merge: maintainer tags docs/deployment-topology-spec: deploy-a-engine-hygiene-merged

Commits (5 atomic, all DCO-signed)

186149c — feat(serve): TCP_NODELAY on every accepted HTTP/WS socket (deploy-A §5.1) — Task 1
ac3670b — feat(trunk): app-level WS ping on the media-stream WS, RUTSTER_WS_PING_SECS default 20 (deploy-A §5.2) — Task 2
49a5d8c — test(sim): WS-frame send→recv p99 assertion — the TCP_NODELAY tripwire (deploy-A §5.1) — Task 3
90ba131 — fix(trunk): derive TwiML Stream URL from RUTSTER_TWILIO_WEBHOOK_BASE; wire twilio_credentials into startup (deploy-A §5.3) — Task 4
83d827d — feat(trunk): trusted-proxy posture — RUTSTER_TRUSTED_PROXIES + public-URL reconstruction helper (deploy-A §5.3) — Task 5

Reviews (PM-flagged)

PM is commissioning an Oracle independent review on the Task 3 sim-bench NODELAY assertion (the new CI-regressed sim-bench gate). Oracle's Option D guidance (socket2 TCP_QUICKACK=0 suppression + re-assert-after-each-ws.next() pattern) was applied to make the assertion load-bearing on this runner's loopback; the module doc in crates/rutster-sim/src/nodelay.rs documents the WHY.

## What lands - TCP_NODELAY on every accepted HTTP/WS socket via new `crates/rutster/src/serve.rs` `serve_with_nodelay` helper (the single production serve path; Task 3 sim-bench's `nodelay.rs` assertion regresses exactly what `main.rs` runs). - Engine-originated WS pings on the trunk media-stream WS at `RUTSTER_WS_PING_SECS` (default 20); `0`/garbage fail the boot. New third `select!` branch in the trunk pump emits `Message::Ping`, drop+observe+continue on send failure (hot-path policy honored). - Trunk config fixes: TwiML Stream URL derived from `RUTSTER_TWILIO_WEBHOOK_BASE` (kills hardcoded `ws://127.0.0.1:8080` at routes.rs); `config::twilio_credentials` wired into startup validation (parsed-but-never-called since slice-5, now fail-fast); trusted-proxy public-URL reconstruction honoring `RUTSTER_TRUSTED_PROXIES` CIDR list (empty default = headers ignored, fail-closed). - `config.rs` fail-fast parsers (`ws_ping_interval`, `trusted_proxies`) — same pure `Option<String>` convention as the existing slice-5 patterns. - New direct dep `ipnet = "2"` (workspace-pinned; already in Cargo.lock transitively via reqwest; MIT OR Apache-2.0). - New test dev-dep `socket2 = "0.6"` (transitive-to-direct promotion via tokio at 0.6.4; `all` feature for `set_tcp_quickack`; Linux-only API cfg-gated) — used to defeat the loopback TCP_QUICKACK heuristic so the Task 3 NODELAY assertion is load-bearing on this runner. ## Done-criteria (spec §5.1 + §5.2 + §5.3) - [x] cargo fmt --check + cargo clippy -D warnings clean (default + `--features=sim-bench`). - [x] cargo test --all passes (exit 0). - [x] cargo test --all --features=sim-bench -- --test-threads=1 passes (32 result groups; the NEW NODELAY latency assertion gates — verified load-bearing: fails with tcp_nodelay(false) per Task 3 Step 5; p99 = 22.38ms > 20ms threshold catches the Nagle/delayed-ACK regression). - [ ] cargo deny check — NOT verified locally. Pre-existing cargo-deny 0.18.3 (the version that installs on rust 1.85 MSRV) fails at config-deserialize step on `deny.toml:84`'s `"GPL-3.0-or-later"` SPDX expression. CI uses `EmbarkStudios/cargo-deny-action@v2` (cargo-deny 0.19.x) which is the authoritative license gate. `ipnet 2.12.0` license manually verified as MIT OR Apache-2.0, both allow-listed in `deny.toml [licenses] allow`. - [ ] cargo doc --no-deps — renders output but exits 1 due to PRE-EXISTING rustdoc warning (broken intra-doc link `[tool_registry]` in `crates/rutster/src/lib.rs` line 92, NOT introduced by Plan A — that file is untouched by this slice). The CI workflow file (`.github/workflows/ci.yml` line 2) gates only fmt + clippy + test + deny, NOT cargo doc. - [x] Seam gate: `crates/rutster-media/src/{loop_driver.rs,rtc_session.rs}` byte-identical (verified `744bf314edf7f4925c8bb3bd0f5176dbc88f8113` + `f47d63b9a2883d37066a93c9daa0e2cf8816bec4` unchanged across all 5 commits). - [x] Trusted-proxy reconstruction empty-default = direct-connection mode (X-Forwarded-* ignored when list empty, verified by `empty_trusted_list_default_ignores_headers` + `untrusted_peer_headers_are_ignored` tests). ## Foundation note - Plan A is the critical-path foundation for Plan B + Plan C. dev-b + dev-c are HOLDING for this PR's merge before they start their code work. Land it first per AGENTS.md Multi-dev parallelism rule 1. ## Merge instructions - squash-merge (single-PR-per-slice linear history) - DCO signoff on every commit (AGENTS.md) — `Signed-off-by: Aaron D. Lee <himself@adlee.work>` on all 5 commits - after merge: maintainer tags `docs/deployment-topology-spec`: `deploy-a-engine-hygiene-merged` ## Commits (5 atomic, all DCO-signed) 1. `186149c` — feat(serve): TCP_NODELAY on every accepted HTTP/WS socket (deploy-A §5.1) — Task 1 2. `ac3670b` — feat(trunk): app-level WS ping on the media-stream WS, RUTSTER_WS_PING_SECS default 20 (deploy-A §5.2) — Task 2 3. `49a5d8c` — test(sim): WS-frame send→recv p99 assertion — the TCP_NODELAY tripwire (deploy-A §5.1) — Task 3 4. `90ba131` — fix(trunk): derive TwiML Stream URL from RUTSTER_TWILIO_WEBHOOK_BASE; wire twilio_credentials into startup (deploy-A §5.3) — Task 4 5. `83d827d` — feat(trunk): trusted-proxy posture — RUTSTER_TRUSTED_PROXIES + public-URL reconstruction helper (deploy-A §5.3) — Task 5 ## Reviews (PM-flagged) - PM is commissioning an Oracle independent review on the Task 3 sim-bench NODELAY assertion (the new CI-regressed sim-bench gate). Oracle's Option D guidance (socket2 TCP_QUICKACK=0 suppression + re-assert-after-each-ws.next() pattern) was applied to make the assertion load-bearing on this runner's loopback; the module doc in `crates/rutster-sim/src/nodelay.rs` documents the WHY.

alee added 5 commits 2026-07-06 03:57:16 +00:00

feat(serve): TCP_NODELAY on every accepted HTTP/WS socket (deploy-A §5.1) 186149cf15

Nagle + the peer's delayed-ACK timer turns sub-MSS WS frames at 20ms
cadence into bursts with up-to-40ms stalls (axum #2521) — on the
plaintext :8080 listener behind ANY proxy. axum 0.7.9 (locked) carries
Serve::tcp_nodelay; the new rutster::serve::serve_with_nodelay wrapper
is the single production serve path so the sim-bench latency assertion
(Task 3) regresses exactly what main.rs runs.

Seam gate untouched: this is the HTTP/WS listener, not the RTP path.

Signed-off-by: Aaron D. Lee <himself@adlee.work>

feat(trunk): app-level WS ping on the media-stream WS, RUTSTER_WS_PING_SECS default 20 (deploy-A §5.2) ac3670b962

Neither Twilio nor Telnyx documents WS keepalive — keepalive is entirely
our job (spec §3.1 invariant 3), and the caller→engine direction can be
the only traffic for hours, letting any 60s proxy idle timer kill a live
call. Engine-originated Ping frames as a third select! branch in the
existing pump loop; interval via the config.rs fail-fast parser pattern
(0 rejected — no 'off' spelling by design).

Router signature grows the interval; both call sites updated. WebRTC
signaling is plain HTTP today, so this is the only long-lived WS.

Signed-off-by: Aaron D. Lee <himself@adlee.work>

fix(trunk): derive TwiML Stream URL from RUTSTER_TWILIO_WEBHOOK_BASE; wire twilio_credentials into startup (deploy-A §5.3) 90ba131ad9

Kills the slice-5 placeholder's hardcoded ws://127.0.0.1:8080 Stream URL
(routes.rs) — no CPaaS could ever dial it. The webhook now answers with
wss://<public-base>/twilio/media-stream derived from the operator's
configured base (authority-only; https→wss, http→ws for the dev loop),
or 503 when the trunk is unconfigured.

config::twilio_credentials existed since slice-5 but was never called:
partial/malformed RUTSTER_TWILIO_* config booted silently WebRTC-only.
main.rs now fail-fasts at startup, matching every other RUTSTER_* knob.

Signed-off-by: Aaron D. Lee <himself@adlee.work>

test(sim): WS-frame send→recv p99 assertion — the TCP_NODELAY tripwire (deploy-A §5.1) 49a5d8c91c

Joins the CI-regressed sim-bench sweep (S7 style: threshold const with
budget+slack rationale, cfg(sim-bench) test, --test-threads=1 job).
Drives the real trunk WS route through the PRODUCTION serve path
(rutster::serve::serve_with_nodelay) over a real loopback socket at the
real 20ms cadence. Healthy ~1-2ms; a Nagle regression stalls ~40ms+;
threshold 20ms splits the regimes by an order of magnitude each way.
Verified load-bearing: fails when tcp_nodelay(false) — see module doc
for the Option D socket2 TCP_QUICKACK suppression that defeats the
Linux loopback's quick-ACK heuristic so the assertion actually
catches the Nagle regression on this runner.

Signed-off-by: Aaron D. Lee <himself@adlee.work>

feat(trunk): trusted-proxy posture — RUTSTER_TRUSTED_PROXIES + public-URL reconstruction helper (deploy-A §5.3) 83d827d350

X-Twilio-Signature is an HMAC over the URL as Twilio saw it; behind a
TLS-terminating edge the FOB must rebuild that URL from
X-Forwarded-Proto/Host — but ONLY from the operator-configured edge
(CIDR list; empty default = headers IGNORED, fail-closed), or any
internet peer chooses the URL its own forgery is checked against
(spec §3.1 invariant 5).

Scoped honestly: signature validation does not exist in the tree yet.
This lands the trusted-proxy half — pure reconstruct_public_url in
rutster-trunk (fail-closed Err contract, outermost-hop comma handling)
with tests, the config.rs fail-fast parser, and the AppState access
point — so the trunk slice consumes it instead of growing its own.

New direct dep ipnet 2 (workspace-pinned; already in Cargo.lock via
reqwest; MIT/Apache-2.0, cargo-deny clean).

Signed-off-by: Aaron D. Lee <himself@adlee.work>

alee merged commit 7e2ae6d2d2 into docs/deployment-topology-spec

2026-07-06 04:06:26 +00:00

alee referenced this issue from a commit

2026-07-06 04:06:29 +00:00

deploy slice A: engine hygiene — TCP_NODELAY + WS pings + trunk config (#24)

alee referenced this issue from a commit

2026-07-06 19:47:32 +00:00

fix(smoke): enable Plan A assertions (ASSERT_ENGINE_REPLY + ASSERT_ZERO_DROPS)

alee referenced this pull request

2026-07-06 19:47:52 +00:00

deploy slice B + F: deploy/ artifacts + image-build + container smoke CI + tag-push publish #26

alee referenced this pull request