# Rutster slice 1 — WebRTC media loopback: the media-core proof - **Status:** Draft (pending review) - **Date:** 2026-06-28 - **Spearhead step:** 1 of 6 (vision-revision §10 / PORT_PLAN "Phasing") - **Origin:** brainstorming session 2026-06-28 - **Related:** [ADR-0002](../../adr/0002-north-star-and-fused-core.md) (fused vertical), [ADR-0003](../../adr/0003-sip-rust-native-trunk.md) (Rust-native stack stance), [ADR-0004](../../adr/0004-license.md) (GPL-3.0-or-later), [ADR-0006](../../adr/0006-ingress-posture.md) (WebRTC-first ingress) --- ## TL;DR Stand up the Rutster Rust workspace and implement spearhead step 1: a browser talks WebRTC to the core; the core terminates DTLS-SRTP, decodes Opus → canonical 16-bit PCM @24 kHz mono, and echoes the PCM back through Opus → DTLS-SRTP to the browser. The user speaks and hears themselves back with no perceptible delay. Slice 1 proves the **media core**: RTP/SRTP termination, codec-to-PCM boundary, the canonical PCM frame, and the sans-IO polling posture. It deliberately omits the tap, the brain, barge-in, the trunk, and the spend cap — steps 2–6 of the spearhead — though it pre-paves the tap by exposing the PCM boundary as a clean trait seam. --- ## 1. Scope ### 1.1 In scope - Cargo workspace scaffold shaped to ADR-0002's fused per-call vertical. - Implementation of spearhead step 1: WebRTC media loopback with Opus⇄PCM termination. - One real-but-minimal crate (`rutster-call-model`): the `Channel` leg object embryo. - Three stub crates (`rutster-signaling-sip`, `rutster-tap`, `rutster-spend`) that lock future boundaries without anticipating code. - axum-based signaling server + a self-contained browser test client. - `cargo-deny`, CI workflow, thorough learner-facing code comments. ### 1.2 Out of scope (with scheduled return) | Deferred item | Returns in | Why deferred | |---|---|---| | Dedicated timing thread for the 20 ms media loop | Spearhead step 4 (barge-in) | ARCHITECTURE.md mandates "never the shared tokio pool"; honored once a reflex needs determinism. Slice 1 has no reflex to time against — a dedicated thread now would be theater. | | TLS on the HTTP signaling surface | Spearhead step 5 (PSTN trunk) | Dev loop is loopback-only; the real security surface (DTLS-SRTP media) is already mandatory. TLS needs a cert story (Vault/KMS per ARCHITECTURE.md) with no home in slice 1. | | Authn / authz / multi-tenancy on `/v1/sessions` | Spearhead step 6 (spend cap) | The boundary's auth posture lands when the spend gate does; together they constitute the trust boundary. | | Trickle ICE | When real-world NATs demand it (likely step 5) | Non-trickle (one POST with offer+candidates, one response with answer+candidates) suffices for local loopback and keeps the dev loop zero-dependency. | | The tap itself (audio routing to an external echo process) | Spearhead step 2 | Slice 1 *pre-paves* the tap by exposing the PCM boundary as `AudioSource`/`AudioSink` traits in `rutster-media`; step 2 implements the WSS tap client behind that seam. | | The brain (STT/LLM/TTS) | Spearhead step 3 | Slice 1 echoes; step 2 swaps echo for an external process, step 3 swaps echo-process for a real brain. | | Barge-in / VAD-driven playout kill | Spearhead step 4 | No reflex to enforce yet; no VAD even on the inbound side. | | PSTN trunk (SIP client) | Spearhead step 5 | WebRTC-only ingress in slice 1; ADR-0003's Rust-native trunk SIP lands with the trunk integration. | | Spend cap / abuse gate | Spearhead step 6 | No trunk yet to gate spend against. | | CDR emission, event bus, OTel traces beyond the per-channel `tracing` span | Later rungs | PORT_PLAN keeps these as services *around* the core; slice 1 has one peer, one channel, no fanout needed. | | Transfer / park / pickup / barge (call features) | Escalation rung 2 | `Channel` is shaped to accept `Option`, `audiohooks: Vec` by addition, so these slot in without breaking slice 1. High on post-slice-1 list. | | Browser-based automated e2e test (Selenium/Playwright) | Post-slice-1 when latency matters | Would balloon the dev loop for a thin-slice proof. Manual test plan documents success criteria instead. | | Latency benchmarking harness | Spearhead step 4 | Latency matters when barge-in needs to beat the brain round-trip; slice 1's bar is "no perceptible delay" (~≤200 ms), not sub-10 ms. | | Fuzz harnesses for wire parsers | Spearhead step 5 (SIP/SDP/RTP) | PORT_PLAN §10 mandates continuous fuzzing of every wire parser; slice 1 has no hostile-bytes surface (browser is trusted) and the fuzz story lands with the SIP trunk. A `fuzz/` placeholder dir pre-paves the layout. | | Resumability / re-INVITE / session migration | Later | Refresh the page → new session. Acceptable for dev loop. | ### 1.3 What this slice does NOT prove It does **not** prove: barge-in, latency determinism, the tap interface itself (only its seam), PSTN trunking, or spend control. It proves **only** the media-core termination + the codec-to-PCM boundary. Every "is X done?" question is answered by the out-of-scope table above. --- ## 2. Workspace layout Cargo workspace at the repo root. One binary crate and five library crates shaped to the ADR-0002 fused vertical. Every crate manifest sets `license = "GPL-3.0-or-later"` and carries an SPDX header (ADR-0004). ``` rutster/ ├── Cargo.toml # [workspace], shared deps via [workspace.dependencies] ├── deny.toml # cargo-deny config: licenses, advisories, bans, sources ├── rust-toolchain.toml # pinned stable; MSRV confirmed against str0m at impl time ├── LEARNING.md # index of "to learn concept X, read file Y" ├── .github/workflows/ci.yml # fmt, clippy -D warnings, test --all, cargo deny check ├── crates/ │ ├── rutster/ # binary: axum signaling server + media driver + static page │ ├── rutster-media/ # REAL slice-1 code: str0m WebRTC + Opus⇄PCM boundary │ ├── rutster-call-model/ # REAL-but-minimal: the Channel/Leg object embryo │ ├── rutster-signaling-sip/ # STUB: doc comment + compile test (step 5 fills in) │ ├── rutster-tap/ # STUB: doc comment + compile test (step 2 fills in) │ └── rutster-spend/ # STUB: doc comment + compile test (step 6 fills in) └── fuzz/ # placeholder cargo-fuzz harness dir (real harnesses: step 5) ``` ### 2.1 Workspace dependency pattern `[workspace.dependencies]` in the root manifest pins versions; member crates reference them with `dep.workspace = true`. Keeps versions unified as crates fill in and prevents accidental version drift caught late. ### 2.2 Stub crate policy `rutster-signaling-sip`, `rutster-tap`, `rutster-spend` each ship as: - `lib.rs` with a `//!` module doc comment: what the crate will hold, why it's deferred, and which spearhead step fills it in. - A `#[cfg(test)] mod tests { #[test] fn crate_compiles() {} }` so CI exercises them. Stubs **do not** anticipate code. They lock the boundary shape; that is their only job. ### 2.3 Dependency direction - `rutster` (binary) depends on `rutster-media`, `rutster-call-model`. - `rutster-media` depends on `rutster-call-model` (the `ChannelId` / `Channel` types). - `rutster-call-model` depends on nothing in the workspace (leaf). - `rutster-tap`, `rutster-spend`, `rutster-signaling-sip` depend on nothing in slice 1 (their future dependency direction is documented in their `//!` comments). --- ## 3. Media core (`rutster-media`) ### 3.1 Components - **`RtcSession`** — owns a `str0m::Rtc` instance (the main sans-IO driver type; str0m has no `Live` struct) plus the per-peer codec state (an Opus decoder + an Opus encoder). One per WebRTC peer. - **`PcmFrame`** — the canonical tap format from ARCHITECTURE.md: 16-bit signed mono PCM @ 24 kHz, fixed frame size (20 ms = 480 samples). This is the single format every future brain/tap consumer speaks. Defined here in slice 1; `rutster-tap` will re-export it once that crate fills in (step 2 — keeps one canonical home). - **Codec pair** — `opus` crate (libopus via FFI; PORT_PLAN §7 disposition for Opus is 🦀 Core (FFI)). Decoder: `opus_decode` → fills a `PcmFrame`; Encoder: `opus_encode` from ring-buffered PCM → Opus payload for str0m's Frame API. ### 3.2 Loop shape (Approach A from brainstorming) str0m exposes two API surfaces: the **Frame API** (default; str0m handles packetization internally, applications push/pop decoded media frames) and the **RTP API** (opt-in via `RtcConfig::set_rtp_mode(true)`; raw RTP packets, the app does packetization). Slice 1 uses the **Frame API** because slice 1's goal is to prove the codec-to-PCM boundary, not to prove RTP packetization. The Frame API gives us decoded audio frames as `MediaData` and accepts encoded audio back; str0m does the RTP/SRTP framing underneath. RTP API is a candidate for step 5 (PSTN trunk) where raw-packet control over the SIP/RTP boundary may matter. ``` str0m Rtc (sans-IO, polled on tokio — see §3.4) ↓ Event::RtpPacket(RtpPacket) { …, payload } (single struct; RID is a field if present) ↓ str0m delivers decoded audio as MediaData via the Frame API decode Opus payload → PcmFrame (24 kHz mono, 20 ms) ↓ buffer PCM into playout ring (slice 1: echo; step 2 taps here) ↓ on each 20 ms tick take 480 samples → encode Opus → push to str0m via Rtc::writer(mid)->Writer::write ``` > **API-surface notes (verified against str0m 0.21 docs):** > - `str0m::Rtc` is the main sans-IO driver; there is no `Live` type. `Rtc::handle_input(Input)` > feeds network input; `Rtc::poll_output() -> Output` is the poll interface, where > `Output::Timeout(Instant)` gives the next-deadline signal we sleep tokio until. > - `Event::RtpPacket(RtpPacket)` is a single-struct variant (RID is a field on `RtpPacket`, > not a second variant payload). > - Inbound/`Event::RtpPacket`: in RTP mode you get raw packets; in slice 1's Frame-mode > the canonical inbound event is `Event::MediaData(MediaData)` containing decoded media. > - Outbound: `StreamTx::write_rtp` exists in RTP mode only; under the Frame API the path is > `Rtc::writer(mid) -> Writer::write(...)`. ### 3.3 The PCM-tap seam The point where decoded `PcmFrame`s emerge from the inbound side and where `PcmFrame`s get enqueued on the outbound side is the **exact point** step 2 splices the tap. Slice 1 echoes both ends together; the seam is made explicit via two traits in `rutster-media`, not buried inside `RtcSession` internals: ```rust pub trait AudioSource: Send + Sync { /// Take the next PCM frame to send to the peer. None = silence. fn next_pcm_frame(&mut self) -> Option; } pub trait AudioSink: Send + Sync { /// Receive a decoded PCM frame from the peer. Must not block. fn on_pcm_frame(&mut self, frame: PcmFrame); } ``` Slice 1 wires an `EchoAudioPipe` (implements both traits) between sink and source. Step 2 replaces `EchoAudioPipe` with a real tap client. No code changes to `RtcSession` itself in step 2 — that's the test of the seam. **Who drives the traits:** the poll loop (§3.4) drives both. On each `poll_output()` cycle: inbound `Event::MediaData` → decode → `sink.on_pcm_frame(PcmFrame)`; the outbound side runs on the `Output::Timeout(Instant)` deadline → `source.next_pcm_frame()` → encode → push to str0m. `RtcSession` owns the `Arc>` (or a concrete `EchoAudioPipe` in slice 1 — no trait object needed yet; step 2 introduces indirection when a real tap client replaces the pipe). ### 3.4 Polling & the tokio-vs-dedicated-thread deviation str0m is sans-IO; its `Rtc` API exposes `handle_input(Input)` (feed network input) and `poll_output() -> Output` (the poll interface, where `Output::Timeout(Instant)` gives the next deadline we sleep tokio until). Slice 1 runs this poll on the tokio runtime. **This is an explicit, documented deviation from ARCHITECTURE.md**, which mandates *dedicated timing threads, never the shared tokio pool*. The deviation is **scoped to slice 1 only** and is acceptable because: 1. Slice 1 has no reflex to time against (no VAD, no barge-in). 2. A dedicated timing thread now would be theater — there is nothing whose determinism needs defending. 3. Step 4 (barge-in) is the scheduled landing point for the dedicated thread. It is listed in the out-of-scope table. The code shape (a single poll function called from a tokio task) makes the step-4 swap to a dedicated thread a localized change, not a re-architecture. The deviation is called out in code (`// DEV-DEVIATION: tokio polling accepted for slice 1; step 4 replaces with dedicated timing thread per ARCHITECTURE.md.`) and in this spec so it cannot be forgotten. ### 3.5 SRTP / DTLS DTLS-SRTP is mandatory from slice 1 (Security-as-product pillar). str0m handles DTLS+ SRTP natively; configured via `RtcConfig::set_dtls_cert` at startup. If no cert is passed, str0m auto-generates one — in slice 1 we explicitly pass our self-signed DTLS cert (see §3.6) to make the cert ownership clear, but the auto-gen path is also acceptable for a dev loop. **No plaintext RTP fallback path exists** in code or config. This is a deliberate floor, not a default. ### 3.6 DTLS certificate Self-signed DTLS cert generated at startup, held in memory, **passed explicitly to `RtcConfig::set_dtls_cert`** (per §3.5 — making cert ownership clear in code rather than relying on str0m's auto-gen default). No cert file on disk in slice 1 (cert rotation / persistence is a step-5 / trunk-integration concern). The cert is regenerated on every server restart — acceptable for slice 1 because the browser fetches a fresh SDP answer each session, so the DTLS fingerprint in the SDP always matches the current cert. No caching layer exists to go stale. ### 3.7 Codec negotiation Answer-only (browser initiates the offer). We advertise Opus in SDP for our single audio m-line; reject video m-lines. The slice's SDP answer is the **embryo of the future SIP SDP path** (PORT_PLAN §1 `res_pjsip_session` + `_sdp_rtp` rows). The SDP code lives in `rutster-media` as a private module (not in `rutster-signaling-sip`) because the SDP we manipulate is WebRTC-ICE-coupled, not SIP-coupled. The future SIP/SDP negotiation lives in `rutster-signaling-sip` (step 5) and may extract shared code from this module at that point. Slice 1's SDP module is a focused ~50-line mapper: parse the browser offer, extract the audio m-line, munge into our answer (Opus only, recv+send, DTLS fingerprint from our cert, ICE ufrag/pwd via str0m's `IceCreds` / `Candidate` types at the crate root — str0m has no public `str0m::ice` module; the ICE-relevant public surface lives at `str0m::`: `Candidate`, `CandidateKind`, `IceCreds`, `IceConnectionState`). ### 3.8 Error handling on the hot path Decode/encode failures are logged + counted via a minimal metric counter. A dropped packet does **not** terminate the peer. Panic-on-bad-bytes is **never** the policy. The slice is short of the fuzzed-parser bar (see out-of-scope) but the posture is "drop + observe, don't crash" so the eventual fuzz harness has clean semantics to test against. Hot-path errors use `thiserror`-derived error enums; the `?` operator is used liberally on the cold path (signaling, setup) but **never** on the 20 ms loop itself (where the policy is match-and-continue). ### 3.9 Default PCM format choice 24 kHz mono, 16-bit signed. Reasons: - 24 kHz is a speech-model-friendly rate (matches Opus's typical wideband mode and common STT input expectations). - ARCHITECTURE.md names 24 kHz default, 16 kHz fallback — slice 1 ships the default only; the fallback is a future-rung concern when the first brain needs it. - Mono: telephony is mono. Stereo costs 2x for no contact-center value. --- ## 4. Signaling & browser ingress (`rutster` binary) ### 4.1 HTTP surface (slice 1) - `POST /v1/sessions` → mint a `RtcSession` (which owns a fresh `Channel`; the `ChannelId` *is* the session id). Returns `{ "session_id": "" }`. - `POST /v1/sessions/:id/offer` (body: browser SDP offer, `Content-Type: application/sdp`, including all ICE candidates — non-trickle) → core produces SDP answer (including its ICE candidates), feeds candidates to str0m, returns the answer as `application/sdp`. - `DELETE /v1/sessions/:id` → tear down: transition `Channel` to `Closing → Closed`, drop the `RtcSession`, close the peer connection cleanly via str0m. - `GET /` → serve the static HTML test client. There is **no separate `/ice` endpoint** in slice 1. Non-trickle ICE bundles all candidates into the SDP offer/answer exchange, so one POST (`/offer`) carries everything. A separate `/ice` endpoint is a step-5 concern (trickle ICE — see the out-of-scope table). ### 4.2 ICE strategy Non-trickle ICE. Browser gathers all candidates, sends offer+candidates in one POST, core returns answer+candidates in one response. One round-trip, simpler server. Trickle ICE is deferred (see out-of-scope table) until real-world NATs demand it, likely with the PSTN trunk in step 5. ### 4.3 Binding & security posture - `0.0.0.0:8080` plaintext HTTP for the **dev loop only**. No TLS in slice 1 — see the out-of-scope rationale. - No authn/authz on `/v1/sessions` in slice 1. No multi-tenancy. Both land with the spend-gate step (step 6) and a real deployment posture. - `Access-Control-Allow-Origin: *` on the static page only; the API is same-origin by virtue of being served from the same host. (No CORS preflight needed because the test client is same-origin.) ### 4.4 Browser test client (`GET /`) A single self-contained HTML file with inline JS, no build step. Behavior: 1. `getUserMedia({ audio: true })`. 2. `new RTCPeerConnection` with a STUN config of `[]` (host candidates only — no STUN server needed for local loopback, keeping the dev loop zero-dependency). 3. Create offer, POST to `/v1/sessions`, get back the answer, set as remote. 4. Play the echoed audio back via an `