slice-1: WebRTC media loopback — the media-core proof #1

Merged

alee merged 12 commits from slice-1-webrtc-loopback into main 2026-06-28 17:20:24 +00:00

Conversation 0 Commits 12 Files Changed 30 +3786 -7

alee commented 2026-06-28 17:15:54 +00:00

Owner

Copy Link

Implements spearhead step 1 of the Rutster build: a browser talks WebRTC to the core; the core terminates DTLS-SRTP, decodes Opus → canonical 16-bit PCM @ 24 kHz mono, echoes the PCM back to the browser.

What lands

• Workspace (ADR-0002-fused vertical shape): one binary + five library crates with [workspace.dependencies] pinning; rust-toolchain.toml pinned to 1.85 (edition 2024 floor, mandated by uuid 1.x → getrandom 0.4.x).
• rutster-media — the media core: str0m 0.21 sans-IO WebRTC + Opus⇄PCM boundary. PcmFrame (canonical 480-sample 24 kHz mono), AudioSource/AudioSink trait seam (pre-paves the step-2 tap), EchoAudioPipe slice-1 wiring, OpusDecoder/OpusEncoder with hot-path match-and-continue (no ? on the 20 ms loop, no unwrap).
• rutster-call-model — the Channel/ChannelId/ChannelState embryo (signaling-state only; media stays internal to rutster-media as a leaf concern).
• Three stub crates (rutster-signaling-sip, rutster-tap, rutster-spend) locking ADR-0002 boundaries without anticipating code — each ships a //! doc naming its spearhead step.
• rutster binary — axum 0.7 signaling on 0.0.0.0:8080 plaintext (dev-loop posture), DashMap<ChannelId, Arc<Mutex<RtcSession>>> session store, four routes (POST /v1/sessions, POST /v1/sessions/:id/offer, DELETE /v1/sessions/:id, GET /), single tokio poll task driving all sessions, graceful shutdown via Ctrl-C / SIGTERM, browser test client served via include_str!.
• cargo-deny + CI — gates on fmt, clippy -D warnings, test --all, deny check; matrix stable + 1.85; deny.toml allows permissive Rust-ecosystem licenses + our own GPL-3.0 (workaround for cargo-deny 0.18.x parser bug on the SPDX form on the pinned toolchain — well-documented inline, revert at MSRV ≥ 1.88).
• LEARNING.md — 10 concept-to-file pointers (floor was 5); fuzz/README.md pre-paves the step-5 harness layout.

Process

Built via subagent-driven-development: 7 tasks, fresh implementer + reviewer pair per task, controller applied two cross-task plan corrections (toolchain bump 1.80→1.85, deny.toml parser workaround), final whole-branch review Approved.

Gate status (spec §6.5 done checklist)

• ✅ cargo test --all — 18 tests across the workspace
• ✅ cargo fmt --check — clean
• ✅ cargo clippy -- -D warnings — clean
• ✅ cargo deny check — 3/4 subcommands clean locally on cargo-deny 0.18.3 (licenses, bans, sources); advisories fails locally on CVSS-4.0 entries the older parser can't read, but CI's 0.19+ action parses them fine
• ⏳ Manual browser e2e — pending per implementer environments (no browser available); accepted under spec §1.2's browser-automation deferral. The browser client, the DELETE route, and the Closing→Closed transition are in place; the README §6.5 manual checklist covers verification
• ✅ Every stub crate compiles; doc-comment names its scheduled spearhead step
• ✅ LEARNING.md indexes 10 "to learn X, read Y" pointers

Deviations

• str0m 0.21's accept_offer() produces the SDP answer natively (DTLS fingerprint + ICE creds + Opus codec) — replaced the spec §3.7 sketch's "50-line SDP munger". str0m evolved since the spec was written.
• opus crate 0.3.1 links system libopus via FFI (audiopus_sys) — amends spec §6.3's "no external deps beyond Rust" with PORT_PLAN §7's "Core (FFI)" rationale. libopus-dev is the one system dep; documented in README + installed in CI.
• Pinned Rust 1.85 (was 1.80 in original plan) because uuid's transitive dep getrandom 0.4 requires edition 2024 (stabilized in 1.85). Documented in the rust-toolchain commit + plan's Task 6 CI matrix.
• str0m's transitive rcgen/time deps needed lock-downgrade for rust 1.85 compat (the action's bundled cargo-deny 0.19+ parses fine).
• will block until MSRV lifts past 1.88 flags in deny.toml — RUSTSEC-2026-0009 ignore (time 0.3.41; fix needs rustc 1.88), thiserror skip entries (workspace bump to v2 clears them), wildcards = warn (path deps lack version = field).

Plan: docs/superpowers/plans/2026-06-28-slice-1-webrtc-loopback.md. Spec: docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md.

Out-of-scope (per spec §1.2)

Slice 1 pre-paves the tap seam but does NOT ship the tap (step 2), the brain (step 3), barge-in/VAD (step 4), PSTN trunk (step 5), or spend cap (step 6). Dedicated timing thread deferred to step 4 (the tokio poll-path is a documented DEV-DEVIATION comment in loop_driver.rs). No TLS on the HTTP surface, no authn/authz, no trickle ICE.

Implements spearhead step 1 of the Rutster build: a browser talks WebRTC to the core; the core terminates DTLS-SRTP, decodes Opus → canonical 16-bit PCM @ 24 kHz mono, echoes the PCM back to the browser. ## What lands - **Workspace** ([ADR-0002](docs/adr/0002-north-star-and-fused-core.md)-fused vertical shape): one binary + five library crates with `[workspace.dependencies]` pinning; `rust-toolchain.toml` pinned to 1.85 (edition 2024 floor, mandated by uuid 1.x → getrandom 0.4.x). - **`rutster-media`** — the media core: `str0m 0.21` sans-IO WebRTC + Opus⇄PCM boundary. `PcmFrame` (canonical 480-sample 24 kHz mono), `AudioSource`/`AudioSink` trait seam (pre-paves the step-2 tap), `EchoAudioPipe` slice-1 wiring, `OpusDecoder`/`OpusEncoder` with hot-path match-and-continue (no `?` on the 20 ms loop, no `unwrap`). - **`rutster-call-model`** — the `Channel`/`ChannelId`/`ChannelState` embryo (signaling-state only; media stays internal to rutster-media as a leaf concern). - **Three stub crates** (`rutster-signaling-sip`, `rutster-tap`, `rutster-spend`) locking ADR-0002 boundaries without anticipating code — each ships a `//!` doc naming its spearhead step. - **`rutster` binary** — axum 0.7 signaling on `0.0.0.0:8080` plaintext (dev-loop posture), `DashMap<ChannelId, Arc<Mutex<RtcSession>>>` session store, four routes (`POST /v1/sessions`, `POST /v1/sessions/:id/offer`, `DELETE /v1/sessions/:id`, `GET /`), single tokio poll task driving all sessions, graceful shutdown via Ctrl-C / SIGTERM, browser test client served via `include_str!`. - **`cargo-deny` + CI** — gates on fmt, clippy -D warnings, test --all, deny check; matrix stable + 1.85; `deny.toml` allows permissive Rust-ecosystem licenses + our own GPL-3.0 (workaround for cargo-deny 0.18.x parser bug on the SPDX form on the pinned toolchain — well-documented inline, revert at MSRV ≥ 1.88). - **`LEARNING.md`** — 10 concept-to-file pointers (floor was 5); `fuzz/README.md` pre-paves the step-5 harness layout. ## Process Built via subagent-driven-development: 7 tasks, fresh implementer + reviewer pair per task, controller applied two cross-task plan corrections (toolchain bump 1.80→1.85, deny.toml parser workaround), final whole-branch review Approved. ## Gate status (spec §6.5 done checklist) - ✅ `cargo test --all` — 18 tests across the workspace - ✅ `cargo fmt --check` — clean - ✅ `cargo clippy -- -D warnings` — clean - ✅ `cargo deny check` — 3/4 subcommands clean locally on cargo-deny 0.18.3 (licenses, bans, sources); `advisories` fails locally on CVSS-4.0 entries the older parser can't read, but CI's 0.19+ action parses them fine - ⏳ Manual browser e2e — pending per implementer environments (no browser available); accepted under spec §1.2's browser-automation deferral. The browser client, the `DELETE` route, and the Closing→Closed transition are in place; the README §6.5 manual checklist covers verification - ✅ Every stub crate compiles; doc-comment names its scheduled spearhead step - ✅ `LEARNING.md` indexes 10 "to learn X, read Y" pointers ## Deviations - str0m 0.21's `accept_offer()` produces the SDP answer natively (DTLS fingerprint + ICE creds + Opus codec) — replaced the spec §3.7 sketch's "50-line SDP munger". str0m evolved since the spec was written. - `opus` crate 0.3.1 links system libopus via FFI (audiopus_sys) — amends spec §6.3's "no external deps beyond Rust" with PORT_PLAN §7's "Core (FFI)" rationale. `libopus-dev` is the one system dep; documented in README + installed in CI. - Pinned Rust 1.85 (was 1.80 in original plan) because uuid's transitive dep getrandom 0.4 requires edition 2024 (stabilized in 1.85). Documented in the rust-toolchain commit + plan's Task 6 CI matrix. - str0m's transitive `rcgen`/`time` deps needed lock-downgrade for rust 1.85 compat (the action's bundled cargo-deny 0.19+ parses fine). - `will block until MSRV lifts past 1.88` flags in deny.toml — `RUSTSEC-2026-0009` ignore (time 0.3.41; fix needs rustc 1.88), `thiserror` skip entries (workspace bump to v2 clears them), `wildcards = warn` (path deps lack `version =` field). Plan: `docs/superpowers/plans/2026-06-28-slice-1-webrtc-loopback.md`. Spec: `docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md`. ## Out-of-scope (per spec §1.2) Slice 1 *pre-paves* the tap seam but does NOT ship the tap (step 2), the brain (step 3), barge-in/VAD (step 4), PSTN trunk (step 5), or spend cap (step 6). Dedicated timing thread deferred to step 4 (the tokio poll-path is a documented `DEV-DEVIATION` comment in `loop_driver.rs`). No TLS on the HTTP surface, no authn/authz, no trickle ICE.

alee added 11 commits 2026-06-28 17:15:55 +00:00

workspace: scaffold + three stub crates (sip/tap/spend) ... 39245a6553

Workspace root, pinned toolchain, and the three stub crates whose only
job in slice 1 is to lock the ADR-0002 boundary shape. Each ships a
lib.rs module doc (what it will hold, why deferred, which spearhead step
fills it) and a crate_compiles test. Spec §2.2.

call-model: Channel + ChannelId + ChannelState (signaling embryo) ... a5d0f90fe2

rutster-call-model is real-but-minimal (spec §5): the unifying leg
object the future API exposes. ChannelId is a Uuid newtype for
type-safety (the slice-1 worked example of the newtype pattern).
Channel is signaling-state only — media lives in rutster-media as a
leaf concern of the Channel, surfaced only when a second consumer needs
to observe it (spec §5.3). ChannelState matches the New→Connecting→
Connected→Closing→Closed flow from §5.4.

toolchain: bump pinned Rust 1.80 → 1.85 (edition 2024 floor) ... 8260c76824

uuid 1.x → getrandom 0.4.x requires edition = "2024", stabilized in Rust 1.85.
The Task 1 pin of 1.80 was written when uuid 1.x resolved to a getrandom that
still compiled on edition 2021; the resolution has shifted. The brief said
'confirm the latest stable at impl time' — bumping the pin is faithful to that.

Plan doc's Task 6 CI matrix also updated: [stable, "1.85"].

Cross-task correction in response to Task 2 implementer's DONE_WITH_CONCERNS
escalation; not a Task 2 code change.

media: PcmFrame + AudioSource/Sink + Opus codec pair ... e09af55e00

PcmFrame is the canonical tap format (16-bit mono @ 24 kHz, 480 samples
per 20 ms frame — ARCHITECTURE.md). AudioSource/AudioSink are the seam
step 2 splices the tap client into (spec §3.3); EchoAudioPipe is the
slice-1 wiring of that seam. OpusDecoder/OpusEncoder wrap the opus
crate's libopus FFI with hot-path match-and-continue (no ? on the 20 ms
loop, spec §3.8); decode/encode return Option<PcmFrame>/Option<Vec<u8>>
so a dropped frame is logged + counted, never propagated to crash the
peer.

lock: pin transitive rcgen/time deps for rust 1.85 ... 27fb64aad5

str0m 0.21's default crypto backend pulls rcgen 0.14.8 and time
0.3.51, both of which require rustc 1.88+. The workspace toolchain
is pinned to 1.85 (edition 2024 floor, the str0m MSRV). Pin rcgen
to 0.14.7 and time to 0.3.41 in Cargo.lock so the str0m tree
builds on the pinned toolchain. No source changes; this only
restricts transitive versions.

media: RtcSession + str0m poll loop (the media core) ... 34c590b48a

RtcSession owns a str0m::Rtc + Opus decoder/encoder + EchoAudioPipe +
a bound UDP socket (spec §3.1, §4.5). accept_offer calls str0m 0.21's
sdp_api().accept_offer() natively — no hand-rolled SDP munger; str0m
fills DTLS fingerprint + ICE creds + Opus codec. The loop driver
drains poll_output per str0m's single-mutation invariant, routes
inbound MediaData through Opus decode + EchoAudioPipe sink, sends
Transmit packets on the UDP socket, and checks the 60 s idle timeout.

DEV-DEVIATION: loop runs on tokio (spec §3.4); step 4 replaces with
a dedicated timing thread per ARCHITECTURE.md.

str0m 0.21 API adjustments from the brief's sketch (verified against
src/str0m-0.21.0):
- SdpAnswer has no .mid() accessor; use answer.media_lines[0].mid()
  via the Deref<Target=Sdp> impl.
- Input::Receive carries the Instant as the first tuple element
  (Input::Receive(now, recv)); handle_input takes a single Input arg.
- Receive constructed via Receive::new(proto, src, dst, buf) (the
  DatagramRecv field is private).
- UDP socket binds 127.0.0.1:0 (Candidate::host rejects the
  unspecified address 0.0.0.0).
- BROWSER_SDP_OFFER fixture restored a=group:BUNDLE 0 (str0m's
  Sdp::assert_consistency rejects SDP without a session group).

binary: axum signaling + DashMap session store + browser test client ... 1f0f64c1c6

Four routes on axum 0.7 per spec §4.1: POST /v1/sessions (mint),
POST /v1/sessions/:id/offer (str0m-native SDP accept), DELETE
/v1/sessions/:id (close), GET / (static HTML client). Session store is
a DashMap<ChannelId, Arc<Mutex<RtcSession>>> (spec §4.5); one tokio
task drives all session poll loops — per-session tasks would pre-pave
the wrong pattern for step 4's dedicated thread. Graceful shutdown
drops the DashMap on Ctrl-C / SIGTERM. Integration test exercises the
REST surface; manual browser e2e per README §6.5.

ci: cargo-deny + GitHub Actions workflow (spec §6.1, §6.2) ... f92e6f467b

deny.toml allows the permissive Rust-ecosystem licenses + our own
GPL-3.0-or-later; bans duplicate versions of tokio/serde/bytes/tracing
to catch dep-tree divergence early; restricts sources to crates.io. CI
runs fmt --check, clippy -D warnings, test --all (matrix: stable +
MSRV 1.85), and cargo deny check on push + PR to main. The CI job
installs libopus-dev — the opus crate's FFI dependency (PORT_PLAN §7
'Core (FFI)' disposition).

docs: LEARNING.md + fuzz/ placeholder + README dev-loop (spec §7, §6.3) ... f765fc2b27

LEARNING.md indexes ten concept-to-file pointers (the spec floor was
five) — the newtype pattern, exhaustive enum match, sans-IO, trait
extension seams, thiserror + hot-path match-and-continue, Arc<Mutex>
vs Arc<RwLock>, DashMap, str0m's single-mutation invariant, graceful
shutdown, include_str!. fuzz/README.md pre-paves the layout (no
hostile-bytes surface in slice 1; harnesses land at step 5 per the
out-of-scope table). README's new dev-loop section documents the
libopus FFI prerequisite and the manual e2e steps.

deny: workaround cargo-deny 0.18.x GPL-3.0-or-later parse bug ... 85a58ab880

cargo-deny 0.18.3 (the latest that installs on rust 1.85) can't parse
the SPDX 'GPL-3.0-or-later' suffix — 'expected a <bare-gnu-license>'.
CI uses cargo-deny-action@v1 (bundles 0.19+) which parses it correctly.

Switching to bare 'GPL-3.0' lets local 'cargo deny check' work on the
pinned toolchain. ADR-0004's GPL-3.0-or-later stance is unchanged; this
is a parser workaround flagged for revert when the MSRV lifts past 1.88
(where cargo-deny 0.19.x installs natively).

Three of four 'cargo deny check' subcommands now pass locally:
  - licenses ok
  - bans ok
  - sources ok
  - advisories: fails locally only because the advisory-db contains CVSS 4.0
    entries (anchor-lang RUSTSEC-2026-0146) that cargo-deny 0.18.3's parser
    rejects. Unfixable on the rust 1.85 toolchain; CI's 0.19+ parses fine.

Surfaced by Task 7's done-checklist run; not a Task 7 code change.

fix: restore doc bar on RtcSession accessors; refresh stale README status ... 3041036d94

Two Important findings from the final whole-branch review:

T4-M3: RtcSession::channel_id/channel_state/is_closed lacked /// docs
(AGENTS.md mandates /// on every public item; learner-facing code is a
primary product). One-line docs added per existing /// style in rtc_session.rs.

T7-M2: README "Status" section still read "Scoping. No code yet" — stale
post-slice-1. Updated to reflect slice-1 implementation status; minimal
edit, surrounding content preserved.

alee added 1 commit 2026-06-28 17:19:22 +00:00

Merge remote-tracking branch 'origin/main' into slice-1-webrtc-loopback ... 52462723d3

# Conflicts:
#	README.md

alee merged commit 9fcdb60548 into main 2026-06-28 17:20:24 +00:00

alee referenced this issue from a commit 2026-06-28 17:20:27 +00:00

Merge pull request 'slice-1: WebRTC media loopback — the media-core proof' (#1) from slice-1-webrtc-loopback into main

alee referenced this issue from a commit 2026-06-28 22:25:58 +00:00

fix(slice-2): widen outer close-bound + strengthen reconnect-test assertion

alee referenced this pull request 2026-06-28 22:29:37 +00:00

Slice 2 — The agent tap: splicing the brain seam #2

alee referenced this issue from a commit 2026-07-02 00:22:01 +00:00

spec+plan(slice-4): revise trigger source — local VAD primary, advisory secondary

alee referenced this pull request 2026-07-03 13:23:49 +00:00

slice-4 (dev-a): Reflex<P> + LocalVadReflex<P> (Task 2 + 2b) #8

alee referenced this issue from a commit 2026-07-03 13:51:55 +00:00

test(slice-4): primary-path barge-in e2e — local VAD kills playout without brain

alee referenced this pull request 2026-07-03 13:52:45 +00:00

slice-4 (dev-a): Task 9 Step 1 — primary-path barge-in e2e (wedge #1) #10

alee referenced this issue from a commit 2026-07-04 01:54:03 +00:00

slice-4 (dev-a): Task 9 Step 1 — primary-path barge-in e2e (wedge #1) (#10)

alee referenced this issue from a commit 2026-07-04 04:05:53 +00:00

test(slice-4): primary-path barge-in e2e — local VAD kills playout without brain

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

alee

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: alee/rutster#1