alee/rutster
0
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

spec+plan(slice-4): barge-in / VAD-driven playout kill on dedicated media thread #6

Merged
alee merged 5 commits from slice-4-barge-in-design into main 2026-07-03 03:07:00 +00:00
Conversation 0 Commits 5 Files Changed 5 +3323
5 changed files with 3323 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

211
docs/superpowers/kickoffs/2026-07-01-slice-4-dev-a-prompt.md Normal file
View File

@@ -0,0 +1,211 @@
# Dev A Kickoff Prompt — slice-4 Plan A (Reflex + LocalVadReflex + MediaThread)
Paste everything below the `---` line into a fresh Claude Code terminal as the first user message.
---
You are a **senior developer** owning the Reflex + thread chain for the slice-4 "barge-in / VAD-driven playout kill" release. A PM in another terminal coordinates you with dev-b. With the relay server running, you communicate via `post_message` / `read_messages` directly — no user copy-paste needed.
## Your scope (the Reflex + thread chain)
You own the **FOB reflex wrapper + the dedicated media thread** — the FOB-owned, in-core pieces of slice-4. Tasks:
- **Task 1** (critical-path foundation — you run this FIRST, nobody parallelizes until it merges)
- **Task 2** (`Reflex<P>` state machine)
- **Task 2b** (`LocalVadReflex<P>` — the primary trigger, the wedge-#1 proof)
- **Task 6** (`MediaThread` — the dedicated std::thread; needs your Task 2+2b AND dev-b's Task 5)
- **Task 7** (`session_map.rs` rewire + `main.rs` + `routes.rs` — needs your Task 6)
- Possibly **Task 9** + **Task 10** (shared — PM assigns; likely you since your chain finishes Task 7 first)
Your work embodies ARCHITECTURE.md's mandate: *"Local real-time reflexes... live in-core because the brain round-trip is too slow to enforce them."* Your `LocalVadReflex` (Task 2b) IS the wedge-#1 proof — the barge-in fires from the FOB's own inspection of caller audio, zero brain round-trip.
## Setup (do this first)
```bash
cd /home/alee/Sources/rutster
git fetch
git checkout main
git pull
git worktree add /home/alee/Sources/rutster.slice-4-dev-a -b slice-4-dev-a-reflex
cd /home/alee/Sources/rutster.slice-4-dev-a
pwd # should print /home/alee/Sources/rutster.slice-4-dev-a
```
**ALL subsequent work happens in `/home/alee/Sources/rutster.slice-4-dev-a`**. Force-cd subagents into this directory — per AGENTS.md (which overrides the default no-comments rule — read it).
Today: 2026-07-01. Project rules in `AGENTS.md` apply — READ THEM IN FULL, especially "Code style (Rust)" (learner-facing comments override the global no-comments rule), "Terminology policy" (inclusive language; ICE is kept verbatim as a protocol name per RFC convention), "Git workflow" (squash-merge default + DCO signoff REQUIRED on every commit via `git commit -s`), "Slice-1 boundaries — what NOT to add (yet)."
## Relay server
A message-bus MCP server is running on `localhost:7110`. You have three native tools:
- `post_message(from, to, kind, body)` — push a message; your `from` is always `"dev-a"`
- `read_messages(for)` — drain your inbox; call with `for="dev-a"` before each task
- `list_pending(for)` — check inbox count without consuming
Recipients: `pm, dev-a, dev-b, dev-c, dev-d, dev-e, dev-f`. Use these instead of asking the user to copy-paste. Before starting each task: `read_messages(for="dev-a")`. After emitting any status/question block: `post_message(from="dev-a", to="pm", kind="status"|"question", body="...")`.
**Fallback:** If the relay MCP tools are not registered in your session, use the Python shim:
```bash
export RELAY_PORT=7110
cd ~/Sources/relay
python3 call.py post_message '{"from":"dev-a","to":"pm","kind":"status","body":"..."}'
python3 call.py read_messages '{"for":"dev-a"}'
```
**Common pitfalls:**
- Prefer single-line `body` content (strict JSON parsers reject embedded `\n`).
- Use single quotes inside f-strings: `{m.get('from')}`.
## Relay polling cadence — MANDATORY (do NOT go head-down)
The #1 failure mode is a dev going head-down on a long run and never checking the inbox — so a PM `HOLD` or `RESCOPE` is never seen. Do not be that dev. The ground can shift under you mid-task (e.g., dev-b's Task 5 signature revision could change the composition site in your Task 6).
**Call `read_messages(for="dev-a")` (or `list_pending(for="dev-a")` for a cheap check) at ALL of these points:**
- Before dispatching EACH subagent — and again the moment it returns.
- Before EACH commit, and at the start + end of every task/step.
- Any time you've been heads-down for more than a few minutes.
**An inbound `Action: HOLD` or `RESCOPE` is an interrupt, not a suggestion:** stop immediately, do NOT dispatch the next subagent, acknowledge with a STATUS UPDATE, and comply before resuming.
## Required reading (in order)
1. `AGENTS.md` — the whole thing (code style, git workflow with DCO, slice boundaries, multi-dev parallelism rules)
2. `docs/ARCHITECTURE.md` — §"Biggest technical risk" + §"Media plane" (the dedicated-thread mandate you're landing)
3. `docs/superpowers/specs/2026-07-01-slice-4-barge-in-design.md` — your scope is Tasks 1, 2, 2b, 6, 7 (+ possibly 9, 10)
4. `docs/superpowers/plans/2026-07-01-slice-4-barge-in.md` — your plan, execute task by task
5. `crates/rutster-media/src/pcm.rs` — the `AudioPipe` trait you'll extend (Task 1) + the `EchoAudioPipe` shape
6. `crates/rutster-media/src/loop_driver.rs` + `crates/rutster-media/src/rtc_session.rs` — **DO NOT TOUCH** (the seam gate, byte-identical to slice-3)
## Execution mode
Use **subagent-driven-development**. Invoke `superpowers:subagent-driven-development` and follow it: fresh subagent per task, two-stage review between tasks.
**Every subagent prompt MUST start with**:
```
cd /home/alee/Sources/rutster.slice-4-dev-a
```
…before any other instruction.
**Between every subagent dispatch, poll the relay** — the gaps between subagents are exactly where a PM directive lands and exactly where head-down devs miss it.
## Your scope and boundaries
**In scope:** Task 1 (`AdvisoryEvent` + `ReflexMetrics` + `barge_in_flush` trait), Task 2 (`Reflex<P>`), Task 2b (`LocalVadReflex<P>`), Task 6 (`MediaThread`), Task 7 (`session_map.rs` rewire). Possibly Task 9 (e2e — PM assigns) and Task 10 (CI seam gate).
**Out of scope (dev-b owns):** Task 3 (`TapAudioPipe::barge_in_flush` override + `TapMetrics.barge_drained_inflight`), Task 4 (`advisory_tx` through `run_tap_client`/`handle_brain_frame`), Task 5 (`spawn_tap_engine` signature change), Task 8 (`MockRealtimeBrain` schedule). If you trip over an out-of-scope issue, file a QUESTION TO PM and keep moving.
**Hard rules:**
- **The seam gate:** `crates/rutster-media/src/loop_driver.rs` + `crates/rutster-media/src/rtc_session.rs` MUST stay byte-identical to slice-3. CI will enforce. Do not touch.
- **Task 1 is critical-path foundation:** nobody parallelizes until Task 1 merges. Land it FIRST, alone, before starting Task 2.
- **DCO signoff:** every commit MUST be `git commit -s` (or `--signoff`). The signoff line uses your human's name + email, matching the git author identity.
- **`LocalVadReflex` (Task 2b) is the wedge-#1 proof:** the PRIMARY-path e2e (Task 9 step 1, if assigned to you) MUST kill playout WITHOUT any brain advisory. Watch this test carefully.
- Do not merge your branch to main. The PM owns merges.
- Do not push `--force` or run `git reset --hard`. Per `AGENTS.md`: ask first.
## Coordination protocol
You are one of 3 terminals. The user's only window into your work is what flows through this terminal and the relay — silence reads as "stuck" even when you're cooking. Narrate.
**Narration discipline.** STATUS UPDATEs at task boundaries are the floor, not the ceiling. Also emit `Status: IN-PROGRESS` updates at meaningful in-flight moments:
- When you dispatch a subagent
- When a subagent returns with a decision worth flagging
- When a sub-task within a phase completes
- When you change direction or hit something unexpected
- When you start a new phase
The `Notes` field should narrate WHAT happened and WHY (decisions taken, surprises, trade-offs) — not just "Phase X done". Three sentences max.
Also print every STATUS UPDATE locally before/after sending it so the user reads it in your own terminal.
**At every task boundary AND every meaningful in-flight moment**: call `read_messages(for="dev-a")` first, then post your update via `post_message(from="dev-a", to="pm", kind="status"|"question", body="...")` and also print it here. Use this format:
```
## STATUS UPDATE — DEV-A
Time: <iso8601 like 2026-07-01T20:30:00-07:00>
Branch: slice-4-dev-a-reflex
Task: <number / short name>
Status: STARTED | IN-PROGRESS | DONE | BLOCKED | REVIEW-READY
Last commit: <short sha + first line of message>
Tests: <green | red (which failed) | N/A>
Notes: <anything PM needs to know — keep to 3 sentences max>
```
**When you need PM input mid-task**: post via `post_message(kind="question")` with format:
```
## QUESTION TO PM — DEV-A
Time: <iso8601>
Context: <what task, what decision point>
Options: <A: ... / B: ... / C: ...>
Recommended: <your pick + one-sentence rationale>
Blocker: yes | no
```
**Coordination point (load-bearing):** Task 6 needs dev-b's Task 5 (`spawn_tap_engine` accepting `advisory_tx` as a parameter). Before starting Task 6's composition site (where you construct the `(advisory_tx, advisory_rx)` pair + wire `LocalVadReflex<Reflex<TapAudioPipe>>`), POST a QUESTION TO PM asking whether dev-b's Task 5 has merged. Hold Task 6's composition work until confirmed.
**You'll receive**: `## DIRECTIVE TO DEV-A` blocks from the PM via relay. Acknowledge and act.
## Ship-it autonomy + simplify discipline
**Hard guardrails:** no `rm` / `rmdir`, no `git push --force` / `--force-with-lease`, no `git reset --hard`, no `git branch -D`, no `git worktree remove`, no `git clean -f*`, no `git checkout -- *`, no `sudo`, no `chmod 777`. If you genuinely need one of these, surface a QUESTION TO PM block.
**Speed without spaghetti — required before every REVIEW-READY:**
- Invoke `superpowers:simplify` (or the project's equivalent code-review skill) on the changed code.
- Do not create parallel implementations of an existing helper.
- Do not add error handling, fallbacks, or validation for scenarios that can't happen (AGENTS.md forbids this).
- Default to learner-facing comments per AGENTS.md code style (this project OVERRIDES the global no-comments convention).
- Half-finished implementations are forbidden. Either ship a complete sub-task or surface a QUESTION TO PM block.
## Authority within the plan
You don't need PM permission to:
- Execute task-to-task per the plan
- Make implementation decisions consistent with the plan and spec
- Write tests, refactor your own code, fix bugs you introduce
- Push commits to your feature branch (with DCO signoff)
You **do** escalate to PM when:
- A scope question outside the plan
- A test you can't make green after honest debugging (don't fudge — debug)
- A discovered bug not in your plan
- Anything destructive (per project rules)
- Before opening the PR for review
## Final steps before REVIEW-READY
Run the project's full validation:
```bash
cargo fmt --all --check
cargo clippy --all --all-targets -- -D warnings
cargo test --all
```
Then push and open the PR:
```bash
git push -u origin slice-4-dev-a-reflex
tea pulls create \
--head slice-4-dev-a-reflex \
--base main \
--title "slice-4 (dev-a): Reflex + LocalVadReflex + MediaThread + session_map rewire" \
--description "## What lands
- Task 1: AdvisoryEvent + ReflexMetrics + barge_in_flush trait (critical-path foundation)
- Task 2: Reflex<P> state machine
- Task 2b: LocalVadReflex<P> (the wedge-#1 primary trigger)
- Task 6: MediaThread (dedicated std::thread)
- Task 7: session_map + main + routes rewire
## What this proves
slice-4 dev-a's chain lands the FOB reflex loop + the dedicated media thread. Combined with dev-b's tap chain, the slice-4 e2e proves wedge #1 (caller speech → kill within 80ms, zero brain round-trip).
## Merge instructions
- squash-merge
- DCO signoff on every commit (AGENTS.md)"
```
Emit a `## STATUS UPDATE` with `Status: REVIEW-READY` and the PR URL.
## First action
After reading: emit a `## STATUS UPDATE` confirming setup complete (worktree created, plan absorbed, on `slice-4-dev-a-reflex`), then start Task 1 of your plan. Task 1 is the critical-path foundation — nobody parallelizes until it merges, so land it solo + REVIEW-READY before starting Task 2.

219
docs/superpowers/kickoffs/2026-07-01-slice-4-dev-b-prompt.md Normal file
View File

@@ -0,0 +1,219 @@
# Dev B Kickoff Prompt — slice-4 Plan B (TapAudioPipe + advisory + MockRealtimeBrain)
Paste everything below the `---` line into a fresh Claude Code terminal as the first user message.
---
You are a **senior developer** owning the tap + advisory chain for the slice-4 "barge-in / VAD-driven playout kill" release. A PM in another terminal coordinates you with dev-a. With the relay server running, you communicate via `post_message` / `read_messages` directly — no user copy-paste needed.
## Your scope (the tap + advisory chain)
You own the **brain↔FOB advisory plumbing + the mock brain's VAD schedule** — the pieces that wire brain-side advisories into dev-a's Reflex. Tasks:
- **Task 3** (`TapAudioPipe::barge_in_flush` override + `TapMetrics.barge_drained_inflight`)
- **Task 4** (`advisory_tx` through `run_tap_client` + `handle_brain_frame`)
- **Task 5** (`spawn_tap_engine` signature change — takes `advisory_tx` as a parameter)
- **Task 8** (`MockRealtimeBrain` advisory schedule)
- Possibly **Task 9** + **Task 10** (shared — PM assigns; likely dev-a since their chain finishes Task 7 first, but you might pick these up if your Task 8 lands before dev-a's Task 6 coordination point)
**You HOLD until dev-a's Task 1 merges** (the critical-path foundation — `AdvisoryEvent` enum + `ReflexMetrics` + `barge_in_flush` trait). Your Task 3 depends on Task 1's `barge_in_flush` trait method. While you wait, read the plan + the slice-2/3 code for context.
Your work wires the SECONDARY/confirmation trigger path (the brain's ASR-quality VAD). The PRIMARY trigger is dev-a's `LocalVadReflex` — but your advisory plumbing IS the path the wedge-#1 e2e test's secondary case exercises.
## Setup (do this first)
```bash
cd /home/alee/Sources/rutster
git fetch
git checkout main
git pull
git worktree add /home/alee/Sources/rutster.slice-4-dev-b -b slice-4-dev-b-tap
cd /home/alee/Sources/rutster.slice-4-dev-b
pwd # should print /home/alee/Sources/rutster.slice-4-dev-b
```
**ALL subsequent work happens in `/home/alee/Sources/rutster.slice-4-dev-b`**. Force-cd subagents into this directory.
Today: 2026-07-01. Project rules in `AGENTS.md` apply — READ THEM IN FULL, especially "Code style (Rust)" (learner-facing comments override the global no-comments rule), "Terminology policy" (inclusive language; ICE is kept verbatim per RFC convention), "Git workflow" (squash-merge default + DCO signoff REQUIRED on every commit via `git commit -s`), "Slice-1 boundaries — what NOT to add (yet)."
## Relay server
A message-bus MCP server is running on `localhost:7110`. You have three native tools:
- `post_message(from, to, kind, body)` — push a message; your `from` is always `"dev-b"`
- `read_messages(for)` — drain your inbox; call with `for="dev-b"` before each task
- `list_pending(for)` — check inbox count without consuming
Recipients: `pm, dev-a, dev-b, dev-c, dev-d, dev-e, dev-f`. Use these instead of asking the user to copy-paste. Before starting each task: `read_messages(for="dev-b")`. After emitting any status/question block: `post_message(from="dev-b", to="pm", kind="status"|"question", body="...")`.
**Fallback:** If the relay MCP tools are not registered in your session, use the Python shim:
```bash
export RELAY_PORT=7110
cd ~/Sources/relay
python3 call.py post_message '{"from":"dev-b","to":"pm","kind":"status","body":"..."}'
python3 call.py read_messages '{"for":"dev-b"}'
```
**Common pitfalls:**
- Prefer single-line `body` content (strict JSON parsers reject embedded `\n`).
- Use single quotes inside f-strings: `{m.get('from')}`.
## Relay polling cadence — MANDATORY (do NOT go head-down)
The #1 failure mode is a dev going head-down on a long run and never checking the inbox — so a PM `HOLD` or `RESCOPE` is never seen. Do not be that dev.
**Call `read_messages(for="dev-b")` (or `list_pending(for="dev-b")`) at ALL of these points:**
- Before dispatching EACH subagent — and again the moment it returns.
- Before EACH commit, and at the start + end of every task/step.
- Any time you've been heads-down for more than a few minutes.
- **Especially while you hold for Task 1:** poll every few minutes so you see dev-a's REVIEW-READY the moment it lands + the PM clears you to start.
**An inbound `Action: HOLD` or `RESCOPE` is an interrupt, not a suggestion:** stop immediately, do NOT dispatch the next subagent, acknowledge with a STATUS UPDATE, and comply before resuming.
## Required reading (in order — read this while holding for Task 1)
1. `AGENTS.md` — the whole thing (code style, git workflow with DCO, slice boundaries)
2. `docs/superpowers/specs/2026-07-01-slice-4-barge-in-design.md` — your scope is Tasks 3, 4, 5, 8 (+ possibly 9, 10)
3. `docs/superpowers/plans/2026-07-01-slice-4-barge-in.md` — your plan, execute task by task
4. `crates/rutster-tap/src/tap_audio_pipe.rs` — the seam object you'll extend in Task 3 (the `barge_in_flush` override)
5. `crates/rutster-tap/src/tap_client.rs` — `handle_brain_frame` (line 323-421) + `run_tap_client` (line 140-270); Task 4 threads `advisory_tx` through both
6. `crates/rutster-tap/src/metrics.rs` — `TapMetrics` struct (Task 3 adds `barge_drained_inflight`)
7. `crates/rutster-brain-realtime/src/mock.rs` — `MockRealtimeBrain` (Task 8 adds the advisory schedule)
8. `crates/rutster-tap/src/protocol.rs` — the wire protocol (the `encode_speech_started`/`encode_speech_stopped` your code forwards in Task 4)
9. `crates/rutster/src/tap_engine.rs` — `spawn_tap_engine` (line 131-216) + `run_engine_loop` (line 240-356); Task 5 changes `spawn_tap_engine`'s signature
## Execution mode
Use **subagent-driven-development**. Invoke `superpowers:subagent-driven-development` and follow it: fresh subagent per task, two-stage review between tasks.
**Every subagent prompt MUST start with**:
```
cd /home/alee/Sources/rutster.slice-4-dev-b
```
…before any other instruction.
**Between every subagent dispatch, poll the relay.**
## Your scope and boundaries
**In scope:** Task 3 (`TapAudioPipe::barge_in_flush` override + `TapMetrics.barge_drained_inflight`), Task 4 (`advisory_tx` through `run_tap_client` + `handle_brain_frame`), Task 5 (`spawn_tap_engine` signature change), Task 8 (`MockRealtimeBrain` advisory schedule). Possibly Task 9 (e2e secondary path) + Task 10 (shared).
**Out of scope (dev-a owns):** Task 1 (foundation — `AdvisoryEvent` + `ReflexMetrics` + the `barge_in_flush` trait method on `AudioPipe`), Task 2 (`Reflex<P>` state machine), Task 2b (`LocalVadReflex<P>`), Task 6 (`MediaThread`), Task 7 (`session_map.rs` rewire). If you trip over an out-of-scope issue, file a QUESTION TO PM and keep moving.
**Hard rules:**
- **The seam gate:** `crates/rutster-media/src/loop_driver.rs` + `crates/rutster-media/src/rtc_session.rs` MUST stay byte-identical to slice-3. You probably won't touch them, but if you find yourself wanting to, STOP — file a QUESTION TO PM.
- **HOLD until dev-a's Task 1 merges.** Task 3 depends on Task 1's `barge_in_flush` trait method. The PM will clear you to start.
- **DCO signoff:** every commit MUST be `git commit -s` (or `--signoff`).
- **Task 5 signature revision (load-bearing):** the plan originally sketched `spawn_tap_engine` returning a 3-tuple `(TapAudioPipe, TapConn, Option<Receiver<AdvisoryEvent>>)`. The revised plan (after the slice-4 adversarial review + the user's "both, local VAD primary" decision) changed this: `spawn_tap_engine` TAKES `advisory_tx: mpsc::Sender<AdvisoryEvent>` as a parameter (the media thread owns the channel + clones the Sender; `tokio::sync::mpsc::Sender` is `Clone`). Follow the plan's **Task 5 revision note**, NOT the original 3-tuple sketch. The plan's Task 6 composition site (dev-a's MediaThread) constructs the `(tx, rx)` pair, clones `tx` for both `spawn_tap_engine` AND `LocalVadReflex::new`. This means your `spawn_tap_engine` signature is `pub fn spawn_tap_engine(session_id, tap_url, app_state, advisory_tx) -> (TapAudioPipe, TapConn)`.
- Do not merge your branch to main. The PM owns merges.
- Do not push `--force` or run `git reset --hard`. Per `AGENTS.md`: ask first.
## Coordination protocol
You are one of 3 terminals. The user's only window into your work is what flows through this terminal and the relay — silence reads as "stuck" even when you're cooking. Narrate.
**Narration discipline.** STATUS UPDATEs at task boundaries are the floor, not the ceiling. Also emit `Status: IN-PROGRESS` updates at meaningful in-flight moments:
- When you dispatch a subagent
- When a subagent returns with a decision worth flagging
- When a sub-task within a phase completes
- When you change direction or hit something unexpected
- When you start a new phase
The `Notes` field should narrate WHAT happened and WHY — not just "Phase X done". Three sentences max.
Also print every STATUS UPDATE locally before/after sending it.
**At every task boundary AND every meaningful in-flight moment**: call `read_messages(for="dev-b")` first, then post your update via `post_message(from="dev-b", to="pm", kind="status"|"question", body="...")` and also print it here. Format:
```
## STATUS UPDATE — DEV-B
Time: <iso8601>
Branch: slice-4-dev-b-tap
Task: <number / short name>
Status: STARTED | IN-PROGRESS | DONE | BLOCKED | REVIEW-READY
Last commit: <short sha + first line of message>
Tests: <green | red (which failed) | N/A>
Notes: <3 sentences max>
```
**When you need PM input mid-task**: post via `post_message(kind="question")` with format:
```
## QUESTION TO PM — DEV-B
Time: <iso8601>
Context: <what task, what decision point>
Options: <A: ... / B: ... / C: ...>
Recommended: <your pick + one-sentence rationale>
Blocker: yes | no
```
**Coordination point (load-bearing):** dev-a's Task 6 (`MediaThread`) needs your Task 5 (`spawn_tap_engine` accepting `advisory_tx`). When dev-a posts a QUESTION TO PM about whether your Task 5 has merged, post a STATUS UPDATE confirming. If you're still mid-Task-5, surface an ETA.
**You'll receive**: `## DIRECTIVE TO DEV-B` blocks from the PM via relay. Acknowledge and act.
## Ship-it autonomy + simplify discipline
**Hard guardrails:** no `rm` / `rmdir`, no `git push --force`, no `git reset --hard`, no `git branch -D`, no `git worktree remove`, no `git clean -f*`, no `git checkout -- *`, no `sudo`, no `chmod 777`. If you genuinely need one of these, surface a QUESTION TO PM block.
**Speed without spaghetti — required before every REVIEW-READY:**
- Invoke `superpowers:simplify` on the changed code.
- Do not create parallel implementations of an existing helper.
- Do not add error handling, fallbacks, or validation for scenarios that can't happen.
- Default to learner-facing comments per AGENTS.md code style.
- Half-finished implementations are forbidden. Either ship a complete sub-task or surface a QUESTION TO PM block.
## Authority within the plan
You don't need PM permission to:
- Execute task-to-task per the plan
- Make implementation decisions consistent with the plan and spec
- Write tests, refactor your own code, fix bugs you introduce
- Push commits to your feature branch (with DCO signoff)
You **do** escalate to PM when:
- A scope question outside the plan
- A test you can't make green after honest debugging
- A discovered bug not in your plan
- Anything destructive
- Before opening the PR for review
## MockRealtimeBrain schedule API (Task 8 open decision)
The plan proposes `set_advisory_schedule(Vec<AdvisoryTrigger>)` where `AdvisoryTrigger { after_audio_in_frames: u32, event: AdvisoryKind }`. If a cleaner shape emerges during implementation (e.g., a free-form `Vec<(trigger_frame_count, AdvisoryEvent)>` queue), use it — but surface the deviation in the STATUS UPDATE Notes so the PM can sanity-check. The slice-4 e2e (Task 9) is the consumer of this API; if you change the shape, Task 9 (whoever picks it up) needs the new name.
## Final steps before REVIEW-READY
Run the project's full validation:
```bash
cargo fmt --all --check
cargo clippy --all --all-targets -- -D warnings
cargo test --all
```
Then push and open the PR:
```bash
git push -u origin slice-4-dev-b-tap
tea pulls create \
--head slice-4-dev-b-tap \
--base main \
--title "slice-4 (dev-b): TapAudioPipe::barge_in_flush + advisory_tx + MockRealtimeBrain schedule" \
--description "## What lands
- Task 3: TapAudioPipe::barge_in_flush override + barge_drained_inflight metric
- Task 4: advisory_tx through run_tap_client + handle_brain_frame
- Task 5: spawn_tap_engine takes advisory_tx parameter (signature revision)
- Task 8: MockRealtimeBrain advisory schedule
## What this enables
The brain's speech_started/speech_stopped advisory reaches dev-a's Reflex via the dedicated advisory_tx side-channel. This is the SECONDARY/confirmation path (the PRIMARY is dev-a's LocalVadReflex). The slice-4 e2e's secondary case exercises this.
## Merge instructions
- squash-merge
- DCO signoff on every commit (AGENTS.md)"
```
Emit a `## STATUS UPDATE` with `Status: REVIEW-READY` and the PR URL.
## First action
After reading + setup: emit a `## STATUS UPDATE` confirming setup complete (worktree created, plan absorbed, on `slice-4-dev-b-tap`, reading slice-2/3 code for context). State explicitly: "HOLD for dev-a Task 1 REVIEW-READY; polling pm inbox." Do NOT start Task 3 until the PM clears you (dev-a's Task 1 must merge first). Poll every few minutes while you read.

150
docs/superpowers/kickoffs/2026-07-01-slice-4-pm-prompt.md Normal file
View File

@@ -0,0 +1,150 @@
# PM Kickoff Prompt — slice-4 barge-in / VAD-driven playout kill
Paste everything below the `---` line into a fresh Claude Code terminal as the first user message.
---
You are the **project manager** for the **slice-4 "barge-in / VAD-driven playout kill"** release. 2 senior developers report to you, each working in their own terminal on a parallel feature branch within a shared worktree. The user runs all 3 terminals and (with the relay server) reads along as you coordinate.
## Setup
- Working directory: `/home/alee/Sources/rutster`
- Branch: stay on `main` (or slice-4-barge-in-design if coordinating the spec). Do not check out feature branches.
- Today: 2026-07-01. Project rules in `AGENTS.md` apply — READ THEM IN FULL, especially the "Multi-agent coordination — the relay" section, "PM session launch checklist," "Multi-dev parallelism" (5-rule checklist), "Session handoff," and "Git workflow" (squash-default + rebase-merge carve-out for stacked branches).
## Relay server
A message-bus MCP server is running on `localhost:7110`. You have three native tools:
- `post_message(from, to, kind, body)` — push a message; `from` is always `"pm"` for you
- `read_messages(for)` — drain your inbox; call with `for="pm"` before each action
- `list_pending(for)` — check inbox count without consuming
Recipients: `pm, dev-a, dev-b, dev-c, dev-d, dev-e, dev-f`. Use these instead of asking the user to copy-paste. After sending any directive, call `post_message(from="pm", to="dev-X", kind="directive", body="...")`.
**Turn-start discipline (load-bearing — AGENTS.md "PM-mode discipline"):** at the start of EVERY turn before responding to the user: (1) drain `pm` inbox via `read_messages(for="pm")`; (2) `list_pending` for each dev role; (3) `git log --oneline --all -10`; (4) surface anything actionable BEFORE the user asks.
**Fallback:** If the relay MCP tools are not registered in your session (this happens when the relay server was not running when your session opened), use the Python shim instead:
```bash
export RELAY_PORT=7110
cd ~/Sources/relay
python3 call.py read_messages '{"for":"pm"}'
python3 call.py post_message '{"from":"pm","to":"dev-a","kind":"directive","body":"..."}'
```
Also consider launching the poller (`~/Sources/relay/poller.py`) per AGENTS.md "PM session launch checklist" — it keeps state warm between turns.
## Required reading (in order)
1. `AGENTS.md` — the whole thing (PM-mode discipline, multi-dev parallelism rules, git workflow, slice-4 boundaries)
2. `docs/ARCHITECTURE.md` — §"Biggest technical risk" (the reflex loop IS the long pole), §"Media plane" (dedicated timing threads mandate this slice lands)
3. `docs/superpowers/specs/2026-07-01-slice-4-barge-in-design.md` — the slice-4 spec (revised 2026-07-01: local VAD primary, advisory secondary)
4. `docs/superpowers/plans/2026-07-01-slice-4-barge-in.md` — the implementation plan (11 tasks; structured for parallel dispatch)
5. `docs/adr/0002-north-star-and-fused-core.md` — the fused-vertical rule (no hop on the per-call hot path — slice-4 re-affirms this)
6. `docs/adr/0008-fob-and-green-zone.md` — FOB/green-zone doctrine (the reflex is a FOB member)
## The two dev scopes (single plan, partitioned)
This is ONE plan (`docs/superpowers/plans/2026-07-01-slice-4-barge-in.md`) split across 2 devs by task partition. The dependency graph is:
- **Task 1** (critical-path foundation: `AdvisoryEvent` + `ReflexMetrics` + `barge_in_flush` trait) MUST land first. Nobody parallelizes until Task 1 merges. **dev-a executes Task 1 first** while dev-b holds.
- After Task 1 merges, fan-out:
- **dev-a** owns the Reflex + thread chain: Task 2 (`Reflex<P>`), Task 2b (`LocalVadReflex<P>` — the wedge-#1 primary trigger), Task 6 (`MediaThread` — needs 2+2b+5), Task 7 (`session_map` rewire — needs 6)
- **dev-b** owns the tap + advisory chain: Task 3 (`TapAudioPipe::barge_in_flush`), Task 4 (`advisory_tx` through `run_tap_client`/`handle_brain_frame`), Task 5 (`spawn_tap_engine` signature — needs 3+4), Task 8 (`MockRealtimeBrain` schedule — needs 5)
- **Coordination point:** Task 6 (dev-a) needs BOTH `Reflex<P>` + `LocalVadReflex<P>` (dev-a's Task 2+2b) AND `spawn_tap_engine` taking `advisory_tx` (dev-b's Task 5). When dev-a is ready to start Task 6, dev-b must have merged Task 5.
- **Shared tasks:** Task 9 (e2e — needs 6+7+8) + Task 10 (CI seam gate — needs 7). Whoever is free first picks them up; PM arbitrates. Likely dev-a (their chain lands Task 7 first).
## Your authority
- Approve or deny scope changes from devs
- Review and merge PRs from each dev's feature branch
- Drive any release-prep work that isn't a feature plan (CHANGELOG, version bumps) — your hands-on work
- **The user authorizes merges via your session** — do not merge without showing the user the diff is green
- Arbitrate the shared tasks (9 + 10) — assign to whichever dev finishes their chain first
- Re-scope if a dev's chain stalls (e.g. dev-b's Task 4 hits unexpected tap_client complexity — you can shift dev-a's finished Task 2 dev to help)
## Your boundaries
- Don't write feature code yourself. Edits to `AGENTS.md` / specs / CHANGELOG are fine.
- Don't deviate from the spec without user approval. The slice-4 spec was just revised after adversarial review (advisory-only → local VAD primary); re-decisions need user input.
- **Don't merge a PR until** the dev says `REVIEW-READY` AND you've run `tea pr view <url>` + `tea pr diff <url>` to confirm green CI + spec-conformance.
- Don't tag without user approval.
- Per AGENTS.md: ask before any git-destructive op (`git push --force`, `git reset --hard`, `git branch -D`).
## The seam gate — broadcast to BOTH devs
`loop_driver.rs` + `rtc_session.rs` MUST stay byte-identical to slice-3. The §8.5 #6 invariant. CI will enforce via `git diff --exit-code main -- crates/rutster-media/src/loop_driver.rs crates/rutster-media/src/rtc_session.rs`. The `barge_in_flush` trait method adds to `pcm.rs`, NOT to those files. The reflex wrapper decorates on the binary side, NOT in `rtc_session.rs`. State this in your opening directive to both devs.
## The wedge-#1 audit (the load-bearing thing)
The slice-4 spec was revised after a 2026-07-01 adversarial review. The reviewer's finding #3: advisory-only (the initial brainstorming decision) puts the brain round-trip in the barge-in trigger path — contradicting ARCHITECTURE.md:79-81 ("the brain round-trip is too slow to enforce them") + README:98-100 ("VAD killing TTS the instant the caller speaks, without the brain"). The revision: `LocalVadReflex<P>` (Task 2b) is the PRIMARY trigger — RMS/energy VAD in `on_pcm_frame`, runs on the dedicated thread in the 20ms loop, zero brain round-trip. The brain's advisory becomes the SECONDARY/confirmation path.
**The proof is Task 9, Step 1** (the PRIMARY-path e2e test): loud caller audio → kill within ≤80ms wallclock, WITHOUT any brain advisory. If this test passes, slice-4 proves wedge #1. If it doesn't, slice-4 doesn't prove the property the spearhead exists for. Watch dev-a's Task 9 step 1 closely.
## Judgment calls in the plan (for your awareness)
- **DCO signoff now required on every commit.** AGENTS.md gained this in the same 2026-07-01 docs PR (#5). Devs MUST `git commit -s` (or `--signoff`) on every commit. The agent signs off with the human's name + email, NOT the agent's identity. If a dev commits without `-s`, surface it before merge.
- **Task 5 signature:** `spawn_tap_engine` takes `advisory_tx: mpsc::Sender<AdvisoryEvent>` as a parameter (the media thread owns the channel; `tokio::sync::mpsc::Sender` is `Clone`, so two senders — the engine + the local VAD — share one receiver). This was revised mid-plan; dev-b should follow the plan's Task 5 revision note, NOT the original 3-tuple-return sketch.
- **Task 6 composition site:** the `Connected` spawn in `media_thread.rs` constructs the `(advisory_tx, advisory_rx)` pair, clones the Sender to `spawn_tap_engine` AND `LocalVadReflex::new`, hands the Receiver to `Reflex::new`. The composition is `LocalVadReflex<Reflex<TapAudioPipe>>`.
- **MockRealtimeBrain schedule API (Task 8):** the plan leaves the exact API shape as an open decision (`set_advisory_schedule(Vec<AdvisoryTrigger>)` proposed). The dev can adjust if a cleaner shape emerges; surface non-trivial deviations.
## Coordination protocol
You are one of 3 terminals. With the relay server running, use `post_message` / `read_messages` directly — you do not need the user to copy-paste messages. Call `read_messages(for="pm")` before every action.
**Narrate to the user in plain prose between tool calls.** The user's only window into the release is the PM terminal output. Don't emit DIRECTIVE blocks silently. When a STATUS UPDATE lands in your inbox, summarize it for the user in a sentence or two before deciding. When you send a directive, state the rationale briefly. One or two sentences per beat is plenty.
**You receive:** `## STATUS UPDATE — DEV-<letter>` or `## QUESTION TO PM — DEV-<letter>` blocks from the relay inbox.
**You emit:** a `## DIRECTIVE TO DEV-<letter>` block — post it via `post_message` AND print it here so the user can see it. Format:
```
## DIRECTIVE TO DEV-<letter>
Time: <iso8601>
Action: PROCEED | HOLD | RESCOPE | REVIEW-COMPLETE | MERGE-APPROVED
Notes: <one paragraph max>
Next: <one concrete instruction or "continue plan">
```
**Confirm your directives are actually seen.** Devs are told to poll their inbox constantly, but a head-down dev can still miss a HOLD/RESCOPE. After posting a HOLD or RESCOPE, watch that dev's next STATUS UPDATE for an explicit acknowledgement.
When asked "status?" by the user at any time, give a current rollup:
```
## RELEASE STATUS — slice-4 barge-in
Devs: <per-dev one-line state>
PM: <what you're working on>
Blockers: <list, or "none">
Next milestone: <e.g., "Task 1 merges", "Task 6 coordination point", "wedge-#1 e2e green">
```
## Reviewing PRs
When a dev posts `Action: REVIEW-READY` with a PR URL:
1. `tea pr view <url>` to read description + CI status
2. `tea pr diff <url>` to read changes
3. Check the diff against the spec + plan acceptance criteria (esp. the seam gate — `loop_driver.rs` + `rtc_session.rs` untouched)
4. If green: post `Action: MERGE-APPROVED` + `tea pr merge <index>` (squash-merge default per AGENTS.md)
5. If red: post `Action: HOLD` with specific concerns
Use the `superpowers:requesting-code-review` skill if you want a deeper independent review.
## Pre-tag checklist
Before tagging `slice-4`:
- [ ] Every dev branch merged to `main` (squash-merge default)
- [ ] All 11 plan tasks landed (verify against the plan's task list)
- [ ] Full test suite green on main: `cargo fmt --check && cargo clippy -- -D warnings && cargo test --all && cargo deny check` (stable + 1.85 matrix)
- [ ] Seam gate passes: `git diff --exit-code main -- crates/rutster-media/src/loop_driver.rs crates/rutster-media/src/rtc_session.rs`
- [ ] User-driven smoke test (the wedge-#1 demo: caller speaks → brain audio stops within 80ms, no advisory needed)
- [ ] Explicit user approval to tag
## First action
1. Call `read_messages(for="pm")` to drain any early inbox messages.
2. `git log --oneline --all -10` + `pgrep -af poller.py` (verify poller alive per AGENTS.md checklist).
3. Emit a `## RELEASE STATUS` block confirming you've absorbed spec + plan, noting the wedge-#1 audit + the seam gate as the load-bearing things to watch.
4. Send opening directives to both devs via `post_message`:
- dev-a: "PROCEED with Task 1 (critical-path foundation). Task 1 MUST land before anyone parallelizes. When REVIEW-READY, post status + open PR."
- dev-b: "HOLD until Task 1 merges. Read the plan, plus the slice-2 `tap_audio_pipe.rs` + slice-3 `tap_client.rs` for context on Tasks 3+4. Surface questions via QUESTION TO PM."
5. Wait for dev-a's REVIEW-READY on Task 1 before clearing dev-b to start Task 3.

1929
docs/superpowers/plans/2026-07-01-slice-4-barge-in.md Normal file
View File

File diff suppressed because it is too large Load Diff

814
docs/superpowers/specs/2026-07-01-slice-4-barge-in-design.md Normal file
View File

@@ -0,0 +1,814 @@
# Rutster slice 4 — Barge-in: VAD-driven playout kill on a dedicated media thread
- **Status:** Draft (pending review)
- **Date:** 2026-07-01
- **Spearhead step:** 4 of 6 (vision-revision §10 / PORT_PLAN "Phasing")
- **Origin:** brainstorming session 2026-07-01
- **Depends on:** [slice 1 — WebRTC media loopback](2026-06-28-slice-1-webrtc-loopback-design.md),
[slice 2 — The agent tap](2026-06-28-slice-2-agent-tap-design.md), and slice-3's
OpenAI Realtime brain (merged as `c30a452` — `MockRealtimeBrain` + translator + the
`speech_started` / `speech_stopped` advisory events). All three must be landed and green.
- **Related:** [ADR-0002](../../adr/0002-north-star-and-fused-core.md) (fused vertical — the
hot-path hop invariant this slice re-affirms), [ADR-0008](../../adr/0008-fob-and-green-zone.md)
(FOB/green-zone doctrine — the reflex is a FOB member: hot-path, differentiating),
[ARCHITECTURE.md §"Biggest technical risk"](../../ARCHITECTURE.md) (the reflex loop *is*
the remaining long pole), [ARCHITECTURE.md §"Media plane"](../../ARCHITECTURE.md)
("Dedicated timing threads for the 20ms loop, **never the shared tokio pool**" — this
slice finally lands that mandate).
---
## TL;DR
Stand up spearhead step 4: the **FOB reflex loop**. Slice 3 pre-paved the advisory signals
(`speech_started` / `speech_stopped` from the brain) and locked the turn-ownership decision
(OpenAI Realtime server-side VAD disabled; the FOB owns turn-taking). Slice 4 **acts** on
the caller's speech with a **local in-core VAD** as the primary trigger — an RMS/energy
detector running in `on_pcm_frame` on the dedicated thread, in the 20 ms loop, with **zero
brain round-trip** between caller speech and playout kill. This is the property ARCHITECTURE.md:79-81
demands ("Local real-time reflexes... live in-core because the brain round-trip is too slow
to enforce them") and the proof wedge #1 rests on ("VAD killing TTS the instant the caller
speaks, without the brain" — README:98-100). Slice-3's `speech_started`/`speech_stopped`
advisory becomes the **secondary/confirmation** signal — the brain's ASR-quality VAD
confirms the local kill slightly later, but the kill itself fires from the FOB's own
inspection of caller audio.
Slice 4 also **graduates the media loop off the tokio pool**: a single dedicated `std::thread`
owns all `RtcSession`s exclusively and drives the 20 ms tick via `Instant::sleep_until`. This
honors ARCHITECTURE.md's "never the shared tokio pool" mandate, which slice-1 explicitly
deferred to "step 4 (barge-in)" (`loop_driver.rs:18-23`). The graduation is load-bearing:
the reflex is the differentiator and the long pole, and its timing discipline demands a
thread that doesn't compete with the axum runtime for scheduling.
The **seam slice 1→3 preserved** (`loop_driver.rs` + `rtc_session.rs` byte-identical) holds
for slice-4 as well: the reflex is a pair of composing `AudioPipe` decorators (`Reflex<P>` +
`LocalVadReflex<P>`) in `rutster-media`, invisibly to `loop_driver::drive`. Only the
binary-side wiring (`session_map.rs` → `media_thread.rs`) changes shape; the media crate's
hot path stays untouched.
---
## 1. Scope
### 1.1 In scope
- Implementation of spearhead step 4: **barge-in / VAD-driven playout kill**, driven by a
**local in-core VAD** (RMS/energy detector in `on_pcm_frame` — the primary trigger, zero
brain round-trip) with the brain's `speech_started`/`speech_stopped` advisory as the
secondary/confirmation signal. This proves wedge #1 ("VAD killing TTS the instant the
caller speaks, without the brain" — README:98-100, ARCHITECTURE.md:79-81).
- A **new `Reflex<P: AudioPipe>` wrapper** (`rutster-media/src/reflex.rs`) that decorates the
pipe the `RtcSession` holds. The reflex owns the mute state machine, the advisory channel
receiver, and the barge-in flush trigger. It is the concrete embodiment of ARCHITECTURE.md's
"local real-time reflexes" row for the barge-in case. Fed by BOTH the local VAD (via the
outer `LocalVadReflex` wrapper that injects `AdvisoryEvent`s into the same mpsc) AND the
brain's advisories.
- A **new `LocalVadReflex<P: AudioPipe>` wrapper** (`rutster-media/src/reflex.rs`) — the
PRIMARY trigger. Decorates `AudioPipe`'s `on_pcm_frame` path: inspects the caller's decoded
PCM samples (computes RMS, compares to a `const` threshold, debounces N consecutive
above-threshold frames), fires `AdvisoryEvent::SpeechStarted { at: Instant::now() }` into
the `Reflex`'s advisory mpsc when local VAD trips. Wraps `Reflex<TapAudioPipe>` (composition:
`LocalVadReflex<Reflex<TapAudioPipe>>` — the decorator pattern from §6.4 pays off here).
Forward-compatible: the threshold is a `const` for the MVP (no tuning framework — that's the
only piece deferred, per §1.2).
- A **new `barge_in_flush` method on the `AudioPipe` trait** (default impl delegates to
`clear_playout_ring`) — the seam object's "kill now" path: clear the playout ring AND drain
the brain-bound `rx_audio_out` channel of any frames queued before the barge so the first
`audio_out` observed post-barge is provably post-barge. `TapAudioPipe` overrides;
`EchoAudioPipe` uses the default.
- A **new `AdvisoryEvent` enum** (`SpeechStarted { at }`, `SpeechStopped { at }`) flowing
over a tokio mpsc from the TapEngine (tokio) to the Reflex (media thread). The engine
pushes the events it already decodes from the brain (slice-3 wired these as log+count;
slice-4 *forwards* them into the reflex).
- A **new dedicated media thread** (`rutster/src/media_thread.rs`) replacing the tokio
`spawn_poll_task`. One `std::thread::spawn` at binary startup owns
`HashMap<ChannelId, RtcSession>` exclusively; all access from axum is via a command
channel (`AcceptOffer`, `Delete`, `Shutdown`). The 20 ms tick is `std::thread::sleep`.
- **Rewired `session_map.rs`** (binary): `SessionEntry.rtc: Arc<Mutex<RtcSession>>` →
`cmd_tx: mpsc::Sender<MediaCmd>`. `create_session`, `post_offer`, `close`,
`spawn_poll_task` all route through the command channel. The async handlers are cold-path;
no cross-thread coordination happens on the 20 ms tick.
- **`MockRealtimeBrain` extension** (`rutster-brain-realtime/src/mock.rs`): gains the ability
to emit `speech_started` / `speech_stopped` on a programmable schedule (e.g. "after N
audio_in frames received, send `speech_started`; after M more, send `speech_stopped`").
- **Barge-in e2e integration test** (extends slice-3's
`crates/rutster/tests/realtime_integration.rs` harness): synthetic WebRTC peer →
MediaThread → TapEngine → MockRealtimeBrain; mock emits `speech_started`; assert playout
goes silent within ≤1 tick (20 ms); mock emits fresh `audio_out`; assert playout resumes.
- New `ReflexMetrics` (`barge_in_count`, `advisory_dropped`, `frames_suppressed`) mirroring
`TapMetrics` shape (atomics, snapshot fn). Threaded through the same `TapConn.metrics`
surface where reasonable, or a new side-car.
- Thorough learner-facing comments on the new std-thread / channel-bridge / wrapper-decorator
patterns (slice-1 §7 standard carries over).
### 1.2 Out of scope (with scheduled return)
| Deferred item | Returns in | Why deferred |
|---|---|---|
| Local VAD **tuning framework** (configurable thresholds, per-environment calibration, adaptive noise floor) | post-spearhead refinement | The VAD itself (RMS/energy detector + debounce) IS in scope for slice-4 (the primary barge-in trigger, proving wedge #1). Only the *tuning* framework — configurable thresholds, calibration UI, adaptive noise floors — is deferred. The MVP ships with a single `const` threshold + N-frame debounce, exercised by the e2e test with a synthetic loud signal. Tuning to real-world noise conditions is post-spearhead. |
| Per-session media threads / threadpool shard | later rung | Single thread covers spearhead scale (loopback dev + low-concurrency PSTN via slice-5). The command-channel seam between axum and the thread makes the graduation to a threadpool shard localized. |
| Trickle ICE | later | Unchanged from slice-1 deferral. |
| Min-mute floor / inter-word-gap debouncing | post-spearhead | `SpeechStopped` is a no-op for mute; a floor timer on resume would protect against brain-yield races (brain emits fresh `audio_out` before the caller's inter-word gap ends). Defer until observed in practice. |
| Brain-side `input_audio_buffer.interrupt` / `clear` on barge | slice-5 or brain-side | Whether the brain should clear its own input buffer on `speech_started` is a brain-UX decision, not a FOB one; the FOB only kills *playout* (its half-duplex gate). The advisory already tells the brain what happened; the brain's response is its own concern. |
| Half-duplex gating beyond playout kill | later rung | Barge-in is the first half-duplex reflex; full HD gating (mixing, jitter buffer interaction, multi-party) arrives with conferencing. |
| TLS on HTTP / WSS | slice-5 | Unchanged. |
| Authn / authz / multi-tenancy | slice-6 | Unchanged. |
| Spend cap / abuse gate | slice-6 | Unchanged. |
| Browser-based automated e2e (Playwright/Selenium) | post-spearhead | Unchanged. The synthetic-peer harness from slice-2/3 is the test vehicle. |
---
## 2. Architecture delta
### 2.1 The reflex wrapper
`Reflex<P: AudioPipe>` is a zero-cost-style decorator around any `AudioPipe`. It sits
between `RtcSession.pipe` (which `loop_driver::drive` calls via `session.pipe.next_pcm_frame()`)
and the concrete pipe (`TapAudioPipe` in production, `EchoAudioPipe` in slice-1's unit tests).
`loop_driver` is oblivious to the wrapper: it still calls `session.pipe.next_pcm_frame()`,
the dynamic dispatch through `Box<dyn AudioPipe>` lands in `Reflex::next_pcm_frame`, which
applies the state machine and delegates to `inner.next_pcm_frame()` per the table in §3.2.
The reflex owns three pieces of state:
- `advisory_rx: mpsc::Receiver<AdvisoryEvent>` — drained sync-non-blocking via `try_recv`
on the 20 ms tick before delegating to `inner`. Fed by TWO senders on the same channel:
the outer `LocalVadReflex` (local VAD, the primary trigger) and the TapEngine task (the
brain's advisory, secondary/confirmation) — both over tokio mpsc.
- `muted: bool` — the kill state. `next_pcm_frame` returns `None` while muted, *unless* the
inner returns `Some` (the resume condition — the first fresh `audio_out` clears mute).
- `barge_epoch: u64` — incremented on every `SpeechStarted`. Load-bearing this slice: the
local VAD (primary) and the brain's advisory (secondary) can both fire on the same barge,
and the epoch distinguishes a genuine re-barge from the slower confirmation landing on the
same event (§3.2). The flush + drain keeps resume race-free regardless.
### 2.2 The dedicated media thread
A single `std::thread::spawn` replaces the tokio `spawn_poll_task`. The thread owns
`HashMap<ChannelId, RtcSession>` **exclusively** — no `Arc<Mutex<RtcSession>>` shared with
axum. All access from the axum handlers is via a command channel:
```rust
enum MediaCmd {
AcceptOffer { id: ChannelId, sdp: String, reply: oneshot::Sender<Result<String, String>> },
Delete { id: ChannelId, reply: oneshot::Sender<()> },
Shutdown { reply: oneshot::Sender<()> },
}
```
The thread loop per 10 ms meta-tick:
1. Drain `cmd_rx` via `try_recv` loop — handle all pending commands before ticking.
2. For each session in the map: drain the per-session `flush_rx` side-channel (slice-2's
existing disconnect-flush signal) BEFORE `run_poll_once`, then call
`RtcSession::run_poll_once(now)` (the unchanged `loop_driver::drive`).
3. After `run_poll_once`, observe `channel.state`:
- `Connected && tap.is_none()` → spawn the TapEngine (tokio task via the
`tokio::runtime::Handle` captured at thread-start) + wire
`Reflex<TapAudioPipe>` as the session's pipe. Mirror of slice-2's spawn seam,
relocated from `session_map.rs::drive_all_sessions` to here.
- `Closed` → remove the entry + drop the session.
4. `std::thread::sleep(Duration::from_millis(10))` — 10 ms meta-tick. (Stable API:
`std::thread::sleep_until` is nightly-only; `sleep(dur)` is the stable path. The 20 ms
outbound encode tick is driven inside `loop_driver::drive` (unchanged); the 10 ms
meta-tick gives finer resolution so str0m's `Timeout` outputs are honored promptly.)
**The tokio ↔ std-thread bridge:** all channels are tokio mpsc/oneshot (constructable on
tokio, drainable via `try_recv`/`blocking_recv` from any thread). The `tokio::runtime::Handle`
captured at `MediaThread::spawn` time is used on the std thread to `handle.spawn(...)` the
TapEngine when the `Connected` transition fires. No async code runs on the std thread itself
— only sync channel ops + `RtcSession::run_poll_once`.
**Why a single thread, not per-session:** spearhead scale. One loopback peer at a time in
dev; even at low PSTN concurrency (slice-5) one thread drives dozens of sessions in 10 ms.
Per-session threads arrive when the threadpool shard model lands (deferred). The
command-channel seam between axum and the thread makes that graduation localized.
### 2.3 The hot-path audit (ADR-0002 honored)
ADR-0002's load-bearing rule: *"the control↔media gRPC hop on the per-call hot path is
removed."* Slice 4 does not re-introduce a hop:
- The reflex's kill decision happens **inside** `Reflex::next_pcm_frame` on the dedicated
thread — no channel send, no cross-thread coordination on the 20 ms tick. The advisory
arrives via a `try_recv` drain (sync, non-blocking).
- axum → media-thread is **cold-path only** (SDP accept, DELETE). None of it runs on the
20 ms tick.
- The brain WS ↔ TapEngine (tokio) path is unchanged from slice-3. The advisory channel
is a *third* mpsc alongside the existing `tx_pcm_in`/`rx_audio_out`/`flush_tx` — same
pattern, additive.
**The fused vertical stays fused.** ADR-0002 honored.
---
## 3. Component design
### 3.1 `AdvisoryEvent` enum
```rust
// crates/rutster-media/src/reflex.rs
/// A turn-event advisory from the brain. The brain decodes its own
/// speech-to-text / VAD results and forwards these; the FOB *owns*
/// turn-taking and acts on them (slice-3 §4.3 — OpenAI Realtime
/// server-side VAD is DISABLED; the FOB's reflex is authoritative).
///
/// Carried over a tokio mpsc from the TapEngine (tokio task) to the
/// `Reflex` wrapper (media thread). Drained sync via `try_recv` on the
/// 20 ms tick — the kill decision lives in the loop, not in a handler.
#[derive(Debug)]
pub enum AdvisoryEvent {
/// The brain detected caller speech. Trigger barge-in: kill playout.
SpeechStarted { at: Instant },
/// The brain detected caller speech ended. Observed + counted; does
/// NOT toggle mute (the resume condition is "first fresh audio_out
/// after the barge", not "speech_stopped" — see §3.2 state table).
SpeechStopped { at: Instant },
}
```
### 3.2 `Reflex<P>` state machine
| Current state | Event | Action | New state |
|---|---|---|---|
| Playing | `SpeechStarted` | `muted=true`; `epoch++`; `inner.barge_in_flush()` (clear ring + drain `rx_audio_out` so stale brain frames queued pre-barge are dropped); `metrics.barge_in_count++` | Muted |
| Muted | `SpeechStarted` (duplicate/re-barge) | `epoch++`; `barge_in_flush()` again (fresh barge resets the "fresh audio" clock); `barge_in_count++` | Muted |
| Muted | `SpeechStopped` | increment `advisory_observed_speech_stopped` counter; **no state change** | Muted |
| Playing | `SpeechStopped` | increment counter; **no state change** | Playing |
| Muted | inner `next_pcm_frame()` returns `Some(f)` (fresh brain audio arrived post-barge) | `muted=false`; return `Some(f)` | Playing |
| Muted | inner `next_pcm_frame()` returns `None` | return `None` (silence); `metrics.frames_suppressed++` | Muted |
**Why `SpeechStopped` is a no-op for mute:** per the resume-semantics decision (resume on
first fresh `audio_out`). The brain's `speech_stopped` is *observed* (counter) but doesn't
gate — this avoids the inter-word-gap problem (caller pauses, VAD fires stopped, brain
un-mutes too early, brain's audio overlaps caller's next word). The resume condition is
"the brain has yielded and started a new response," which is provably signaled by the first
`audio_out` frame after the barge — not by the caller's silence.
**Why `epoch`:** with two concurrent trigger sources this slice — the local VAD (primary)
and the brain's advisory (secondary/confirmation, §6.1) — the epoch disambiguates "is this
barge a re-barge of the same event, or a new one" when both race into the same advisory
channel. The local VAD trips first; the brain's slower ASR-grade advisory lands ~300 ms
later on the *same* event and must not be counted as a fresh barge. The epoch is that
disambiguator — load-bearing now, not a forward-compat seam.
### 3.3 `AudioPipe` trait extension
```rust
// crates/rutster-media/src/pcm.rs — additive method on `AudioPipe`
/// Barge-in flush: clear the playout ring AND drain the inbound brain
/// audio queue of any frames queued before the barge. Called by `Reflex`
/// on `SpeechStarted`. The drain of `rx_audio_out` is what makes the
/// resume condition race-free: the first `audio_out` observed post-barge
/// is provably post-barge (frames queued pre-barge are dropped here).
///
/// Default impl delegates to `clear_playout_ring` — sufficient for
/// pipes without an inbound queue to drain (like `EchoAudioPipe`).
fn barge_in_flush(&mut self) {
self.clear_playout_ring();
}
```
`TapAudioPipe` overrides:
```rust
// crates/rutster-tap/src/tap_audio_pipe.rs
fn barge_in_flush(&mut self) {
// Clear the playout ring (drops buffered brain-proposed frames).
self.playout_ring.clear();
// Drain rx_audio_out of any frames the engine task queued before
// the barge. Without this, a stale frame in the mpsc would un-mute
// immediately on the next tick — defeating the "first fresh audio_out"
// resume condition. Hot-path: try_recv loop, bounded, no blocking.
while self.rx_audio_out.try_recv().is_ok() {
self.metrics.barge_drained_inflight.fetch_add(1, Ordering::Relaxed);
}
}
```
### 3.4 `LocalVadReflex<P>` — the primary trigger (FOB-local, zero brain round-trip)
The decorator that proves wedge #1. Wraps any `AudioPipe`; inspects the caller's decoded PCM
in `on_pcm_frame`, computes RMS energy, fires `AdvisoryEvent::SpeechStarted` into the inner
`Reflex`'s advisory channel when local VAD trips. Composes as
`LocalVadReflex<Reflex<TapAudioPipe>>` — the outer wrapper does local-VAD; the inner wrapper
applies the mute state machine to the advisory stream (which now has TWO sources: the local
VAD + the brain's advisory, both feeding the same mpsc).
```rust
// crates/rutster-media/src/reflex.rs
/// RMS energy threshold for caller-speech detection. The MVP ships with
/// a single tuned-for-synthetic-loud-signal const; the tuning framework
/// (per-environment calibration, adaptive noise floor) is deferred per
/// slice-4 §1.2. The synthetic-peer e2e test sends a frame with samples
/// well above this threshold so the trip is deterministic.
///
/// i16 max is 32767; a |sample| average of ~500 (~1.5% of full scale) is
/// a quiet-but-unmistakable signal; room-tone background sits well below.
/// The debounce (N consecutive frames above threshold) filters transient
/// spikes (clicks, nose-breaths) without slowing the trip materially.
pub const VAD_RMS_THRESHOLD: f64 = 500.0;
/// Number of consecutive above-threshold frames required before the VAD
/// trips. At 20 ms/frame, N=3 = 60 ms of above-threshold audio — well
/// below the brain's ~300 ms ASR-VAD latency, comfortably instant for
/// the wedge-#1 demonstration. Tunable in a later slice; const for MVP.
pub const VAD_DEBOUNCE_FRAMES: u32 = 3;
pub struct LocalVadReflex<P: AudioPipe> {
inner: P,
/// The advisory_tx the LocalVadReflex pushes into when local VAD
/// trips. This is the SAME channel the brain's advisories arrive
/// on (the Reflex holds the paired rx) — so the Reflex's mute state
/// machine sees both sources uniformly via `drain_advisories`.
advisory_tx: mpsc::Sender<AdvisoryEvent>,
/// Consecutive above-threshold frame count (the debounce counter).
above_threshold_streak: u32,
/// True once the VAD has tripped for the current caller-speech
/// burst. Reset to false when the streak breaks (caller stopped)
/// — so a new burst trips a fresh `SpeechStarted`.
vad_armed: bool,
}
impl<P: AudioPipe> LocalVadReflex<P> {
pub fn new(inner: P, advisory_tx: mpsc::Sender<AdvisoryEvent>) -> Self {
Self {
inner,
advisory_tx,
above_threshold_streak: 0,
vad_armed: true, // armed on construction
}
}
/// Compute RMS energy of a PCM frame. The frame is 480 i16 samples
/// @ 24 kHz; RMS = sqrt(mean(sample²)). Cheap: ~480 multiplications,
/// one division, one sqrt — well under the 20 ms tick budget.
fn rms(frame: &PcmFrame) -> f64 {
let sum_sq: u64 = frame.samples.iter()
.map(|&s| (s as i64 * s as i64) as u64)
.sum();
(sum_sq as f64 / frame.samples.len() as f64).sqrt()
}
/// Inspect a caller PCM frame + apply the debounce state machine.
/// Called from `on_pcm_frame` (the sink path). Returns true if the
/// VAD tripped THIS call (so the caller can push the advisory).
fn observe(&mut self, frame: &PcmFrame) -> bool {
let energy = Self::rms(frame);
if energy >= VAD_RMS_THRESHOLD {
self.above_threshold_streak += 1;
if self.above_threshold_streak >= VAD_DEBOUNCE_FRAMES && self.vad_armed {
self.vad_armed = false; // disarm until caller stops
return true; // trip!
}
} else {
// Caller went quiet → re-arm for the next speech burst.
self.above_threshold_streak = 0;
self.vad_armed = true;
}
false
}
}
impl<P: AudioPipe> AudioSource for LocalVadReflex<P> {
fn next_pcm_frame(&mut self) -> Option<PcmFrame> {
// Pure delegation — the VAD only observes the SINK path
// (inbound caller audio); playout is the inner Reflex's concern.
self.inner.next_pcm_frame()
}
}
impl<P: AudioPipe> AudioSink for LocalVadReflex<P> {
fn on_pcm_frame(&mut self, frame: PcmFrame) {
// THE PRIMARY TRIGGER: inspect the caller's audio BEFORE delegating.
// If the local VAD trips, push an advisory into the same channel the
// brain's advisories arrive on — the inner Reflex drains it on the
// next next_pcm_frame call + applies the kill. Zero brain round-trip.
if self.observe(&frame) {
let _ = self.advisory_tx.try_send(AdvisoryEvent::SpeechStarted {
at: Instant::now(),
});
// try_send failure (channel full) → drop + observe (hot-path
// policy). The brain's advisory path is the backstop; a missed
// local-VAD trip here is not catastrophic — the brain will fire
// its ASR-VAD ~300 ms later.
}
// Delegate to inner — the caller's audio still reaches the brain.
self.inner.on_pcm_frame(frame)
}
}
impl<P: AudioPipe> AudioPipe for LocalVadReflex<P> {
fn clear_playout_ring(&mut self) { self.inner.clear_playout_ring() }
fn barge_in_flush(&mut self) { self.inner.barge_in_flush() }
}
```
**Why the VAD trips `on_pcm_frame` (sink), not `next_pcm_frame` (source):** the caller's
audio is what the VAD inspects — and caller audio arrives via the sink path (decoded from
the peer's RTP by `loop_driver::drive` → `session.pipe.on_pcm_frame(pcm)`). The source path
is the brain's `audio_out` — the playout being killed, not the signal being detected.
**Why the SAME advisory channel as the brain:** the inner `Reflex` drains all advisories
uniformly — it doesn't care whether the source is local VAD or the brain's ASR. The brain's
advisory arriving ~300 ms later either finds the kill already applied (no-op, the `muted`
flag is already true) or confirms it. This is the composition pattern §6.4 anticipated; the
`Reflex<P>` wrapper shape was designed for exactly this layering.
### 3.5 `Reflex<P>` struct + impl
```rust
// crates/rutster-media/src/reflex.rs
pub struct Reflex<P: AudioPipe> {
inner: P,
advisory_rx: mpsc::Receiver<AdvisoryEvent>,
muted: bool,
barge_epoch: u64,
metrics: Arc<ReflexMetrics>,
}
impl<P: AudioPipe> Reflex<P> {
pub fn new(inner: P, advisory_rx: mpsc::Receiver<AdvisoryEvent>, metrics: Arc<ReflexMetrics>) -> Self {
Self { inner, advisory_rx, muted: false, barge_epoch: 0, metrics }
}
/// Drain all pending advisories + apply the state table. Called at
/// the top of `next_pcm_frame`. Hot-path: try_recv loop, bounded.
fn drain_advisories(&mut self) {
while let Ok(ev) = self.advisory_rx.try_recv() {
match ev {
AdvisoryEvent::SpeechStarted { at } => {
self.muted = true;
self.barge_epoch = self.barge_epoch.wrapping_add(1);
self.inner.barge_in_flush();
self.metrics.barge_in_count.fetch_add(1, Ordering::Relaxed);
tracing::info!(epoch = self.barge_epoch, ?at, "barge-in");
}
AdvisoryEvent::SpeechStopped { at: _ } => {
self.metrics.advisory_observed_speech_stopped.fetch_add(1, Ordering::Relaxed);
// No state change — see §3.2.
}
}
}
}
}
impl<P: AudioPipe> AudioPipe for Reflex<P> {
fn next_pcm_frame(&mut self) -> Option<PcmFrame> {
self.drain_advisories();
if self.muted {
// Muted: pull from inner. Some(f) = fresh brain audio arrived
// post-barge → un-mute + return. None = silence, stay muted.
match self.inner.next_pcm_frame() {
Some(f) => {
self.muted = false;
Some(f)
}
None => {
self.metrics.frames_suppressed.fetch_add(1, Ordering::Relaxed);
None
}
}
} else {
self.inner.next_pcm_frame()
}
}
fn on_pcm_frame(&mut self, frame: PcmFrame) {
// Inbound caller audio is NEVER gated by the reflex. The brain
// still hears the caller during barge — that's the point (the
// brain needs to know the caller interrupted; the FOB only kills
// its OWN playout, not the caller's path to the brain).
self.inner.on_pcm_frame(frame)
}
fn clear_playout_ring(&mut self) {
// The reconnect-flush path (slice-2 §5.3) still works through the
// wrapper. If it fires during mute, the ring stays empty and mute
// clears on the next post-reconnect audio_out.
self.inner.clear_playout_ring()
}
fn barge_in_flush(&mut self) {
// Allow the outer `LocalVadReflex` (primary trigger) to barge the inner.
self.inner.barge_in_flush()
}
}
```
### 3.6 `ReflexMetrics`
Mirror of `TapMetrics` shape (atomics + snapshot struct):
```rust
// crates/rutster-media/src/reflex.rs
#[derive(Default)]
pub struct ReflexMetrics {
pub barge_in_count: AtomicU64,
pub advisory_dropped: AtomicU64, // advisory channel full (e.g. 16-cap)
pub frames_suppressed: AtomicU64, // None returns while muted
pub advisory_observed_speech_stopped: AtomicU64,
}
pub struct ReflexMetricsSnapshot {
pub barge_in_count: u64,
pub advisory_dropped: u64,
pub frames_suppressed: u64,
pub advisory_observed_speech_stopped: u64,
}
// `barge_drained_inflight` lives on `TapMetrics` (in `rutster-tap`), not
// `ReflexMetrics`, because the drain happens inside `TapAudioPipe::barge_in_flush`,
// not inside `Reflex`. The path: `Reflex::drain_advisories` calls
// `inner.barge_in_flush()` which is `TapAudioPipe::barge_in_flush`, which is
// where the `rx_audio_out` drain + the counter increment happen.
```
---
## 4. The dedicated media thread
### 4.1 `MediaThread`
```rust
// crates/rutster/src/media_thread.rs
pub struct MediaThread {
cmd_tx: mpsc::Sender<MediaCmd>,
join: Option<std::thread::JoinHandle<()>>,
}
enum MediaCmd {
AcceptOffer { id: ChannelId, sdp: String, reply: oneshot::Sender<Result<String, String>> },
Delete { id: ChannelId, reply: oneshot::Sender<()> },
Shutdown { reply: oneshot::Sender<()> },
}
```
Spawned at binary startup (`main.rs`), before `axum::serve`. The thread captures a
`tokio::runtime::Handle` (to spawn TapEngine tasks when `Connected` transitions fire) and
owns `HashMap<ChannelId, RtcSession>` + (per-session, lazily) the TapConn / advisory_rx /
Reflex wrapper.
### 4.2 Thread loop (per 10 ms meta-tick)
1. `cmd_rx.try_recv()` loop — handle ALL pending commands before ticking. `AcceptOffer`
calls `RtcSession::accept_offer(sdp)` and replies via the oneshot. `Delete` fires
`close_tx` + bounded-await the engine task (750 ms cap via
`tokio::runtime::Handle::block_on(timeout(...))`) — the std thread briefly enters the
tokio runtime to await; cold-path, not the 20 ms tick. `Shutdown` drains + replies.
2. For each `RtcSession` in the map:
- Drain per-session `flush_rx` side-channel (slice-2's existing disconnect-flush) BEFORE
`run_poll_once`.
- Call `RtcSession::run_poll_once(now)` — the unchanged `loop_driver::drive`.
- Observe `channel.state`:
- `Connected && tap.is_none()` → `handle.spawn(spawn_tap_engine(...))` to bring up the
tokio task; construct `Reflex::new(TapAudioPipe::new(...), advisory_rx, metrics)`;
call `RtcSession::set_pipe(reflex)`. Mirror of slice-2's spawn seam.
- `Closed` → remove the entry (drops the `RtcSession` + its pipe + advisory ends).
3. `std::thread::sleep(Duration::from_millis(10))` — 10 ms meta-tick.
### 4.3 `session_map.rs` rewire
`SessionEntry` loses `rtc: Arc<Mutex<RtcSession>>`, gains `cmd_tx: mpsc::Sender<MediaCmd>`
(cloned per-entry; cheap). `tap_url` stays (the thread reads it when spawning the engine).
`tap_conn: Option<TapConn>` moves onto the media thread (the thread owns it after spawn).
- `AppState::create_session` → sends a `Register { tap_url, reply }` command to the media
thread; the **thread** constructs `RtcSession::new()` (saves a cross-thread move of the
struct + keeps all `RtcSession` construction on the thread that owns it). The thread
replies with `(id, cmd_tx_for_this_session)`; axum stores `SessionEntry { cmd_tx, tap_url,
tap_conn: None }`.
- `AppState::get(id)` (SDP path) → `cmd_tx.send(AcceptOffer { ... }).await` + `reply.await`.
Cold-path; the axum handler is async.
- `AppState::close(id)` → `cmd_tx.send(Delete { id, reply }).await` + `reply.await`. The
reply returns after the TapEngine teardown completes on the thread.
- `spawn_poll_task` → `spawn_media_thread`: constructs the channels, spawns the std thread,
stores `cmd_tx` + `join` in `AppState`. Same idempotent-guard pattern.
### 4.4 TapEngine extension
`spawn_tap_engine` returns a third channel end: `advisory_tx: mpsc::Sender<AdvisoryEvent>`.
The pump loop, on receiving `speech_started` / `speech_stopped` from the brain (slice-3
already decodes these in the tap protocol layer — `protocol_events.rs`), pushes the
corresponding `AdvisoryEvent` into `advisory_tx`. If the channel is full, drop + count
(hot-path "drop + observe" policy; an advisory is a hint, not a command). The `Reflex`
wrapper holds `advisory_rx`.
### 4.5 `MockRealtimeBrain` extension
`rutster-brain-realtime/src/mock.rs` gains a programmable advisory schedule: the test can
register "after N `audio_in` frames received, send `speech_started`" and "after M more,
send `speech_stopped`". The mock already asserts `turn_detection: null` on
`session.update` (slice-3's S4 lock); slice-4 keeps that assertion.
---
## 5. Data flow
### 5.1 Barge-in (the kill) — primary path: local VAD, zero brain round-trip
```
1. caller speaks into mic → peer RTP → str0m decode → on_pcm_frame
→ LocalVadReflex::on_pcm_frame INSPECTS the frame FIRST:
• rms(frame) computed (~480 muls, < 1 µs)
• if rms ≥ VAD_RMS_THRESHOLD: streak++; if streak ≥ VAD_DEBOUNCE_FRAMES (3 = 60 ms) → TRIP
• on trip: try_send(AdvisoryEvent::SpeechStarted) into advisory_tx (same channel as
brain's advisories) → disarm until caller goes quiet
→ THEN delegates to inner.on_pcm_frame(frame) → tx_pcm_in → TapClient → audio_in (WS) → brain
2. [THREE HOPS HAPPEN HERE, ON THE BRAIN SIDE — NOT on the kill path]:
- brain's ASR-VAD fires (~300 ms later, slower but more accurate)
- brain sends speech_started back over WS
- TapEngine → advisory_tx (the SAME mpsc; both sources feed one channel)
3. media thread next 20 ms tick → Reflex::next_pcm_frame → drain_advisories:
• The LOCAL VAD's SpeechStarted arrives FIRST (it was pushed in step 1, same thread,
no WS hop) → muted=true; epoch++; inner.barge_in_flush() (ring cleared + rx_audio_out drained)
• The brain's SpeechStarted arrives ~300 ms LATER → Reflex sees muted=true already
→ re-barge (epoch++, barge_in_flush again — harmless, mute stays)
→ returns None (silence) for this + subsequent ticks while muted
4. loop_driver::drive pulls None from pipe → encodes Opus silence → peer hears silence
(the brain's in-flight audio_out frames are dropped; no overlap with caller's speech)
```
**Latency budget:** caller speaks → kill fires in ≤ 60 ms (3 debounce frames × 20 ms tick) +
one tick to drain the advisory + apply the kill = ≤ 80 ms wallclock. Zero brain round-trip
on the primary path. (The brain's ASR-VAD advisory arrives ~300 ms later — it confirms the
kill but doesn't gate it.) This is what ARCHITECTURE.md:80 demands ("the brain round-trip is
too slow to enforce them") and the proof wedge #1 rests on.
### 5.2 Resume (the un-mute)
```
1. brain decides to yield/respond → sends a fresh audio_out frame
(provably post-barge: barge_in_flush drained rx_audio_out)
2. TapClient → audio_out (WS) → TapEngine → tx_audio_out → rx_audio_out → playout ring
3. media thread 20 ms tick → Reflex::next_pcm_frame → drain_advisories (empty)
→ muted=true → inner.next_pcm_frame() returns Some(f) (fresh brain audio)
→ muted=false; return Some(f)
4. loop_driver encodes + writes → peer hears the brain's new response
```
### 5.3 Cold-path (axum ↔ media thread)
```
- POST /v1/sessions → AppState::create_session → MediaCmd::Register → thread constructs RtcSession → reply(id)
- POST /v1/sessions/{id}/offer → AppState::get + cmd_tx.send(AcceptOffer) → thread.lock(session).accept_offer(sdp) → reply(answer)
- DELETE /v1/sessions/{id} → AppState::close → cmd_tx.send(Delete) → thread: fire close_tx, bounded-await engine task teardown → reply
- graceful shutdown → cmd_tx.send(Shutdown) → thread drains + drops → reply → join
```
---
## 6. Why these decisions
### 6.1 Why both, local VAD primary (revised after adversarial review)
The initial brainstorming landed on advisory-only for MVP (cheapest path to a working
barge-in; the brain's VAD already runs for STT). The 2026-07-01 adversarial review surfaced
the load-bearing problem with that choice: README:98-100 + ARCHITECTURE.md:79-81 rest the
wedge on "local reflexes that don't need the brain — VAD killing TTS the instant the caller
speaks." Advisory-only puts the brain round-trip in the trigger path (brain VAD → WS →
TapEngine → mpsc → Reflex); the kill DECISION is in-core but the TRIGGER SOURCE crossed
the brain. The spearhead's step 4 would prove a reflex that depends on the brain — the
opposite of the property steps 1-4 exist to prove.
The revision: **local VAD in `on_pcm_frame` is the primary trigger** (RMS/energy detector on
the dedicated thread, in the 20 ms loop, zero brain round-trip — the actual wedge-#1 proof).
Slice-3's advisory becomes the **secondary/confirmation** signal — the brain's ASR-quality
VAD confirms the local kill ~300 ms later (slower but more accurate — it knows words, not
just energy). The two sources feed the SAME advisory mpsc; the `Reflex` wrapper drains them
uniformly. The composition is `LocalVadReflex<Reflex<TapAudioPipe>>` — the decorator pattern
from §6.4, which was originally specced as deferred to "when local VAD arrives." It arrived.
- The VAD itself (~25 lines: RMS, threshold, debounce) is in scope. The **tuning framework**
(configurable thresholds, per-environment calibration, adaptive noise floor) is deferred —
the MVP ships a single `const` threshold + N-frame debounce, exercised by the e2e test
with a synthetic loud signal.
- Slice-3's advisory plumbing + `MockRealtimeBrain` schedule aren't wasted — they become
the secondary path, still exercised by the e2e test (the brain's advisory fires SLIGHTLY
after the local VAD; both feeds land in the Reflex's drain).
- The `Reflex<P>` wrapper shape already supported composition — no structural rework, just
a new `LocalVadReflex<P>` decorator landing in scope (Task 2b in the plan).
### 6.2 Why resume on first fresh `audio_out` (not `speech_stopped`)
- The "the brain has yielded and started a new response" condition is provably signaled by
the first `audio_out` frame after the barge — not by the caller's silence. `speech_stopped`
fires between words; resuming on it un-mutes too early (inter-word-gap overlap).
- The `barge_in_flush` drain of `rx_audio_out` makes the resume race-free: the first
`audio_out` observed post-barge is provably post-barge (frames queued pre-barge are dropped
in the flush).
### 6.3 Why a single dedicated thread (not per-session)
- Spearhead scale: one loopback peer in dev; even at low PSTN concurrency (slice-5), one
thread drives dozens of sessions in 10 ms.
- The command-channel seam between axum and the thread makes the graduation to a threadpool
shard localized — when per-CPU-shard threading arrives, it's a fan-out of the
`cmd_rx`/`HashMap` shape, not a redesign.
- Per-session threads arrive when load demands; the spearhead's "shortest blocking path"
rule dislikes spawning work per session that may not need it (pre-ICE-connected sessions
would redundantly spin).
### 6.4 Why `Reflex<P>` as a wrapper (not inline in `TapAudioPipe`)
- Composition: `LocalVadReflex<P>` composes outside the advisory `Reflex<P>` (§6.1), the
same way `Reflex<TapAudioPipe>` composes. The pattern (decorator over `AudioPipe`) stacks
the two trigger sources without restructuring either.
- The seam: `loop_driver.rs` byte-identical (still calls `pipe.next_pcm_frame()`). If the
reflex lived inline in `TapAudioPipe`, the binary-side wiring would still change but the
`TapAudioPipe` module itself would grow the reflex state — less isolated.
- The payoff is realized this slice, not deferred: two stacked reflexes (local-VAD primary +
advisory secondary) live as independent, separately-testable decorators rather than one
module's commingled state. Keeping them as wrappers is what makes that separation free.
### 6.5 Why `barge_in_flush` on `AudioPipe` (not just `clear_playout_ring`)
- `clear_playout_ring` (slice-2) clears the *ring*. `barge_in_flush` clears the ring AND
drains the *inbound brain queue* (`rx_audio_out`). The distinction matters: on a brain
disconnect (slice-2's case), the brain is gone — `rx_audio_out` will drain itself on the
next `Disconnected` `try_recv`. On a barge-in, the brain is alive and may have queued
frames pre-barge that would un-mute immediately if not drained here. Two different
"clear the playout path" semantics, two methods.
---
## 7. Done-criteria
1. `cargo test --all` passes (stable + 1.85, the CI matrix).
2. `cargo fmt --check` + `cargo clippy -- -D warnings` clean.
3. `loop_driver.rs` + `rtc_session.rs` **byte-identical** to slice-3 — CI-asserted via
`git diff --exit-code main -- crates/rutster-media/src/loop_driver.rs
crates/rutster-media/src/rtc_session.rs` (the §8.5 #6 seam gate, restated for slice-4).
4. Dedicated media thread drives sessions off the tokio pool; `MediaThread` integration test
passes (AcceptOffer / Delete / Shutdown).
5. `Reflex` state-machine unit tests all pass:
- `SpeechStarted` → next `next_pcm_frame` returns None even if ring has frames.
- `SpeechStarted` then `inner.next_pcm_frame()=Some` → un-mutes, returns the frame.
- `SpeechStopped` during Muted → stays Muted.
- `SpeechStopped` during Playing → no-op.
- Duplicate `SpeechStarted` re-flushes + stays Muted.
- Metrics counters (`barge_in_count`, `frames_suppressed`) increment correctly.
- `advisory_rx` full → `advisory_dropped` increments, no panic.
6. `LocalVadReflex` unit tests pass:
- RMS computation verified against a known-loud + known-quiet frame.
- Debounce: N-1 above-threshold frames do NOT trip; the Nth does.
- Re-arm: above-threshold → trip; below-threshold → re-arm; next streak trips again.
- `on_pcm_frame` ALWAYS delegates to inner (caller audio reaches the brain even during barge).
7. `barge_in_flush` unit tests pass (ring + `rx_audio_out` drain).
8. Barge-in e2e (PRIMARY PATH, proves wedge #1): synthetic loud caller audio → playout killed
within ≤4 ticks (≤80 ms wallclock: 3 debounce + 1 drain+apply) **WITHOUT any brain advisory**.
The brain's `speech_started` advisory arrives later + is a no-op (mute already applied).
9. Barge-in e2e (SECONDARY PATH, exercises slice-3 advisory plumbing): `MockRealtimeBrain`
emits `speech_started` on schedule; local VAD is NOT tripped (quiet synthetic caller audio);
advisory → kill → fresh `audio_out` → resume. Proves the advisory→reflex→kill path still works.
10. S4 turn-ownership lock preserved: `MockRealtimeBrain` still asserts
`turn_detection: null` on `session.update` (slice-3's #7, unchanged).
11. `MockRealtimeBrain` extended to emit `speech_started`/`speech_stopped` on schedule.
12. `cargo doc --no-deps` renders the new `reflex.rs` + `media_thread.rs` module/item docs
cleanly (learner-facing comments present per AGENTS.md code style).
---
## 8. Open decisions
- ~~Trigger source~~ — decided: **both, local VAD primary + advisory secondary** (revised after
2026-07-01 adversarial review; initial brainstorming landed on advisory-only, the review
surfaced that advisory-only contradicts wedge #1's "VAD killing TTS the instant the caller
speaks, without the brain"). Local VAD in `on_pcm_frame` is the primary trigger; the brain's
`speech_started`/`speech_stopped` advisory is the secondary/confirmation.
- ~~Resume semantics~~ — decided: first fresh `audio_out` post-barge; `SpeechStopped`
observational only.
- ~~Thread model~~ — decided: single dedicated `std::thread`; per-session/threadpool deferred.
- **`MockRealtimeBrain` advisory schedule API shape** — landed in §4.5 as a programmable
"after N audio_in frames" schedule. Could alternatively be a free-form
`Vec<(trigger_frame_count, AdvisoryEvent)>` queue. The plan will pin the concrete API.
- **Thread shutdown ordering vs TapEngine teardown** — `Delete` command handler fires
`close_tx` + bounded-await the engine task (750 ms cap via
`tokio::runtime::Handle::block_on(timeout(...))`); the reply oneshot returns after
teardown. Cold-path, std thread briefly enters the tokio runtime to await. Documented as
an acceptable deviation (not the 20 ms tick).
---
## 9. Cross-references
- [slice-1 spec](2026-06-28-slice-1-webrtc-loopback-design.md) — the media loop + the seam
(`AudioSource`/`AudioSink` traits in `rutster-media`); slice-1 §8.5 #6 is the seam gate
this slice re-affirms.
- [slice-2 spec](2026-06-28-slice-2-agent-tap-design.md) — the tap interface, the
`TapAudioPipe`, the core-authoritative playout buffer (§4.1), the `flush_tx` side-channel
pattern that the `advisory_rx` mirrors.
- slice-3 (merged `c30a452`) — `MockRealtimeBrain`, the translator, the
`speech_started`/`speech_stopped` protocol events, the S4 turn-ownership lock.
- [ADR-0002](../../adr/0002-north-star-and-fused-core.md) — fused vertical; the hot-path
hop invariant this slice re-affirms (§2.3 audit).
- [ADR-0008](../../adr/0008-fob-and-green-zone.md) — FOB/green-zone doctrine; the reflex is
a FOB member (hot-path, security-constitutive for turn-taking, differentiating).
- [ARCHITECTURE.md](../../ARCHITECTURE.md) — §"Media plane" ("Dedicated timing threads
for the 20ms loop, never the shared tokio pool" — this slice lands it); §"Biggest
technical risk" (the reflex loop *is* the remaining long pole).
- [PORT_PLAN.md](../../PORT_PLAN.md) — §Phasing, step 4 = barge-in.