Slice 2 hardens the presumptive agent-tap shape from ARCHITECTURE.md §"Agent tap" (PORT_PLAN §10 open decision) against a working implementation: - Replace slice-1's in-process EchoAudioPipe with a real out-of-process brain reached over WSS; core-as-client, no inbound tap port (ADR-0006 posture). - Versioned JSON event protocol (v1) over WS text frames with base64 PCM. Event-named + JSON-over-text-WS so the step-3 OpenAI-Realtime adapter is a translation shim, not a gRPC-bridge project. - Decoupled TapEngine (cold-path tokio task) owns the WS connection; TapAudioPipe is a thin sync wrapper the AudioSource/AudioSink seam holds. RtcSession's media loop is byte-identical to slice-1 — the seam-test payoff. - Core-authoritative playout buffer (5-frame/100ms ring; drop-oldest on overflow, silence on underflow) — the place where "brain proposes, core disposes" lives. - Bounded-backoff reconnect (250ms→5s cap, infinite retries); Channel stays Connected; stateless-brain reconnect contract. - Both a Python reference echo brain (examples/echo_brain/, not in CI) and a Rust echo brain crate (crates/rutster-tap-echo, the in-process test server + standalone dev binary). - ws:// loopback-only enforced; wss:// URL accepted at schema, 501 at connect (cert/mTLS impl deferred to step 6). Out-of-scope table (§1.2) locks the spearhead sequencing: real brain (3), barge-in/VAD-driven playout kill (4), PSTN trunk (5), spend cap (6), wss:// TLS posture (6), binary PCM mode (future-rung), byte-endian negotiation (tracked open decision §9).
739 lines
46 KiB
Markdown
739 lines
46 KiB
Markdown
# Rutster slice 2 — The agent tap: splicing the brain seam
|
||
|
||
- **Status:** Draft (pending review)
|
||
- **Date:** 2026-06-28
|
||
- **Spearhead step:** 2 of 6 (vision-revision §10 / PORT_PLAN "Phasing")
|
||
- **Origin:** brainstorming session 2026-06-28
|
||
- **Depends on:** [slice 1 — WebRTC media loopback](2026-06-28-slice-1-webrtc-loopback-design.md)
|
||
(all slice-1 code must be landed and green)
|
||
- **Related:** [ADR-0002](../../adr/0002-north-star-and-fused-core.md) (fused vertical),
|
||
[ADR-0004](../../adr/0004-license.md) (GPL-3.0-or-later),
|
||
[ADR-0006](../../adr/0006-ingress-posture.md) (core-as-client tap posture),
|
||
[ARCHITECTURE.md §"Agent tap"](../../ARCHITECTURE.md) (the presumptive tap shape this
|
||
slice hardens)
|
||
|
||
---
|
||
|
||
## TL;DR
|
||
|
||
Stand up spearhead step 2: rip out the in-process `EchoAudioPipe` from slice 1 and
|
||
splice in a real **external brain** reached over WebSocket. The core dials out
|
||
(core-as-client; brain-as-server; **no inbound tap port on the core**), speaks a small
|
||
versioned JSON event protocol, and owns a core-authoritative playout buffer where the
|
||
brain *proposes* audio (`AudioOut` frames) and the core *disposes* (drops-oldest on
|
||
overflow, emits silence on underflow).
|
||
|
||
Slice 2 proves the **tap interface**: the same WSS plumbing that today echoes will, in
|
||
step 3, carry a real STT/LLM/TTS brain. It deliberately omits the real brain (step 3),
|
||
barge-in / VAD-driven playout kill (step 4), the PSTN trunk (step 5), and spend control
|
||
(step 6) — but it pre-paves the OpenAI-Realtime adapter shape by choosing an event-named
|
||
JSON protocol that translates cleanly to OpenAI's event taxonomy.
|
||
|
||
The seam slice 1 pre-paved (`AudioSource` / `AudioSink` traits in `rutster-media`) is the
|
||
**test of this slice**: `RtcSession`'s media-loop path changes by exactly one line — swap
|
||
`EchoAudioPipe` for `TapAudioPipe` — and `loop_driver.rs` does not change at all.
|
||
|
||
---
|
||
|
||
## 1. Scope
|
||
|
||
### 1.1 In scope
|
||
|
||
- Implementation of spearhead step 2: WebRTC WebRTC peer → core terminates DTLS-SRTP,
|
||
decodes Opus → canonical PCM @ 24 kHz mono, ships PCM **over WSS** to an external echo
|
||
brain, receives PCM back, encodes + plays out via str0m. The user speaks and hears
|
||
themselves back, **routed through an out-of-process brain**, with no perceptible delay
|
||
(~≤250 ms; slice-1's 200 ms + tap round-trip + 100 ms playout buffer headroom).
|
||
- A small **versioned JSON event protocol** for the tap wire (envelope + event types:
|
||
`hello`, `audio_in`, `audio_out`, `session_end`, `bye`, `error`).
|
||
- **`TapEngine`**: a cold-path tokio task per session that owns the WSS connection and
|
||
shovels PCM between WS frames and the playout buffer.
|
||
- **`TapAudioPipe`**: a thin sync wrapper that the existing `AudioSource` / `AudioSink`
|
||
seam holds — `RtcSession` swaps it in for `EchoAudioPipe`, nothing else changes there.
|
||
- **Core-authoritative playout buffer** (bounded ring; drop-oldest on overflow; silence
|
||
on underflow) — the concrete embodiment of "brain proposes, core disposes."
|
||
- `tap: Option<TapHandle>` field on `Channel` (locked-in by slice-1 §5.2). `TapHandle`
|
||
is a zero-cost marker newtype; the binary looks up the live tap connection by the
|
||
channel's existing `ChannelId` in an internal `DashMap<ChannelId, TapConn>`, so
|
||
`rutster-call-model` stays a leaf (no tokio dep).
|
||
- Two-source tap URL config: `RUTSTER_TAP_URL` env default + optional per-call
|
||
`tap_url` in `POST /v1/sessions` body. `ws://` loopback-only enforced; `wss://`
|
||
URL accepted by the schema but cert/mTLS impl deferred to step 6.
|
||
- **Bounded-backoff reconnect** on brain disconnect: 250 ms → 500 ms → 1 s → 2 s → cap at
|
||
5 s, infinite retries. `Channel` stays `Connected` throughout; playout falls to silence
|
||
during outage; reconnect re-`hello`s with the same `session_id` (stateless brain
|
||
contract).
|
||
- **Python reference echo brain** (`examples/echo_brain/`) — the canonical foreign-language
|
||
brain demo (README-documented, runnable, **not in CI**).
|
||
- **Rust echo brain crate** (`crates/rutster-tap-echo`) — a dual-purpose binary crate:
|
||
a standalone dev-loop binary (`cargo run -p rutster-tap-echo`) and an in-process
|
||
`EchoServer` used by integration tests.
|
||
- Two new deps: `tokio-tungstenite` (WS client + server), `serde_json` if not already
|
||
pulled. New workspace member crate. Thorough learner-facing comments on the new
|
||
async/mpsc/ring-buffer patterns (slice-1 §7 standard carries over).
|
||
|
||
### 1.2 Out of scope (with scheduled return)
|
||
|
||
| Deferred item | Returns in | Why deferred |
|
||
|---|---|---|
|
||
| `wss://` cert validation / mTLS / brain cert pinning | Step 6 (spend cap) | Same rationale as slice-1 HTTP TLS — TLS needs the cert story from ARCHITECTURE.md (Vault/KMS), which lands with the real trust boundary + authz. `ws://` 127.0.0.1 dev loop is sufficient to prove the tap interface. Slice-2 accepts `wss://` URLs at the schema level but the connect attempt returns a clear `501 NOT IMPLEMENTED` so the seam is reserved without ratifying a half-impl. |
|
||
| Authn / authz on the `tap_url` override | Step 6 | Inherits slice-1's "no auth yet" posture; authz on who-may-set-`tap_url` lands with the spend gate. Spec §4.5 flags this as a known gap. |
|
||
| Barge-in / VAD-driven playout kill | Step 4 | No reflex to enforce yet; the playout buffer in slice-2 just queues + drops on overflow, doesn't kill on caller speech. |
|
||
| Real brain (STT / LLM / TTS, OpenAI Realtime, etc.) | Step 3 | Slice-2 proves the *interface* and the core-authoritative playout posture; step 3 swaps echo → real brain. |
|
||
| OpenAI-Realtime adapter | Step 3 | The translation shim from our protocol → OpenAI Realtime's event schema is step-3 work. Slice-2's protocol is designed with that translation in mind (event-named, versioned, JSON-over-text-WS — same shape OpenAI Realtime uses). |
|
||
| Re-INVITE / session migration / resumability | Later | Refresh the page → new session, same as slice-1. Tap reconnect reuses the `ChannelId` as `session_id`, but there's no in-flight call preservation across a server restart. |
|
||
| CDR / event bus / OTel beyond per-Channel `tracing` spans | Step 5 | Single peer, single brain; no fanout yet. Tap reconnect + error counters go to logs + counters only. |
|
||
| Binary PCM mode (raw LE i16 over WS binary frames) | Future-rung | Base64 inside text JSON is ~33% overhead (960 B PCM → ~1.3 KB JSON → ~65 KB/s). Acceptable for dev loop; the wire format reserves `v: 2` for a binary mode. |
|
||
| Byte-endian negotiation | Tracked in §9 (Open decisions) | v1 emits the host's native endian as raw bytes inside base64. Documented as a v1 simplification; v2 should nail explicit LE byte order. |
|
||
| Per-tenant tap routing / multi-brain selection | Step 6 | One tap URL per call in slice-2 (env default + per-call override). Multi-brain routing is a deployment-posture concern. |
|
||
| Predictive dialer / spend cap / abuse gate | Step 6 | No spend surface yet; brain is an in-loopback echo, no metering to gate. |
|
||
| Trickle ICE, transfer / park / pickup, browser automation, fuzz harnesses | (unchanged from slice-1 §1.2) | Still deferred for the same reasons. |
|
||
|
||
### 1.3 What this slice does NOT prove
|
||
|
||
It does **not** prove: a real brain (only an echo process), barge-in playout kill, latency
|
||
determinism under reflex timing, PSTN trunking, spending controls, multi-tenancy on the
|
||
tap URL, or wss:// TLS posture in production. It proves **only** the tap interface: the
|
||
WS dial-out, the versioned event protocol, the playout buffer posture, the reconnect
|
||
behavior, and the seam-test (slice-1's `RtcSession` accepts a new pipe at the
|
||
`AudioSource`/`AudioSink` boundary with zero internal change).
|
||
|
||
---
|
||
|
||
## 2. Workspace layout (delta on slice-1)
|
||
|
||
One new workspace member, one new `examples/` dir, new `[workspace.dependencies]` entries.
|
||
|
||
```
|
||
rutster/
|
||
├── Cargo.toml # +[workspace.dependencies]: tokio-tungstenite, futures-util (and serde_json if not already pulled)
|
||
├── crates/
|
||
│ ├── rutster/ # binary: wires TapEngine per session
|
||
│ │ ├── src/main.rs
|
||
│ │ ├── src/session_map.rs # unchanged shape; ChannelId → RtcSession
|
||
│ │ ├── src/routes.rs # POST /v1/sessions body gains optional tap_url
|
||
│ │ ├── src/tap_engine.rs # NEW: spawns + supervises the per-session WSS task
|
||
│ │ └── static/index.html # minor: surface tap connection status in the <pre>
|
||
│ ├── rutster-media/ # swap EchoAudioPipe → TapAudioPipe at construction (in the binary, not here)
|
||
│ │ ├── src/pcm.rs # PcmFrame stays; EchoAudioPipe stays (slice-1 unit tests + dev-loop fallback)
|
||
│ │ ├── src/rtc_session.rs # UNCHANGED — the seam test
|
||
│ │ └── src/loop_driver.rs # UNCHANGED — calls sink/source, no awareness of tap
|
||
│ ├── rutster-call-model/ # +tap: Option<TapHandle> field; +TapHandle(()) marker newtype
|
||
│ │ └── src/lib.rs
|
||
│ ├── rutster-tap/ # FILLED IN (was stub): protocol + TapClient + TapAudioPipe
|
||
│ │ ├── src/lib.rs # module docs, error enum, re-export PcmFrame from rutster-media
|
||
│ │ ├── src/protocol.rs # JSON event schema, version field, frame codec
|
||
│ │ ├── src/tap_client.rs # WS connection driver (runs inside the TapEngine task)
|
||
│ │ └── src/tap_audio_pipe.rs # AudioSource + AudioSink impl over mpsc + playout ring
|
||
│ ├── rutster-tap-echo/ # NEW crate: the Rust reference echo brain + test server
|
||
│ │ ├── src/lib.rs # EchoServer::start(addr) -> JoinHandle + EchoHandle (test driver)
|
||
│ │ └── src/main.rs # standalone binary: ws://127.0.0.1:<port>, echo audio_in → audio_out
|
||
│ ├── rutster-signaling-sip/ # STUB (unchanged)
|
||
│ └── rutster-spend/ # STUB (unchanged)
|
||
└── examples/
|
||
└── echo_brain/
|
||
├── README.md # how to run, what the protocol is, pointer to this spec
|
||
├── echo_brain.py # canonical foreign-language brain (websockets lib, ~80 lines)
|
||
└── requirements.txt # websockets
|
||
```
|
||
|
||
### 2.1 Dependency direction (delta from slice-1 §2.3)
|
||
|
||
- `rutster-tap` → `rutster-media` (for `PcmFrame`), per slice-1 §3.1's promise that
|
||
"`rutster-tap` will re-export it once that crate fills in (step 2)." `PcmFrame` re-export
|
||
preserved from `rutster-media`; one canonical home remains.
|
||
- `rutster` (binary) → `rutster-tap` (new; for `TapAudioPipe`/`TapClient` types) and
|
||
→ `rutster-tap-echo` (dev-binary + integration-test `EchoServer`).
|
||
- `rutster-tap-echo` → `rutster-tap` (reuses the protocol types — proves the wire types
|
||
are reusable from a separate brain implementation, the contract-test for "anyone can
|
||
write a brain in Rust").
|
||
- `rutster-call-model` **stays a leaf**; `TapHandle` is a zero-sized marker newtype
|
||
there (no tokio dep). The binary maps the channel's `ChannelId → mpsc::Sender` / `Receiver` via an internal
|
||
`DashMap<ChannelId, TapConn>` — the call model carries only the marker, not the
|
||
connections.
|
||
- `rutster-media` ↔ `rutster-tap`: **only via the trait seam** — `rutster-media` defines
|
||
`AudioSource` / `AudioSink` / `PcmFrame`; `rutster-tap` implements the traits.
|
||
`rutster-media` does **not** depend on `rutster-tap` (and never will — that would
|
||
invert the canonical-home of `PcmFrame` and pull the loopback peer into the tap story).
|
||
|
||
### 2.2 Why keep `EchoAudioPipe` in `rutster-media`
|
||
|
||
Slice-1's in-process echo pipe isn't deleted in slice-2. Rationale:
|
||
|
||
1. `rutster-media`'s slice-1 unit tests use it directly to exercise the codec + loop
|
||
driver without a network. Deleting it would break slice-1's tests-as-learning-aids.
|
||
2. A `--features=echo` dev mode on the binary (routes audio through `EchoAudioPipe`
|
||
instead of `TapAudioPipe`) keeps the dev loop fully zero-network-dependency when the
|
||
tap isn't needed (e.g. reproducing a slice-1 bug). Default = tap; feature = echo.
|
||
3. Two impls of the same trait is the cleanest possible documentation that the seam
|
||
*is* a seam.
|
||
|
||
### 2.3 Why one new crate for the Rust echo brain (not an `examples/` file)
|
||
|
||
Both artifacts ship (Python in `examples/`, Rust as a crate) — this dual is the
|
||
brainstorming resolution of the "showcase Rust binaries + external Python" goal:
|
||
|
||
- **`crates/rutster-tap-echo`** is a real workspace member. It runs `cargo fmt`,
|
||
`cargo clippy -D warnings`, `cargo test`, and `cargo deny check` like every other
|
||
crate. It reuses `rutster-tap`'s protocol types — **the contract-test that the wire
|
||
types are reusable from outside the core**. It doubles as the in-process `EchoServer`
|
||
for integration tests (with hooks to inject deliberate disconnects, malformed frames,
|
||
underflow, overflow).
|
||
- **`examples/echo_brain/echo_brain.py`** is the canonical foreign-language brain demo,
|
||
hand-rolled from the documented protocol text. It proves the wire format is
|
||
language-agnostic and matches the "brain is a Python script" persona from
|
||
ARCHITECTURE.md. **Not in CI** (Python would violate the zero-non-Rust-dev-deps dev
|
||
loop). README-documented runnable: `pip install websockets && python
|
||
examples/echo_brain/echo_brain.py`.
|
||
|
||
---
|
||
|
||
## 3. Tap wire protocol (`rutster-tap/src/protocol.rs`)
|
||
|
||
A minimal **versioned JSON event protocol over WS text frames**. Every frame is one JSON
|
||
object. PCM payloads are base64-encoded raw bytes of the `PcmFrame`'s `[i16; 480]`
|
||
in-memory repr (host-endian — see §9 Open Decisions for the v1 simplification; on a
|
||
little-endian host this is LE).
|
||
|
||
### 3.1 Envelope (common to all messages)
|
||
|
||
```jsonc
|
||
{
|
||
"v": 1, // protocol_version (integer; this slice ships v1)
|
||
"type": "<event_name>", // string, one of the names in §3.2 / §3.3
|
||
"seq": <uint>, // per-direction monotonic counter, starts at 0; gaps = loss
|
||
"ts": <uint> // monotonic ms since the direction's session_start (clock = sender's; advisory)
|
||
}
|
||
```
|
||
|
||
- `seq` is **per-direction** (core maintains its own egress counter; brain maintains its
|
||
own). The receiver detects gaps by tracking the last-seen `seq` and counting skips.
|
||
Out-of-order frames are treated as loss (slice-2 has no reorder buffer — WS guarantees
|
||
per-connection ordering anyway, so out-of-order would only happen across a reconnect;
|
||
the reconnect path resets both `seq` counters to 0). A mismatch → logged + counter
|
||
incremented; the frame is **not** dropped on `seq` gap (latency > perfect-ordering here).
|
||
- `ts` is advisory; no wall-clock sync assumed between core and brain.
|
||
|
||
### 3.2 Messages — core → brain (egress from core's POV)
|
||
|
||
| `type` | payload fields | when |
|
||
|---|---|---|
|
||
| `hello` | `{ "session_id": "<uuid>", "sample_rate": 24000, "channels": "mono", "frame_ms": 20 }` | first message after WS connect; declares the canonical PCM format (will not change mid-session; re-sent on reconnect with the same `session_id` — see §5.3) |
|
||
| `audio_in` | `{ "pcm": "<base64>", "samples": 480 }` | on each decoded `PcmFrame` from the peer — the peer's mic → brain direction |
|
||
| `session_end` | `{ "reason": "hangup" \| "idle_timeout" \| "shutdown" }` | core tearing down the call; brain should expect a WS close frame to follow |
|
||
| `bye` | `{ "reason": "..." }` | graceful protocol-level close initiated by the core before the WS close frame |
|
||
| `error` | `{ "code": "<slug>", "message": "..." }` | protocol-level error from the core (e.g. a malformed brain frame was received); the call stays up, this is an FYI |
|
||
|
||
### 3.3 Messages — brain → core (the "brain proposes" direction)
|
||
|
||
| `type` | payload fields | when |
|
||
|---|---|---|
|
||
| `hello` | `{ "session_id": "<uuid>" }` (echo back) | brain acks the session handshake |
|
||
| `audio_out` | `{ "pcm": "<base64>", "samples": 480 }` | brain-proposed outbound audio — **advisory**; core enqueues in the playout ring (§4.2) |
|
||
| `bye` | `{ "reason": "..." }` | graceful brain-initiated exit; core enters the reconnect path (§5.2) |
|
||
| `error` | `{ "code": "<slug>", "message": "..." }` | brain errors; the call stays up; core logs + counter |
|
||
|
||
### 3.4 Invariants and forward-compat
|
||
|
||
- **Sample-count invariant:** every `audio_in` / `audio_out` declares `samples: 480`
|
||
(20 ms @ 24 kHz mono). The receiver validates; mismatched frames are logged + counted
|
||
+ dropped (hot-path "drop + observe" policy from slice-1 §3.8 — *not* a
|
||
connection-terminating error).
|
||
- **Versioning:** `v: 1`. Unknown envelope fields are ignored (forwards-compat for
|
||
additive changes). Unknown `type` values are logged + counted + dropped (not fatal).
|
||
A future v2 negotiates via a `v` upgrade on `hello`.
|
||
- **Single text-JSON mode in v1** — base64 inside text JSON is the only mode. ~33% wire
|
||
overhead is acceptable at 24 kHz mono i16 (960 B PCM/frame → ~1.3 KB JSON → ~65 KB/s).
|
||
The protocol asserts `v: 1` in every envelope so a future `v: 2` binary mode is a
|
||
clean break, not a legacy compat hazard.
|
||
|
||
### 3.5 Why JSON + base64 over binary length-prefixed framing
|
||
|
||
ARCHITECTURE.md names WSS as the presumptive transport because "the consumer is a Python
|
||
script / a browser / an OpenAI-Realtime-style speech-to-speech API for which
|
||
event-framed WSS is already the de-facto protocol." A JSON event envelope is the natural
|
||
mapping onto that ecosystem — both OpenAI Realtime's events and a hand-rolled Python
|
||
brain's `json.loads` want the same shape. A binary length-prefixed framing would be
|
||
cheaper on the wire (~33% smaller, ~50 µs less encode/decode) but would force every
|
||
brain — including the canonical Python reference and the step-3 OpenAI adapter — to
|
||
implement a byte-parser instead of `json.loads`. The wire overhead (65 KB/s) is
|
||
negligible at slice-2's scale and the brain-authoring ergonomics dominate. The Open
|
||
Decisions entry (§9) tracks the binary-mode re-evaluation for a later rung.
|
||
|
||
---
|
||
|
||
## 4. Tap plumbing (`rutster-tap`)
|
||
|
||
Three modules, three responsibilities. The split is the "Approach B — Decoupled
|
||
TapEngine" decision from the brainstorming session: keep `TapAudioPipe` a thin sync
|
||
wrapper over mpsc, isolate all WSS awareness in the `TapClient` (which the `TapEngine`
|
||
task runs), and let the binary spawn + supervise the task.
|
||
|
||
### 4.1 The seam: `TapAudioPipe` (`src/tap_audio_pipe.rs`)
|
||
|
||
The sync object `RtcSession` holds and the `loop_driver` calls via the trait seam.
|
||
|
||
```rust
|
||
pub struct TapAudioPipe {
|
||
// Core → brain (inbound decoded PCM from peer):
|
||
tx_pcm_in: mpsc::Sender<PcmFrame>, // fed by AudioSink::on_pcm_frame; drained by TapClient (audio_in WS frames)
|
||
|
||
// Brain → core (playout buffer for outbound PCM to encode + push to str0m):
|
||
playout_ring: std::collections::VecDeque<PcmFrame>, // bounded at TAP_PLAYOUT_FRAMES (5)
|
||
rx_audio_out: mpsc::Receiver<PcmFrame>, // fed by TapClient (audio_out WS frames → ring)
|
||
|
||
// Optional counters (loss, overflow, underflow) — hot-path drop+observe posture.
|
||
metrics: TapMetrics,
|
||
}
|
||
|
||
impl AudioSource for TapAudioPipe {
|
||
/// Take the next brain-proposed PCM frame to send to the peer. None = silence.
|
||
/// Drains the playout ring; underflow returns None (silence), overflow dropped earlier at enqueue.
|
||
fn next_pcm_frame(&mut self) -> Option<PcmFrame> {
|
||
match self.rx_audio_out.try_recv() {
|
||
Ok(frame) => Some(frame), // happy path: brain-proposed audio
|
||
Err(mpsc::TryRecvError::Empty) => None, // underflow → loop_driver emits Opus silence
|
||
Err(mpsc::TryRecvError::Disconnected) => None, // engine task gone → silence; reconnect is the engine's job
|
||
}
|
||
}
|
||
}
|
||
|
||
impl AudioSink for TapAudioPipe {
|
||
/// Receive a decoded PCM frame from the peer. Must not block (slice-1 §3.3 contract).
|
||
/// Forwards to the engine task via mpsc; if the channel is full (engine task slow / gone),
|
||
/// drops + counts (hot-path policy: drop + observe, don't crash, don't block).
|
||
fn on_pcm_frame(&mut self, frame: PcmFrame) {
|
||
if self.tx_pcm_in.try_send(frame).is_err() {
|
||
self.metrics.inbound_dropped.fetch_add(1, Ordering::Relaxed);
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**Playout ring policy:**
|
||
|
||
- **Capacity**: `TAP_PLAYOUT_FRAMES = 5` (= 100 ms at 20 ms / frame). Enough to absorb
|
||
brain jitter without introducing perceptible delay. Documented as a tunable *constant*
|
||
for slice-2 (no runtime config; a future-rung concern).
|
||
- **Overflow** (brain pushes faster than 20 ms / tick): drop **oldest**, log + counter.
|
||
Drop-oldest is the lowest-latency-correct posture — a brain pushing too fast means the
|
||
late frames are staler than the fresh ones; shedding the late frames keeps the buffer
|
||
at-or-behind real-time. (Drop-newest would accumulate growing latency — wrong
|
||
posture for a real-time media path.)
|
||
- **Underflow** (tick fires, ring empty): `next_pcm_frame` returns `None`;
|
||
`loop_driver` emits an Opus silence frame (already what slice-1 does on `None`).
|
||
|
||
### 4.2 TapClient (`src/tap_client.rs`)
|
||
|
||
The async object that owns the WSS connection. Lives only inside the `TapEngine` task —
|
||
the media loop never sees it.
|
||
|
||
```rust
|
||
pub struct TapClient {
|
||
ws: WebSocketStream<...>, // tokio_tungstenite client WS
|
||
session_id: ChannelId, // re-sent in hello on reconnect
|
||
rx_pcm_in: mpsc::Receiver<PcmFrame>, // drains inbound PCM → audio_in frames
|
||
tx_audio_out: mpsc::Sender<PcmFrame>, // feeds playout ring from audio_out frames
|
||
seq_egress: u64, // per-direction counter, starts at 0
|
||
last_seq_ingress: Option<u64>, // for gap detection
|
||
metrics: TapMetrics, // shared with TapAudioPipe
|
||
}
|
||
```
|
||
|
||
The pump loop (simplified): `tokio::select!` over (a) `rx_pcm_in.recv()` → build
|
||
`audio_in` JSON → `ws.send()`; (b) `ws.next()` → deserialize → on `audio_out`, push
|
||
to `tx_audio_out`; on `hello`, ack-tracking; on `bye` / `error`, log + counter; on
|
||
unknown `type`, log + counter + drop. Every send bumps `seq_egress`; every receive
|
||
checks `seq` against `last_seq_ingress` (gap → counter).
|
||
|
||
The TapClient never decides to reconnect itself — reconnect is the `TapEngine`'s job
|
||
(§4.3). On any WS close / error, TapClient returns from its pump loop; the engine
|
||
rebuilds it. This keeps "the connection" and "the reconnect policy" as separate concerns
|
||
— the one knows the wire, the other knows the backoff.
|
||
|
||
### 4.3 `TapEngine` (in `crates/rutster/src/tap_engine.rs`, lives in the binary)
|
||
|
||
The task supervisor. Spawned by the binary at the `Channel::Connected` transition; aborted
|
||
on `Channel::Closing`.
|
||
|
||
```rust
|
||
pub fn spawn_tap_engine(
|
||
session_id: ChannelId,
|
||
tap_url: Url, // validated ws:// 127.0.0.1 OR wss:// (which 501s in slice-2)
|
||
tx_pcm_in: mpsc::Receiver<PcmFrame>, // inbound PCM (drained from peer via TapAudioPipe::on_pcm_frame)
|
||
tx_audio_out: mpsc::Sender<PcmFrame>, // outbound PCM (playout ring feed)
|
||
close: oneshot::Receiver<()>, // aborted on Channel::Closing
|
||
) -> JoinHandle<()>
|
||
```
|
||
|
||
Loop:
|
||
|
||
1. `tokio_tungstenite::connect_async(tap_url)` with bounded timeout (2 s). On failure →
|
||
exponential backoff (`250 ms → 500 ms → 1 s → 2 s → cap 5 s`, infinite retries) and
|
||
retry. (Playout ring stays empty → `TapAudioPipe::next_pcm_frame` returns `None`
|
||
→ silence; the call survives.)
|
||
2. On connect: send `hello`, await brain `hello` (bounded 2 s; on timeout → close + retry).
|
||
3. On handshake: enter the `TapClient` pump loop. The pump runs until WS close, WS error,
|
||
or the `close` oneshot fires.
|
||
4. On any close/error (not the `close` oneshot): flush the playout buffer (drains
|
||
`tx_audio_out`'s outstanding queue — the `TapAudioPipe` end will see `Disconnected`
|
||
and emit silence until the new TapClient reconnects via a fresh mpsc), reset both
|
||
`seq` counters, re-enter step 1.
|
||
|
||
**Why this isn't the step-4 forbidden "dedicated timing thread":** the TapEngine task
|
||
does cold-path network I/O on tokio's shared runtime pool. It is *not* the 20 ms media
|
||
loop (which slice-1 §3.4 already runs on tokio as a scoped deviation; step 4 lands the
|
||
dedicated-timing-thread swap there, not here). ARCHITECTURE.md's "dedicated timing
|
||
threads, not the shared tokio pool" applies to the *timed media work* — adding a network
|
||
I/O supervisor task in slice-2 doesn't widen slice-1's documented deviation.
|
||
|
||
### 4.4 Wire-validation posture
|
||
|
||
`ws://` schemes must resolve to `127.0.0.1` or `localhost` — enforced as a hard runtime
|
||
check at session-create time (returns `400 Bad Request` from `POST /v1/sessions` if
|
||
violated, with a clear error message). `wss://` URLs are accepted by the schema and
|
||
rejected at connect time with a logged `501 NOT IMPLEMENTED` and a counter — the seam
|
||
is reserved for step 6 without ratifying a half-impl.
|
||
|
||
---
|
||
|
||
## 5. Lifecycle & failure mode
|
||
|
||
### 5.1 Session lifecycle (slice-2 delta on slice-1)
|
||
|
||
1. `POST /v1/sessions` — body now *optionally* carries `{"tap_url": "ws://..."}`. If
|
||
absent, falls back to `RUTSTER_TAP_URL` env (default `ws://127.0.0.1:8081/echo`).
|
||
Core validates the scheme (§4.4).
|
||
2. `POST /v1/sessions/:id/offer` — unchanged from slice-1; SDP answer returned.
|
||
3. On ICE+DTLS `Connected` (the slice-1 transition): the binary spawns a **`TapEngine`
|
||
task** for this `ChannelId`. The task dials the brain URL per §4.3.
|
||
`Channel.tap = Some(TapHandle)` is set; the binary maps the `ChannelId` to the
|
||
engine's mpsc handles in an internal `DashMap<ChannelId, TapConn>`.
|
||
4. `Channel.state` transitions still drive the loopback peer (slice-1's machine
|
||
unchanged). The `Connected→Closing→Closed` path is unchanged except for the additional
|
||
tap teardown step in step 5 below.
|
||
5. `DELETE /v1/sessions/:id` or peer-close → `Closing`: the binary sends `session_end`
|
||
over the tap WS, awaits brain `bye` (bounded 500 ms), closes the WS, drops the engine
|
||
task (the `close` oneshot fires + `JoinHandle::abort`). `Channel.tap = None`. Then
|
||
slice-1's `Closing → Closed` path runs.
|
||
|
||
### 5.2 Failure mode
|
||
|
||
- **WS connect failure / brain unreachable at session-Connect:** the `TapEngine` task
|
||
retries with bounded exponential backoff (250 ms → 500 ms → 1 s → 2 s → cap at 5 s;
|
||
infinite retries; a live call must self-heal). During retries: playout ring stays
|
||
empty → silence. **Channel stays `Connected`** throughout. A counter tracks retry
|
||
count.
|
||
- **Brain `bye` or WS close mid-call:** same as connect-failure — enter the backoff /
|
||
reconnect loop. On reconnect: re-send `hello` with the same `session_id` (the
|
||
`ChannelId` — §5.3). Playout ring is flushed on disconnect (drops stale brain audio);
|
||
resumes filling as the brain sends `audio_out` again.
|
||
- **Brain protocol error** (malformed frame, unknown `type`, bad `samples` count):
|
||
log + counter + drop the frame; **do not** disconnect. Hot-path "drop + observe"
|
||
policy from slice-1 §3.8, extended to the tap wire.
|
||
- **Core-side teardown (DELETE / peer-close / SIGTERM):** the TapEngine sends
|
||
`session_end`, awaits `bye` (bounded 500 ms), closes the WS, aborts the task. A
|
||
brain that doesn't `bye` back in time just gets a WS close — acceptable.
|
||
|
||
### 5.3 The stateless-brain reconnect contract
|
||
|
||
Slice-2's brain contract is **stateless**: both the Python and Rust echo brains hold no
|
||
per-call state across reconnects. On reconnect:
|
||
|
||
1. Core sends a fresh `hello` with the **same `session_id` (== the `ChannelId`)**.
|
||
2. Brain acks with `hello`.
|
||
3. Both sides reset `seq` counters to 0.
|
||
4. Playout ring has already been flushed on disconnect; the first `audio_out` from the
|
||
brain starts a fresh playout.
|
||
|
||
A real brain (step 3) is free to use `session_id` to resume state (e.g. an LLM
|
||
conversation context) but slice-2 does **not** require or test that: the contract is
|
||
"the brain may have forgotten everything; the core survives." This is the right
|
||
resilience posture for the eventual real brain (which may also crash / restart) and
|
||
the simplest thing to prove the tap interface with.
|
||
|
||
---
|
||
|
||
## 6. Call-model delta (`rutster-call-model`)
|
||
|
||
Slice-1 §5.2 promised `Channel` grows `tap: Option<TapHandle>` "with step 2." Slice 2
|
||
delivers that field add — a backwards-compatible field add, no slice-1 code is thrown
|
||
away.
|
||
|
||
```rust
|
||
pub struct Channel {
|
||
pub id: ChannelId,
|
||
pub state: ChannelState,
|
||
pub direction: Direction,
|
||
pub created_at: Instant,
|
||
pub tap: Option<TapHandle>, // NEW (slice-2). None until Connected, set on Connected, cleared on Closing.
|
||
}
|
||
|
||
pub struct TapHandle(()); // zero-cost marker: a tap is attached. The binary looks up the live
|
||
// connection by channel.id (ChannelId == session_id per §5.3) in its DashMap.
|
||
// Zero-sized so Option<TapHandle> compiles to a bool; no extra new UUID minted.
|
||
```
|
||
|
||
The `Channel` stays signaling-state only — it holds a `TapHandle` (a marker), not the
|
||
connection. The mpsc connections live in the binary's tap registry
|
||
(`DashMap<ChannelId, TapConn>`), keyed by the channel's existing `ChannelId`. This keeps
|
||
`rutster-call-model` a leaf with no tokio dep, and matches slice-1's "media state lives
|
||
internal to `rutster-media`, not on the `Channel`" framing — the tap connection is
|
||
similarly internal to the binary, not the `Channel`.
|
||
|
||
`TapHandle` is `Option<...>` (not always-`Some`) so a `Channel` can exist before the tap
|
||
attaches (New, Connecting states) and after it detaches (Closing, Closed). The None
|
||
transition on Closing is the tap teardown signal the binary acts on.
|
||
|
||
The `Channel`'s `id` field *is* the `session_id` carried in the tap's `hello` messages
|
||
(§5.3) and the lookup key for the binary's tap registry — no separate `TapId` newtype.
|
||
This means slice-2's design assumes a **1:1 mapping** between a `Channel` and its tap
|
||
connection (one tap per call). Multi-tap-per-channel (e.g. recording taps beside a brain
|
||
tap) is a future-rung concern — when it appears, that's the trigger to mint a separate
|
||
`TapId` newtype. For slice-2 (one brain per call), the existing `ChannelId` is sufficient
|
||
and avoiding the extra newtype is the right YAGNI call.
|
||
|
||
The `ChannelState` machine and `Direction` enum are unchanged from slice-1. The tap
|
||
attach/detach is a side-effect of the existing `Connecting → Connected` and `Connected
|
||
→ Closing` transitions, not a new state.
|
||
|
||
---
|
||
|
||
## 7. HTTP API delta (`rutster` binary)
|
||
|
||
### 7.1 `POST /v1/sessions` (delta on slice-1 §4.1)
|
||
|
||
Body now optionally carries a `tap_url`:
|
||
|
||
```json
|
||
{
|
||
"tap_url": "ws://127.0.0.1:8081/echo"
|
||
}
|
||
```
|
||
|
||
- Body is optional; absent body → `tap_url = RUTSTER_TAP_URL` env default.
|
||
- Body present, no `tap_url` field → same as absent body (env default).
|
||
- Body present, `tap_url` field → env default overridden; scheme validated per §4.4.
|
||
Returns `400 Bad Request` on a non-loopback `ws://` URL or an unparseable URL.
|
||
- `wss://` URLs pass schema validation and `501 NOT IMPLEMENTED` at connect time
|
||
(deferred to step 6).
|
||
|
||
Response unchanged from slice-1: `{ "session_id": "<uuid>" }`.
|
||
|
||
### 7.2 Other routes
|
||
|
||
Unchanged from slice-1: `POST /v1/sessions/:id/offer`, `DELETE /v1/sessions/:id`,
|
||
`GET /`. The static `index.html` gets a minor update to surface tap connection status
|
||
(`Connecting → Connected → Reconnecting`, with retry count) in the existing `<pre>`
|
||
debug area.
|
||
|
||
### 7.3 The `tap_url` authn gap (flagged)
|
||
|
||
Slice-2 inherits slice-1's "no authn/authz" posture. The `tap_url` override means any
|
||
caller can point the core's tap at an arbitrary URL — a privilege that will require
|
||
authn/authz in step 6. Slice-2's segment is local dev loop only (no production
|
||
deployment); the gap is documented, not closed. The spec's `wss://` reservation (accept
|
||
URL, 501 at connect) and `127.0.0.1`-only `ws://` enforcement bound the surface — a
|
||
malicious local caller is on a trusted host.
|
||
|
||
---
|
||
|
||
## 8. CI, dev loop, testing (delta on slice-1 §6)
|
||
|
||
### 8.1 New `[workspace.dependencies]` (Cargo.toml)
|
||
|
||
- `tokio-tungstenite = "0.24"` (WS client + server; the binary's `TapEngine` and
|
||
`rutster-tap-echo`'s standalone server both use this).
|
||
- `futures-util = "0.3"` (the `Sink`/`Stream` traits for `WebSocketStream`).
|
||
- `serde_json = "1"` (if not already pulled by slice-1's axum dep tree; verify at impl
|
||
time — if a duplicate-version ban trips in `cargo deny`, prefer the version axum
|
||
already pulls).
|
||
|
||
Member crates reference these with `dep.workspace = true`.
|
||
|
||
### 8.2 CI (`.github/workflows/ci.yml`)
|
||
|
||
Unchanged structure from slice-1: `cargo fmt --check`, `cargo clippy -D warnings`,
|
||
`cargo test --all`, `cargo deny check`. The new `rutster-tap` and `rutster-tap-echo`
|
||
crates join `--all`. **No Python in CI** — the Python brain is README-documented only.
|
||
The Rust `rutster-tap-echo`'s in-process `EchoServer` powers the integration test;
|
||
no network service is launched by CI.
|
||
|
||
### 8.3 Dev loop
|
||
|
||
- `cargo run -p rutster-tap-echo` → starts the Rust echo brain on `127.0.0.1:8081`. Or:
|
||
`python examples/echo_brain/echo_brain.py` (after `pip install websockets`) for the
|
||
foreign-language brain.
|
||
- `cargo run` (or `cargo run -p rutster`) → starts axum on `0.0.0.0:8080`, dials out to
|
||
`$RUTSTER_TAP_URL` (default `ws://127.0.0.1:8081/echo`) on each session.
|
||
- Browser → `http://localhost:8080/` → click "Start call" → grant mic → speak → hear
|
||
yourself back, routed through the external brain.
|
||
- `RUST_LOG=rutster=debug cargo run` for verbose tracing including tap connect /
|
||
reconnect / counter events.
|
||
- `--features=echo` on the binary (§2.2): bypasses the tap entirely, routes audio
|
||
through `EchoAudioPipe` (zero-network-dependency dev mode for slice-1 reproduction).
|
||
|
||
### 8.4 Testing strategy
|
||
|
||
- **Unit tests in `rutster-tap`:**
|
||
- Message (de)serialization round-trips for every `type` (golden JSON fixtures in
|
||
`tests/fixtures/`).
|
||
- `samples != 480` validation drops the frame; counter increments.
|
||
- Unknown `type` dropped + counter increments.
|
||
- Playout ring: overflow drops oldest (not newest); underflow returns `None`.
|
||
- `seq` gap detection increments a loss counter.
|
||
- `TapAudioPipe` end-to-end under a mock TapClient (no network): push PCM via
|
||
`on_pcm_frame`, assert it lands on the `tx_pcm_in` mpsc; push PCM via the
|
||
`tx_audio_out` mpsc, assert `next_pcm_frame` returns it.
|
||
- **Unit tests in `rutster-tap-echo`:**
|
||
- The standalone binary is thin; its echo logic is a `pub fn echo_frame(...) -> ...`
|
||
on the lib, independently unit-tested (recv `audio_in` → send `audio_out` with
|
||
same PCM; on `bye`/`session_end` → close cleanly).
|
||
- **Unit tests in `rutster-media` (unchanged from slice-1):** Opus⇄PCM roundtrip; SDP
|
||
munger; `RtcSession` driven by synthetic str0m `Input`. The slice-1 `EchoAudioPipe`
|
||
is still exercised here — `TapAudioPipe` is integration-tested via
|
||
`rutster-tap-echo`.
|
||
- **Integration test in `rutster` binary crate:** spin up the axum server (ephemeral
|
||
port) + the in-process `EchoServer` (ephemeral port) — set `RUTSTER_TAP_URL`. Drive
|
||
a synthetic WebRTC peer (extending slice-1's `reqwest` + hand-rolled SDP, or
|
||
`webrtc-rs` client if slice-1 landed it): push PCM into the core via the WebRTC peer
|
||
→ assert echo frames come back through the tap (`EchoServer` exposes its sent /
|
||
received frames for inspection) → assert they're re-encoded and pushed to str0m.
|
||
Plus: delete the channel → assert `session_end` / `bye` handshake. Plus (reconnect
|
||
path test): kill the `EchoServer` mid-test → assert `Channel` stays `Connected`,
|
||
tap `Reconnecting` counter increments, playout goes silent; restart the
|
||
`EchoServer` → assert reconnect succeeds and audio resumes.
|
||
- **Manual e2e test plan (README):**
|
||
1. `cargo run -p rutster-tap-echo` (the Rust echo brain on `:8081`).
|
||
2. `cargo run` (core on `:8080`).
|
||
3. Browser → `http://localhost:8080/` → speak → hear yourself echoed **through the
|
||
external brain** within ~250 ms (slice-1's 200 ms + tap round-trip + playout
|
||
headroom).
|
||
4. Kill the echo brain → server logs `tap disconnected, reconnecting`, audio goes
|
||
silent, browser shows `Reconnecting (attempt N)`; restart the echo brain → audio
|
||
resumes; `Channel` stayed `Connected` throughout.
|
||
5. Repeat steps 1–3 with the Python brain (`python
|
||
examples/echo_brain/echo_brain.py`) → same outcome (proves language-agnostic
|
||
protocol).
|
||
6. `cargo test --all` green; `cargo fmt --check` / `cargo clippy -D warnings` /
|
||
`cargo deny check` green.
|
||
|
||
### 8.5 Slice 2 "done" criteria
|
||
|
||
The slice is complete when, on a clean checkout:
|
||
|
||
1. `cargo test --all` passes (unit + integration). The new `rutster-tap` and
|
||
`rutster-tap-echo` crates test green alongside slice-1's suite.
|
||
2. `cargo fmt --check`, `cargo clippy -D warnings`, `cargo deny check` all pass.
|
||
3. `cargo run` (with `rutster-tap-echo` running) → browser, speak, hear echo
|
||
**through the external brain** within ~250 ms.
|
||
4. Kill echo brain mid-call → server reconnects with bounded backoff (visible in
|
||
logs + browser `<pre>`), audio resumes on brain restart, `Channel` never left
|
||
`Connected`.
|
||
5. **Both** `rutster-tap-echo` (Rust) **and** `examples/echo_brain/echo_brain.py`
|
||
(Python) successfully interop against the core — proves the protocol is
|
||
language-agnostic.
|
||
6. `rutster-media`'s `loop_driver.rs` and `rtc_session.rs` are **byte-identical in
|
||
their media-loop paths** to slice-1 (the seam-test: the only diff in `rutster-media`
|
||
is that `EchoAudioPipe` is retained alongside `TapAudioPipe`, not replacing it). A
|
||
`git diff v<slice-1-tag> -- crates/rutster-media/src/loop_driver.rs
|
||
crates/rutster-media/src/rtc_session.rs` shows no hunks in the media-loop
|
||
functions (doc-comment edits are permitted; no behavior change).
|
||
7. `LEARNING.md` grows ≥3 new pointers: `mpsc`/`oneshot` patterns →
|
||
`crates/rutster/src/tap_engine.rs`; `VecDeque` as a bounded ring →
|
||
`crates/rutster-tap/src/tap_audio_pipe.rs`; async WS connect + `Sink`/`Stream` →
|
||
`crates/rutster-tap/src/tap_client.rs`.
|
||
|
||
---
|
||
|
||
## 9. Open decisions (tracked)
|
||
|
||
- **Binary PCM mode (v: 2).** Base64-in-text-JSON is the v1 wire format. ~33% overhead
|
||
is acceptable for the dev loop; the protocol reserves `v: 2` for a binary length-
|
||
prefixed mode (raw LE i16 over WS binary frames) for a later rung. Re-evaluate when
|
||
(a) a real brain (step 3) hits bandwidth ceilings, or (b) the fuzz harness (step 5)
|
||
wants to fuzz a binary parser.
|
||
- **Byte-endian negotiation.** v1 emits the host's native endian as raw bytes inside
|
||
the base64 payload. On a little-endian host (today's typical target) this is LE; on
|
||
a hypothetical big-endian host it would be BE without warning. The protocol should
|
||
nail explicit LE byte order in v2; for v1 it's documented as a simplification.
|
||
Today: every dev-loop brain (Python `websockets` lib, Rust on x86_64/aarch64) is
|
||
little-endian, so the risk is theoretical.
|
||
- **Tap protocol ADR.** PORT_PLAN §10 lists the agent-tap protocol as "presumptively
|
||
WSS + core-as-client + clean PCM + core-authoritative playout," *not* a decided ADR
|
||
yet. Slice-2 **hardens** the presumptive shape against a working implementation. The
|
||
spec's "Implementation lands, then ADR ratifies the wire shape" is the deliberate
|
||
sequence — an ADR-0007 (or similar) capturing slice-2's ratified decisions (JSON
|
||
envelope, base64 v1, core-authoritative playout, stateless reconnect contract) is
|
||
a post-implementation follow-up, not a slice-2 deliverable. The spec's existence is
|
||
the ratification for slice-2; the ADR is the durable form.
|
||
- **`wss://` cert/mTLS posture.** Slice-2 reserves the `wss://` URL scheme (accepted at
|
||
schema validation, `501 NOT IMPLEMENTED` at connect) so step 6 doesn't need a schema
|
||
change. The actual cert-validation / mTLS / brain-cert-pinning impl is step-6 work.
|
||
|
||
---
|
||
|
||
## 10. Out-of-scope re-check (against AGENTS.md slice-2 expectations)
|
||
|
||
AGENTS.md's "Slice-1 boundaries — what NOT to add (yet)" lists items deferred to specific
|
||
later spearhead steps. For slice 2 the equivalent table is §1.2 above. The cross-check:
|
||
|
||
- ❌ Dedicated timing thread for media loop → still step 4. Slice-2 adds the TapEngine
|
||
task but it's a cold-path I/O supervisor, not a timed media loop; slice-1's scoped
|
||
deviation for the 20 ms loop is unchanged.
|
||
- ❌ TLS on HTTP signaling surface → still step 5.
|
||
- ❌ Authn/authz / multi-tenancy on `/v1/sessions` → step 6. Slice-2 inherits slice-1's
|
||
no-auth posture (§7.3 flags the new `tap_url` override gap).
|
||
- ❌ Trickle ICE → unchanged.
|
||
- ❌ The brain itself (STT/LLM/TTS) → step 3. Slice-2 ships only echo brains.
|
||
- ❌ Barge-in / VAD-driven playout kill → step 4. Slice-2's playout buffer queues +
|
||
drops on overflow; doesn't kill on caller speech.
|
||
- ❌ PSTN trunk → still step 5.
|
||
- ❌ Spend cap → still step 6.
|
||
- ❌ CDR / event bus / OTel beyond per-Channel tracing → still step 5.
|
||
- ❌ Browser automation / Playwright → still post-slice-1.
|
||
- ❌ Docker / compose → still later-rung.
|
||
- ❌ Transfer / park / pickup / barge → still escalation rung 2.
|
||
|
||
If an agent proposes adding any of these in slice 2, the right answer is "no, see the
|
||
slice-2 spec §1.2."
|
||
|
||
---
|
||
|
||
## 11. Key design decisions (summary of the brainstorming session)
|
||
|
||
| Decision | Choice | Rejected alternatives | Why |
|
||
|---|---|---|---|
|
||
| Tap architecture | **B. Decoupled TapEngine** — `TapAudioPipe` is a thin sync wrapper over mpsc + ring; `TapClient` (inside the engine task) owns the WSS connection; `RtcSession` only swaps `EchoAudioPipe` → `TapAudioPipe`. | A. In-pipe tap (WSS task owned by the AudioPipe); C. Explicit `tap` field on `RtcSession` bypassing the seam | Honors slice-1 §3.3's promise verbatim ("no code changes to `RtcSession` itself in step 2"); keeps the 20 ms loop pure (only mpsc + ring-touch); cold-path I/O task ≠ the step-4 forbidden "dedicated timing thread"; reconnect is localized to the engine. |
|
||
| Wire protocol | **Own minimal versioned JSON event protocol**, base64 PCM in text WS frames. | Adopt OpenAI Realtime event schema verbatim; binary length-prefixed framing | ARCHITECTURE.md names WSS as presumptive transport because the consumer is a Python script / OpenAI-Realtime-style API; JSON event envelope is the natural mapping onto that ecosystem. Avoids vendor lock-in at our central interface; the step-3 OpenAI adapter translates. ~33% wire overhead is acceptable at 65 KB/s. |
|
||
| Reference echo brain | **Both**: Python `examples/echo_brain/` (canonical foreign-language) + Rust `crates/rutster-tap-echo` (showcase + integration-test `EchoServer`). | Python-only; Rust-only | Python proves the protocol is language-agnostic and matches the "brain is a Python script" persona; Rust proves the wire types are reusable from outside the core and doubles as the in-process test server. The dual is the user's "showcase Rust binaries + external scripts" goal. |
|
||
| TLS on tap | **`ws://` loopback only, `wss://` deferred to step 6.** Hard runtime check on `127.0.0.1`/`localhost`; `wss://` URL accepted by schema, `501 NOT IMPLEMENTED` at connect. | Always-`wss://` even for localhost (self-signed CA burden); plaintext-`ws://`-only with no `wss://` codepath | Matches slice-1's "TLS needs a cert story" stance; keeps the dev loop zero-cert. Reserves the `wss://` seam so step 6 doesn't need a schema change. |
|
||
| Tap URL config | **Env default + per-call override.** `RUTSTER_TAP_URL` env (default `ws://127.0.0.1:8081/echo`); `POST /v1/sessions` body optional `tap_url` overrides. | Env-only; per-call-only | Env default = simplest dev loop; per-call override demonstrates the routing seam (the precursor to multi-brain routing in step 6); authn on the override is a flagged step-6 gap (§7.3). |
|
||
| Brain failure | **Silence + bounded-backoff reconnect (infinite retries).** 250 ms → 500 ms → 1 s → 2 s → cap 5 s; Channel stays Connected; playout flushed on disconnect; reconnect re-`hello`s with same `session_id`; stateless brain contract. | Silence + give-up; tear-down-the-call | Proves "tap is advisory; core disposes" hard — the call outlives the brain. Self-healing is the right posture for a real brain (which may also crash / restart). |
|
||
| Playout buffer policy | **Drop-oldest on overflow, silence on underflow.** Capacity 5 frames (100 ms). | Drop-newest; larger / smaller capacity | Drop-oldest is lowest-latency-correct (sheds stale frames, keeps buffer at-or-behind real-time); 5 frames absorbs brain jitter without introducing perceptible delay. Capacity is a tunable *constant* in slice-2 (no runtime config). |
|
||
| `EchoAudioPipe` fate | **Retained** in `rutster-media` alongside `TapAudioPipe`; `--features=echo` dev-mode on the binary uses it. | Delete it | Slice-1 unit tests still use it (no network); dev-loop zero-network-dep fallback for slice-1 bug repro; two impls of the same trait is the cleanest documentation that the seam *is* a seam. |
|
||
| `TapHandle` on the `Channel` | **Zero-cost marker newtype `TapHandle(())`; binary looks up the live connection by the channel's existing `ChannelId` in a `DashMap<ChannelId, TapConn>`.** `rutster-call-model` stays a leaf (no tokio dep). | Embed the mpsc handles directly on the Channel; mint a separate `TapId(Uuid)` newtype for the lookup key | Matches slice-1's framing: the `Channel` carries signaling state + markers to media-state-holders, not the media state itself. Keeps the call-model crate pure (no runtime deps). The 1:1 mapping of channel↔tap in slice-2 means `ChannelId` *is* the right lookup key — a separate `TapId` is YAGNI until multi-tap-per-channel appears. |
|
||
|
||
---
|
||
|
||
## 12. References
|
||
|
||
- [README.md](../../../README.md) — north star, capability ladder
|
||
- [ARCHITECTURE.md §"Agent tap"](../../ARCHITECTURE.md) — the presumptive tap shape this
|
||
slice hardens
|
||
- [PORT_PLAN.md](../../PORT_PLAN.md) — capability checklist + thin-slice phasing;
|
||
§10 "WASM demoted, agent tap is the extension point"; §10 open decision on the tap
|
||
protocol
|
||
- [Slice 1 — WebRTC media loopback](2026-06-28-slice-1-webrtc-loopback-design.md) —
|
||
this slice's foundation; §1.2 out-of-scope table schedules the tap for step 2;
|
||
§3.3 promises the seam; §5.2 promises the `tap` field
|
||
- [Vision-revision spec](2026-06-26-vision-revision-design.md) — the pressure-test that
|
||
produced the architecture
|
||
- [ADR-0002](../../adr/0002-north-star-and-fused-core.md) — fused vertical; agent tap
|
||
as extension point
|
||
- [ADR-0004](../../adr/0004-license.md) — GPL-3.0-or-later
|
||
- [ADR-0006](../../adr/0006-ingress-posture.md) — core-as-client tap posture
|
||
(tap is egress, opposite security posture to inbound ingress)
|
||
- [AGENTS.md](../../../AGENTS.md) — code style, error handling, slice-boundaries
|
||
cross-check (§ "Slice-1 boundaries — what NOT to add (yet)")
|