e8ba2f3f32
examples/openai_realtime_brain/ -- the canonical foreign-language OpenAI Realtime brain (Python, ~150 lines, websockets lib only). Not in CI (zero-non-Rust-dev-deps dev loop per AGENTS.md). Mirrors the slice-2 echo_brain pattern and proves the protocol is language-agnostic. Hand-rolled against the spec §4.2 mapping rather than using the openai Python SDK: the SDK is unused by the wire-shape (both sides are plain WSS), so dropping it removes a dep that the plan skeleton imported but never called. session.update is sent with turn_detection=null (S4) matching the Rust brain. LEARNING.md gains 5 new pointers under a Slice 3 heading: async-trait in trait objects, the translator pure-function layer, additive protocol extension via #[serde(other)], the FOB-boundary side-channel mpsc, and the WS subprotocol handshake (openai_client.rs). README.md gains the Slice 3 dev loop section (mock mode without an OpenAI key, real-OpenAI mode, and the Python brain alternative). .gitignore learns about __pycache__/*.pyc (the Python brains aren't in CI but byte-compiled artifacts should never be tracked).
117 lines
6.6 KiB
Markdown
117 lines
6.6 KiB
Markdown
# LEARNING.md — to learn concept X, read file Y
|
|
|
|
This index maps a Rust concept you might be learning to the file where
|
|
slice 1 makes the concept concrete. Each entry is a worked example you
|
|
can read in `cargo doc --open` plus the source file itself.
|
|
|
|
## Concepts + pointers
|
|
|
|
- **Newtype pattern (type-safety via single-field wrappers)** →
|
|
`crates/rutster-call-model/src/lib.rs` — `ChannelId(Uuid)`. The newtype
|
|
stops us from mixing up a `ChannelId` with some future `SessionId` at
|
|
the type-system level. Compile-enforced where a comment could only ask.
|
|
|
|
- **`enum` for closed state sets + exhaustive `match`** →
|
|
`crates/rutster-call-model/src/lib.rs` — `ChannelState` (New →
|
|
Connecting → Connected → Closing → Closed). Exhaustiveness checking
|
|
forces every `match` to consider each state; adding a state later
|
|
surfaces every site that needs handling.
|
|
|
|
- **Sans-IO pattern (no I/O inside the library; input via method calls,
|
|
output via return values)** → `crates/rutster-media/src/loop_driver.rs`
|
|
— the str0m poll loop. `Rtc::handle_input` takes a network packet as a
|
|
struct argument, not from a socket the library owns; `poll_output`
|
|
returns `Transmit` packets the caller sends. Fully testable without a
|
|
network — str0m integration tests use this property to drive faster
|
|
than realtime.
|
|
|
|
- **Trait design for extension points (a futures-compatible seam)** →
|
|
`crates/rutster-media/src/pcm.rs` — the `AudioSource` / `AudioSink`
|
|
traits. Slice 1 wires an `EchoAudioPipe` between them; step 2 swaps
|
|
that for a real WSS tap client without touching `RtcSession`. The
|
|
traits describe *what* the splice point does, not *how* it's filled.
|
|
|
|
- **Error enums with `thiserror` + hot-path match-and-continue** →
|
|
`crates/rutster-media/src/lib.rs` (`MediaError`) and
|
|
`crates/rutster-media/src/opus_codec.rs` (`OpusDecoder::decode` returns
|
|
`Option<PcmFrame>`). Cold path: `thiserror`-derived enum + `?`. Hot
|
|
path: match-and-continue, never `?`, never panic — "drop + observe,
|
|
don't crash" (spec §3.8).
|
|
|
|
- **`Arc<Mutex<T>>` vs `Arc<RwLock<T>>` — when each is right** →
|
|
`crates/rutster/src/session_map.rs`. The `RtcSession` lives behind
|
|
`Arc<Mutex<...>>` because every access mutates it (str0m's `&mut self`
|
|
contract) — `RwLock`'s read-mode would be useless. Comment on the
|
|
struct explains the trade-off.
|
|
|
|
- **`DashMap` for sharded concurrent maps** →
|
|
`crates/rutster/src/session_map.rs`. `DashMap` shards its inner map so
|
|
two handlers operating on different `ChannelId`s don't contend;
|
|
`HashMap` wrapped in a single `Mutex` would serialize every access.
|
|
|
|
- **str0m 0.21's single-mutation invariant** →
|
|
`crates/rutster-media/src/loop_driver.rs`. Mutate (handle_input /
|
|
Writer::write) → drain `poll_output` to `Output::Timeout` → next
|
|
mutate. Violating this leaves str0m in an inconsistent state.
|
|
|
|
- **tokio graceful shutdown via signal handlers** →
|
|
`crates/rutster/src/main.rs` (`shutdown_signal`). Ctrl-C / SIGTERM
|
|
drops the AppState; the AppState drops the DashMap; the DashMap drops
|
|
every RtcSession. No in-flight call preservation in slice 1.
|
|
|
|
- **`include_str!` for embedding static assets** →
|
|
`crates/rutster/src/routes.rs` (`include_str!("../static/index.html")`).
|
|
The HTML test client is compiled into the binary at build time — no
|
|
separate file to ship, no disk IO to serve it.
|
|
|
|
- **`mpsc` + `oneshot` for cold-path task supervision** → `crates/rutster/src/tap_engine.rs`
|
|
— how a spawned tokio task is supervised + cancelled via `oneshot::Receiver`.
|
|
- **`VecDeque` as a bounded playout ring with drop-oldest policy** → `crates/rutster-tap/src/tap_audio_pipe.rs`
|
|
— why a manual ring (not `mpsc`) when the overflow policy is drop-oldest, not drop-newest.
|
|
- **Async WS connect + `Sink`/`Stream` traits** → `crates/rutster-tap/src/tap_client.rs`
|
|
— `tokio_tungstenite::connect_async`, `WebSocketStream`, the `SinkExt`/`StreamExt`
|
|
extension traits, `tokio::select!` over inbound + outbound + close.
|
|
- **`Box<dyn Trait + Send>` field widening (the seam test)** → `crates/rutster-media/src/rtc_session.rs`
|
|
— why the `pipe` field type changed from `EchoAudioPipe` to
|
|
`Box<dyn AudioSource + AudioSink + Send>` so `loop_driver`'s call sites
|
|
are byte-identical (slice-2 §8.5 #6).
|
|
- **Zero-sized marker newtype for state flags** → `crates/rutster-call-model/src/lib.rs`
|
|
— `TapHandle(())` compiles `Option<TapHandle>` to a single `bool`; no
|
|
runtime cost for the type-system marker.
|
|
|
|
### Slice 3 — OpenAI Realtime brain
|
|
|
|
- **`async-trait` patterns / async fns in trait objects** →
|
|
`crates/rutster/src/tool_registry.rs` — the `Tool` trait's `async fn call`
|
|
and how `async-trait` lowers it to `Pin<Box<dyn Future + Send>>` (the same
|
|
boxing the slice-1 `pipe` widening used, now applied to trait methods).
|
|
- **OpenAI Realtime adapter + event translation (pure-function layer)** →
|
|
`crates/rutster-brain-realtime/src/translator.rs` — the §4.2 event mapping
|
|
table as pure `fn`s: tap `audio_in` ⇄ OpenAI `input_audio_buffer.append`,
|
|
`response.audio.delta` ⇄ `audio_out`, `function_call_arguments.done` ⇄
|
|
`function_call`. Testable without a network (the pump is in a separate file).
|
|
- **Tap protocol additive extension + forward-compat via `#[serde(other)]`** →
|
|
`crates/rutster-tap/src/protocol.rs` — how slice-3 adds `speech_started`,
|
|
`speech_stopped`, `function_call`, `function_call_output`, `tools.update`
|
|
without breaking slice-2 brains: unknown variants deserialize to a catch-all
|
|
arm instead of erroring (forward-compat by construction).
|
|
- **Side-channel mpsc for FOB-boundary dispatch** →
|
|
`crates/rutster/src/session_map.rs` — how `drive_all_sessions` drains a
|
|
function-call mpsc side channel into the tap WS writer without blocking the
|
|
20 ms media loop. The pattern: hot loop polls `try_recv`; cold path spawns.
|
|
- **HTTP request builder for WS subprotocol handshake** →
|
|
`crates/rutster-brain-realtime/src/openai_client.rs` — why we hand-roll the
|
|
WS upgrade `Request` instead of letting `connect_async` build it: to set
|
|
`Authorization` + `OpenAI-Beta` headers on the WS handshake (tungstenite
|
|
doesn't expose subprotocol/auth headers via the simple `connect_async(url)`
|
|
entry point — see `openai_headers` / `openai_realtime_url`).
|
|
|
|
## How to read
|
|
|
|
1. `cargo doc --open` — every module has a `//!` doc comment; the doc
|
|
tree is the high-level map.
|
|
2. Pick a concept above; open the named file. The first occurrence of
|
|
each non-obvious pattern has a `//` comment explaining *why*.
|
|
3. Cross-ref back to the spec sections cited inline (`spec §3.8`,
|
|
`ADR-0002`, etc.) for the architecture-level rationale.
|