examples/openai_realtime_brain/ -- the canonical foreign-language OpenAI Realtime brain (Python, ~150 lines, websockets lib only). Not in CI (zero-non-Rust-dev-deps dev loop per AGENTS.md). Mirrors the slice-2 echo_brain pattern and proves the protocol is language-agnostic. Hand-rolled against the spec §4.2 mapping rather than using the openai Python SDK: the SDK is unused by the wire-shape (both sides are plain WSS), so dropping it removes a dep that the plan skeleton imported but never called. session.update is sent with turn_detection=null (S4) matching the Rust brain. LEARNING.md gains 5 new pointers under a Slice 3 heading: async-trait in trait objects, the translator pure-function layer, additive protocol extension via #[serde(other)], the FOB-boundary side-channel mpsc, and the WS subprotocol handshake (openai_client.rs). README.md gains the Slice 3 dev loop section (mock mode without an OpenAI key, real-OpenAI mode, and the Python brain alternative). .gitignore learns about __pycache__/*.pyc (the Python brains aren't in CI but byte-compiled artifacts should never be tracked).
6.6 KiB
LEARNING.md — to learn concept X, read file Y
This index maps a Rust concept you might be learning to the file where
slice 1 makes the concept concrete. Each entry is a worked example you
can read in cargo doc --open plus the source file itself.
Concepts + pointers
-
Newtype pattern (type-safety via single-field wrappers) →
crates/rutster-call-model/src/lib.rs—ChannelId(Uuid). The newtype stops us from mixing up aChannelIdwith some futureSessionIdat the type-system level. Compile-enforced where a comment could only ask. -
enumfor closed state sets + exhaustivematch→crates/rutster-call-model/src/lib.rs—ChannelState(New → Connecting → Connected → Closing → Closed). Exhaustiveness checking forces everymatchto consider each state; adding a state later surfaces every site that needs handling. -
Sans-IO pattern (no I/O inside the library; input via method calls, output via return values) →
crates/rutster-media/src/loop_driver.rs— the str0m poll loop.Rtc::handle_inputtakes a network packet as a struct argument, not from a socket the library owns;poll_outputreturnsTransmitpackets the caller sends. Fully testable without a network — str0m integration tests use this property to drive faster than realtime. -
Trait design for extension points (a futures-compatible seam) →
crates/rutster-media/src/pcm.rs— theAudioSource/AudioSinktraits. Slice 1 wires anEchoAudioPipebetween them; step 2 swaps that for a real WSS tap client without touchingRtcSession. The traits describe what the splice point does, not how it's filled. -
Error enums with
thiserror+ hot-path match-and-continue →crates/rutster-media/src/lib.rs(MediaError) andcrates/rutster-media/src/opus_codec.rs(OpusDecoder::decodereturnsOption<PcmFrame>). Cold path:thiserror-derived enum +?. Hot path: match-and-continue, never?, never panic — "drop + observe, don't crash" (spec §3.8). -
Arc<Mutex<T>>vsArc<RwLock<T>>— when each is right →crates/rutster/src/session_map.rs. TheRtcSessionlives behindArc<Mutex<...>>because every access mutates it (str0m's&mut selfcontract) —RwLock's read-mode would be useless. Comment on the struct explains the trade-off. -
DashMapfor sharded concurrent maps →crates/rutster/src/session_map.rs.DashMapshards its inner map so two handlers operating on differentChannelIds don't contend;HashMapwrapped in a singleMutexwould serialize every access. -
str0m 0.21's single-mutation invariant →
crates/rutster-media/src/loop_driver.rs. Mutate (handle_input / Writer::write) → drainpoll_outputtoOutput::Timeout→ next mutate. Violating this leaves str0m in an inconsistent state. -
tokio graceful shutdown via signal handlers →
crates/rutster/src/main.rs(shutdown_signal). Ctrl-C / SIGTERM drops the AppState; the AppState drops the DashMap; the DashMap drops every RtcSession. No in-flight call preservation in slice 1. -
include_str!for embedding static assets →crates/rutster/src/routes.rs(include_str!("../static/index.html")). The HTML test client is compiled into the binary at build time — no separate file to ship, no disk IO to serve it. -
mpsc+oneshotfor cold-path task supervision →crates/rutster/src/tap_engine.rs— how a spawned tokio task is supervised + cancelled viaoneshot::Receiver. -
VecDequeas a bounded playout ring with drop-oldest policy →crates/rutster-tap/src/tap_audio_pipe.rs— why a manual ring (notmpsc) when the overflow policy is drop-oldest, not drop-newest. -
Async WS connect +
Sink/Streamtraits →crates/rutster-tap/src/tap_client.rs—tokio_tungstenite::connect_async,WebSocketStream, theSinkExt/StreamExtextension traits,tokio::select!over inbound + outbound + close. -
Box<dyn Trait + Send>field widening (the seam test) →crates/rutster-media/src/rtc_session.rs— why thepipefield type changed fromEchoAudioPipetoBox<dyn AudioSource + AudioSink + Send>soloop_driver's call sites are byte-identical (slice-2 §8.5 #6). -
Zero-sized marker newtype for state flags →
crates/rutster-call-model/src/lib.rs—TapHandle(())compilesOption<TapHandle>to a singlebool; no runtime cost for the type-system marker.
Slice 3 — OpenAI Realtime brain
async-traitpatterns / async fns in trait objects →crates/rutster/src/tool_registry.rs— theTooltrait'sasync fn calland howasync-traitlowers it toPin<Box<dyn Future + Send>>(the same boxing the slice-1pipewidening used, now applied to trait methods).- OpenAI Realtime adapter + event translation (pure-function layer) →
crates/rutster-brain-realtime/src/translator.rs— the §4.2 event mapping table as purefns: tapaudio_in⇄ OpenAIinput_audio_buffer.append,response.audio.delta⇄audio_out,function_call_arguments.done⇄function_call. Testable without a network (the pump is in a separate file). - Tap protocol additive extension + forward-compat via
#[serde(other)]→crates/rutster-tap/src/protocol.rs— how slice-3 addsspeech_started,speech_stopped,function_call,function_call_output,tools.updatewithout breaking slice-2 brains: unknown variants deserialize to a catch-all arm instead of erroring (forward-compat by construction). - Side-channel mpsc for FOB-boundary dispatch →
crates/rutster/src/session_map.rs— howdrive_all_sessionsdrains a function-call mpsc side channel into the tap WS writer without blocking the 20 ms media loop. The pattern: hot loop pollstry_recv; cold path spawns. - HTTP request builder for WS subprotocol handshake →
crates/rutster-brain-realtime/src/openai_client.rs— why we hand-roll the WS upgradeRequestinstead of lettingconnect_asyncbuild it: to setAuthorization+OpenAI-Betaheaders on the WS handshake (tungstenite doesn't expose subprotocol/auth headers via the simpleconnect_async(url)entry point — seeopenai_headers/openai_realtime_url).
How to read
cargo doc --open— every module has a//!doc comment; the doc tree is the high-level map.- Pick a concept above; open the named file. The first occurrence of
each non-obvious pattern has a
//comment explaining why. - Cross-ref back to the spec sections cited inline (
spec §3.8,ADR-0002, etc.) for the architecture-level rationale.