docs(slice-3): Python reference brain + LEARNING.md + README dev loop (spec §7.5)
examples/openai_realtime_brain/ -- the canonical foreign-language OpenAI Realtime brain (Python, ~150 lines, websockets lib only). Not in CI (zero-non-Rust-dev-deps dev loop per AGENTS.md). Mirrors the slice-2 echo_brain pattern and proves the protocol is language-agnostic. Hand-rolled against the spec §4.2 mapping rather than using the openai Python SDK: the SDK is unused by the wire-shape (both sides are plain WSS), so dropping it removes a dep that the plan skeleton imported but never called. session.update is sent with turn_detection=null (S4) matching the Rust brain. LEARNING.md gains 5 new pointers under a Slice 3 heading: async-trait in trait objects, the translator pure-function layer, additive protocol extension via #[serde(other)], the FOB-boundary side-channel mpsc, and the WS subprotocol handshake (openai_client.rs). README.md gains the Slice 3 dev loop section (mock mode without an OpenAI key, real-OpenAI mode, and the Python brain alternative). .gitignore learns about __pycache__/*.pyc (the Python brains aren't in CI but byte-compiled artifacts should never be tracked).
This commit is contained in:
27
LEARNING.md
27
LEARNING.md
@@ -79,6 +79,33 @@ can read in `cargo doc --open` plus the source file itself.
|
||||
— `TapHandle(())` compiles `Option<TapHandle>` to a single `bool`; no
|
||||
runtime cost for the type-system marker.
|
||||
|
||||
### Slice 3 — OpenAI Realtime brain
|
||||
|
||||
- **`async-trait` patterns / async fns in trait objects** →
|
||||
`crates/rutster/src/tool_registry.rs` — the `Tool` trait's `async fn call`
|
||||
and how `async-trait` lowers it to `Pin<Box<dyn Future + Send>>` (the same
|
||||
boxing the slice-1 `pipe` widening used, now applied to trait methods).
|
||||
- **OpenAI Realtime adapter + event translation (pure-function layer)** →
|
||||
`crates/rutster-brain-realtime/src/translator.rs` — the §4.2 event mapping
|
||||
table as pure `fn`s: tap `audio_in` ⇄ OpenAI `input_audio_buffer.append`,
|
||||
`response.audio.delta` ⇄ `audio_out`, `function_call_arguments.done` ⇄
|
||||
`function_call`. Testable without a network (the pump is in a separate file).
|
||||
- **Tap protocol additive extension + forward-compat via `#[serde(other)]`** →
|
||||
`crates/rutster-tap/src/protocol.rs` — how slice-3 adds `speech_started`,
|
||||
`speech_stopped`, `function_call`, `function_call_output`, `tools.update`
|
||||
without breaking slice-2 brains: unknown variants deserialize to a catch-all
|
||||
arm instead of erroring (forward-compat by construction).
|
||||
- **Side-channel mpsc for FOB-boundary dispatch** →
|
||||
`crates/rutster/src/session_map.rs` — how `drive_all_sessions` drains a
|
||||
function-call mpsc side channel into the tap WS writer without blocking the
|
||||
20 ms media loop. The pattern: hot loop polls `try_recv`; cold path spawns.
|
||||
- **HTTP request builder for WS subprotocol handshake** →
|
||||
`crates/rutster-brain-realtime/src/openai_client.rs` — why we hand-roll the
|
||||
WS upgrade `Request` instead of letting `connect_async` build it: to set
|
||||
`Authorization` + `OpenAI-Beta` headers on the WS handshake (tungstenite
|
||||
doesn't expose subprotocol/auth headers via the simple `connect_async(url)`
|
||||
entry point — see `openai_headers` / `openai_realtime_url`).
|
||||
|
||||
## How to read
|
||||
|
||||
1. `cargo doc --open` — every module has a `//!` doc comment; the doc
|
||||
|
||||
Reference in New Issue
Block a user