docs(slice-3): Python reference brain + LEARNING.md + README dev loop (spec §7.5)
examples/openai_realtime_brain/ -- the canonical foreign-language OpenAI Realtime brain (Python, ~150 lines, websockets lib only). Not in CI (zero-non-Rust-dev-deps dev loop per AGENTS.md). Mirrors the slice-2 echo_brain pattern and proves the protocol is language-agnostic. Hand-rolled against the spec §4.2 mapping rather than using the openai Python SDK: the SDK is unused by the wire-shape (both sides are plain WSS), so dropping it removes a dep that the plan skeleton imported but never called. session.update is sent with turn_detection=null (S4) matching the Rust brain. LEARNING.md gains 5 new pointers under a Slice 3 heading: async-trait in trait objects, the translator pure-function layer, additive protocol extension via #[serde(other)], the FOB-boundary side-channel mpsc, and the WS subprotocol handshake (openai_client.rs). README.md gains the Slice 3 dev loop section (mock mode without an OpenAI key, real-OpenAI mode, and the Python brain alternative). .gitignore learns about __pycache__/*.pyc (the Python brains aren't in CI but byte-compiled artifacts should never be tracked).
This commit is contained in:
32
README.md
32
README.md
@@ -19,6 +19,38 @@ cargo run
|
||||
Open <http://localhost:8080/> → click "Start call" → grant mic → hear yourself echo.
|
||||
Full walkthrough + troubleshooting: **[`docs/QUICKSTART.md`](docs/QUICKSTART.md)**.
|
||||
|
||||
### Slice 3 dev loop — OpenAI Realtime brain
|
||||
|
||||
The dev loop *without* real OpenAI credentials (no API key required):
|
||||
|
||||
```
|
||||
cargo run -p rutster-brain-realtime --features=mock # brain on :8082
|
||||
cargo run # core on :8080
|
||||
```
|
||||
|
||||
Open <http://localhost:8080/> → click "Start call" → speak → hear the
|
||||
mock-brain reply within ~250 ms (the mock echoes audio back, no real OpenAI
|
||||
RTT; this exercises the full brain→core audio round-trip + the new
|
||||
function_call dispatch path).
|
||||
|
||||
With real OpenAI Realtime:
|
||||
|
||||
```
|
||||
export OPENAI_API_KEY=sk-... # or OPENAI_API_KEY_FILE=/var/secrets/openai
|
||||
cargo run -p rutster-brain-realtime
|
||||
cargo run
|
||||
```
|
||||
|
||||
Speak → end-to-end speech-to-speech with OpenAI Realtime within ~700 ms
|
||||
(slice-1's 200 ms + tap round-trip + OpenAI latency + 100 ms playout buffer).
|
||||
|
||||
For the foreign-language brain demo (Python, not in CI):
|
||||
|
||||
```
|
||||
pip install -r examples/openai_realtime_brain/requirements.txt
|
||||
OPENAI_API_KEY=sk-... python examples/openai_realtime_brain/openai_realtime_brain.py
|
||||
```
|
||||
|
||||
> **Status:** Slice 1 (WebRTC media loopback) is the active build target. The workspace is
|
||||
> landing task-by-task on the `slice-1-webrtc-loopback` branch. Design:
|
||||
> [`docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md`](docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md).
|
||||
|
||||
Reference in New Issue
Block a user