Slice 3 — OpenAI Realtime brain: swap echo for the brain (#4)
All checks were successful
CI / fmt (push) Successful in 1m40s
CI / clippy (push) Successful in 2m24s
CI / test (1.85) (push) Successful in 5m8s
CI / test (stable) (push) Successful in 5m20s
CI / deny (push) Successful in 1m34s

This commit was merged in pull request #4.
This commit is contained in:
2026-07-01 22:25:09 +00:00
parent 2f3f92ec6b
commit c30a45232d
36 changed files with 7227 additions and 20 deletions

View File

@@ -19,6 +19,38 @@ cargo run
Open <http://localhost:8080/> → click "Start call" → grant mic → hear yourself echo.
Full walkthrough + troubleshooting: **[`docs/QUICKSTART.md`](docs/QUICKSTART.md)**.
### Slice 3 dev loop — OpenAI Realtime brain
The dev loop *without* real OpenAI credentials (no API key required):
```
cargo run -p rutster-brain-realtime --features=mock # brain on :8082
cargo run # core on :8080
```
Open <http://localhost:8080/> → click "Start call" → speak → hear the
mock-brain reply within ~250 ms (the mock echoes audio back, no real OpenAI
RTT; this exercises the full brain→core audio round-trip + the new
function_call dispatch path).
With real OpenAI Realtime:
```
export OPENAI_API_KEY=sk-... # or OPENAI_API_KEY_FILE=/var/secrets/openai
cargo run -p rutster-brain-realtime
cargo run
```
Speak → end-to-end speech-to-speech with OpenAI Realtime within ~700 ms
(slice-1's 200 ms + tap round-trip + OpenAI latency + 100 ms playout buffer).
For the foreign-language brain demo (Python, not in CI):
```
pip install -r examples/openai_realtime_brain/requirements.txt
OPENAI_API_KEY=sk-... python examples/openai_realtime_brain/openai_realtime_brain.py
```
> **Status:** Slice 1 (WebRTC media loopback) is the active build target. The workspace is
> landing task-by-task on the `slice-1-webrtc-loopback` branch. Design:
> [`docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md`](docs/superpowers/specs/2026-06-28-slice-1-webrtc-loopback-design.md).