Brainstorming output for spearhead step 3: swap slice-2's echo brain
for a real speech-to-speech brain (OpenAI Realtime) reached through
slice-2's existing tap interface. The core dials out (core-as-client,
unchanged); slice-2's tap protocol carries audio + additive new event
types (speech_started/stopped, function_call, function_call_output,
tools.update); the brain process translates between our protocol and
OpenAI Realtime's event taxonomy.
The seam slice-2 pre-paved (AudioSource / AudioSink traits + TapAudioPipe
+ TapEngine) is the test of this slice: RtcSession's media-loop path
changes by zero lines; the TapEngine's spawn/reconnect/teardown logic
is untouched; only rutster-tap/src/protocol.rs grows new event types
and a new tool_registry.rs module lands in the binary.
Locks the S4 turn-ownership decision per ADR-0008 (vision-sanity-check
filed 2026-06-28): the reflex loop is FOB (hot-path + differentiating),
the brain is green-zone (per ADR-0008 explicit classification), the tap
is core-authoritative (slice-2 §4.1). The only doctrine-consistent
choice is to disable OpenAI Realtime's server-side turn detection
(session.update with turn_detection: null) and let the FOB own
turn-taking — pre-paving step 4's barge-in on a clean foundation.
Brainstorming decisions ratified:
- Approach A: rutster-brain-realtime crate (mirrors rutster-tap-echo),
protocol-as-contract (no speculative trait crate).
- Scope: speech + interruption signals (advisory) + function-calling
plumbing.
- Built-in tool registry in the core (FOB); hangup is the only wired
tool, others reply not_implemented.
- Env-var + file-path API-key posture (OPENAI_API_KEY +
OPENAI_API_KEY_FILE).
- --features=mock dev mode with in-process mocked OpenAI Realtime.
Pre-implementation: TDD via writing-plans skill is the next step once
the user reviews this spec.
1a029d7044