- licenses.allow: add "GPL-3.0-or-later" alongside the bare "GPL-3.0"
workaround. cargo-deny 0.19.x (shipped by cargo-deny-action@v2, now on
CI) correctly parses -or-later SPDX; 0.18.x (local rust 1.85 MSRV) still
needs the bare form. When MSRV lifts past 1.88 drop the bare entry.
- advisories.ignore: add RUSTSEC-2024-0421 (idna <= 0.5.0 Punycode
privilege-escalation). rutster never uses idna for hostname comparison
in any privilege-check path; idna is transitive only (url -> reqwest
-> str0m/axum). Bumping past 0.5.0 needs upstream url refresh that
hasn't propagated to our pinned lines. Revisit at MSRV > 1.88.
Unblocks slice-3 PR #4 CI deny gate (was: 8 license rejections on our
own GPL-3.0-or-later crates + 1 idna advisory).
Three end-to-end tests exercising the full slice-3 stack in-process:
MockRealtimeBrain + brain-process-equivalent shim (riding on the public
run_openai_pump library function) + spawn_tap_engine + TapAudioPipe.
No subprocess, no real OpenAI credentials, no browser.
- audio_round_trip_pushes_pcm_and_receives_canned_response (§7.5 #3):
push PcmFrame via TapAudioPipe -> engine -> brain shim ->
run_openai_pump -> translator -> input_audio_buffer.append ->
MockRealtimeBrain -> canned response.audio.delta -> translator ->
audio_out tap frame -> TapAudioPipe.next_pcm_frame. Catches any break
in the audio pipeline; completes in ms.
- function_call_hangup_dispatches_and_closes_session (§7.4 + §7.5 #3):
ToolRegistry.dispatch("hangup") -> HangupTool.call -> AppState::close
-> session removed from DashMap, returns Ok({channel_state: Closing}).
Drives the registry via the public Arc<Mutex<ToolRegistry>> handle on
TapConn (in-process drain_function_calls is private to session_map,
its behavior is covered by session_map.rs:451's inline test).
- s4_brain_sends_session_update_with_turn_detection_null_end_to_end
(§7.5 #7): the brain process equivalent sends session.update with
turn_detection: null on pump startup (run_openai_pump's first action).
MockRealtimeBrain rejects non-null with a typed OpenAI-shaped error
(mock.rs:150). Stack comes up without WS errors + engine
reconnect_attempts stays 0 -> S4 honored end-to-end through the
actual translator.
Note: AGENTS.md says browser-driven e2e is deferred post-slice-1, so
the test pushes PCM directly through TapAudioPipe (the slice-2
tap_integration.rs harness pattern), not via a real str0m peer.
Files added: crates/rutster/tests/realtime_integration.rs (new test).
Files modified: crates/rutster/Cargo.toml adds rutster-brain-realtime
(mock feature) as dev-dep so the test can drive run_openai_pump +
MockRealtimeBrain in-process; Cargo.lock regenerated.
Seam test honored: loop_driver.rs and rtc_session.rs untouched.
All 91 workspace tests green. cargo fmt --all --check +
cargo clippy --all --all-targets -- -D warnings clean.
slice-3 §7.4: external golden JSON fixtures + a new integration-test
file locking the exact on-wire bytes produced by each slice-3 encode_*
function. The inline round-trip tests in protocol.rs use String::contains
(substring) which cannot catch field-order drift, stray fields, or number
formatting changes — all of which the Envelope Serialize impl promises to
keep stable (protocol.rs:115-118: "field order is pinned to make the wire
bytes deterministic"). These fixtures make that promise load-bearing.
5 fixtures cover the 5 new event kinds (spec §3.2-3.3):
speech_started, speech_stopped, function_call,
function_call_output, tools.update.
Each fixture asserts (a) encode_* output equals the fixture bytes exactly
(string equality, not serde_json::Value equality — the latter would
erase the field-order pinning the serializer guarantees), and (b) the
fixture decodes back to the expected DecodedPayload variant with the
expected fields.
Plus a forwards-compat re-assertion: unknown wire type -> DecodedPayload::Unknown
(via FrameKind's #[serde(other)]), not an error — the foundation of the
slice-2 §3.4 "drop + observe" posture that lets a future-protocol brain
not crash an older core.
On first run the golden test caught real drift: serde_json (without
preserve_order) alphabetizes nested object keys, so tools.update's tools
array serializes description-before-name, not input order. Fixture
corrected to lock the actual contract.
11 new tests, all green. cargo fmt + clippy -D warnings clean.
slice-3 §5.3 dev mode: an in-process fake OpenAI Realtime WS server
(MockRealtimeBrain) behind --features=mock. Asserts session.update has
turn_detection: null (S4 — spec §4.3) on handshake; exports a typed
OpenAI-shaped error on violation so the brain fails fast + visibly.
On each input_audio_buffer.append replies with a canned
response.audio.delta carrying 480 zeroed samples as base64 LE i16
(same wire shape as slice-2's audio_out). Powers the offline dev loop
(cargo run -p rutster-brain-realtime --features=mock) + the integration
test (no real OpenAI credentials, no network calls).
Also adds the binary main.rs (spec §4.2 + §5.3): binds the tap-side WS
server on RUTSTER_TAP_BIND (default 127.0.0.1:8082), accepts per-call
tap connections, handles the hello handshake, splits the WS into sink +
stream, dials the OpenAI side (real wss://api.openai.com in default
mode; MockRealtimeBrain url in mock mode), and bridges via
run_openai_pump + two mpsc::channel<String> pairs.
Integration test brain_mock_round_trip drives MockRealtimeBrain +
run_openai_pump end-to-end: pushes a tap audio_in frame through the
pump, observes the canned response.audio.delta arrive + get translated
back as a tap audio_out frame on the pump's outbound mpsc. Verifies
the dev-loop 'actually start + interop on a basic audio round-trip'
contract the directive's Task 4 calls out, without requiring a real
WebRTC peer (same constraint as slice-1/2's browser-driven e2e).
Files: crates/rutster-brain-realtime/src/{lib,mock,main}.rs,
crates/rutster-brain-realtime/tests/brain_mock_round_trip.rs.
NOT touched: loop_driver.rs, rtc_session.rs (seam §7.5 #6).
Gates: cargo fmt OK. cargo clippy --all --tests --features=mock
-D warnings OK. cargo test --all --features=mock OK (56 tests).
cargo deny check has pre-existing environmental failure (CVSS 4.0
unsupported in advisory-db; same on main).
Dev loop verified: cargo run -p rutster-brain-realtime --features=mock
starts the brain (mock on ephemeral port, tap server on :8082).
./target/release/rutster with RUTSTER_TAP_URL=ws://127.0.0.1:8082/realtime
starts the core on :8080. POST /v1/sessions succeeds. (Full audio
round-trip requires a browser-driven WebRTC peer, same constraint as
slice-1/2 — covered by the integration test instead.)
slice-3 §5.2 + §6: the binary's poll task now drains the brain's
function_call proposals from rx_function_call, dispatches through the
per-channel ToolRegistry (HangupTool wired at spawn_tap_engine time),
and writes function_call_output replies back through tx_function_call_output
which run_tap_client forwards as tap WS frames to the brain.
TapClient: handle_brain_frame now forwards function_call events to a
new tx_function_call mpsc side-channel instead of dropping them.
run_tap_client adds a select! arm draining rx_function_call_output +
sending each as a tap frame. Advisory events (speech_started/stopped,
tools.update) still log + count (slice-3 deferred-action posture).
TapEngine: spawn_tap_engine now takes AppState + constructs a per-channel
ToolRegistry (spec §6.2) with HangupTool pre-registered (§6.3). TapConn
gains rx_function_call, tx_function_call_output, tool_registry fields.
session_map: drive_all_sessions calls drain_function_calls in the same
cycle as the slice-2 §5.3 step 4 flush drain (one extra channel, same
cycle); the helper spawns each dispatch as its own task so the 750 ms
hangup teardown bound (AppState::close) can't stall the 10 ms poll
cadence.
files touched: crates/rutster-tap/src/{lib,tap_client}.rs,
crates/rutster/src/{session_map,tap_engine}.rs,
crates/rutster/tests/tap_integration.rs ( AppState arg ),
crates/rutster-brain-realtime/src/translator.rs (clippy needless_borrow ).
NOT touched: loop_driver.rs, rtc_session.rs (seam test §7.5 #6).
gates: cargo fmt --check OK. cargo clippy --all --tests -D warnings OK.
cargo test --all OK. cargo deny check has pre-existing environmental
failure (CVSS 4.0 unsupported in advisory-db; same on main).
examples/openai_realtime_brain/ -- the canonical foreign-language OpenAI
Realtime brain (Python, ~150 lines, websockets lib only). Not in CI
(zero-non-Rust-dev-deps dev loop per AGENTS.md). Mirrors the slice-2
echo_brain pattern and proves the protocol is language-agnostic.
Hand-rolled against the spec §4.2 mapping rather than using the openai
Python SDK: the SDK is unused by the wire-shape (both sides are plain
WSS), so dropping it removes a dep that the plan skeleton imported but
never called. session.update is sent with turn_detection=null (S4)
matching the Rust brain.
LEARNING.md gains 5 new pointers under a Slice 3 heading: async-trait
in trait objects, the translator pure-function layer, additive protocol
extension via #[serde(other)], the FOB-boundary side-channel mpsc, and
the WS subprotocol handshake (openai_client.rs).
README.md gains the Slice 3 dev loop section (mock mode without an
OpenAI key, real-OpenAI mode, and the Python brain alternative).
.gitignore learns about __pycache__/*.pyc (the Python brains aren't in
CI but byte-compiled artifacts should never be tracked).
FOB-boundary dispatch for brain-proposed tool calls (ADR-0007: 'rutster
mediates both the provider call-control API and the brain tap, so the
brain never holds the wire'). Slice-3's only wired tool is hangup
(fires AppState::close → existing slice-2 teardown); other tool names
reply not_implemented.
TDD: 5 tests (dispatch known + unknown tool, catalog listing, hangup
schema shape, hangup_call fires AppState::close).
Builds session.update with turn_detection: null on handshake (S4,
encoded in the translator's build_openai_session_update; Task 4).
Runs a select! loop over tap-side input (audio_in → append,
function_call_output → conversation.item.create) and OpenAI-side
input (response.audio.delta → tap audio_out, speech_started/stopped
→ tap speech_started/stopped, function_call_arguments.done → tap
function_call). 401 surfaces as OpenAiClientError::AuthFailed.
URL + headers shape tested directly. Full pump loop tested via
MockRealtimeBrain in Task 10 — neither side of this pump is a real
network endpoint in unit tests.
Pure functions, no I/O, no call state. The OpenAI-side WS client
(Task 5) and the tap-side WS server (Task 9) call these per event.
S4 turn-ownership decision (spec §4.3, load-bearing per ADR-0008)
encoded in build_openai_session_update: turn_detection: null. OpenAI's
server-side VAD is disabled; the FOB reflex loop (step 4) owns
turn-taking; tap playout stays core-authoritative (slice-2 §4.1).
S4 test verifies the load-bearing assertion
(v['session']['turn_detection'] == Null).
Audio is pass-through — OpenAI's PCM base64 (LE i16 24 kHz mono) is
indentical to slice-2's tap PCM wire shape (spec §3.5); decode +
re-encode as a PcmFrame so the playout ring stays byte-aligned for
slice-2's playout-buffer invariants (samples: 480).
Five new event types in slice-2's v1 protocol. Forwards-compatible
per slice-2 §3.4: FrameKind's #[serde(other)] Unknown absorbs the
new types in old brains (they log + count + drop). No wire-format
break, no version bump.
New kid on the dispatch: tools.update — brain declares its catalog
on hello so the core's tool registry can validate function_call
events by name (Task 6). The 'function_call'/'function_call_output'
pair is the FOB-boundary dispatch contract the brain's translator
(Task 4) wires to OpenAI Realtime's
response.function_call_arguments.done / conversation.item.create.
TDD: 6 tests fail-red on missing impl, pass-green after this task.
Slice-2's existing protocol tests stay green — the additions are
purely additive.
Workspace member 8 of 8. Library + binary; mirrored slice-2's
rutster-tap-echo shape. Three stub modules (api_key, translator,
openai_client) filled in by Tasks 3-5. No behavioral code yet —
this task only locks the crate boundary + the new async-trait dep
(the Tool trait in Task 6 needs it for async-fns-in-trait-objects).
The RtcSession module doc cited the dropped rutster-signaling-sip crate
and a 'future SIP SDP path' that ADR-0007 superseded. Re-anchor on
ADR-0007: no first-party SIP/SDP — the trunk is rented (CPaaS media-leg)
or out-of-tree SBC, so rutster's SDP is WebRTC-only via str0m. Final
straggler of the rutster-signaling-sip → rutster-trunk rename.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KhhqKG4cra7d1PBoVbj9UJ
Brainstorming output for spearhead step 3: swap slice-2's echo brain
for a real speech-to-speech brain (OpenAI Realtime) reached through
slice-2's existing tap interface. The core dials out (core-as-client,
unchanged); slice-2's tap protocol carries audio + additive new event
types (speech_started/stopped, function_call, function_call_output,
tools.update); the brain process translates between our protocol and
OpenAI Realtime's event taxonomy.
The seam slice-2 pre-paved (AudioSource / AudioSink traits + TapAudioPipe
+ TapEngine) is the test of this slice: RtcSession's media-loop path
changes by zero lines; the TapEngine's spawn/reconnect/teardown logic
is untouched; only rutster-tap/src/protocol.rs grows new event types
and a new tool_registry.rs module lands in the binary.
Locks the S4 turn-ownership decision per ADR-0008 (vision-sanity-check
filed 2026-06-28): the reflex loop is FOB (hot-path + differentiating),
the brain is green-zone (per ADR-0008 explicit classification), the tap
is core-authoritative (slice-2 §4.1). The only doctrine-consistent
choice is to disable OpenAI Realtime's server-side turn detection
(session.update with turn_detection: null) and let the FOB own
turn-taking — pre-paving step 4's barge-in on a clean foundation.
Brainstorming decisions ratified:
- Approach A: rutster-brain-realtime crate (mirrors rutster-tap-echo),
protocol-as-contract (no speculative trait crate).
- Scope: speech + interruption signals (advisory) + function-calling
plumbing.
- Built-in tool registry in the core (FOB); hangup is the only wired
tool, others reply not_implemented.
- Env-var + file-path API-key posture (OPENAI_API_KEY +
OPENAI_API_KEY_FILE).
- --features=mock dev mode with in-process mocked OpenAI Realtime.
Pre-implementation: TDD via writing-plans skill is the next step once
the user reviews this spec.
