Brainstorming output for spearhead step 3: swap slice-2's echo brain
for a real speech-to-speech brain (OpenAI Realtime) reached through
slice-2's existing tap interface. The core dials out (core-as-client,
unchanged); slice-2's tap protocol carries audio + additive new event
types (speech_started/stopped, function_call, function_call_output,
tools.update); the brain process translates between our protocol and
OpenAI Realtime's event taxonomy.
The seam slice-2 pre-paved (AudioSource / AudioSink traits + TapAudioPipe
+ TapEngine) is the test of this slice: RtcSession's media-loop path
changes by zero lines; the TapEngine's spawn/reconnect/teardown logic
is untouched; only rutster-tap/src/protocol.rs grows new event types
and a new tool_registry.rs module lands in the binary.
Locks the S4 turn-ownership decision per ADR-0008 (vision-sanity-check
filed 2026-06-28): the reflex loop is FOB (hot-path + differentiating),
the brain is green-zone (per ADR-0008 explicit classification), the tap
is core-authoritative (slice-2 §4.1). The only doctrine-consistent
choice is to disable OpenAI Realtime's server-side turn detection
(session.update with turn_detection: null) and let the FOB own
turn-taking — pre-paving step 4's barge-in on a clean foundation.
Brainstorming decisions ratified:
- Approach A: rutster-brain-realtime crate (mirrors rutster-tap-echo),
protocol-as-contract (no speculative trait crate).
- Scope: speech + interruption signals (advisory) + function-calling
plumbing.
- Built-in tool registry in the core (FOB); hangup is the only wired
tool, others reply not_implemented.
- Env-var + file-path API-key posture (OPENAI_API_KEY +
OPENAI_API_KEY_FILE).
- --features=mock dev mode with in-process mocked OpenAI Realtime.
Pre-implementation: TDD via writing-plans skill is the next step once
the user reviews this spec.
Design record for the operator console — the downstream non-technical
operator's surface. **Not scheduled for current build;** lands after the
spearhead (steps 1-6) and the ACD/escalation rungs (capability ladder
rungs 1-2). The current build target is slice-2; the next build target
after slice-2 is slice-3 (the OpenAI Realtime adapter). An agent that
proposes building the console now is wrong — point it at spec §0.
Records the visual-brainstorming session's resolved decisions so they
don't get re-litigated at implementation time:
§1 Principles (load-bearing):
- GUI is a pure API client — **green-zone, never an insider** (ADR-0008);
holds no privilege a third party couldn't get from the public API.
- Config-as-code is the single source of truth; the UI is a reconciled
*view* (the k8s model — declarative desired-state in git, the engine
reconciles; console + CLI + git are views over it).
- Operator persona, not builder. The builder authors flows in **code +
an AI pair** (ARCHITECTURE 'DX spine'); this console is NOT the
authoring surface for the technical builder. **No drag-drop authoring
canvas** — a deliberately-closed decision (vision-revision §8).
- Ships in the batteries-included distro and the all-in-one image
(the reference GUI is bundled — 'boom' includes a usable console).
§2 IA — Ops Console shell: left nav + center panel + persistent live rail.
Dense single-screen; for an operator who lives in it. Live rail always
visible regardless of the selected section.
§3 Dashboards — Live Ops (default landing) + Quality (sibling view). Maps
the k8s split: Live Ops = live operational state; Quality = the data-owned
eval loop (capability ladder rungs 3→4).
§4 Flows editor — master/detail + a generated, navigable preview map.
The preview map is a *view* rendered from the config, not a drag-drop
authoring canvas. Earns its place by showing branching (the escalation
split) the linear stage-list flattens.
§5 Declarative flow schema (illustrative YAML; format is open). The config
the console edits via the API; the builder edits in git; the engine
reconciles. The map is generated from this structure.
§6 Other sections (conventional): Agents · Numbers · Queues · People ·
Recordings · Settings.
§7 Architecture: separate app, public API only; never reaches into the FOB.
Config writes go through the reconcile path. Live data rides the event
stream off Valkey (ADR-0005); never the 20 ms media loop.
§8 v1 vs. later scoping. §9 open decisions (flow config format, map
rendering depth, quality metrics, live transport, framework).
Cross-refs ADR-0002 (fused vertical + composable platform), ADR-0006
(WebRTC ingress / escalation UX), ADR-0007 (rented trunk / Numbers
config), ADR-0008 (FOB / green-zone — the UI is green-zone),
ARCHITECTURE.md 'DX spine' + 'GUI & extension architecture', PORT_PLAN §5
(contact-center domain).
Mechanical propagation of the crate rename ( rutster-signaling-sip →
rutster-trunk ) and the repo URL fix ( github.com/anomalyco →
git.adlee.work/alee ) through the documents that name them:
- docs/DEVELOPMENT.md: crate-layout sketch + stub-crate description.
- docs/superpowers/plans/2026-06-28-slice-1-webrtc-loopback.md: the
workspace members list in the plan's binding-values section —
repository URL updated.
- docs/superpowers/plans/2026-06-28-slice-2-agent-tap.md: workspace
members list in binding-values + crate-layout sketch in the file-
structure section (rutster-signaling-sip → rutster-trunk).
- docs/superpowers/specs/2026-06-28-slice-2-agent-tap-design.md: §2.1
workspace-layout sketch's STUB crate-row.
No content changes beyond the substitutions; the slice-2 spec/plan
body (protocol, TapAudioPipe, TapClient, TapEngine, lifecycle, done-
criteria) is untouched.
Patches the agent-tap design with seven findings from PM-side review
before the implementation plan gets written. Resolves invariant-breakers
and deferred-decision debt that would compound across steps 3-6.
- P1 (high): §8.5 #6 'byte-identical loop_driver.rs' was the wrong bar
for the seam test. Reworded: the trait-method *call sites* are
unchanged; impl bodies are free to differ. That's what 'the seam
held' actually means — the contract is the test.
- P2 (high): tap: Option<TapHandle> and ChannelState were parallel
state machines without an invariant tying them. Added an explicit
state-pair table: tap is Some iff state == Connected. Lookups tolerate
transient inconsistency; the source-of-truth is the pair, not either
field alone.
- P3 (medium): v1 wire format was host-endian (silent big-endian
hazard on paper). Nailed LE explicitly via i16::to_le_bytes /
i16::from_le_bytes. v2 can negotiate endianness if a big-endian
brain ever materializes; v1 contract is LE-only. Added a §3.4
byte-order invariant alongside the sample-count invariant.
- P4 (medium): wss:// URLs were accepted by schema but returned
501 at connect time (UX: 2 s into a call). Now rejected at
POST /v1/sessions with 400 + clear message — fail fast at
session-create. Updated §4.4, §7.1, §7.3, §9, and the decisions
table for consistency; removed all stale '501 at connect' claims.
- P5 (low): TapEngine spawn owner was unspecified. Added explicit
ownership: session_map::drive_all_sessions (the binary's poll
task) observes the Connected transition from run_poll_once and
spawns the engine. Keeps loop_driver.rs behaviorally unchanged
(the §8.5 #6 seam test still holds).
- P6 (low): 'two new deps' in §1.1 undercounted. Updated list:
tokio-tungstenite, futures-util, url, serde_json (if not already
pulled).
- P7 (low): ADR-0007 was 'post-implementation follow-up, not a
slice-2 deliverable.' Tightened: strongly recommended to land
alongside or immediately after slice-2 implementation — once
step 3 ships against an unpinned protocol, the wire shape will
silently drift. The spec lists the decisions ADR-0007 should
capture (LE byte order added to the list).
Slice 2 hardens the presumptive agent-tap shape from ARCHITECTURE.md §"Agent tap"
(PORT_PLAN §10 open decision) against a working implementation:
- Replace slice-1's in-process EchoAudioPipe with a real out-of-process brain
reached over WSS; core-as-client, no inbound tap port (ADR-0006 posture).
- Versioned JSON event protocol (v1) over WS text frames with base64 PCM.
Event-named + JSON-over-text-WS so the step-3 OpenAI-Realtime adapter is a
translation shim, not a gRPC-bridge project.
- Decoupled TapEngine (cold-path tokio task) owns the WS connection; TapAudioPipe
is a thin sync wrapper the AudioSource/AudioSink seam holds. RtcSession's media
loop is byte-identical to slice-1 — the seam-test payoff.
- Core-authoritative playout buffer (5-frame/100ms ring; drop-oldest on overflow,
silence on underflow) — the place where "brain proposes, core disposes" lives.
- Bounded-backoff reconnect (250ms→5s cap, infinite retries); Channel stays
Connected; stateless-brain reconnect contract.
- Both a Python reference echo brain (examples/echo_brain/, not in CI) and a
Rust echo brain crate (crates/rutster-tap-echo, the in-process test server
+ standalone dev binary).
- ws:// loopback-only enforced; wss:// URL accepted at schema, 501 at connect
(cert/mTLS impl deferred to step 6).
Out-of-scope table (§1.2) locks the spearhead sequencing: real brain (3),
barge-in/VAD-driven playout kill (4), PSTN trunk (5), spend cap (6), wss:// TLS
posture (6), binary PCM mode (future-rung), byte-endian negotiation (tracked
open decision §9).
uuid 1.x → getrandom 0.4.x requires edition = "2024", stabilized in Rust 1.85.
The Task 1 pin of 1.80 was written when uuid 1.x resolved to a getrandom that
still compiled on edition 2021; the resolution has shifted. The brief said
'confirm the latest stable at impl time' — bumping the pin is faithful to that.
Plan doc's Task 6 CI matrix also updated: [stable, "1.85"].
Cross-task correction in response to Task 2 implementer's DONE_WITH_CONCERNS
escalation; not a Task 2 code change.
Verified str0m 0.21 + opus 0.3.1 API surfaces via docs.rs subagents;
patched the plan with the real signatures and several design-level fixes.
API-surface patches (str0m 0.21, verified):
- Global Constraints: full verified API surface documented inline
(Rtc::new takes Instant; SdpOffer::from_sdp_string is the entry
point, NOT from_str_unchecked; add_local_candidate returns
Option<&Candidate>; Writer::write takes rtp_time not media_time;
MediaTime has no add(Duration) -> use mt + MediaTime::from(d);
payload_params returns impl Iterator; MediaData.data is Arc<[u8]>).
- Task 4 accept_offer: rewritten with from_sdp_string + correct error
mapping; RtcSessionError::SdpOffer changed to String (collapses parse
+ accept failures uniformly).
- Task 4 loop_driver: MediaTime::add -> + MediaTime::from(Duration);
media.data deref coercion documented.
Design-level patches:
- Task 4: RtcSession::new_for_test -> pub fn new() (single idiomatic
constructor, no test/prod split).
- Task 4: added accept_offer_transitions_channel_to_connecting test
(the transition was claimed but untested in the original plan).
- Task 5: dropped reqwest from workspace deps (unused — integration
test uses tower::ServiceExt::oneshot); added tower as workspace dep
+ binary crate dev-dependency.
- Task 5: removed duplicate DELETE /v1/sessions route (was incorrectly
chained via .delete() on the collection route AND on /v1/sessions/:id
— only the latter is correct).
- Task 5: clarified pub mod requirement (must be pub because the
integration test references modules via absolute paths).
- Global Constraints: added task/PR strategy (one commit per task,
merged in numeric order, granular history is load-bearing for the
learning-codebase goal).
Self-review section updated to reflect the patches.
Plan is now implementation-ready.
API-surface corrections (verified against str0m 0.21 docs):
- str0m::Live -> str0m::Rtc (no Live type exists)
- needs_input_timeout() -> Rtc::poll_output() returning Output::Timeout(Instant)
- RtcConfig::with_dtls_cert -> RtcConfig::set_dtls_cert (also: str0m auto-gens
a cert if none passed; slice 1 still passes explicitly for ownership clarity)
- Event::RtpPacket(pkt, rid) -> Event::RtpPacket(RtpPacket) { single struct,
RID is a field }
- str0m::ice::IceAgent -> no public 'ice' module; ICE types at crate root:
Candidate, CandidateKind, IceCreds, IceConnectionState
- str0m.write_rtp() on Rtclive -> Frame API path: Rtc::writer(mid) -> Writer::write
New design decision surfaced by API verification:
- Media API path: Frame API (default) vs RTP API (set_rtp_mode(true)).
Slice 1 uses Frame API — the proof target is the codec-to-PCM boundary,
not RTP packetization. RTP API is a candidate for step 5.
Design-level fixes from close reading:
- §3.3 AudioSource/AudioSink: clarify who drives the traits (the poll loop
drives both; sink on inbound MediaData, source on Output::Timeout deadline)
- §3.6: clarify cert/CrossRef — fresh in-memory cert each restart is safe
because browser fetches fresh SDP each session, no caching layer to go stale
- §4.1: add Content-Type: application/sdp to /offer (browser-side fetch needs
it)
- §4.5: resolve 5-min-vs-60s idle-timeout contradiction (60s, not 5 min)
- §4.5: add graceful-shutdown story (drop DashMap on SIGINT, browsers see
dead peer — acceptable for dev loop)
- §6.4: fix 'mocked str0m Rtc' — str0m's Rtc is a concrete struct, not a
trait; sans-IO means drive real Rtc with synthetic Inputs, not mock it.
Stronger than mocking: production path exercised exactly.
Resolves 5 load-bearing API errors + 3 design-level ambiguities. Spec is now
implementation-ready pending user review.
Folds the 2026-06-26 vision-revision pressure-test into the canonical
docs. The spec moves from Proposed to Ratified; the decisions it produced
land in the docs it said it would amend, with ADRs recording each
load-bearing reversal.
- README.md: reframe north star (AI-era contact-center engine, not
Asterisk successor); persona; revised pillars (add data-ownership,
demote WASM, promote spend-control); update 'what it is/isn't'.
- ARCHITECTURE.md: replace three-plane framing with fused per-call
vertical + composable horizontal platform; remove control<->media
hot-path gRPC hop; make the agent tap the central interface; add
DX spine + GUI-as-API-client + k8s declarative/operational model.
Also: 'too slow to police' -> 'too slow to enforce' (terminology).
- PORT_PLAN.md: recharacterize as capability checklist (not template);
graduate contact-center capabilities to first-class domain; Rust-native
trunk SIP rows; WASM demoted; thin-slice + capability ladder phasing.
- ADR-0001: marked Superseded by ADR-0003.
- ADR-0002 (new): north star + fused per-call core.
- ADR-0003 (new): Rust-native trunk SIP, no SBC shield.
- ADR-0004 (new): GPL-3.0-or-later license.
- ADR-0005 (new): Valkey as event bus + state store.
- ADR-0006 (new): WebRTC-first ingress; SIP endpoint deferred.
- vision-revision spec: status -> Ratified 2026-06-26.
- Clarify RtcSession/Channel ownership: ChannelId IS the session id;
RtcSession owns both str0m Live + codecs and the Channel. Fixes ambiguity.
- Drop the separate /ice endpoint: non-trickle ICE bundles candidates
into the SDP offer/answer, so the endpoint was redundant. Fixes
contradiction.
- Clarify idle timeout = 'no RTP for 60s' (not 5 min blanket), and that
no per-session tokio task is spawned (pre-paves the wrong pattern
for step 4's dedicated thread).
- 'reflex to police' -> 'reflex to enforce' (terminology: avoid
authoritarian verbs).
Workspace scaffold + spearhead step 1: browser talks WebRTC to the core,
core terminates DTLS-SRTP, decodes Opus to canonical 16-bit PCM @24 kHz mono,
echoes back. Proves the media core + the codec-to-PCM boundary (pre-paves
the tap). Stubs lock the ADR-0002 fused-vertical boundaries for steps
2/5/6.
Drafted in brainstorming session 2026-06-28.
Records the decisions from pressure-testing the four founding docs:
- North-star reframe: the engine for the AI-era contact center, not an
Asterisk-protocol successor (Asterisk's map = checklist, not template)
- Fused per-call vertical + composable horizontal platform (tightens the
three-plane model; removes the control<->media hot-path gRPC hop)
- Rust-native trunk SIP, IP-allowlisted; drops the Kamailio/rtpengine C
shield (reverses ADR-0001, to be formalized as ADR-0003)
- Pillars re-weighted: +data-ownership, spend-control promoted to
constitutive, WASM demoted out of the core story
- Developer-first DX spine (config-as-code, AEL lesson, engine+batteries)
- GUI as pure API client (FreePBX model) + k8s declarative/operational split
- Sequenced thin-slice first proof + capability ladder
For review before applying to README/ARCHITECTURE/PORT_PLAN and writing
ADR-0002 (identity) and ADR-0003 (SIP).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUtgQTo3wHGXP1TixHr4gd
