diff --git a/LEARNING.md b/LEARNING.md index b680e43..174e166 100644 --- a/LEARNING.md +++ b/LEARNING.md @@ -106,6 +106,44 @@ can read in `cargo doc --open` plus the source file itself. doesn't expose subprotocol/auth headers via the simple `connect_async(url)` entry point — see `openai_headers` / `openai_realtime_url`). +## Slice 4½ — benchmark + simulation harness + +- **Measurement discipline: the caller's clock** → + `crates/rutster-sim/src/sim_audio_pipe.rs` — the `AudioPipe` test-double + that IS the caller. Both onset + receipt timestamps are captured inside + the SimAudioPipe (the wall-clock the *caller* started speaking + the + wall-clock the *caller* heard the reply). The harness cannot lie about + latency because the only clock it uses is the caller's (spec §2.2 — the + load-bearing design choice). +- **Post-hoc p50/p99 computation + dedup of noise captures** → + `crates/rutster-sim/src/latency.rs` — the `LatencyProbe` consumes + `Capture` events + pairs `CallerLoudOnset` with the next + `BargeKillObserved` (kill-time) and the next `CallerHeardReply` + (mouth-to-ear). Captures without a prior onset are silently dropped + (the SimAudioPipe captures BargeKillObserved unconditionally on empty + ring; the probe is the dedup gate). +- **In-process concurrency sweep + the doctrine-drift detector** → + `crates/rutster-sim/src/concurrency.rs` — the `ConcurrencyRunner` spawns + N `SimCall`s at levels [1, 10, 50] (the spearhead-scale envelope) + + aggregates per-call latencies. Per spec §2.4: 1 isolates the baseline, + 10 is the warm working set, 50 is the saturation point. +- **Atomic accumulators on the hot path: `compare_exchange_weak`** → + `crates/rutster-sim/src/tick_lag.rs` — `TickLagStats` keeps a per-tick + max + overrun counts via `AtomicU64`. The CAS loop on `max` is the + idiomatic pattern for "atomic max update" in Rust (a single + load-then-CAS-on-fail loop, ordering::Relaxed because stat counters + don't need cross-thread synchronization ordering). 3 atomic ops per tick + (max CAS + conditional fetch_add + unconditional fetch_add) — no Mutex, + lock-free hot path. +- **`pub(crate)` visibility for cross-module helper** → + `crates/rutster-sim/src/latency.rs` — `percentile_ms` is `pub(crate)` + (not `pub` nor private). The `ConcurrencyRunner` (sibling module) needs + it to compute p50/p99 over the *merged sample across N probes* — merging + `kill_times()` samples per-probe (not concatenating `Capture` vectors) + avoids the interleaved-captures-corrupt-LatencyProbe-pairing problem + (each probe has its own `Option` pairing cursor; merging + captures across probes would interleave their cursors). + ## How to read 1. `cargo doc --open` — every module has a `//!` doc comment; the doc diff --git a/crates/rutster-sim/scenarios/loud-barge.toml b/crates/rutster-sim/scenarios/loud-barge.toml new file mode 100644 index 0000000..f9461a3 --- /dev/null +++ b/crates/rutster-sim/scenarios/loud-barge.toml @@ -0,0 +1,27 @@ +# Scenario: loud-barge (slice-4½ spec §5.3 entry #1) +# +# Drives the PRIMARY barge-in path (slice-4 §5.1): the caller speaks +# one loud burst of audio; the local VAD trips; playout dies; no brain +# advisory needed. Asserts the wedge-#1 path: "VAD killing TTS the +# instant the caller speaks, without the brain." +# +# 20 frames @ 20 ms = 400 ms of speech — comfortably past the +# VAD_DEBOUNCE_FRAMES=3 (60 ms) debounce threshold; the VAD trips +# deterministically. +# +# Threshold assertion (S7): N ∈ [1, 10, 50] concurrent SimCalls — all +# must pass BARGE_IN_KILL_TIME_P99_MS = 80 ms at p99 (60 ms budget + +# 20 ms observer slack). + +name = "loud-barge" + +[[steps]] +kind = "speak_loud" +frames = 20 + +[[steps]] +kind = "await_reply" +frames = 0 + +[[steps]] +kind = "end" diff --git a/crates/rutster-sim/scenarios/quiet-advisory.toml b/crates/rutster-sim/scenarios/quiet-advisory.toml new file mode 100644 index 0000000..11f7cad --- /dev/null +++ b/crates/rutster-sim/scenarios/quiet-advisory.toml @@ -0,0 +1,31 @@ +# Scenario: quiet-advisory (slice-4½ spec §5.3 entry #2) +# +# Drives the SECONDARY barge-in path (slice-4 §5.2): the caller speaks +# sub-VAD-threshold audio (zeroed PCM, energy=0, well below +# VAD_RMS_THRESHOLD=500.0); the local VAD cannot trip; the kill must +# come from the brain's slower ASR-VAD advisory path. +# +# In the slice-4½ sim harness's standalone-wiring mode, the brain side +# is a fake-brain tokio task (no real MockRealtimeBrain WS server) — +# this scenario exercises the advisory path via the LocalVadReflex's +# no-op observation of quiet frames (no trip) + the awaited reply from +# the brain task's seed reply. The kill_time + mouth_to_ear metrics +# therefore measure the harness's own latencies, not real brain-side +# ASR-VAD latency (the latter is deferred to the post-spearhead +# refinement tier per spec §1.2 + §8.6 — paired with MockRealtimeBrain +# composition + LLM-driven callers). +# +# Threshold assertion (S7): N=1 only (the secondary-path focus). + +name = "quiet-advisory" + +[[steps]] +kind = "speak_quiet" +frames = 20 + +[[steps]] +kind = "await_reply" +frames = 0 + +[[steps]] +kind = "end" diff --git a/crates/rutster-sim/scenarios/sustained-call.toml b/crates/rutster-sim/scenarios/sustained-call.toml new file mode 100644 index 0000000..a04c385 --- /dev/null +++ b/crates/rutster-sim/scenarios/sustained-call.toml @@ -0,0 +1,49 @@ +# Scenario: sustained-call (slice-4½ spec §5.3 entry #3) +# +# Drives the multi-barge / fatigue / sustained-load check. Three loud +# bursts + two quiet interludes + end: +# - loud cycle 1 → barge fires → local VAD re-arms on quiet +# - quiet cycle 1 → no barge, VAD re-armed for next loud +# - loud cycle 2 → second barge fires +# - quiet cycle 2 → re-arm +# - loud cycle 3 → third barge fires +# +# Threshold assertion (S7): the per-barge kill_times should drift +# ≤ 1.5× across the three bar cycles (anti-fatigue). The second + third +# bar's kill time should not be significantly longer than the first's +# (no resource exhaustion, no GC pressure, no growing lock contention +# across the slice's lifetime). +# +# See slice-4 §6.1 for the barge_epoch disambiguation mechanism that +# makes "fresh re-barge" vs "late confirmation of the bar already in +# flight" distinguishable — the Reflex::barge_epoch increments on each +# SpeechStarted. + +name = "sustained-call" + +[[steps]] +kind = "speak_loud" +frames = 10 + +[[steps]] +kind = "speak_quiet" +frames = 10 + +[[steps]] +kind = "speak_loud" +frames = 10 + +[[steps]] +kind = "speak_quiet" +frames = 10 + +[[steps]] +kind = "speak_loud" +frames = 10 + +[[steps]] +kind = "await_reply" +frames = 0 + +[[steps]] +kind = "end"