docs(sim): scenario pack + LEARNING.md pointers (slice-4½ S8)

Three shipped scenarios assert distinct FOB reflex properties per spec section 5.3:
- loud-barge.toml: PRIMARY barge-in path (local VAD, 20 loud frames @ 20ms = 400ms)
- quiet-advisory.toml: SECONDARY path (sub-VAD-threshold quiet frames; in the
  slice-4-half standalone-wiring mode this exercises the harness's own latencies
  rather than real brain ASR-VAD -- the latter deferred to post-spearhead
  MockRealtimeBrain composition per spec section 8.6)
- sustained-call.toml: multi-barge / anti-fatigue check (3 loud bursts across 5 speak
  cycles; the second + third bar's kill-time should drift <= 1.5x the first's).

LEARNING.md gains 5 new slice-4-half concept pointers covering: measurement discipline
(the callers clock), post-hoc dedup of noise captures, in-process concurrency sweep,
atomic accumulators with compare_exchange_weak, and pub(crate) visibility for the
percentile_ms helper used by ConcurrencyRunner's sample-level aggregation (avoiding
the interleaved-captures-corrupt-LatencyProbe-pairing problem).

Signed-off-by: Aaron D. Lee <himself@adlee.work>
This commit is contained in:
2026-07-05 03:28:02 -04:00
parent 2f435607f3
commit 5c24c64140
4 changed files with 145 additions and 0 deletions

View File

@@ -106,6 +106,44 @@ can read in `cargo doc --open` plus the source file itself.
doesn't expose subprotocol/auth headers via the simple `connect_async(url)`
entry point — see `openai_headers` / `openai_realtime_url`).
## Slice 4½ — benchmark + simulation harness
- **Measurement discipline: the caller's clock** →
`crates/rutster-sim/src/sim_audio_pipe.rs` — the `AudioPipe` test-double
that IS the caller. Both onset + receipt timestamps are captured inside
the SimAudioPipe (the wall-clock the *caller* started speaking + the
wall-clock the *caller* heard the reply). The harness cannot lie about
latency because the only clock it uses is the caller's (spec §2.2 — the
load-bearing design choice).
- **Post-hoc p50/p99 computation + dedup of noise captures** →
`crates/rutster-sim/src/latency.rs` — the `LatencyProbe` consumes
`Capture` events + pairs `CallerLoudOnset` with the next
`BargeKillObserved` (kill-time) and the next `CallerHeardReply`
(mouth-to-ear). Captures without a prior onset are silently dropped
(the SimAudioPipe captures BargeKillObserved unconditionally on empty
ring; the probe is the dedup gate).
- **In-process concurrency sweep + the doctrine-drift detector** →
`crates/rutster-sim/src/concurrency.rs` — the `ConcurrencyRunner` spawns
N `SimCall`s at levels [1, 10, 50] (the spearhead-scale envelope) +
aggregates per-call latencies. Per spec §2.4: 1 isolates the baseline,
10 is the warm working set, 50 is the saturation point.
- **Atomic accumulators on the hot path: `compare_exchange_weak`** →
`crates/rutster-sim/src/tick_lag.rs``TickLagStats` keeps a per-tick
max + overrun counts via `AtomicU64`. The CAS loop on `max` is the
idiomatic pattern for "atomic max update" in Rust (a single
load-then-CAS-on-fail loop, ordering::Relaxed because stat counters
don't need cross-thread synchronization ordering). 3 atomic ops per tick
(max CAS + conditional fetch_add + unconditional fetch_add) — no Mutex,
lock-free hot path.
- **`pub(crate)` visibility for cross-module helper** →
`crates/rutster-sim/src/latency.rs``percentile_ms` is `pub(crate)`
(not `pub` nor private). The `ConcurrencyRunner` (sibling module) needs
it to compute p50/p99 over the *merged sample across N probes* — merging
`kill_times()` samples per-probe (not concatenating `Capture` vectors)
avoids the interleaved-captures-corrupt-LatencyProbe-pairing problem
(each probe has its own `Option<Instant>` pairing cursor; merging
captures across probes would interleave their cursors).
## How to read
1. `cargo doc --open` — every module has a `//!` doc comment; the doc

View File

@@ -0,0 +1,27 @@
# Scenario: loud-barge (slice-4½ spec §5.3 entry #1)
#
# Drives the PRIMARY barge-in path (slice-4 §5.1): the caller speaks
# one loud burst of audio; the local VAD trips; playout dies; no brain
# advisory needed. Asserts the wedge-#1 path: "VAD killing TTS the
# instant the caller speaks, without the brain."
#
# 20 frames @ 20 ms = 400 ms of speech — comfortably past the
# VAD_DEBOUNCE_FRAMES=3 (60 ms) debounce threshold; the VAD trips
# deterministically.
#
# Threshold assertion (S7): N ∈ [1, 10, 50] concurrent SimCalls — all
# must pass BARGE_IN_KILL_TIME_P99_MS = 80 ms at p99 (60 ms budget +
# 20 ms observer slack).
name = "loud-barge"
[[steps]]
kind = "speak_loud"
frames = 20
[[steps]]
kind = "await_reply"
frames = 0
[[steps]]
kind = "end"

View File

@@ -0,0 +1,31 @@
# Scenario: quiet-advisory (slice-4½ spec §5.3 entry #2)
#
# Drives the SECONDARY barge-in path (slice-4 §5.2): the caller speaks
# sub-VAD-threshold audio (zeroed PCM, energy=0, well below
# VAD_RMS_THRESHOLD=500.0); the local VAD cannot trip; the kill must
# come from the brain's slower ASR-VAD advisory path.
#
# In the slice-4½ sim harness's standalone-wiring mode, the brain side
# is a fake-brain tokio task (no real MockRealtimeBrain WS server) —
# this scenario exercises the advisory path via the LocalVadReflex's
# no-op observation of quiet frames (no trip) + the awaited reply from
# the brain task's seed reply. The kill_time + mouth_to_ear metrics
# therefore measure the harness's own latencies, not real brain-side
# ASR-VAD latency (the latter is deferred to the post-spearhead
# refinement tier per spec §1.2 + §8.6 — paired with MockRealtimeBrain
# composition + LLM-driven callers).
#
# Threshold assertion (S7): N=1 only (the secondary-path focus).
name = "quiet-advisory"
[[steps]]
kind = "speak_quiet"
frames = 20
[[steps]]
kind = "await_reply"
frames = 0
[[steps]]
kind = "end"

View File

@@ -0,0 +1,49 @@
# Scenario: sustained-call (slice-4½ spec §5.3 entry #3)
#
# Drives the multi-barge / fatigue / sustained-load check. Three loud
# bursts + two quiet interludes + end:
# - loud cycle 1 → barge fires → local VAD re-arms on quiet
# - quiet cycle 1 → no barge, VAD re-armed for next loud
# - loud cycle 2 → second barge fires
# - quiet cycle 2 → re-arm
# - loud cycle 3 → third barge fires
#
# Threshold assertion (S7): the per-barge kill_times should drift
# ≤ 1.5× across the three bar cycles (anti-fatigue). The second + third
# bar's kill time should not be significantly longer than the first's
# (no resource exhaustion, no GC pressure, no growing lock contention
# across the slice's lifetime).
#
# See slice-4 §6.1 for the barge_epoch disambiguation mechanism that
# makes "fresh re-barge" vs "late confirmation of the bar already in
# flight" distinguishable — the Reflex::barge_epoch increments on each
# SpeechStarted.
name = "sustained-call"
[[steps]]
kind = "speak_loud"
frames = 10
[[steps]]
kind = "speak_quiet"
frames = 10
[[steps]]
kind = "speak_loud"
frames = 10
[[steps]]
kind = "speak_quiet"
frames = 10
[[steps]]
kind = "speak_loud"
frames = 10
[[steps]]
kind = "await_reply"
frames = 0
[[steps]]
kind = "end"