docs(sim): scenario pack + LEARNING.md pointers (slice-4½ S8)
Three shipped scenarios assert distinct FOB reflex properties per spec section 5.3: - loud-barge.toml: PRIMARY barge-in path (local VAD, 20 loud frames @ 20ms = 400ms) - quiet-advisory.toml: SECONDARY path (sub-VAD-threshold quiet frames; in the slice-4-half standalone-wiring mode this exercises the harness's own latencies rather than real brain ASR-VAD -- the latter deferred to post-spearhead MockRealtimeBrain composition per spec section 8.6) - sustained-call.toml: multi-barge / anti-fatigue check (3 loud bursts across 5 speak cycles; the second + third bar's kill-time should drift <= 1.5x the first's). LEARNING.md gains 5 new slice-4-half concept pointers covering: measurement discipline (the callers clock), post-hoc dedup of noise captures, in-process concurrency sweep, atomic accumulators with compare_exchange_weak, and pub(crate) visibility for the percentile_ms helper used by ConcurrencyRunner's sample-level aggregation (avoiding the interleaved-captures-corrupt-LatencyProbe-pairing problem). Signed-off-by: Aaron D. Lee <himself@adlee.work>
This commit is contained in:
38
LEARNING.md
38
LEARNING.md
@@ -106,6 +106,44 @@ can read in `cargo doc --open` plus the source file itself.
|
||||
doesn't expose subprotocol/auth headers via the simple `connect_async(url)`
|
||||
entry point — see `openai_headers` / `openai_realtime_url`).
|
||||
|
||||
## Slice 4½ — benchmark + simulation harness
|
||||
|
||||
- **Measurement discipline: the caller's clock** →
|
||||
`crates/rutster-sim/src/sim_audio_pipe.rs` — the `AudioPipe` test-double
|
||||
that IS the caller. Both onset + receipt timestamps are captured inside
|
||||
the SimAudioPipe (the wall-clock the *caller* started speaking + the
|
||||
wall-clock the *caller* heard the reply). The harness cannot lie about
|
||||
latency because the only clock it uses is the caller's (spec §2.2 — the
|
||||
load-bearing design choice).
|
||||
- **Post-hoc p50/p99 computation + dedup of noise captures** →
|
||||
`crates/rutster-sim/src/latency.rs` — the `LatencyProbe` consumes
|
||||
`Capture` events + pairs `CallerLoudOnset` with the next
|
||||
`BargeKillObserved` (kill-time) and the next `CallerHeardReply`
|
||||
(mouth-to-ear). Captures without a prior onset are silently dropped
|
||||
(the SimAudioPipe captures BargeKillObserved unconditionally on empty
|
||||
ring; the probe is the dedup gate).
|
||||
- **In-process concurrency sweep + the doctrine-drift detector** →
|
||||
`crates/rutster-sim/src/concurrency.rs` — the `ConcurrencyRunner` spawns
|
||||
N `SimCall`s at levels [1, 10, 50] (the spearhead-scale envelope) +
|
||||
aggregates per-call latencies. Per spec §2.4: 1 isolates the baseline,
|
||||
10 is the warm working set, 50 is the saturation point.
|
||||
- **Atomic accumulators on the hot path: `compare_exchange_weak`** →
|
||||
`crates/rutster-sim/src/tick_lag.rs` — `TickLagStats` keeps a per-tick
|
||||
max + overrun counts via `AtomicU64`. The CAS loop on `max` is the
|
||||
idiomatic pattern for "atomic max update" in Rust (a single
|
||||
load-then-CAS-on-fail loop, ordering::Relaxed because stat counters
|
||||
don't need cross-thread synchronization ordering). 3 atomic ops per tick
|
||||
(max CAS + conditional fetch_add + unconditional fetch_add) — no Mutex,
|
||||
lock-free hot path.
|
||||
- **`pub(crate)` visibility for cross-module helper** →
|
||||
`crates/rutster-sim/src/latency.rs` — `percentile_ms` is `pub(crate)`
|
||||
(not `pub` nor private). The `ConcurrencyRunner` (sibling module) needs
|
||||
it to compute p50/p99 over the *merged sample across N probes* — merging
|
||||
`kill_times()` samples per-probe (not concatenating `Capture` vectors)
|
||||
avoids the interleaved-captures-corrupt-LatencyProbe-pairing problem
|
||||
(each probe has its own `Option<Instant>` pairing cursor; merging
|
||||
captures across probes would interleave their cursors).
|
||||
|
||||
## How to read
|
||||
|
||||
1. `cargo doc --open` — every module has a `//!` doc comment; the doc
|
||||
|
||||
27
crates/rutster-sim/scenarios/loud-barge.toml
Normal file
27
crates/rutster-sim/scenarios/loud-barge.toml
Normal file
@@ -0,0 +1,27 @@
|
||||
# Scenario: loud-barge (slice-4½ spec §5.3 entry #1)
|
||||
#
|
||||
# Drives the PRIMARY barge-in path (slice-4 §5.1): the caller speaks
|
||||
# one loud burst of audio; the local VAD trips; playout dies; no brain
|
||||
# advisory needed. Asserts the wedge-#1 path: "VAD killing TTS the
|
||||
# instant the caller speaks, without the brain."
|
||||
#
|
||||
# 20 frames @ 20 ms = 400 ms of speech — comfortably past the
|
||||
# VAD_DEBOUNCE_FRAMES=3 (60 ms) debounce threshold; the VAD trips
|
||||
# deterministically.
|
||||
#
|
||||
# Threshold assertion (S7): N ∈ [1, 10, 50] concurrent SimCalls — all
|
||||
# must pass BARGE_IN_KILL_TIME_P99_MS = 80 ms at p99 (60 ms budget +
|
||||
# 20 ms observer slack).
|
||||
|
||||
name = "loud-barge"
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_loud"
|
||||
frames = 20
|
||||
|
||||
[[steps]]
|
||||
kind = "await_reply"
|
||||
frames = 0
|
||||
|
||||
[[steps]]
|
||||
kind = "end"
|
||||
31
crates/rutster-sim/scenarios/quiet-advisory.toml
Normal file
31
crates/rutster-sim/scenarios/quiet-advisory.toml
Normal file
@@ -0,0 +1,31 @@
|
||||
# Scenario: quiet-advisory (slice-4½ spec §5.3 entry #2)
|
||||
#
|
||||
# Drives the SECONDARY barge-in path (slice-4 §5.2): the caller speaks
|
||||
# sub-VAD-threshold audio (zeroed PCM, energy=0, well below
|
||||
# VAD_RMS_THRESHOLD=500.0); the local VAD cannot trip; the kill must
|
||||
# come from the brain's slower ASR-VAD advisory path.
|
||||
#
|
||||
# In the slice-4½ sim harness's standalone-wiring mode, the brain side
|
||||
# is a fake-brain tokio task (no real MockRealtimeBrain WS server) —
|
||||
# this scenario exercises the advisory path via the LocalVadReflex's
|
||||
# no-op observation of quiet frames (no trip) + the awaited reply from
|
||||
# the brain task's seed reply. The kill_time + mouth_to_ear metrics
|
||||
# therefore measure the harness's own latencies, not real brain-side
|
||||
# ASR-VAD latency (the latter is deferred to the post-spearhead
|
||||
# refinement tier per spec §1.2 + §8.6 — paired with MockRealtimeBrain
|
||||
# composition + LLM-driven callers).
|
||||
#
|
||||
# Threshold assertion (S7): N=1 only (the secondary-path focus).
|
||||
|
||||
name = "quiet-advisory"
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_quiet"
|
||||
frames = 20
|
||||
|
||||
[[steps]]
|
||||
kind = "await_reply"
|
||||
frames = 0
|
||||
|
||||
[[steps]]
|
||||
kind = "end"
|
||||
49
crates/rutster-sim/scenarios/sustained-call.toml
Normal file
49
crates/rutster-sim/scenarios/sustained-call.toml
Normal file
@@ -0,0 +1,49 @@
|
||||
# Scenario: sustained-call (slice-4½ spec §5.3 entry #3)
|
||||
#
|
||||
# Drives the multi-barge / fatigue / sustained-load check. Three loud
|
||||
# bursts + two quiet interludes + end:
|
||||
# - loud cycle 1 → barge fires → local VAD re-arms on quiet
|
||||
# - quiet cycle 1 → no barge, VAD re-armed for next loud
|
||||
# - loud cycle 2 → second barge fires
|
||||
# - quiet cycle 2 → re-arm
|
||||
# - loud cycle 3 → third barge fires
|
||||
#
|
||||
# Threshold assertion (S7): the per-barge kill_times should drift
|
||||
# ≤ 1.5× across the three bar cycles (anti-fatigue). The second + third
|
||||
# bar's kill time should not be significantly longer than the first's
|
||||
# (no resource exhaustion, no GC pressure, no growing lock contention
|
||||
# across the slice's lifetime).
|
||||
#
|
||||
# See slice-4 §6.1 for the barge_epoch disambiguation mechanism that
|
||||
# makes "fresh re-barge" vs "late confirmation of the bar already in
|
||||
# flight" distinguishable — the Reflex::barge_epoch increments on each
|
||||
# SpeechStarted.
|
||||
|
||||
name = "sustained-call"
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_loud"
|
||||
frames = 10
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_quiet"
|
||||
frames = 10
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_loud"
|
||||
frames = 10
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_quiet"
|
||||
frames = 10
|
||||
|
||||
[[steps]]
|
||||
kind = "speak_loud"
|
||||
frames = 10
|
||||
|
||||
[[steps]]
|
||||
kind = "await_reply"
|
||||
frames = 0
|
||||
|
||||
[[steps]]
|
||||
kind = "end"
|
||||
Reference in New Issue
Block a user