The critical-path foundation for the benchmark + simulation harness. Scenario is a TOML-deserializable scripted-caller data type; ScenarioStep covers speak_loud / speak_quiet / pause / await_reply / end. Determinism is the point -- reproducible thresholds in CI (ADR-0010). All other sim modules land as stubs here + fill in across S2-S7. Threshold consts (BARGE_IN_KILL_TIME_P99_MS = 80.0, MOUTH_TO_EAR_P99_MS = 700.0, TICK_LAG_MAX_MS = 10.0, TICK_OVERRUN_PCT_MAX = 1.0, SWEEP_CONCURRENCIES = [1,10,50]) land now per the plan's S1 step 2 note (used by S5/S6/S7 wiring). Adds toml = 0.8 to workspace.dependencies (the first consumer; spec §1.1 claim of pre-existing membership was inaccurate). Task S1 of slice-4½ -- everything else depends on this landing. Signed-off-by: Aaron D. Lee <himself@adlee.work>
251 lines
9.3 KiB
Rust
251 lines
9.3 KiB
Rust
//! # Scenario — the scripted-caller data type
|
|
//!
|
|
//! See `docs/superpowers/specs/2026-07-05-slice-4-half-benchmark-sim-design.md`
|
|
//! §3.1.
|
|
//!
|
|
//! A `Scenario` is a sequence of `ScenarioStep`s read from a TOML file under
|
|
//! `crates/rutster-sim/scenarios/*.toml`. Deterministic by construction —
|
|
//! the entire point is reproducible thresholds in CI (LLM-driven callers
|
|
//! land in a post-spearhead refinement tier; see §1.2).
|
|
//!
|
|
//! # Why TOML (not YAML, not RON)
|
|
//!
|
|
//! `serde` + `toml` is already a workspace member. TOML keeps the scenario
|
|
//! file readable as a one-shot script (a sequence of named steps + numbers);
|
|
//! YAML would invite flow-mapping complexity this format doesn't need.
|
|
//!
|
|
//! # Why `#[serde(tag = "kind")]` (internally-tagged enum)
|
|
//!
|
|
//! Each step in the scenario TOML is one TOML table:
|
|
//!
|
|
//! ```toml
|
|
//! [[steps]]
|
|
//! kind = "speak_loud"
|
|
//! frames = 20
|
|
//! ```
|
|
//!
|
|
//! `serde`'s internally-tagged enum representation (`tag = "kind"`) reads the
|
|
//! `kind` key to dispatch to the matching enum variant. This is the idiomatic
|
|
//! shape for "list of named, differently-shaped records" in TOML — the
|
|
//! alternative (externally-tagged) would require a redundant table layer
|
|
//! (`[[steps]] variant = { speak_loud = { frames = 20 } }`) that hurts
|
|
//! readability for no benefit. See <https://serde.rs enum-representations>.
|
|
//!
|
|
//! `rename_all = "snake_case"` maps the Rust `SpeakLoud` variant to the
|
|
//! TOML `speak_loud` tag — matches the convention used in slice-4's
|
|
//! `AdvisoryEvent` enum (the precedent this file follows).
|
|
|
|
use std::path::Path;
|
|
|
|
/// The scripted-caller scenario. Read from a TOML file. Deterministic.
|
|
///
|
|
/// # Example
|
|
///
|
|
/// ```toml
|
|
/// name = "loud-barge"
|
|
///
|
|
/// [[steps]]
|
|
/// kind = "speak_loud"
|
|
/// frames = 20
|
|
///
|
|
/// [[steps]]
|
|
/// kind = "await_reply"
|
|
/// frames = 0
|
|
///
|
|
/// [[steps]]
|
|
/// kind = "end"
|
|
/// ```
|
|
///
|
|
/// The `SimAudioPipe::new(scenario, ..)` constructor consumes the
|
|
/// `steps` vector front-to-back during `on_pcm_frame` (the sink path —
|
|
/// the caller "speaks") and `next_pcm_frame` (the source path — the
|
|
/// caller "hears" brain replies, advancing `AwaitReply` steps).
|
|
#[derive(Debug, Clone, serde::Deserialize, PartialEq, Eq)]
|
|
pub struct Scenario {
|
|
/// Human-readable identifier; surfaces in CI failure messages
|
|
/// ("scenario loud-barge failed: p99 kill-time 84ms > 80ms").
|
|
pub name: String,
|
|
/// Time-ordered sequence of caller actions. The `SimAudioPipe`
|
|
/// consumes them in order during `on_pcm_frame` (for speak/pause
|
|
/// steps) and `next_pcm_frame` (for `AwaitReply` barriers).
|
|
pub steps: Vec<ScenarioStep>,
|
|
}
|
|
|
|
/// One axis of caller behavior. A scenario is a time-ordered sequence
|
|
/// of these. The `SimAudioPipe` consumes them in order during
|
|
/// `on_pcm_frame`.
|
|
///
|
|
/// # Why an enum (not a struct with a `kind` field)
|
|
///
|
|
/// The steps have *different fields* (`SpeakLoud { frames }` vs `End`
|
|
/// has none). A struct-with-kind-field would require `Option<u32>` for
|
|
/// every variant-irrelevant field — losing type safety for no ergonomic
|
|
/// gain. The enum approach makes the variant's payload explicit at the
|
|
/// type level; `serde`'s internally-tagged representation keeps the TOML
|
|
/// shape flat + readable.
|
|
#[derive(Debug, Clone, serde::Deserialize, PartialEq, Eq)]
|
|
#[serde(tag = "kind", rename_all = "snake_case")]
|
|
pub enum ScenarioStep {
|
|
/// Send N loud PCM frames (sample value 1000, well above
|
|
/// `VAD_RMS_THRESHOLD = 500.0`). Triggers the local VAD via
|
|
/// slice-4's `LocalVadReflex::on_pcm_frame` — the primary barge-in
|
|
/// path (slice-4 §5.1). The `SimAudioPipe`'s sink path emits one
|
|
/// loud `PcmFrame` per `on_pcm_frame` call while this step is
|
|
/// active; on step entry it captures `Capture::CallerLoudOnset`.
|
|
SpeakLoud { frames: u32 },
|
|
/// Send N zero frames (sample value 0, well below
|
|
/// `VAD_RMS_THRESHOLD`). Drives the mock-brain advisory path
|
|
/// (slice-4 §5.2 secondary path): `MockRealtimeBrain` sees
|
|
/// "no caller audio for M frames" + emits an advisory →
|
|
/// `Reflex::muted = true`. The wedge cares about LOUD barge
|
|
/// measurement; quiet onsets are unscored (no `Capture`).
|
|
SpeakQuiet { frames: u32 },
|
|
/// Insert N zero frames before the next step (silence pacing).
|
|
/// Used by the `sustained-call.toml` scenario (5 minutes of talk
|
|
/// with 3 barges) to space barge cycles apart.
|
|
Pause { frames: u32 },
|
|
/// Wait until the harness receives M "ear" frames before advancing.
|
|
/// Barrier semantics: brain's reply must arrive before the next
|
|
/// caller action. The `SimAudioPipe`'s source path
|
|
/// (`next_pcm_frame`) decrements this counter for each `Some(frame)`
|
|
/// returned; on reaching zero, advances.
|
|
AwaitReply { frames: u32 },
|
|
/// End the scenario. The `SimAudioPipe`'s `next_pcm_frame` returns
|
|
/// `None` thereafter; the `SimCall` (S4) detects end-of-scenario +
|
|
/// terminates its tick loop.
|
|
End,
|
|
}
|
|
|
|
/// Errors surfaced during scenario loading. Cold-path; OK to be
|
|
/// `thiserror`-derived (the hot path goes through
|
|
/// `SimAudioPipe::on_pcm_frame` which never reads files).
|
|
///
|
|
/// `#[from]` on the variants auto-implements `From<io::Error>` and
|
|
/// `From<toml::de::Error>` so `?`-propagation Just Works in
|
|
/// `Scenario::load`.
|
|
#[derive(Debug, thiserror::Error)]
|
|
pub enum ScenarioError {
|
|
#[error("scenario file read failed: {0}")]
|
|
Io(#[from] std::io::Error),
|
|
#[error("scenario TOML parse failed: {0}")]
|
|
Parse(#[from] toml::de::Error),
|
|
}
|
|
|
|
impl Scenario {
|
|
/// Load a scenario from a TOML file. Cold-path.
|
|
///
|
|
/// Wraps `std::fs::read_to_string` + `toml::from_str` behind the
|
|
/// `ScenarioError` enum so callers can `?`-propagate both failure
|
|
/// modes uniformly. The `path: impl AsRef<Path>` bound follows
|
|
/// the std-library convention: it accepts `&str`, `String`,
|
|
/// `PathBuf`, `&Path` — matching how scenarios are loaded from
|
|
/// CLI args or test fixtures.
|
|
pub fn load(path: impl AsRef<Path>) -> Result<Self, ScenarioError> {
|
|
let raw = std::fs::read_to_string(path)?;
|
|
Self::from_toml(&raw)
|
|
}
|
|
|
|
/// Parse a scenario from an in-memory TOML string.
|
|
///
|
|
/// Split out from `load` so unit tests can construct scenarios
|
|
/// without touching the filesystem (filesystem-isolated unit
|
|
/// tests are the std pattern in this codebase — see slice-4's
|
|
/// `reflex.rs` tests).
|
|
pub fn from_toml(s: &str) -> Result<Self, ScenarioError> {
|
|
Ok(toml::from_str(s)?)
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn scenario_parses_minimal_end_only() {
|
|
// The trivial scenario: just one `End` step. Exercises the
|
|
// internally-tagged enum's bare-variant shape (`kind = "end"`
|
|
// with no payload fields).
|
|
let toml = r#"
|
|
name = "trivial"
|
|
[[steps]]
|
|
kind = "end"
|
|
"#;
|
|
let s = Scenario::from_toml(toml).expect("parse");
|
|
assert_eq!(s.name, "trivial");
|
|
assert_eq!(s.steps, vec![ScenarioStep::End]);
|
|
}
|
|
|
|
#[test]
|
|
fn scenario_parses_loud_barge_shape() {
|
|
// The canonical loud-barge scenario from spec §5.3. Verifies
|
|
// the three-step shape (speak_loud → await_reply → end) parses
|
|
// to the expected variant sequence with payload fields intact.
|
|
let toml = r#"
|
|
name = "loud-barge"
|
|
[[steps]]
|
|
kind = "speak_loud"
|
|
frames = 20
|
|
[[steps]]
|
|
kind = "await_reply"
|
|
frames = 0
|
|
[[steps]]
|
|
kind = "end"
|
|
"#;
|
|
let s = Scenario::from_toml(toml).expect("parse");
|
|
assert_eq!(s.name, "loud-barge");
|
|
assert_eq!(
|
|
s.steps,
|
|
vec![
|
|
ScenarioStep::SpeakLoud { frames: 20 },
|
|
ScenarioStep::AwaitReply { frames: 0 },
|
|
ScenarioStep::End,
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn scenario_parses_sustained_call_shape() {
|
|
// The sustained-call scenario (spec §5.3 entry #3) alternates
|
|
// speak_loud + speak_quiet. Verifies both payload-bearing
|
|
// variants parse correctly in sequence.
|
|
let toml = r#"
|
|
name = "sustained"
|
|
[[steps]]
|
|
kind = "speak_loud"
|
|
frames = 10
|
|
[[steps]]
|
|
kind = "speak_quiet"
|
|
frames = 10
|
|
[[steps]]
|
|
kind = "speak_loud"
|
|
frames = 10
|
|
[[steps]]
|
|
kind = "end"
|
|
"#;
|
|
let s = Scenario::from_toml(toml).expect("parse");
|
|
assert_eq!(s.steps.len(), 4);
|
|
assert!(matches!(s.steps[0], ScenarioStep::SpeakLoud { frames: 10 }));
|
|
assert!(matches!(
|
|
s.steps[1],
|
|
ScenarioStep::SpeakQuiet { frames: 10 }
|
|
));
|
|
assert!(matches!(s.steps[2], ScenarioStep::SpeakLoud { frames: 10 }));
|
|
assert!(matches!(s.steps[3], ScenarioStep::End));
|
|
}
|
|
|
|
#[test]
|
|
fn scenario_unknown_kind_errors() {
|
|
// An unknown `kind` tag (typo, future-extension tag, etc.)
|
|
// must surface as a `Parse` error rather than silently
|
|
// defaulting. This is the contract `serde`'s internally-tagged
|
|
// enum provides: unknown tags fail the deserialize rather
|
|
// than producing a `None`-ish default.
|
|
let toml = r#"
|
|
name = "bad"
|
|
[[steps]]
|
|
kind = "ship_a_real_caller"
|
|
"#;
|
|
assert!(Scenario::from_toml(toml).is_err());
|
|
}
|
|
}
|