Files
rutster/docs/reviews/2026-06-28-vision-sanity-check.md
Aaron D. Lee 60b65ab893
Some checks failed
CI / fmt (push) Successful in 1m16s
CI / clippy (push) Successful in 2m19s
CI / test (stable) (push) Has been cancelled
CI / deny (push) Has been cancelled
CI / test (1.85) (push) Has started running
docs(reviews): salvage June pre-pivot reviews from retired branches
These four reviews were filed on slice-1-review-fixes and
strategic-reviews-post-pivot-rescore but never reached main; both
branches are being retired in the post-slice-4 cleanup. Kept for
lineage: the 06-29 post-pivot re-score is where ADR-0007/0008 era
strategy was re-scored, and the 07-03 reviews build on these.

Sources: slice-1-review-fixes @ d2ef53b (gtm-path, vision-sanity-check,
slice-1-claude-adversarial-assessment), strategic-reviews-post-pivot-
rescore @ ea27167 (post-pivot re-score).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012QndwfhjyTiZcUYp87dwW8
Signed-off-by: Aaron D. Lee <himself@adlee.work>
2026-07-04 11:32:29 -04:00

142 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Claude Adversarial Sanity-Check — Rutster Vision & Strategy
- **Reviewer:** Claude (Opus 4.8), peer-level read of the strategy docs (not the code — see the companion `2026-06-28-slice-1-claude-adversarial-assessment.md` for the code).
- **Targets:** `docs/superpowers/specs/2026-06-26-vision-revision-design.md`, `docs/ARCHITECTURE.md`, ADRs 00020006, as of `main@22d3f03`.
- **Question asked:** "Sanity-check the *idea*." This is a strategy critique, not an architecture-correctness review.
## How to use this document (for the GLM-5.2 reviewers)
Same contract as the code assessment: **these are claims to refute, not conclusions to accept.** Strategy has no unit tests, so each critique below carries three fields instead of a falsification test:
- **Steelman (the doc's defense):** the strongest version of the current position. Try to make it win.
- **What would change the verdict:** the evidence or decision that flips the critique.
- **Decision implied:** the concrete thing to change *if* the critique holds — so this stays actionable, not philosophical.
The author already pressure-tested this vision once (the 2026-06-26 revision reversed ADR-0001). Hold it to that standard: a critique you can dissolve should be struck with reasoning recorded.
---
## Verdict
**Stronger than "decent."** This is an unusually coherent strategy precisely because it already cannibalized its own founding premises — reversing the Kamailio/rtpengine C shield instead of rationalizing it is the tell that the thinking is honest. The **identity, architecture, and security posture are sound.** The vulnerability is **not the bet** — it's **sequencing and emphasis**: the pitch leads with the one leg a funded competitor can contest, and the moat-grade capabilities sit on the far side of a long solo valley. Both are fixable by reordering, not by changing the thesis.
Confidence: high on the architecture/security being sound; medium on the strategic critiques (they're judgment calls about a market in motion, which is exactly why they're written to be argued with).
---
## TL;DR — strategic findings
| # | Type | One-line |
|---|------|----------|
| S1 | **Emphasis** | The technical wedge leads with no-GC determinism — the weakest, most benchmarkable leg — on ground LiveKit Agents will contest directly |
| S2 | **Sequencing (dominant risk)** | The spearhead (steps 16) lands at Vapi/Retell parity; every *differentiating* capability is past it, deep in the capability ladder — a long solo valley before the moat exists |
| S3 | **Positioning** | The wedge is "strongest for regulated" but regulated buyers are the most allergic to bus-factor-1 infra; first adopters are the builder persona, not the compliance prize |
| S4 | **Integration risk** | Core-authoritative VAD vs. the brain's own server-side turn detection (OpenAI Realtime) — "AudioOut advisory" meets a vendor with opinions |
| S5 | **Scope/timing** | Solo attack on a 1.18M-LOC-category problem against a window that may be closing; the long pole is integration/hardening, which AI assistance collapses least |
| S6 | **Reframe** | The self-improvement loop (own the call → auto-labeled takeover data) is the durable moat and is buried under determinism in the wedge framing |
**Net:** S2 is the one that matters. S1 and S6 are the same fix from two directions (move emphasis off determinism, onto the boundary + data loop). S3S5 are clear-eyes caveats to record, not blockers.
---
## S1 — The technical wedge leads with its weakest leg
**Type: Emphasis. Confidence: medium-high.**
§3 frames the technical wedge as **(1) no-GC real-time determinism, (2) one secure auditable boundary, (3) operational simplicity**, and calls determinism *"the one thing a Go SFU + external agent structurally can't match on quality."* That ordering puts the most contestable claim first:
- The dominant latency is the **brain round-trip** (hundreds of ms to a speech-to-speech API) — which rutster does not own. No-GC buys **tail smoothness** in the local loop, real but second-order to the RTT.
- The latency-critical reflex is **local VAD barge-in** — and competitors can also run local VAD. Barge-in *quality* in a no-GC loop is an edge, but it's a number a competitor can benchmark against, not a category they're locked out of.
- **LiveKit Agents** is the specific threat. The doc characterizes LiveKit as "horizontal media infra, no contact-center domain" — accurate for LiveKit *core* — but LiveKit is actively building the agent/turn-taking layer. Leading on determinism points the pitch at exactly the ground they're moving onto.
The **structural** legs — single auditable boundary (#2) and data-ownership — can't be benchmarked away. Those should lead.
- **Steelman:** for tight turn-taking the floor latency *does* matter, and a no-GC loop genuinely gives more predictable barge-in under load; "structurally can't match" is about the **whole** (self-host + audit + reflex), not the reflex alone — §3's honest caveat already says this.
- **What would change the verdict:** evidence that no-GC barge-in is a *primary* buying criterion (not a refinement) for the target persona — e.g., a head-to-head where GC jitter visibly degrades a competitor's turn-taking in normal use, not a synthetic stress test.
- **Decision implied:** reorder §3 — lead the technical wedge with the auditable boundary + data-ownership; keep determinism as the third support, framed as "quality refinement," not "the thing they can't match."
## S2 — The differentiation valley (dominant strategic risk)
**Type: Sequencing. Confidence: high that the valley exists; medium on its severity.**
Walk the spearhead (§10) to its end: *"I called my Rust box, an AI answered, it barged in, and it can't blow my budget."* Motivating and real — but at that milestone the product is **at Vapi/Retell feature-parity while behind them on convenience** (they're managed; this is DIY self-host). Everything *categorically* different is **past** the spearhead, in the capability ladder (§11): human escalation (rung 2), containment measurement (rung 3), the self-improvement loop (rung 4), plus the actual contact-center domain library (ACD/queues/CDR/supervisor).
So a solo builder spends enormous effort to reach "another AI voice bot you happen to host yourself" **before** reaching anything that is a moat. Motivation, validation, and early adoption all have to survive that crossing.
- **Steelman:** the spearhead's job is to prove the *combination is technically real*, not to be differentiated — and "I called my Rust box and an AI answered" is the momentum fuel a solo build needs. Differentiation can follow once the core is proven.
- **What would change the verdict:** if "self-hostable AI voice bot" is *itself* enough of a draw for the §2 persona (data-ownership alone sells it), the valley is shallow and this de-rates to a caveat.
- **Decision implied:** pull **rung-2 human takeover** forward to sit adjacent to the spearhead. "AI answers, and the instant it flounders a human seamlessly barges in via the audiohook" is the earliest demo that is **not** Vapi-parity and proves the contact-center thesis — and §11 already notes rung 2 reuses the audiohook primitive PORT_PLAN kept. This reorders the proof so differentiation appears before momentum runs out. (Tradeoff: it widens the spearhead — weigh against "brutally thin.")
## S3 — The compliance paradox
**Type: Positioning. Confidence: medium-high.**
§3 says the wedge is "strongest for regulated/high-trust (PCI/HIPAA/TCPA)." True for *fit* — but regulated buyers are the **most** risk-averse about brand-new, single-maintainer infra in the call path. Self-host cuts both ways: it removes "my data on a vendor's cloud" (good) and adds "we now operate unproven Rust telephony ourselves with bus-factor 1" (bad, for exactly a compliance-conscious org). The realistic first adopters are the **§2 builders** (doing it for a less-regulated client, or internal/experimental use), who *reach* the compliance segment later — the same arc Asterisk walked.
- **Steelman:** self-hosting + auditability + data-ownership are *literally the buying criteria* in regulated spaces, and a builder-led bottom-up motion is how open infra always enters the enterprise — this is a feature of the path, not a bug.
- **What would change the verdict:** a regulated design partner willing to route real (or even shadow) traffic early — that would prove the compliance segment is reachable before the project is "proven," collapsing the paradox.
- **Decision implied:** state explicitly that **regulated is the eventual prize reached via the builder persona, not the beachhead.** Keep the §2 builder as the actual day-1 customer in all positioning, so the messaging doesn't chase a buyer who won't move first.
## S4 — Turn-taking ownership at the tap
**Type: Integration risk. Confidence: medium (depends on brain vendor).**
ARCHITECTURE.md is right that the core should be authoritative on VAD/barge-in ("the tap carries the *results* of reflexes, not the responsibility"; "`AudioOut` advisory / core-authoritative"). But the most likely first brain — **OpenAI Realtime** — does **its own** server-side VAD and turn detection by default. Integrating step 34 means either:
- disable the brain's turn detection and feed it clean, locally-detected turns (preserves the reflex-authoritative principle, but is *more* integration work and fights the API's grain), or
- accept split-brain turn-taking where local VAD and the brain's VAD can disagree (double-triggers, dropped barge-ins).
This is precisely where the elegant "core disposes" principle meets a vendor with opinions, and it's a *design* decision, not an implementation detail.
- **Steelman:** the tap is deliberately core-as-client and format-canonical specifically so the core can drive turns and treat the brain as a dumb speech-to-speech transducer — Realtime supports disabling server VAD, so the clean path is available.
- **What would change the verdict:** a step-3 prototype showing Realtime-with-server-VAD-disabled gives acceptable turn quality fed by local VAD — then it's just integration, not a tension.
- **Decision implied:** make "who owns turn detection" an explicit decision in the step-3 (brain) design doc, defaulting to **core-authoritative, brain VAD off**, with the integration cost budgeted. Don't let it be discovered at wiring time.
## S5 — Solo scope vs. category timing
**Type: Scope/timing. Confidence: medium.**
The thin-slice sequencing and AI-pair leverage are the right mitigations and they're real. The honest framing: this is a **1.18M-LOC-category** problem attacked solo, and the long pole isn't writing code (AI helps most there) — it's the **integration/hardening surface** of real-time telephony: NAT traversal, carrier signaling quirks (re-INVITE for hold/transfer, DTMF transport negotiation, PRACK/early media, IP-auth vs registration trunks, SRTP keying on the trunk side), codec edge cases, behavior under load, and security-hardening every wire parser. AI assistance collapses *that* the least. Meanwhile the "AI-era contact center is white space" window is open **now** but contested (incumbents bolting on AI; Vapi/Retell maturing and funded).
- **Steelman:** "engine not product" bounds the scope hard, the capability ladder is genuinely shippable rung-by-rung, and a focused solo builder with AI leverage in 2026 is materially faster than the team-years Asterisk took — the comparison to 1.18M LOC is unfair because most of that is breadth rutster deliberately rejects.
- **What would change the verdict:** sustained shipping velocity through steps 15 (the trunk client especially) on the planned timeline — evidence the hardening pole is shorter than feared.
- **Decision implied:** ruthlessly sequence toward the **differentiated** demo (see S2), not breadth — every month spent on capability that overlaps Vapi is a month the window can close. Treat the SIP-trunk hardening (ADR-0003) as the schedule risk it is; WebRTC-first ordering already hedges it.
## S6 — Reframe: the self-improvement loop is the actual moat
**Type: Reframe. Confidence: high that it's the most durable advantage.**
The capability-ladder rung 4 — **own the whole call → every human takeover auto-labeled "AI failed here → human did this" → containment analytics + eval sets + fine-tune export, all on the operator's own infra** — is the one advantage that is:
1. **structurally impossible** for cloud bots (they don't own your data) and CCaaS (not their business model),
2. a **compounding data-network-effect** (better the more you use it), and
3. **uniquely enabled** by self-hosting + owning the entire call.
That is a durable moat in a way no-GC determinism is not. The north-star one-liner already names it ("a closed ML loop that learns from every escalation") — but the §3 *technical-wedge* framing buries it under determinism. S1 and S6 are the same correction from opposite ends: **the wedge is the auditable boundary + the data loop; determinism and simplicity are supporting cast.**
- **Steelman:** the loop is rung 4 — years out — so leading with it oversells what exists; determinism is what the *spearhead* actually demonstrates, so it's honest to lead with it now.
- **What would change the verdict:** nothing about *whether* the loop is the moat; the open question is *timing of the claim* — see decision.
- **Decision implied:** separate "what the spearhead proves" (the combination works) from "what the wedge is" (boundary + data-ownership + loop). Lead strategy/positioning with the latter even while the spearhead demonstrates the former. Don't let the demo's emphasis become the strategy's emphasis.
---
## Architecture decisions — spot-check (mostly sound; recorded for completeness)
These are not strategy critiques; they're the architecture calls the strategy rests on, and they hold up:
- **Remove the control↔media gRPC hot-path hop / fused per-call vertical** (ADR-0002) — correct; that hop was always a latency + failure-mode liability.
- **Spend gate co-located with trunk termination** — genuinely strong and *under-sold*; "a runaway brain can't exceed spend because it doesn't hold the wire" is a demonstrable structural property a 3-vendor stack cannot match. Promote it in the pitch.
- **Core-as-client tap, no inbound port; core-authoritative playout** — correct security instinct; deletes an attack class.
- **WebRTC-first ingress** (ADR-0006) — smart de-risking; first-call never blocks on SIP.
- **Rust-native trunk SIP, no C shield** (ADR-0003) — right call for the scoped use case (interop tail collapses to a few cooperative carriers; parser-security thesis becomes literally true). Mild caveat: even cooperative carriers carry a provider-independent signaling-state-machine complexity that "a few documented providers" slightly undersells — the cost is accepted, just don't let the framing make it sound small.
- **Valkey event bus, 20 ms loop never on the bus, bus not source-of-truth for billing-critical state** (ADR-0005) — correct discipline.
---
## Verification checklist for the GLM-5.2 reviewers
1. **S2 is the load-bearing critique.** Decide whether "self-hostable AI voice bot" is independently compelling to the §2 persona. If yes, S2 de-rates and the spearhead stands as-is. If no, pull rung-2 takeover forward. Everything else is secondary to this call.
2. **S1 + S6 are one decision:** move the wedge's emphasis off determinism onto boundary + data loop — or refute that determinism is benchmarkable/contestable.
3. **S4 belongs in the step-3 design doc**, not here — make sure it lands there as an explicit turn-ownership decision.
4. Push back on **S3/S5** with evidence (a regulated design partner; demonstrated velocity) — those are the critiques most likely to dissolve with a single data point.
5. Strike anything you can dissolve, with reasoning recorded — same standard the vision held itself to when it reversed ADR-0001.