Closes the ground-truth loop. deploy/opencode-plugin/router-outcome.js hooks
tool.execute.after, watches for test and build commands, and reports pass/fail
to /outcome. opencode already runs your tests; this is what makes the result
reach routing.
Command detection is deliberately narrow -- pytest, npm test, cargo, go, ruff,
mypy, tsc and friends. A failing `ls` says nothing about model quality, and a
false signal is worse than none because it trains the router on noise. Verdict
comes from exit status plus text signatures for tools that exit 0 while
reporting failures, with "0 failed" and "no errors" guarded against. The router
being unreachable never breaks a session.
Attribution is the hard part, and two assumptions failed under test.
The first fingerprint design keyed on the system prompt. One real opencode run
produced TWO distinct keys, because it runs several agents with different
prompts -- so that fingerprint identifies AGENTS, not sessions, and would have
refused every single run forever. A permanent false positive dressed as
safety.
The second assumption was that opencode states its project root up front.
Capturing a real request showed it does not. Directory is now derived from the
file paths an agent touches across the whole conversation, counting every
ancestor so the shared project root wins over any one subdirectory, and
stripping trailing filenames so a file is never mistaken for a directory. A
single mention is not enough; corroboration is required.
When a report cannot be matched by directory and more than one conversation
was active in the window, /outcome answers 409 and records nothing. Refusing
beats guessing: a misattributed failure penalizes a model for work it never
did, and this project has already recorded false failures twice from harness
bugs that took measurement to catch.
The window is 120 seconds, not 30 minutes. At 30 it swept in traffic from
earlier in the same work session and refused a legitimate report -- observed
directly, not theorised.
Tests 232 -> 243.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xTPER7K8fNyKiuqNvTCTa
6e729ad670