Files
6krrt/tests/test_iteration.py
adlee-was-taken e1fd65f3e4 feat: tier is an iteration budget, spent on evidence rather than on a hunch
A tier used to mean only a capability floor. It now also buys corrective
attempts after a verification failure: tier 1 gets one shot, tier 2 one retry,
tier 3 two. Interactive requests are capped below their tier regardless,
because every retry doubles time-to-answer and in interactive use latency IS a
quality loss.

escalation.preemptive_on_low_confidence now defaults to FALSE. Bumping the
tier because the classifier was unsure of its own call pays frontier prices
before anything has gone wrong. Spending after a check has actually failed is
better on both mandates: the cheap attempt usually succeeds and costs nothing
extra, and when it fails there is evidence rather than a hunch.

Retries are matched to the failure, because the causes differ. Truncation
raises the token budget on the same model -- a different one would run out
too. Malformed output escalates to the next-ranked candidate, since more
tokens will not make unparseable output parse. 'ok' and 'unverifiable' buy
nothing; retrying unverifiable would burn quota across the majority of prose
traffic for no signal at all.

Testing it live exposed that the truncation branch was UNREACHABLE as first
written. Through /v1 the token cap is either the client's, which is not ours
to override, or absent -- and when absent the model hit its own ceiling, so
doubling changes nothing. It now escalates to a candidate with a larger output
ceiling, which is the actionable move, and declines when no such candidate
exists rather than wasting an attempt against a fixed kWh quota.

Known limit, recorded in CLAUDE.md: retry does not reach the streaming path.
Once bytes have gone to the client there is nothing to take back, and
buffering to allow correction would cost streaming itself. opencode streams,
so the main workflow gets verification and feedback but not correction.

Tests 211 -> 227.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xTPER7K8fNyKiuqNvTCTa
2026-08-17 22:25:54 -04:00

4.5 KiB