f0d9bfb81c
Both verification paths had mirror halves of the same blind spot, and real traffic is what found it. On the first genuine agent session through this router -- 63 completions that shipped a working feature with 349 passing tests, clean mypy and clean ruff -- the structural checker recorded "malformed: empty response" 29 times and the local LLM checker called "cuts off mid-sentence" on 8 of the 9 answers it graded. Both were describing the same thing from opposite sides: a turn that ends by calling a tool. Its text content is empty, or a half-sentence before the call, and both are correct behaviour rather than a defect. verify_response and worth_local_check now take has_tool_calls, supplied on the non-streaming path from message.tool_calls and accumulated on the streaming path from delta.tool_calls. In verify_response the check outranks even finish_reason == 'length', because stopping mid-sentence at a call boundary is a call boundary, not a budget overrun. worth_local_check declines outright, which also stops paying ~6s of local inference to mis-grade a tool call. Had feedback.py run against those rows it would have applied ~12 false failures to the two models that had just done the work. That is the FOURTH harness bug in this project that would have scored the rig rather than the model, and the first one caught by real traffic instead of a synthetic test. The pre-fix rows are kept with model_attributable = 0 so the record survives without steering routing. client_capped was over-applied in the same area. It marked EVERY verdict non-attributable whenever the client set max_tokens, and opencode always sets it, so genuine failures were invisible to feedback for the entire main workflow. A client's token cap explains a truncated verdict and nothing else; a model emitting unparseable code owes nothing to the client's budget. It is now scoped to exactly that verdict. Also recorded: outcome attribution resolves the session directory by path histogram, and on this session that picked .venv/.../site-packages/c2pa 24 times over ~/Sources/fieldwitness 22, because reading a dependency's source outweighed editing the project. It degrades safely -- 31 reports accepted, 3 refused as ambiguous rather than misattributed -- but the heuristic needs to weight writes over reads. Tests 243 -> 249. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WSkcSD2Jzkxo1Kw27ecfXJ
13 KiB
13 KiB