2afdb40e2b
A fallback-classified request is recorded with task_category=general_chat. general_chat is a FULLY SCORED category -- 13 proficiency rows on the live DB, kimi-k2.7-code-fast at 0.95 -- and it is also the configured fallback_category. So without a guard, POST /outcome reports on mislabelled outage traffic fold into proficiency(model, general_chat) through feedback.py and drag real scores toward whatever traffic happened to be flowing while the classifier was down. That pollutes the one table the router treats as ground truth. report_outcome now resolves the decision's classification_source and passes model_attributable accordingly. feedback.py is NOT touched: its existing `AND model_attributable = 1` filter already implements keep-the-record, don't-steer-routing -- the same treatment the tool-call false-failures got. Attributable: classifier, cached, override, classifier_cloud -- the category was measured on this request, or declared by a client who knows. Not attributable: fallback (a guess), session_stale and session_history (a real classification of a DIFFERENT request, already past the staleness bound the design itself set). The asymmetry is deliberate. A degraded source is good enough to route one visibly-flagged request; a proficiency score is consulted by every future request, so attribution must not trust more than routing does. FAILS OPEN on an unknown decision row. Withholding ground truth requires positive knowledge the category was a guess, and an over-applied exclusion already starved feedback once (client_capped). The test pinning that passes on main by design -- it guards a future over-correction, not a fixed defect. Also adds a drift test: the exclusion set must equal the Classification source Literal minus the trusted four, so a new degraded source cannot be added without deciding its attribution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRQXz5SYZYVWscxS1QqF6U
8.5 KiB
8.5 KiB