92031dbd0b
A declined answer left one number behind and it was in a log line, so confidence_min could not be tuned from data. route_decisions.confidence cannot say it either: the chat path re-routes through the override branch, which hard-codes 1.0, and 43,804 of 43,856 live rows hold exactly that. Measured on 2026-10-04 under local_decision (qwen3.5:4b): 769 of 2,557 turns (30%) had no fresh classification, against 0% for local_llm and local_encoder. The cause was recorded in only 4 of them. - classifier_confidence, classifier_coverage and classifier_reject on route_decisions, filled from a ClassifierAttempt carried on Classification. Both accepted and declined answers carry one, so the two distributions can be compared around the floor. Reason codes are listed in docs/data-model.md. - ClassifierRejected (a RuntimeError subclass, messages unchanged) replaces the plain RuntimeErrors at the six floor-miss sites and the two local_decision refusals, so the number and reason travel out of the raise site. - The chat path captures the classifier's verdict before the re-route and passes it to persist_route_decision (attempt_of), like it already does for source. - Admin decisions page: the source badge's tooltip shows the attempt, and session_history / session_stale get an amber badge instead of neutral grey. TUI detail popup and the live event carry the same three keys. - The /metrics degraded-share warning lists the recorded reasons instead of claiming the classifier "has been failing", which was wrong for a classifier that answers and is declined. - degraded_warn_threshold must be in (0, 1] and degraded_warn_min at least 1, refused at load: a value above 1 could never fire. The three columns arrive by ALTER and are NULL on every earlier row; metrics selects them only when present, so the live DB reads NULL until its restart. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa
60 lines
2.3 KiB
Python
60 lines
2.3 KiB
Python
"""Why the primary classifier's answer was not used, as data.
|
|
|
|
A classifier that declines to answer is the router's most consequential quiet
|
|
failure: the request is then routed on a borrowed or guessed label. Before this
|
|
module the reason existed only as a log line, and the number that decided it
|
|
(how far below ``confidence_min`` the answer was) existed nowhere durable, so a
|
|
threshold could not be tuned from the database. ``ClassifierRejected`` carries
|
|
both out of the raise site, and the ``REASON_*`` strings are the stable codes
|
|
stored in ``route_decisions.classifier_reject``.
|
|
|
|
Stdlib only, and it imports nothing from this package, so ``local_decision`` can
|
|
raise it without importing ``dispatcher`` (which imports ``local_decision``).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Final
|
|
|
|
# The model answered, and the dispatcher declined the answer.
|
|
REASON_BELOW_CONFIDENCE: Final = "below_confidence_min"
|
|
REASON_BELOW_COVERAGE: Final = "below_coverage_min"
|
|
REASON_NO_LOGPROBS: Final = "no_logprobs"
|
|
|
|
# The model did not answer usefully.
|
|
REASON_TIMEOUT: Final = "timeout"
|
|
REASON_TRANSPORT: Final = "transport_error"
|
|
REASON_PARSE: Final = "parse_error"
|
|
REASON_PRIMARY_FAILED: Final = "primary_failed"
|
|
|
|
# The model was never asked. Followed by the skip reason, e.g.
|
|
# ``skipped_gaming_mode`` or ``skipped_backoff``.
|
|
REASON_SKIPPED_PREFIX: Final = "skipped_"
|
|
|
|
|
|
class ClassifierRejected(RuntimeError):
|
|
"""The classifier answered, but the answer was refused.
|
|
|
|
A ``RuntimeError`` subclass on purpose: ``classify()`` already treats a
|
|
``RuntimeError`` from the primary as "the primary failed" and walks the
|
|
cascade, so existing handlers keep working and only the data is new. The
|
|
message text is unchanged from the plain ``RuntimeError`` it replaces,
|
|
because it is what the ``fallback`` log line prints.
|
|
|
|
``confidence`` and ``coverage`` are whatever the raise site knew, and None
|
|
where it did not: a coverage rejection happens before a confidence exists.
|
|
"""
|
|
|
|
def __init__(
|
|
self,
|
|
reason: str,
|
|
message: str,
|
|
*,
|
|
confidence: float | None = None,
|
|
coverage: float | None = None,
|
|
) -> None:
|
|
super().__init__(message)
|
|
self.reason = reason
|
|
self.confidence = confidence
|
|
self.coverage = coverage
|