test(dispatcher): tighten the local_decision backoff tests #107

alee · 2026-10-03T03:48:45Z

alee commented

2026-10-03 03:48:45 +00:00

What

Test-only follow-up to #106. No production code changes: one file, tests/test_classifier_modes_dispatch.py, +50 / -7. It was written during review of the backoff fix and was meant to land inside #106, but #106 merged first.

Why

Review of the local_decision backoff tests found two weak spots.

A skip that re-stamps the clock would have passed. Both circuit tests asserted only _last_classifier_failure > 0 after the skipped second call. A skip that called _record_failure() would hold the circuit open for as long as traffic flows, which is the exact failure the backoff section of CLAUDE.md describes. Both tests now capture the timestamp after the first failure and assert it is unchanged after the second call.
The dispatcher's own confidence_min raise had no test. test_local_decision_low_confidence_does_not_open_circuit actually tripped the parse_logprobs coverage floor (coverage 0.0116 below minimum 0.3), not confidence_min. It is split in two:
- test_local_decision_below_confidence_min_does_not_open_circuit (new): four equal letters give coverage about 0.99 and confidence about 0.25, so the verdict lands below confidence_min after a healthy answer.
- test_local_decision_coverage_floor_does_not_open_circuit: the old test, renamed, with its comment corrected to say what it triggers.

Verification

Each mutation below was applied to a scratch export of the dispatcher, and the intended tests failed:

mutation	tests that fail
a skip records a failure before raising `_ClassifierSkipped`	both circuit tests (unmetered, metered)
the dispatcher's `confidence_min` raise records a failure	`below_confidence_min` only
record on any exception, not just `requests.RequestException`	`below_confidence_min` and `coverage_floor`

tests/test_classifier_modes_dispatch.py: 36 passed.
The cherry-pick applies cleanly onto current main (cc72f20).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

## What Test-only follow-up to #106. No production code changes: one file, `tests/test_classifier_modes_dispatch.py`, +50 / -7. It was written during review of the backoff fix and was meant to land inside #106, but #106 merged first. ## Why Review of the `local_decision` backoff tests found two weak spots. 1. **A skip that re-stamps the clock would have passed.** Both circuit tests asserted only `_last_classifier_failure > 0` after the skipped second call. A skip that called `_record_failure()` would hold the circuit open for as long as traffic flows, which is the exact failure the backoff section of `CLAUDE.md` describes. Both tests now capture the timestamp after the first failure and assert it is **unchanged** after the second call. 2. **The dispatcher's own `confidence_min` raise had no test.** `test_local_decision_low_confidence_does_not_open_circuit` actually tripped the `parse_logprobs` coverage floor (`coverage 0.0116 below minimum 0.3`), not `confidence_min`. It is split in two: - `test_local_decision_below_confidence_min_does_not_open_circuit` (new): four equal letters give coverage about 0.99 and confidence about 0.25, so the verdict lands below `confidence_min` after a healthy answer. - `test_local_decision_coverage_floor_does_not_open_circuit`: the old test, renamed, with its comment corrected to say what it triggers. ## Verification Each mutation below was applied to a scratch export of the dispatcher, and the intended tests failed: | mutation | tests that fail | |---|---| | a skip records a failure before raising `_ClassifierSkipped` | both circuit tests (unmetered, metered) | | the dispatcher's `confidence_min` raise records a failure | `below_confidence_min` only | | record on any exception, not just `requests.RequestException` | `below_confidence_min` and `coverage_floor` | - `tests/test_classifier_modes_dispatch.py`: 36 passed. - The cherry-pick applies cleanly onto current `main` (`cc72f20`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

alee added 1 commit 2026-10-03 03:48:46 +00:00

test(dispatcher): tighten the local_decision backoff tests a6fc80c96a

The review of the backoff fix found two weak spots, both now closed:

- The circuit tests asserted only that _last_classifier_failure was still set
  after the skipped second call. A skip that re-stamped the clock (holding the
  circuit open for as long as traffic flows) would have passed. Both tests now
  assert the timestamp is unchanged.
- test_local_decision_low_confidence_does_not_open_circuit tripped the
  parse_logprobs coverage floor, not confidence_min, so the dispatcher's own
  confidence raise had no test. Split in two: a new test that actually lands
  below confidence_min (four equal letters, coverage 0.99, confidence 0.25),
  and the old one renamed and re-documented as the coverage-floor case.

Mutation-checked in a scratch export: a skip that records, a confidence raise
that records, and recording on any exception each fail the intended tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

alee merged commit 1ca9ad1ba6 into main

2026-10-03 05:42:16 +00:00

alee referenced this issue from a commit

2026-10-03 05:42:17 +00:00

Merge pull request 'test(dispatcher): tighten the local_decision backoff tests' (#107) from test/local-decision-tighten-backoff-tests into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: alee/6krrt#107