adlee-was-taken a6fc80c96a test(dispatcher): tighten the local_decision backoff tests ...

The review of the backoff fix found two weak spots, both now closed:

- The circuit tests asserted only that _last_classifier_failure was still set
  after the skipped second call. A skip that re-stamped the clock (holding the
  circuit open for as long as traffic flows) would have passed. Both tests now
  assert the timestamp is unchanged.
- test_local_decision_low_confidence_does_not_open_circuit tripped the
  parse_logprobs coverage floor, not confidence_min, so the dispatcher's own
  confidence raise had no test. Split in two: a new test that actually lands
  below confidence_min (four equal letters, coverage 0.99, confidence 0.25),
  and the old one renamed and re-documented as the coverage-floor case.

Mutation-checked in a scratch export: a skip that records, a confidence raise
that records, and recording on any exception each fail the intended tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

2026-10-02 23:46:49 -04:00

..

fixtures

fix(local-decision): fix logprobs read path to top-level Ollama key + fixture test

2026-09-28 20:09:46 -04:00

conftest.py

feat(admin): re-poll the catalog after an allowlist edit

2026-09-09 19:16:08 -04:00

test_admin_allowlist_and_fallback_contracts.py

fix(admin): validate cloud base_urls, and let an allowlist note be corrected

2026-09-08 18:39:17 -04:00

test_admin_allowlist_repoll.py

feat(admin): re-poll the catalog after an allowlist edit

2026-09-09 19:16:08 -04:00

test_admin_allowlist.py

test(allowlist): full suite — config, poller, admin, integration

2026-09-06 18:33:14 -04:00

test_admin_bodies_are_strict.py

fix(admin): unknown request keys are errors, and base providers are read-only

2026-09-08 18:30:06 -04:00

test_admin_classifier_config.py

feat(admin): include decision block in classifier-config GET response

2026-09-29 00:19:06 -04:00

test_admin_cloud_fallback_config.py

feat(admin): add classifier.cloud_fallback control to Controls page

2026-09-07 18:47:40 -04:00

test_admin_config.py

feat(admin): local_vision.enabled gets a persisted control

2026-09-28 22:57:28 -04:00

test_admin_frontend.py

feat(admin): add local_decision mode fields to classifier card UI

2026-09-29 00:04:54 -04:00

test_admin_health.py

fix(admin): decisions tile reports route decisions and client outcomes

2026-09-26 03:03:30 -04:00

test_admin_history.py

refactor: move config data files into config/ and update all references

2026-08-30 00:03:05 -04:00

test_admin_js_units.py

fix(admin): wire Block button onclick to blockModelConfirm(event)

2026-09-27 00:48:00 -04:00

test_admin_knob_coverage.py

feat(admin): local_vision.enabled gets a persisted control

2026-09-28 22:57:28 -04:00

test_admin_models.py

feat(admin): add inline block confirm with reason, shared across panels

2026-09-26 23:25:00 -04:00

test_admin_nav_is_complete.py

refactor(admin): render nav links from navbar.js instead of eight copies

2026-09-26 03:03:06 -04:00

test_admin_override_binds_everywhere.py

fix: an admin availability override binds every path, and "stale" works

2026-09-08 18:33:50 -04:00

test_admin_proficiency.py

refactor(admin): render nav links from navbar.js instead of eight copies

2026-09-26 03:03:06 -04:00

test_admin_profile_is_default.py

fix(admin): the profiles page claimed a restart the API says is not needed

2026-09-12 11:54:25 -04:00

test_admin_profiles.py

feat(admin): set the routing default from the profiles page; fold the allowlist

2026-09-08 20:49:36 -04:00

test_admin_provider_readonly.py

fix(admin): unknown request keys are errors, and base providers are read-only

2026-09-08 18:30:06 -04:00

test_admin_providers.py

fix(admin): return provider provenance flags in list and detail endpoints

2026-09-06 02:02:57 -04:00

test_admin_routing_override.py

fix: an admin availability override binds every path, and "stale" works

2026-09-08 18:33:50 -04:00

test_admin_runtime.py

feat(admin): one knob table merges runtime and persisted state

2026-09-26 03:03:30 -04:00

test_admin_snapshot.py

fix(admin): decisions tile reports route decisions and client outcomes

2026-09-26 03:03:30 -04:00

test_admin_triggers.py

refactor: move config data files into config/ and update all references

2026-08-30 00:03:05 -04:00

test_admin_watchdog.py

fix: import sort order in test_admin_watchdog.py

2026-09-26 23:43:08 -04:00

test_allowlist_filter_structure.py

feat(admin): set the routing default from the profiles page; fold the allowlist

2026-09-08 20:49:36 -04:00

test_allowlist.py

test(allowlist): full suite — config, poller, admin, integration

2026-09-06 18:33:14 -04:00

test_apply_tiering.py

refactor: move config data files into config/ and update all references

2026-08-30 00:03:05 -04:00

test_balance_config.py

feat(config): DispatchProvider.balance_url with fail-closed parser/telemetry validators

2026-09-06 12:47:49 -04:00

test_baseline_report.py

feat(baseline_report): incumbent-free counterfactual for the challenger dial

2026-09-13 20:11:56 -04:00

test_blocked_routing.py

fix: ruff style compliance on new files

2026-09-26 13:15:25 -04:00

test_capabilities.py

test: pin iter_image_url_values contract on well-typed-but-unparseable input

2026-08-23 16:02:25 -04:00

test_chat_completions.py

feat(admin): local_vision.enabled gets a persisted control

2026-09-28 22:57:28 -04:00

test_cheapest_classifier_candidate.py

feat(classifier): configurable backend -- local_llm / cloud_llm / local_encoder

2026-09-05 23:53:44 -04:00

test_circuit_breaker.py

fix: gate streaming breaker on stream_completed, register content-fault warning tripwire

2026-09-09 22:55:26 -04:00

test_classifier_backoff.py

fix(tests): migrate backoff/modes/gaming fixtures to per-provider refusal state

2026-09-06 02:43:19 -04:00

test_classifier_candidate_categories.py

fix(local-encoder): update test references from confidence_threshold to confidence_min

2026-09-23 01:22:59 -04:00

test_classifier_cascade.py

test(dispatch): multi-provider pin resolution, telemetry, and refusal tests

2026-09-06 01:43:52 -04:00

test_classifier_input.py

fix: resolve context-pruning-and-framing review findings

2026-08-23 22:23:08 -04:00

test_classifier_modes_config.py

Revert "fix(config): switch local_decision default model to mistral-nemo:12b to avoid qwen3.5 reasoning model"

2026-09-29 18:00:51 -04:00

test_classifier_modes_dispatch.py

test(dispatcher): tighten the local_decision backoff tests

2026-10-02 23:46:49 -04:00

test_config_endpoints.py

fix(config): adoption_window_seconds comments match the validator

2026-09-20 21:43:21 -04:00

test_config_watchdog.py

test: config validation tests for new watchdog config sections

2026-09-26 11:42:05 -04:00

test_config.py

feat(config): incumbent knobs — challenger dial, refresh TTL, pricing floor; gated off

2026-09-13 19:38:12 -04:00

test_context_prune.py

fix(pinch): cap outsized tool results even inside the protected window

2026-09-06 19:10:24 -04:00

test_conversation_identity.py

fix(conversation): first occurrence of a duplicate header wins

2026-09-20 21:27:52 -04:00

test_conversation_wiring.py

feat(dispatcher): session_key from X-Router-Conversation, agent and parent recorded

2026-09-20 20:21:09 -04:00

test_credit_attenuation_config.py

feat(config): objective.credit_attenuation block, off by default

2026-09-06 12:50:27 -04:00

test_credit_attenuation_routing.py

fix(test): reprice attenuation fixture so OFF→ON flip is observable

2026-09-06 14:12:54 -04:00

test_decisions_page_structure.py

fix(admin): drop the Kind column, keep the 0.2% it was hiding

2026-09-10 22:21:09 -04:00

test_dispatcher_helpers.py

test: make the HTTP fakes exercise the boundary instead of modelling it

2026-09-08 14:05:16 -04:00

test_dispatcher_local_energy.py

fix(local-encoder): update test references from confidence_threshold to confidence_min

2026-09-23 01:22:59 -04:00

test_docs_content.py

docs: client conversation headers and conversation_id

2026-09-20 20:28:22 -04:00

test_eval_classifier.py

fix(local-decision): add classify_category() and move letter mapping out of eval harness

2026-09-29 18:10:59 -04:00

test_eval_heldout.py

feat(eval): promote held-out set to evals/heldout.yaml

2026-09-28 19:56:55 -04:00

test_eval_proficiency_logging.py

fix(eval): call_model's call site had a positional/keyword argument collision

2026-09-06 22:36:10 -04:00

test_eval_scoring.py

test(eval_scoring): bound infinite-loop test timeout to 1s

2026-09-01 23:05:09 -04:00

test_events.py

fix(events): guard publish_decision against concurrent subscriber churn

2026-08-28 19:02:47 -04:00

test_exploration.py

test(proficiency): step-2 gate (persisted-cost inertness) + property tests for feedback + exploration

2026-09-01 22:10:22 -04:00

test_feedback_preview.py

fix(feedback): align dry-run preview count with projected snapshot; emit CSV header on empty backlog

2026-09-18 00:00:19 -04:00

test_feedback.py

fix(feedback): align dry-run preview count with projected snapshot; emit CSV header on empty backlog

2026-09-18 00:00:19 -04:00

test_gaming_mode.py

fix(tests): migrate backoff/modes/gaming fixtures to per-provider refusal state

2026-09-06 02:43:19 -04:00

test_incumbent_routing.py

test(dispatcher): flag gate proves DB-query absence off, dial kwarg presence on

2026-09-13 20:53:34 -04:00

test_iteration.py

feat(iteration): budget retries in prompt re-bills; prefer cache-preserving same-model retry on malformed

2026-09-18 00:32:55 -04:00

test_leaderboard.py

refactor(leaderboard): thread provider into apply_priors from config key

2026-09-04 20:55:41 -04:00

test_load_candidates.py

refactor: move config data files into config/ and update all references

2026-08-30 00:03:05 -04:00

test_local_decision_descriptions.py

docs(local-decision): gate G measurement results and description tuning

2026-09-28 21:11:36 -04:00

test_local_decision_parse.py

feat(local-decision): add classify_choice with Ollama /api/chat

2026-09-28 20:12:38 -04:00

test_local_dispatch_fallback.py

test: make the HTTP fakes exercise the boundary instead of modelling it

2026-09-08 14:05:16 -04:00

test_local_dispatch.py

test(local_dispatch, proficiency): honor new default_provider validation

2026-09-04 21:01:04 -04:00

test_local_encoder.py

fix(local-encoder): dispatch pooling strategy by model_id, not load order

2026-09-23 16:20:15 -04:00

test_local_energy.py

feat: session attribution + local energy ledger (Phase 1/2)

2026-09-01 00:20:10 -04:00

test_logs.py

feat: one line per request saying what was decided, and why

2026-08-22 21:52:58 -04:00

test_measured_cache_rates.py

feat(dispatcher): measured per-model cache rates with TTL + independent pricing floor

2026-09-13 19:47:24 -04:00

test_metrics_calibration_and_latency.py

feat(metrics): report the cost-estimator's error and the latency nobody could see

2026-09-16 14:34:27 -04:00

test_metrics_endpoint.py

feat(metrics): report the cost-estimator's error and the latency nobody could see

2026-09-16 14:34:27 -04:00

test_metrics.py

fix(admin): decisions tile reports route decisions and client outcomes

2026-09-26 03:03:30 -04:00

test_models_page_structure.py

fix(admin): tighten the models table and show what routing turns on

2026-09-08 22:26:39 -04:00

test_multi_provider_dispatch.py

feat(telemetry): a NULL cached_tokens now says which kind of nothing it was

2026-09-13 11:04:02 -04:00

test_multi_provider_poller.py

feat(poller): add allowlist filter+prune for openrouter provider

2026-09-06 18:11:01 -04:00

test_no_header_snapshot.py

test(dispatcher): baseline no-header route_decisions snapshot fixture

2026-09-20 20:03:07 -04:00

test_notifier.py

fix: ruff style compliance on new files

2026-09-26 13:15:25 -04:00

test_openrouter_poller.py

feat(poller): fetch and normalize OpenRouter catalog

2026-09-06 01:16:27 -04:00

test_outcome_attribution.py

feat(outcome): attribute a report to an exact conversation

2026-09-20 20:26:16 -04:00

test_plans_declare_status.py

docs(plans): give every plan a Status line so the queue is greppable

2026-09-08 18:55:16 -04:00

test_poller_freshness.py

feat(poller): multi-provider main loop with failure isolation and tiering

2026-09-06 01:29:15 -04:00

test_poller_parsing.py

fix(poller): cap the output reserve so a catalog field cannot zero a model

2026-09-09 18:34:35 -04:00

test_prefix_probe.py

feat(telemetry): a prefix-stability probe that stores no prefix

2026-09-13 11:37:59 -04:00

test_proficiency.py

fix(proficiency): peer_rate is the sample-weighted pooled rate per plan §2

2026-09-08 21:24:58 -04:00

test_progress_backtest.py

fix: use sys.executable for backtest subprocess instead of worktree-path-based VENV_PYTHON

2026-09-26 14:58:01 -04:00

test_progress_detect.py

fix: canonicalize args for dup/slow, pass RO keywords to backtest

2026-09-26 14:46:29 -04:00

test_provider_balance.py

feat: store total_credits_usd and total_usage_usd from balance poller

2026-09-10 22:05:12 -04:00

test_route_decisions.py

feat(route-decisions): agent and parent_key columns, surfaced on purpose

2026-09-20 19:54:14 -04:00

test_router_cli.py

feat(cli): router_cli one-shot routing-decision printer

2026-08-23 20:19:46 -04:00

test_router_wall_clock.py

feat(telemetry): router-observed wall-clock and TTFT columns, with plan docs

2026-09-13 12:25:39 -04:00

test_routing_profiles_integration.py

fix(api): /route and /dispatch honour routing.default_profile

2026-09-09 21:19:09 -04:00

test_routing.py

test(routing): pin the load-bearing challenger clamp across the dial class

2026-09-13 20:53:34 -04:00

test_scoring.py

feat: retarget router to NeuralWatt-only, add serving-class routing and an OpenAI-compatible dispatcher

2026-08-12 00:03:43 -04:00

test_seed_local_dispatch.py

fix: local dispatch metering + seed module LOC ceiling

2026-09-02 02:09:59 -04:00

test_seed_sweep.py

refactor(seed_energy): cloud provider from config key

2026-09-04 20:56:15 -04:00

test_session_cache.py

feat(session_cache): pure in-memory classification cache module

2026-08-26 20:58:12 -04:00

test_session_identity.py

feat: session attribution + local energy ledger (Phase 1/2)

2026-09-01 00:20:10 -04:00

test_suite_ignores_machine_overlay.py

fix(config): stop the test suite reading the machine's config.local.yaml

2026-09-08 18:51:22 -04:00

test_task_set.py

fix(evals,seed): unanswerable diff_checking question + two seed-script defects

2026-09-03 19:33:14 -04:00

test_textcut.py

fix: cut text on character boundaries, and write up the mutation-point audit

2026-09-08 14:15:35 -04:00

test_tiering.py

fix: price is a market signal, not a capability measurement

2026-08-21 19:03:17 -04:00

test_truncation_is_cluster_safe.py

fix: cut text on character boundaries, and write up the mutation-point audit

2026-09-08 14:15:35 -04:00

test_tui_schema_drift.py

docs(tests): drift-test comments match the registries

2026-09-20 21:43:21 -04:00

test_tui_warnings.py

feat(metrics): a cache-rate series, and an expiry check on assumed_cache_rate

2026-09-13 11:19:13 -04:00

test_tui.py

fix(tui): render blank ctx cell instead of literal "None"

2026-09-18 23:31:07 -04:00

test_verification.py

fix: an agent turn that calls a tool is not a failed answer

2026-08-21 19:02:07 -04:00

test_watchdog.py

feat(watchdog): enrich alert summary with calls, cost, and model evidence

2026-09-26 23:19:47 -04:00