6krrt/plans/named-routing-profiles.md

Named routing profiles

Status: done -- BUILTIN_PROFILES plus overlay profiles

Status: FINAL — decision-complete. §3 was answered by the user on 2026-09-03 (filter only). Written 2026-09-03 against main at c2893b5.

What this is

Today the router has exactly one behaviour: classify, hard-filter, rank quality-first with cost as a tiebreak. The user wants named alternatives — locality, onlycheaps, bigboybritches — alongside a default that stays the feature-rich automatic router it is now.

The mechanism already exists and is half-built. auto:batch is a profile: a named variant of auto that changes one hard-filter parameter (latency_tolerance), documented in README and shipped for months. This plan generalises that one hard-coded special case into a named, configurable set. It is deliberately NOT a new ranking path.

1. Where it slots in, precisely

Three call sites, all already doing most of the work:

• src/dispatcher.py:2937 — wants_routing = bare in (ROUTER_MODEL, ROUTER_MODEL_BATCH). An exact membership test against two constants. This becomes a parse: split auto:<profile>, look the profile up, reject an unknown one.
• src/dispatcher.py:2955 — latency = BATCH if requested == ROUTER_MODEL_BATCH else INTERACTIVE. This is the existing profile, expressed as a ternary. It must become one case of the general mechanism, not survive alongside it.
• src/routing.py:313 select_candidates — already takes exclude_models: set[str], task_category, latency_tolerance, allowed_access_levels, min_tool_proficiency. A profile supplies or overrides these. All the filtering machinery is already there.

routing.py is a pure module with injected dependencies and rejection_reason is the single copy of the rules (is_eligible is a thin wrapper over it). Keep that: a profile must not introduce a second place where eligibility is decided.

2. The one genuinely missing primitive

select_candidates has a denylist (exclude_models) and no allowlist. Every profile the user named is naturally an allowlist:

• locality → only provider = 'ollama-local' rows
• onlycheaps → only rows under a cost bar
• bigboybritches → only frontier rows

Add one parameter, restrict_to: set[str] | None = None, threaded through rejection_reason → is_eligible → select_candidates, with None meaning "unrestricted" (NOT an empty set, which must mean "nothing eligible"). Give it its own rejection_reason string so a 422 says which profile emptied the candidate set — the catalog-staleness incident is the precedent: an empty candidate set with an opaque reason cost ~19 hours to diagnose.

Prefer expressing profiles as predicates over catalog columns (provider, cost, tier) rather than hard-coded model-id lists, so a profile does not go stale the moment the poller adds a row. A literal id list stays available for the cases where that is genuinely what the user means.

3. THE OPEN DECISION — filter only, or objective overrides too?

DECIDED 2026-09-03 by the user: FILTER ONLY. A profile restricts the candidate set and nothing else. Ranking stays quality-first with cost as the tiebreak, one rule for the whole system. Do NOT implement objective overrides, and do NOT add a quality_tolerance field to a profile definition — if a future preference-shaped profile needs one, that is a separate plan with its own justification.

The reasoning is preserved below because it explains WHY filter-only is sufficient, which is not obvious.

A profile is unambiguously a candidate-set filter. The question is whether it may ALSO override objective.* (quality_tolerance, max_energy_per_request).

It matters concretely, and we have the measurement: plans/local-dispatch-inert-and-test-coupling.md established that the local model scores 0.767 on file_summarization against cloud's 0.95 — a 0.183 gap against a quality_tolerance of 0.1 — so local never routes under quality-first ranking.

• A pure-filter locality still works: restrict the set to local rows and the best local model wins by default, because there is no cloud row left to lose to. This covers the user's stated case.
• A mixed profile that prefers local without excluding cloud does NOT work as a pure filter. It reproduces the dormancy exactly.

So: pure filters satisfy the three named profiles. Objective overrides are only needed for preference-shaped profiles nobody has asked for yet.

Recommendation: ship filter-only. It is the smaller change, it satisfies every named use case, and it keeps one ranking rule in the system. Add overrides later if a real preference-shaped profile appears. If the user wants overrides now, they must be scoped strictly to the profile and never mutate global config — and quality_tolerance in particular describes measurement noise on 2-3 samples, so overriding it means asserting a preference through a knob that means something else.

4. Naming and selection

Profiles are selected the way auto:batch already is: the model field.

auto                  -> default profile
auto:batch            -> MUST keep working, byte-identical behaviour
auto:locality         -> named profile
<real model id>       -> passthrough, unchanged

auto:batch is load-bearing: it is in README, docs/, and possibly user clients. Reimplement it as a profile whose definition sets latency_tolerance: batch, and pin its behaviour with a test that would fail if the general mechanism changed it. Do not leave the ternary in place beside the new path — two mechanisms for one behaviour is how they drift.

An unknown profile (auto:nonsense) must fail loudly with a 422 naming the valid profiles. It must NOT silently fall back to default: a user who mistypes auto:locallity would otherwise get cloud routing and a surprising bill, with nothing indicating why.

Config lives under a new top-level profiles: key. Config is strict (extra="forbid"), so every field needs declaring; an unknown key is an error by design.

5. Observability

route_decisions must record which profile served each decision, or the proficiency loop cannot tell a bigboybritches win from a default one and will read profile-forced selections as evidence of model quality. This is the same reasoning that made the local fallback record kind='local_dispatch_fallback' rather than chat.

Add a profile column (code-side migration in ensure_route_decisions, as every prior column addition did — config/schema.sql is CREATE TABLE IF NOT EXISTS and does not alter a live DB). Surface it in the admin decisions table and its filters, which already filter by kind, category and tier.

6. Interactions to get right

• eligible_categories (local dispatch) is a per-MODEL filter; a profile is a per-REQUEST filter. They compose as AND. A locality profile must not bypass eligible_categories — that gate is what keeps a summarization-grade model away from code, and nemotron-mini:4b detecting 0/4 bugs is why.
• The local-dispatch fallback (kind='local_dispatch_fallback') is a degraded path, not a profile. Leave it alone. Under locality the local model is the primary choice, so the fallback should simply never fire.
• Circuit breaker and admin overrides already feed exclude_models. Profiles must AND with them, never replace them: a profile must not resurrect a model an operator deprecated or the breaker has marked down.
• exploration (epsilon-greedy) picks from eligible candidates. Confirm it reads the profile-filtered set, or exploration will select models the profile excluded.

Non-goals

• No new ranking algorithm. Quality-first with cost as tiebreak stays.
• No change to objective.quality_tolerance's global value.
• No per-category preference logic (that was §1(a) of the previous plan and is explicitly out of scope here).
• No changes to the local-dispatch fallback path.
• Do not touch the classifier: profiles select candidates, not categories.

Success criteria

• auto and auto:batch behave exactly as today, pinned by tests that would catch a regression in the general mechanism.
• At least the three named profiles ship, defined as predicates over catalog columns where possible.
• An unknown profile returns 422 naming the valid ones; it never silently degrades to default.
• A profile that empties the candidate set returns a rejection reason naming the profile.
• route_decisions.profile is populated and visible in the admin decisions table.
• Profiles AND with circuit-breaker exclusions, admin overrides, and eligible_categories — proven by a test for each, not by inspection.
• rejection_reason remains the single copy of the eligibility rules.
• Full suite green with local_energy.enabled both true and false (the Plan 5 invariant must not regress).
• The user's config/config.yaml tariff lines remain uncommitted and verbatim.