# Named routing profiles Status: done -- BUILTIN_PROFILES plus overlay profiles **Status: FINAL — decision-complete. §3 was answered by the user on 2026-09-03 (filter only).** Written 2026-09-03 against `main` at `c2893b5`. ## What this is Today the router has exactly one behaviour: classify, hard-filter, rank quality-first with cost as a tiebreak. The user wants named alternatives — `locality`, `onlycheaps`, `bigboybritches` — alongside a `default` that stays the feature-rich automatic router it is now. **The mechanism already exists and is half-built.** `auto:batch` is a profile: a named variant of `auto` that changes one hard-filter parameter (`latency_tolerance`), documented in `README` and shipped for months. This plan generalises that one hard-coded special case into a named, configurable set. It is deliberately NOT a new ranking path. ## 1. Where it slots in, precisely Three call sites, all already doing most of the work: - **`src/dispatcher.py:2937`** — `wants_routing = bare in (ROUTER_MODEL, ROUTER_MODEL_BATCH)`. An exact membership test against two constants. This becomes a *parse*: split `auto:`, look the profile up, reject an unknown one. - **`src/dispatcher.py:2955`** — `latency = BATCH if requested == ROUTER_MODEL_BATCH else INTERACTIVE`. This is the existing profile, expressed as a ternary. It must become one case of the general mechanism, not survive alongside it. - **`src/routing.py:313` `select_candidates`** — already takes `exclude_models: set[str]`, `task_category`, `latency_tolerance`, `allowed_access_levels`, `min_tool_proficiency`. A profile supplies or overrides these. **All the filtering machinery is already there.** `routing.py` is a pure module with injected dependencies and `rejection_reason` is the single copy of the rules (`is_eligible` is a thin wrapper over it). Keep that: a profile must not introduce a second place where eligibility is decided. ## 2. The one genuinely missing primitive `select_candidates` has a **denylist** (`exclude_models`) and no **allowlist**. Every profile the user named is naturally an allowlist: - `locality` → only `provider = 'ollama-local'` rows - `onlycheaps` → only rows under a cost bar - `bigboybritches` → only frontier rows Add **one** parameter, `restrict_to: set[str] | None = None`, threaded through `rejection_reason` → `is_eligible` → `select_candidates`, with `None` meaning "unrestricted" (NOT an empty set, which must mean "nothing eligible"). Give it its own `rejection_reason` string so a 422 says *which* profile emptied the candidate set — the catalog-staleness incident is the precedent: an empty candidate set with an opaque reason cost ~19 hours to diagnose. Prefer expressing profiles as **predicates over catalog columns** (provider, cost, tier) rather than hard-coded model-id lists, so a profile does not go stale the moment the poller adds a row. A literal id list stays available for the cases where that is genuinely what the user means. ## 3. THE OPEN DECISION — filter only, or objective overrides too? **DECIDED 2026-09-03 by the user: FILTER ONLY.** A profile restricts the candidate set and nothing else. Ranking stays quality-first with cost as the tiebreak, one rule for the whole system. Do NOT implement objective overrides, and do NOT add a `quality_tolerance` field to a profile definition — if a future preference-shaped profile needs one, that is a separate plan with its own justification. The reasoning is preserved below because it explains WHY filter-only is sufficient, which is not obvious. A profile is unambiguously a candidate-set filter. The question is whether it may ALSO override `objective.*` (`quality_tolerance`, `max_energy_per_request`). It matters concretely, and we have the measurement: `plans/local-dispatch-inert-and-test-coupling.md` established that the local model scores 0.767 on `file_summarization` against cloud's 0.95 — a 0.183 gap against a `quality_tolerance` of 0.1 — so local **never routes** under quality-first ranking. - A **pure-filter** `locality` still works: restrict the set to local rows and the best local model wins by default, because there is no cloud row left to lose to. This covers the user's stated case. - A **mixed** profile that *prefers* local without excluding cloud does NOT work as a pure filter. It reproduces the dormancy exactly. So: pure filters satisfy the three named profiles. Objective overrides are only needed for preference-shaped profiles nobody has asked for yet. **Recommendation: ship filter-only.** It is the smaller change, it satisfies every named use case, and it keeps one ranking rule in the system. Add overrides later if a real preference-shaped profile appears. If the user wants overrides now, they must be scoped strictly to the profile and never mutate global config — and `quality_tolerance` in particular describes measurement noise on 2-3 samples, so overriding it means asserting a preference through a knob that means something else. ## 4. Naming and selection Profiles are selected the way `auto:batch` already is: the model field. ``` auto -> default profile auto:batch -> MUST keep working, byte-identical behaviour auto:locality -> named profile -> passthrough, unchanged ``` `auto:batch` is load-bearing: it is in `README`, `docs/`, and possibly user clients. **Reimplement it as a profile whose definition sets `latency_tolerance: batch`, and pin its behaviour with a test that would fail if the general mechanism changed it.** Do not leave the ternary in place beside the new path — two mechanisms for one behaviour is how they drift. An unknown profile (`auto:nonsense`) must fail **loudly** with a 422 naming the valid profiles. It must NOT silently fall back to `default`: a user who mistypes `auto:locallity` would otherwise get cloud routing and a surprising bill, with nothing indicating why. Config lives under a new top-level `profiles:` key. Config is strict (`extra="forbid"`), so every field needs declaring; an unknown key is an error by design. ## 5. Observability `route_decisions` must record which profile served each decision, or the proficiency loop cannot tell a `bigboybritches` win from a `default` one and will read profile-forced selections as evidence of model quality. This is the same reasoning that made the local fallback record `kind='local_dispatch_fallback'` rather than `chat`. Add a `profile` column (code-side migration in `ensure_route_decisions`, as every prior column addition did — `config/schema.sql` is `CREATE TABLE IF NOT EXISTS` and does not alter a live DB). Surface it in the admin decisions table and its filters, which already filter by kind, category and tier. ## 6. Interactions to get right - **`eligible_categories`** (local dispatch) is a per-MODEL filter; a profile is a per-REQUEST filter. They compose as AND. A `locality` profile must not bypass `eligible_categories` — that gate is what keeps a summarization-grade model away from code, and `nemotron-mini:4b` detecting 0/4 bugs is why. - **The local-dispatch fallback** (`kind='local_dispatch_fallback'`) is a degraded path, not a profile. Leave it alone. Under `locality` the local model is the primary choice, so the fallback should simply never fire. - **Circuit breaker and admin overrides** already feed `exclude_models`. Profiles must AND with them, never replace them: a profile must not resurrect a model an operator deprecated or the breaker has marked down. - **`exploration`** (epsilon-greedy) picks from eligible candidates. Confirm it reads the profile-filtered set, or exploration will select models the profile excluded. ## Non-goals - No new ranking algorithm. Quality-first with cost as tiebreak stays. - No change to `objective.quality_tolerance`'s global value. - No per-category preference logic (that was §1(a) of the previous plan and is explicitly out of scope here). - No changes to the local-dispatch fallback path. - Do not touch the classifier: profiles select candidates, not categories. ## Success criteria - `auto` and `auto:batch` behave **exactly** as today, pinned by tests that would catch a regression in the general mechanism. - At least the three named profiles ship, defined as predicates over catalog columns where possible. - An unknown profile returns 422 naming the valid ones; it never silently degrades to `default`. - A profile that empties the candidate set returns a rejection reason naming the profile. - `route_decisions.profile` is populated and visible in the admin decisions table. - Profiles AND with circuit-breaker exclusions, admin overrides, and `eligible_categories` — proven by a test for each, not by inspection. - `rejection_reason` remains the single copy of the eligibility rules. - Full suite green with `local_energy.enabled` both true and false (the Plan 5 invariant must not regress). - The user's `config/config.yaml` tariff lines remain uncommitted and verbatim.