alee/6krrt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(routing): add named routing profiles #24

Merged
alee merged 10 commits from feat/named-routing-profiles into main 2026-09-04 14:10:18 +00:00
Conversation 0 Commits 10 Files Changed 16 +1841 -50
alee commented 2026-09-04 13:54:55 +00:00
Owner
Copy Link

feat(routing): named routing profiles

Implements auto:<profile> routing for the LLM router. A profile is a candidate-set filter (not a ranking or objective override). Built-ins:

  • auto / auto:default — normal quality-first routing, -flex rows excluded
  • auto:batch — admits -flex rows for overnight/async work
  • auto:locality — restricts to ollama-local provider only
  • auto:onlycheaps — limits to models priced at ≤ $0.50 per 1M completion tokens
  • auto:bigboybritches — restricts to tier-3 (frontier) models

Operators can add custom profiles under the profiles: key in config/config.yaml. Each entry accepts provider, min_tier/max_tier, max_cost_per_1m_completion, latency_tolerance, and allowed_model_ids. Unknown profile names raise HTTP 422 with a sorted list of valid names.

Verification

  • Full test suite: 1101 passed (both local_energy.enabled: true and local_energy.enabled: false)
  • Final verification wave F1–F4: all APPROVE
  • /v1/models lists auto, auto:default, auto:batch, auto:locality, auto:onlycheaps, auto:bigboybritches
  • Admin decisions page: Profile column and filter present

Commits

  • cc149e4 feat(routing): add restrict_to allowlist primitive
  • b884cd1 feat(config): add named routing profiles configuration
  • 09d5cce feat(dispatcher): resolve auto: built-in/config profiles
  • 9b52b2a feat(db): migrate route_decisions.profile and surface in metrics
  • b7f89e6 feat(admin): add Profile column and filter to decisions page
  • c6392a6 test(profiles): add integration tests for preserved behaviors
  • d48d1e1 docs(profiles): document named routing profiles

Notes

  • Profiles compose with all existing guards: circuit-breaker exclusions, admin-deprecated overrides, and eligible_categories.
  • The routing debug loop currently calls rejection_reason without the restrict_to argument, so profile-excluded rows are logged with reason "-" instead of "profile_excluded". This is a minor observability gap worth a follow-up but does not affect routing correctness.
  • /route silently ignores unknown request fields. This means passing model: "auto:locality" to /route is treated as the default route rather than rejected. This is pre-existing behavior; strict unknown-field rejection could be a future enhancement.
# feat(routing): named routing profiles Implements `auto:<profile>` routing for the LLM router. A profile is a candidate-set filter (not a ranking or objective override). Built-ins: - `auto` / `auto:default` — normal quality-first routing, `-flex` rows excluded - `auto:batch` — admits `-flex` rows for overnight/async work - `auto:locality` — restricts to `ollama-local` provider only - `auto:onlycheaps` — limits to models priced at ≤ $0.50 per 1M completion tokens - `auto:bigboybritches` — restricts to tier-3 (frontier) models Operators can add custom profiles under the `profiles:` key in `config/config.yaml`. Each entry accepts `provider`, `min_tier`/`max_tier`, `max_cost_per_1m_completion`, `latency_tolerance`, and `allowed_model_ids`. Unknown profile names raise HTTP 422 with a sorted list of valid names. ## Verification - Full test suite: **1101 passed** (both `local_energy.enabled: true` and `local_energy.enabled: false`) - Final verification wave F1–F4: all APPROVE - `/v1/models` lists `auto`, `auto:default`, `auto:batch`, `auto:locality`, `auto:onlycheaps`, `auto:bigboybritches` - Admin decisions page: Profile column and filter present ## Commits - `cc149e4` feat(routing): add restrict_to allowlist primitive - `b884cd1` feat(config): add named routing profiles configuration - `09d5cce` feat(dispatcher): resolve auto:<profile> built-in/config profiles - `9b52b2a` feat(db): migrate route_decisions.profile and surface in metrics - `b7f89e6` feat(admin): add Profile column and filter to decisions page - `c6392a6` test(profiles): add integration tests for preserved behaviors - `d48d1e1` docs(profiles): document named routing profiles ## Notes - Profiles compose with all existing guards: circuit-breaker exclusions, admin-deprecated overrides, and `eligible_categories`. - The routing debug loop currently calls `rejection_reason` without the `restrict_to` argument, so profile-excluded rows are logged with reason `"-"` instead of `"profile_excluded"`. This is a minor observability gap worth a follow-up but does not affect routing correctness. - `/route` silently ignores unknown request fields. This means passing `model: "auto:locality"` to `/route` is treated as the default route rather than rejected. This is pre-existing behavior; strict unknown-field rejection could be a future enhancement.
alee added 9 commits 2026-09-04 14:05:15 +00:00
docs(plans): profiles are filter-only (user decision) e975c70e03 
A profile restricts the candidate set; ranking stays quality-first with cost as
the tiebreak. Filter-only is sufficient for all three named profiles because a
pure-filter 'locality' leaves no cloud row to lose to -- the 0.767-vs-0.95
dormancy only blocks MIXED sets that prefer local without excluding cloud.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRQXz5SYZYVWscxS1QqF6U
docs(plans): capability-aware ceiling warnings + reactive rejection detector 60c054bcf1 
From a live incident on 2026-09-04: two image requests 422'd with
"tier >= 1; context >= 242486 tokens; interactive; vision-capable model".
Admin overrides had kimi-k3/-fast/-flex deprecated, and those are the only
vision-capable rows with enough context (782,324). Every remaining active
vision model tops out at 192,500. Incident #3 recurring through a dimension
the existing detector does not model.

The existing machinery is fine and must not be rebuilt: warnings ARE surfaced
(under coverage.warnings, not a top-level key -- a first pass at this analysis
got that wrong), context_ceilings already applies admin deprecations, and
demand_ceiling_warnings already has the correct demand-relative shape.

Gap 1: ceilings bucket by (tier, latency_tolerance) only. During the incident
the tier-1 interactive ceiling was still 782,324 across all models, so every
check stayed silent while the vision-capable ceiling had collapsed to 192,500.
Add vision and json_mode sub-ceilings -- the two hard filters that fail closed
on NULL and can independently empty the set.

Gap 2, and the more valuable half: nothing reads route_decisions' recorded
rejections. A rejection-rate warning would have caught this in minutes without
modelling any capability dimension, and would catch the next failure through a
dimension nobody predicted. Predictive checks only catch what you thought of.

Non-goals are explicit: observability only, no routing changes, and do NOT
auto-revert admin overrides -- those deprecations are deliberate operator cost
decisions. Warn, do not act.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRQXz5SYZYVWscxS1QqF6U
feat(routing): add restrict_to allowlist primitive 877f706996 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat(config): add named routing profiles configuration bd73302ae4 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat(dispatcher): resolve auto:<profile> built-in/config profiles 320a8c96e8 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat(db): migrate route_decisions.profile and surface in metrics 5aded24965 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat(admin): add Profile column and filter to decisions page 0e3df6e61e 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
test(profiles): add integration tests for preserved behaviors 7ded9ef4d0 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
docs(profiles): document named routing profiles a872124bc5 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
alee force-pushed feat/named-routing-profiles from d48d1e1e70 to a872124bc5 2026-09-04 14:05:15 +00:00 Compare
alee merged commit d08aed3fd4 into main 2026-09-04 14:10:18 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
alee
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: alee/6krrt#24
Delete Branch "feat/named-routing-profiles"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?