3523dcf93e
plans/ held 58 documents and exactly one said whether it was open. The rest
mixed finished work, reviews of shipped work, parked specs and genuinely
pending ones, with nothing distinguishing them, so "how many plans are in
the queue" had no answer short of reading all 58.
Now `grep -H '^Status:' plans/*.md` is the answer:
50 done 3 in progress 2 planned 2 reference 1 parked
Statuses were derived rather than guessed: CLAUDE.md's own built list and
"What's NOT built yet" section, plus checking the subject exists in the
code. A review of work that shipped counts as done -- it records what was
found, it is not a request for anything. `reference` separates the two docs
that are conventions rather than work items (admin-design-standards,
admin-work-framework), which otherwise read as permanently-open plans.
The vocabulary is deliberately five words. A larger one invites "mostly
done" and "blocked-ish", which is how the directory became unreadable.
test_plans_declare_status.py keeps it from rotting: a new plan without a
marker fails, as does an unknown status, one buried below the eighth line,
or an open status with no reason -- "planned" alone is the state that rots,
since nobody can tell later whether it waits on a decision, a dependency,
or just nobody's turn.
Also updates the sweep plan with what landed and what did not, including
that #9 was not a defect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRQXz5SYZYVWscxS1QqF6U
172 lines
8.8 KiB
Markdown
172 lines
8.8 KiB
Markdown
# Admin portal: profile visibility and selection
|
|
|
|
Status: done -- profile CRUD in admin.py
|
|
|
|
**Status: FINAL — decision-complete. §C was answered by the user on
|
|
2026-09-04: A, B and C are all in scope.** Written 2026-09-04 against `main` at `d08aed3`.
|
|
|
|
## The principle this serves
|
|
|
|
Stated by the user: the admin portal does not need 100% coverage of every
|
|
configurable lever, but **the main features should each have basic
|
|
functionality and configurability there**. Named routing profiles shipped in
|
|
PR #24 with no portal surface at all — they exist only as code defaults and an
|
|
`auto:<name>` string a client has to know to send. That fails the bar.
|
|
|
|
## What exists today
|
|
|
|
- `BUILTIN_PROFILES` in `src/dispatcher.py:110-116` — five profiles, defined as
|
|
predicates over catalog columns:
|
|
|
|
| profile | definition |
|
|
|---|---|
|
|
| `default` | `RoutingProfile()` |
|
|
| `batch` | `latency_tolerance="batch"` |
|
|
| `locality` | `provider="ollama-local"` |
|
|
| `bigboybritches` | `min_tier=3` |
|
|
| `onlycheaps` | `max_cost_per_1m_completion=0.5` |
|
|
|
|
- `config.RoutingProfile` (`src/config.py:197`) — fields `provider`,
|
|
`min_tier`, `max_tier`, `latency_tolerance`,
|
|
`max_cost_per_1m_completion`, `allowed_model_ids`. All optional.
|
|
- A `profiles:` config section is supported but **no block exists in
|
|
`config/config.yaml`**, so today every profile is a code default.
|
|
- **No `routing.default_profile` knob.** Bare `auto` resolves to the built-in
|
|
`default` and there is no way to change that without editing code.
|
|
- `admin.py:_CONFIG_ALLOWLIST` — dotted config paths an operator may edit.
|
|
Writes go through comment-preserving `ruamel.yaml` round-trip, are validated
|
|
through `RouterConfig` before any byte reaches disk, and back up first.
|
|
`routing.default_flex_preference` is already on it and is the **exact
|
|
precedent** for what Part B needs.
|
|
- Admin pages: `index.html`, `models.html`, `decisions.html`, `controls.html`.
|
|
`decisions.html` already has a Profile column and filter (PR #24).
|
|
|
|
## Part A — Show what profiles are and what they resolve to
|
|
|
|
Read-only, and the highest value per unit of work.
|
|
|
|
A profile's *definition* is not the useful fact; what it **currently admits**
|
|
is. Those differ in ways that are invisible today. Measured on the live
|
|
catalog on 2026-09-04:
|
|
|
|
- `bigboybritches` (`min_tier=3`) matches **7** models — but only **4** are
|
|
reachable interactively, because three are `-flex` rows and the default
|
|
`latency_tolerance=interactive` filters them at the hard-filter stage.
|
|
- `locality` matches **1** model, and only for `file_summarization` /
|
|
`diff_checking`, because per-model `eligible_categories` ANDs on top.
|
|
- `onlycheaps` matches **4**, and the cheapest is the LOCAL model at
|
|
$0.229/1M completion — a measured electricity figure sitting in the same
|
|
column as cloud list prices.
|
|
|
|
None of that is derivable from reading `min_tier=3`.
|
|
|
|
**Add a Profiles panel** (new `profiles.html`, or a section on `models.html` —
|
|
implementer's choice, state which and why) showing per profile: its name,
|
|
whether it is built-in or config-defined, its definition rendered as fields,
|
|
the **count and list of models it currently admits**, and the count reachable
|
|
under `interactive` specifically.
|
|
|
|
Compute admitted sets by calling `routing.select_candidates` with the
|
|
profile's `restrict_to`, exactly as live routing does — do NOT reimplement the
|
|
predicate. `metrics.context_ceilings` is the precedent: it delegates to
|
|
`select_candidates` so its numbers cannot drift from routing's.
|
|
|
|
**A profile that currently admits ZERO models must be visibly flagged.** That
|
|
is the shape of the 422 that cost ~19 hours in incident #3, and a profile can
|
|
reach it silently through admin deprecations — exactly what happened to the
|
|
vision-capable set on 2026-09-04.
|
|
|
|
## Part B — Let the operator choose the default profile
|
|
|
|
Add `routing.default_profile` (string, defaults to `"default"`), and put it on
|
|
`_CONFIG_ALLOWLIST` so it is editable from `controls.html` alongside
|
|
`routing.default_flex_preference`.
|
|
|
|
Effect: bare `auto` — which is what `opencode.json` and every existing client
|
|
sends — resolves to the operator's chosen profile instead of the hard-coded
|
|
`default`. That is what makes profiles *usable* without touching client
|
|
config, and it is the smallest change that turns them from a curiosity into an
|
|
operational lever.
|
|
|
|
Constraints:
|
|
|
|
- Validate against the known profile set at config load. An unknown name must
|
|
fail loudly at load, consistent with how `auto:nonsense` returns 422 rather
|
|
than silently degrading.
|
|
- `auto:<name>` on a request still wins over the default. The default only
|
|
fills in for a bare `auto`.
|
|
- The control must be a **select populated from the live profile list**, not a
|
|
free-text field. A typo here silently redirects all default traffic.
|
|
- Warn in the UI if the chosen default currently admits zero models.
|
|
|
|
## Part C — SCOPE DECISION: create and edit custom profiles?
|
|
|
|
**DECIDED 2026-09-04 by the user: IN SCOPE.** Ship A, B and C together.
|
|
|
|
The cost note below still stands and should shape sequencing — do A and B
|
|
first so the read-only view and the default selector exist before CRUD is
|
|
layered on. C is most of the work; treat it as its own wave.
|
|
|
|
The honest cost difference: `_CONFIG_ALLOWLIST` edits **named scalars**. A
|
|
profile is a structured object with six optional fields, one of them a set of
|
|
model ids. Persisting one means new machinery — a nested-write path through
|
|
the ruamel round-trip, a create/delete flow, and a UI for six fields rather
|
|
than a text input. That is most of the work in this plan.
|
|
|
|
**Sequencing, not deferral:** build A and B before C. A and B make profiles
|
|
visible, understandable and selectable — the "basic functionality and
|
|
configurability" bar — at a fraction of the cost, and they give C something to
|
|
build on: the read-only panel from A is where a created profile is verified,
|
|
and B's `default_profile` is what C's delete-guard has to protect. Landing C
|
|
first would mean writing CRUD against a surface nobody can see.
|
|
|
|
### C requirements, now that it is in scope
|
|
|
|
- **Reuse the validated-write path**: comment-preserving `ruamel.yaml`
|
|
round-trip, `RouterConfig` validation before any byte reaches disk, backup
|
|
first. Do not add a second way to write `config.yaml`.
|
|
- **Built-ins are not editable or deletable.** Render them read-only. A
|
|
config-defined profile whose name collides with a built-in must be **rejected
|
|
at config load** with a clear message, not silently override or be silently
|
|
ignored. Ambiguity about which definition is live would be worse than either.
|
|
- **`allowed_model_ids` needs a multi-select populated from the live catalog**,
|
|
not free text. A typo'd model id yields a profile that silently admits fewer
|
|
models than intended — the failure this plan exists to make visible.
|
|
- **Deleting the profile named by `routing.default_profile` must be refused**,
|
|
with a message saying which setting depends on it. Same for a rename. The
|
|
alternative is a config that fails to load on next start, which is the
|
|
refuse-at-load behaviour of Part B turned into an outage.
|
|
- **Creating a profile that admits zero models is allowed but must warn** on
|
|
save, consistent with Part A's flag. It may be deliberate (a profile for a
|
|
model not yet in the catalog); it must not be silent.
|
|
- Deleting or editing a profile takes effect on reload like any other persisted
|
|
config edit. Say so in the UI rather than implying it is instant.
|
|
|
|
## Non-goals
|
|
|
|
- No ranking changes. Profiles filter; they do not reorder. (Settled in
|
|
`plans/named-routing-profiles.md` §3: filter-only.)
|
|
- No new profile *fields* on `RoutingProfile`.
|
|
- Do not surface profiles as runtime-only toggles that reset on restart. A
|
|
default profile is a deployment preference and belongs in persisted config.
|
|
- Do not let the portal auto-fix a zero-model profile by relaxing it. Warn;
|
|
the operator decides.
|
|
- Do not touch `auto:batch` resolution.
|
|
|
|
## Success criteria
|
|
|
|
- The portal lists every profile — built-in and config-defined — with its
|
|
definition and the models it **currently admits**, computed via
|
|
`routing.select_candidates`, not a reimplementation.
|
|
- A profile admitting zero models is visibly flagged.
|
|
- `routing.default_profile` is editable from `controls.html` as a select
|
|
populated from the live profile list, persists through the validated
|
|
round-trip write, and takes effect for bare `auto` after reload.
|
|
- An invalid `default_profile` fails at config load with a clear message.
|
|
- A test proves `auto:<name>` still overrides the configured default.
|
|
- Full suite green with `local_energy.enabled` both true and false.
|
|
- The user's `config/config.yaml` tariff lines remain uncommitted and verbatim.
|
|
**Note for the implementer: this plan writes to `config/config.yaml`
|
|
programmatically. The write path must preserve unrelated lines — the tariff
|
|
included — and that is worth an explicit test, not an assumption.**
|