6krrt/docs/api.md

> API surface reference. Back to [README](../README.md).

## API Endpoints

The dispatcher binds `127.0.0.1:8080`. **No auth of its own** — loopback
is the only thing standing between the open internet and your billing
allowance.

| Method | Path | Description |
|---|---|---|
| `GET` | `/health` | Catalog/reachability status, scoring coverage, warnings (including models that can never enter the candidate set) |
| `GET` | `/metrics` | Aggregated observability JSON: quota burn, coverage, recent decisions, per-model totals, verdict mix, top proficiency, cost-estimator calibration, router-observed latency; loopback-only, no auth |
| `GET` | `/events/decisions` | Server-Sent Events stream of routing decisions for the TUI's live feed; replays recent decisions, then streams new ones as they happen |
| `POST` | `/route` | Classify task, rank candidates, return selected model — **no provider call, no cost** |
| `POST` | `/dispatch` | Same as `/route`, plus complete the provider call, stream response, log observation |
| `GET` | `/v1/models` | OpenAI-compatible model list (router virtual models + catalog) |
| `POST` | `/v1/chat/completions` | OpenAI-compatible completions — routes then proxies, **streaming supported** |
| `POST` | `/outcome` | Client reports whether a completion actually worked — the only signal that knows the answer did the job, not just that it parsed |
| `GET` | `/admin` | Loopback-only web management portal (read-only dashboards, operational triggers, runtime toggles, allowlisted config edits) |

`/metrics` returns a single JSON object with these top-level keys:

- `quota` — per-provider billing shape from `quota_accounts()`: `period` (billing window with start/next_reset/elapsed_fraction/source), `accounts[]` (list of per-provider dicts with `provider`, `shape`, `spend_usd`, and type-specific blocks: `plan` for metered_plan, `pool` for prepaid_credit, `burn` for burn-rate metrics, `credit` when a balance URL was polled, `energy` for kWh/calls), `spend` (aggregate spend with by_provider_usd, total_usd, estimated_usd, and estimate_ratio), and `alarm` (plan-pace or stale-reading alert with kind/severity/headline). The old `by_provider`/`total_balance_usd` shape and flat top-level keys were removed.
- `coverage` — routable-model counts (`routable_models` counts active models filtered
  on `access_level IN (routing.allowed_access_levels)`, excluding any access-gated rows
  whose `access_level` is not in that list) with energy and proficiency data, plus a
  `selection` sub-key reporting which active models (unfiltered by access-level) the
  router has actually picked over the configured window, and a `warnings` array that
  now also includes entries for active models that can never enter the candidate set
  (structural-gate failures: zero context window or broken tier derived from the
  catalog, detected via `routing.rejection_reason` under the most permissive request
  — one token, tier 1, batch latency). Existing warnings cover missing energy data,
  missing proficiency data, stale catalog age, capacity-demand ceiling breaches, and
  classifier source degradation.
  A `cache` sub-key carries the prefix-cache series from `cache_rate_series()`:
  `window_hours`, `assumed_cache_rate`, aggregate `observations` /
  `prompt_tokens` / `cached_prompt_tokens` / `cache_rate`, and a `by_model` list
  of the same figures per `(provider, model_id)`. The rate is
  `sum(cached_prompt_tokens) / sum(prompt_tokens)` over rows the provider
  actually reported a count for (`cached_tokens_source = 'reported'`), excluding
  `seed_reference` sweeps. Two warnings read it: `cache rate:` when the
  aggregate diverges from `objective.assumed_cache_rate` by more than
  `objective.cache_rate_warn_margin`, and `cache rate outlier:` when one
  `(provider, model_id)` group does. Both need
  `objective.cache_rate_warn_min_observations` reported-cache rows first.
- `recent_decisions` — last 50 rows from `route_decisions`
- `per_model` — per-model aggregates over the last 30 days of `energy_observations`
- `verdict_mix` — counts by verification verdict over the last 7 days
- `top_proficiency` — top models by `blended_score` for `coding_general`
- `generated_at` — ISO8601 timestamp
- `pinch` — context-pruning aggregation (when `pinch.enabled`): `calls_30d`, `pruned_calls_30d`, and `share_pruned` (share of decisions pruned), plus median and total `tokens_saved` over 30 days and estimated dollars saved at the blended rate — exposes no conversation text
- `cost_calibration` — from `cost_estimate_calibration()`: how far
  `routing.estimated_cost` is from the provider's own bill, per
  `(provider, model_id)`. **Reported only — nothing applies it to
  `estimated_cost` or to ranking.** `route_decisions` joined to
  `energy_observations` on `request_id` *and* on the selected model and
  provider (the id alone pairs a failover's estimate with another model's
  bill), over `objective.cost_calibration_window_hours`, excluding
  `seed_reference` on both sides and rows with no positive `cost_usd` or
  `est_cost_usd`. Keys: `window_hours`, `min_observations`, `observations`,
  `est_cost_usd`, `billed_cost_usd`, `overestimate_ratio` (est/billed),
  `correction_factor` (billed/est — what *would* be multiplied in), `spread`
  with `spread_low` / `spread_high`, and a `by_model` list of the same figures
  plus `est_per_request_usd`, `billed_per_request_usd` and `sufficient`.
  `spread` is the figure that matters: a uniform scale error reorders nothing,
  while the spread in that error does. Groups below
  `objective.cost_calibration_min_observations` are listed with
  `sufficient: false` and excluded from `spread`. No warning class reads it.
- `latency` — from `latency_series()`: p50 and p95 of the router-observed
  `router_wall_seconds` and `router_ttft_seconds` per `(provider, model_id)`
  over `objective.latency_window_hours`, excluding `seed_reference`. **A
  report, not an objective — nothing in the ranking reads it.** These are the
  router's own clock, not the provider's `duration_seconds`, which is why
  OpenRouter models are visible here at all (OpenRouter reports no duration).
  Keys: `window_hours`, `min_observations`, aggregate `wall_observations` /
  `ttft_observations` / `wall_p50` / `wall_p95` / `ttft_p50` / `ttft_p95`, and
  a `by_model` list of the same plus `wall_sufficient` / `ttft_sufficient`.
  The wall and TTFT counts are independent because TTFT is streaming-only; a
  group can be sufficient on one and not the other. Degrades to an empty
  series on a database that predates the two columns. No warning class reads
  it.

It exposes no conversation text, prompts, or `session_dir`; it is bound to
loopback and unauthenticated exactly like `/health`.

Input to `/route` and `/dispatch` can include `task_category`, `task_tier`,
and `required_context_tokens` overrides — these skip the classifier, useful
for testing routing without the classifier in the loop.

Named routing profiles (`auto:<profile>`). The general form is
`auto:<profile>`; `auto` alone is shorthand for `auto:default`. The router
resolves the profile name against the built-in set and any custom entries in
`config/config.yaml` under `profiles:`. Profiles narrow the candidate set but
do not override the quality-first ranking objective.

| Profile | Effect |
|---|---|
| `auto:default` | Normal quality-first routing, `-flex` rows excluded |
| `auto:batch` | Admits `-flex` rows for overnight and async work |
| `auto:locality` | Restricts to the `ollama-local` provider only |
| `auto:onlycheaps` | Limits to models priced at ≤ $0.50 per 1M completion tokens |
| `auto:bigboybritches` | Restricts to tier-3 (frontier) models only |

**From opencode:** a profile is just another entry under
`provider.llm-router.models` in `opencode.json` — the object's key is the
literal model id opencode sends upstream, so `auto:batch` already works this
way (see the repo-local `opencode.json`). To make `auto:locality`,
`auto:onlycheaps`, or `auto:bigboybritches` selectable from opencode's model
picker instead of typing an override, add one entry per profile:

```json
"auto:onlycheaps": {
  "name": "auto (cost ceiling)",
  "limit": { "context": 782324, "output": 16384 },
  "modalities": { "input": ["text", "image"] }
}
```

`limit`/`modalities` can be copied from the `auto` entry unchanged — a
profile narrows *which models* are candidates, not the context window or
input types the router itself accepts. See [clients.md](clients.md) for the
full opencode wiring.

An unknown profile name raises HTTP 422 and lists the valid names.

The `profile` field is also accepted on `POST /route` and `POST /dispatch`
to select a profile inline per request.

Ask for **any real model id** in `/v1/chat/completions` and it dispatches
directly, still logged — routing is transparent, not opaque.

**Streaming** (chunk-by-chunk proxy): Tokens render as they arrive. Neuralwatt
emits energy and cost as SSE **comment** lines (`: energy {...}`) before
`data: [DONE]` — ordinary clients ignore comments, so the stream flows
untouched while the router scrapes telemetry on the way past. Without this,
streamed calls would log no energy at all.

**Verification headers** (non-streaming): When streaming is not used, the
structural verification verdict surfaces in the `X-Router-Verification` header
so a client can inspect it without parsing the response body. Valid values:
`ok`, `truncated`, `malformed`, `unverifiable`, `none`.

**Capability 422s**: When no model survives the hard filters, the 422 names the
active constraints. That now includes "vision-capable model" or
"json-mode-capable model" when the request carried images or a JSON-mode
`response_format`, alongside the existing context/tier/latency/tool reasons.

**`POST /outcome`**: everything else the router records is a proxy — structural
checks know whether code *parses*, the local LLM check guesses whether prose
*looks* right, neither knows whether the answer did the job. The client does,
because it ran the tests:

```bash
# id comes from the completion body, or any stream chunk
curl -s localhost:8080/outcome -H 'content-type: application/json' \
  -d '{"request_id":"chatcmpl-...","ok":false,"detail":"tests failed"}'
```

An unknown `request_id` returns `404` rather than being quietly accepted, so a
client whose reports go nowhere finds out. The lookup checks
`energy_observations` first (cloud), then `local_energy_observations` (local
rows carry the same `request_id`/`session_dir` attribution) — cloud wins on a
rare id collision.

Unlike the structural/local-LLM checks — which only ever record failures —
`/outcome` folds **both** directions into `proficiency` via `feedback.py`: a
`false` report counts against the model same as any other verification failure,
but a `true` report counts too. It's also the only quality signal that survives
streaming, since a retry can't reach a response whose bytes are already gone,
while a report arrives afterward and works either way.

## Pinning and auto behavior for local models

`model: "auto"` will route eligible tasks to `qwen2.5-coder-router:14b` when the
classifier returns `file_summarization` or `diff_checking` and the local row
survives the filters; otherwise a cloud model is selected as usual. Pin any
local `model_id` (e.g. `qwen2.5-coder-router:14b`) and the request goes straight
to that Ollama tag. `/v1/models` lists local rows with `owned_by: "ollama-local"`.