c3484f05e5
Drops OpenRouter entirely. The provider column and (model_id, provider) key stay so a second provider needs no migration. Verified against the live API — the poller's field mappings were previously unconfirmed guesses and turned out correct. Routing correctness: - Tier on metadata.reasoning.default_enabled, not capabilities.reasoning. The latter only means "the endpoint accepts a reasoning param" and is true for 17 of 19 rows, which put 17 models in tier 3 and left tier 1 empty. Cost is now checked before the reasoning rule so $0.28/1M models can reach tier 1. Distribution goes from 2/17 to 4/6/9. - Capture serving class. NeuralWatt ships ~6 base models as 19 rows whose id suffixes are three orthogonal dimensions (hence glm-5.2-short-fast-flex): -flex is discounted async held during peak, -fast is reasoning disabled or capped, -short is a 200K pool. They carry identical catalog pricing, so without these columns all 7 GLM rows tie exactly and an interactive request could land on a preemptible row. Latency tolerance is a hard filter, not a weight. Suffixes match whole segments so deepseek-v4-flash is not read as a -fast row. - Exclude access-gated models. 6 of 19 rows are grant-gated or canary, marked only in prose, and would 403 at dispatch. - Log the provider's real billed cost and carbon rather than a tokens x list-price estimate, and score eco on carbon per design doc §4. New dispatcher.py exposes /health, /route (dry run, no spend), /dispatch, plus an OpenAI-compatible /v1/models and /v1/chat/completions so any normal client can use it. Streaming is proxied chunk by chunk; NeuralWatt emits energy and cost as SSE comment lines, which clients ignore and the router reads on the way past — otherwise streamed calls would log no energy at all. Classifier now gets the allowed category list injected from config (it was returning invented labels that join against nothing) and runs at temperature 0, because the same prompt was classifying tier 2 then tier 1 and routing to different models. Ships systemd user units. The poller timer is load-bearing, not housekeeping: stale_after_days is 3 with exclude_stale true, so an unpolled catalog eventually marks every row stale and the router returns no candidates at all. Documents the finding that most affects this project: NeuralWatt bills a flat $8.00/kWh, not per token. List price ranks models backwards — on the same prompt kimi-k2.7-code-fast ($4/1M) cost 10x more than kimi-k3-fast ($15/1M). scoring.cost_score still reads list price; re-basing it is the open call. Tests 28 -> 74. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xTPER7K8fNyKiuqNvTCTa
6.8 KiB
6.8 KiB