feat(monitoring): routing-decision persistence, /metrics, monitoring TUI, router-cli #4

alee · 2026-08-24T00:24:01Z

alee commented

2026-08-24 00:24:01 +00:00

Adds a monitoring/observability stack to the local LLM model router:

route_decisions persistence: every routing decision (route/dispatch/chat/passthrough/local-vision) is recorded best-effort to a new route_decisions table, gated by logging.log_route_decisions, never persisting session_dir or conversation text, and never regenerating the live DB (guarded CREATE TABLE IF NOT EXISTS ensured on the write path).
metrics.py | GET /metrics: quota_burn/scoring_coverage moved into a read-only aggregator (breaking a would-be circular import), plus recent_decisions/per_model/verdict_mix/top_proficiency; /health imports from it; GET /metrics exposes a 7-key, window-bounded, loopback-only JSON view.
tui.py: a Textual terminal dashboard over /metrics - quota burn vs plan, per-model cost/energy/carbon, verdict mix, recent decisions, health warnings - with auto-refresh, last-good error resilience, and keyboard controls. Pins textual as a deliberate UI-only dependency.
router_cli.py: one-shot routing-decision printer (route-only, no provider spend).

Full offline test suite green (505 passed). No auth added; nothing beyond pinned textual; live DB never regenerated.

Adds a monitoring/observability stack to the local LLM model router: - **route_decisions persistence**: every routing decision (route/dispatch/chat/passthrough/local-vision) is recorded best-effort to a new `route_decisions` table, gated by `logging.log_route_decisions`, never persisting session_dir or conversation text, and never regenerating the live DB (guarded `CREATE TABLE IF NOT EXISTS` ensured on the write path). - **metrics.py | GET /metrics**: `quota_burn`/`scoring_coverage` moved into a read-only aggregator (breaking a would-be circular import), plus `recent_decisions`/`per_model`/`verdict_mix`/`top_proficiency`; `/health` imports from it; `GET /metrics` exposes a 7-key, window-bounded, loopback-only JSON view. - **tui.py**: a Textual terminal dashboard over /metrics - quota burn vs plan, per-model cost/energy/carbon, verdict mix, recent decisions, health warnings - with auto-refresh, last-good error resilience, and keyboard controls. Pins textual as a deliberate UI-only dependency. - **router_cli.py**: one-shot routing-decision printer (route-only, no provider spend). Full offline test suite green (505 passed). No auth added; nothing beyond pinned textual; live DB never regenerated.

alee added 3 commits 2026-08-24 00:24:01 +00:00

feat(cli): router_cli one-shot routing-decision printer b84b4839dc

Post a task to POST /route and print the full decision tree
(classification, candidates, selected model, est cost, rejections)
without dispatching a provider call. Supports --category/--tier/--context
overrides and --json. Route-only and no-spend.

feat(tui): Textual monitoring dashboard over GET /metrics 8ef8c401d6

A terminal dashboard that polls the router's GET /metrics and renders
quota burn vs plan, per-model cost/energy/carbon, verdict mix, recent
routing decisions, and health warnings. Auto-refresh with last-good-data
error resilience and keyboard controls (q/Q/Ctrl+C quit, r refresh,
1-5 focus panels). Pins textual (deliberate UI-only dependency, imported
only by tui.py, never by the service).

feat(metrics): persist routing decisions and expose GET /metrics bfb0ff7c00

The router previously recorded only completions (energy_observations),
not the routing decisions behind them, so 'how routing is performing' was
not answerable from data. This adds:

- route_decisions table + idempotent ensure_route_decisions (guarded
  CREATE TABLE IF NOT EXISTS, never regenerates a live DB) gated by
  logging.log_route_decisions; every decision kind (route/dispatch/chat/
  passthrough/local-vision) is persisted best-effort via
  persist_route_decision (never fails a request; only session_key, never
  session_dir). The table is ensured on the write path (mirroring
  proficiency_store._write -> ensure_columns) so a live DB that predates
  the feature migrates safely.
- metrics.py aggregator moved quota_burn/scoring_coverage in from the
  dispatcher (breaking a would-be circular import) and adds
  recent_decisions/per_model/verdict_mix/top_proficiency; /health now
  imports them and GET /metrics exposes the 7-key JSON (window-bounded,
  loopback-only, no auth).
- observed_at indexes on energy_observations/verifications.

alee merged commit 3519923d98 into main

2026-08-24 00:52:23 +00:00

alee referenced this issue from a commit

2026-08-24 00:52:25 +00:00

Merge pull request 'feat(monitoring): routing-decision persistence, /metrics, monitoring TUI, router-cli' (#4) from neuralwatt-router-service into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: alee/6krrt#4