6krrt/docs/operations.md

Logging and traceability. Back to README.

Logging and Traceability

One route line per request says what was decided; one dispatch line says what it cost. Both carry a trace id, and rid/sess are columns in energy_observations, so a journal line pivots to its database row and back.

route id=r9116d9 cat=coding_refactor tier=2 ctx=500 src=classifier tools=0
  latency=interactive cand=8 pick=deepseek-v4-flash est_usd=0.00016296 ms=1868
dispatch id=r9116d9 model=deepseek-v4-flash rid=chatcmpl-... sess=c090d751
  p_tok=59511 c_tok=415 kwh=4.8e-05 usd=0.000783 verdict=ok total_ms=5692

At debug it also says why — the classifier's answer and timing, every candidate that was dropped and by which filter, and the ranking with scores:

classify id=r9116d9 cat=coding_refactor tier=2 confidence=0.95 ms=1863
filter   id=r9116d9 model=gemma-4-31b reason=tier(1<2)
filter   id=r9116d9 model=glm-5.2-short reason=access_level(preview)
filter   id=r9116d9 model=kimi-k3-flex reason=latency_class(flex)
rank     id=r9116d9 pos=0 model=deepseek-v4-flash prof=1 est_usd=0.00016296

No conversation text is logged at any level, prompt or answer — prompts here run 60k–150k tokens and the journal is on disk. A test enforces it.

Set the level in config/config.yaml (logging.level), or override it without touching a tracked file:

systemctl --user edit llm-router      # Environment="LLM_ROUTER_LOG_LEVEL=debug"
systemctl --user restart llm-router

Reading it back:

journalctl --user -u 'llm-router*' -f            # quote the glob in zsh
journalctl --user -u llm-router -f -o cat        # message only
journalctl --user -u llm-router -p warning       # severity actually filters
journalctl --user -u llm-router --grep 'chatcmpl-abc123'   # one request's trail
journalctl --user -u llm-router --grep ' id=r9116d9'       # every stage of it

-p works because the service emits journald priority prefixes when — and only when — systemd owns its stderr. Run uvicorn in a terminal and the lines come out clean.

Inspect what a dispatch cost / burned

Each dispatch writes an energy_observations row; a read-only query shows what the last few cost and burned:

sqlite3 -header -column router.db \
  "SELECT model_id, prompt_tokens, completion_tokens, energy_kwh, carbon_g_co2eq, cost_usd
     FROM energy_observations ORDER BY id DESC LIMIT 10;"