6krrt/plans/no-progress-lift-a2.md

Lift A.2: loops and stalling models, surfaced with the lever beside them

Status: planned -- brief ready; worktree created from origin/main 5aaf1e6 (after #103) Date: 2026-09-26 This brief is the contract. Lift A (PR #102) is the base; do NOT read plans/no-progress-detection.md (reference only).

Goal

Meet the surfacing half of North Star #4 (CLAUDE.md): "Waste is surfaced in the admin portal, and stopping it is one click." Lift A built the detection, the alerts and the data. The portal does not yet show the evidence or put the lever beside it:

• Loops panel (admin/frontend/index.html#loops, loadLoops) shows a severity badge, the raw key opencode-loop:<root id> and opened_at. Nothing an operator can act on.
• Models page has no per-model stall rollup, no Block button beside any evidence, and no Blocked list. blocked exists only as a fourth value in the Model Availability override dropdown.

The data is already there. On the live DB, 18 of 19 watchdog_verdicts rows carry model_id, and all carry calls_since_landed and cost_since_landed_usd, now that conversation identity arrives (fixed in #102).

Out of scope: detection changes, recovery modes, alert channels other than desktop, Lift B, and any change to how blocked is enforced (it already excludes routing and pinned requests via _admin_excluded_models).

Facts to build on (checked on origin/main 2026-09-26)

• watchdog_verdicts: tick_id, session_id, session_root, agent, model_id, provider, flagged, dup, top, top_what, landed, slow, coverage, calls_since_landed, cost_since_landed_usd, llm_second_opinion, created_at. One row per judged session per tick. agent holds the opencode session TITLE, not the agent slug. cost_since_landed_usd is cumulative, so it grows tick by tick for the same session.
• watchdog_alerts: dedup_key (opencode-loop:<root id>), state, severity, flagged_ticks, opened_at, last_fired_at, resolved_at.
• admin_model_overrides: model_id, provider, availability, reason, updated_at. POST /admin/api/models/{model_id:path}/{provider}/availability takes {availability, reason} (_AvailabilityBody, src/admin.py ~885); DELETE on the same path clears the override (~2116).
• Existing endpoints: GET /admin/api/watchdog/status, /loops (~2834), /channels GET/POST, POST /test-alert.
• route_decisions.session_key is "c:" + opencode session id and route_decisions.agent is the agent slug when identity arrives.
• Models page cards: "Model Availability" (models.html ~226) and "Per-model usage" (~310). JS unit harness: tests/test_admin_js_units.py (marker-delimited pure functions, run under node).

Components

1. Loops panel shows the evidence (index.html#loops)

• Extend GET /admin/api/watchdog/loops to return each OPEN alert joined to its root's most recent FLAGGED verdict (any session in that root). Fields: dedup_key, root_id, session_id, title, agent_slug, model_id, provider, severity, flagged_ticks, opened_at, top_what, dup, top, coverage, calls_since_landed, cost_since_landed_usd, llm_second_opinion. agent_slug comes from the latest route_decisions.agent for "c:" + session_id (NULL if none). Also return resolved (alerts resolved in the last 24h, same fields, capped at 20).
• One row per open alert: severity badge, title (truncate ~40ch, full title in title=), model, top repeated target, calls since last landed, $ since last landed ($0.00, n/a when NULL), open for (relative time), and a Block model button (component 3 flow) when model_id is known.
• Resolved rows below, dimmed. Session ids truncated with a tooltip.
• Empty state stays "No open alerts".

2. Per-model stall rollup (models.html, new card "Stalls by model")

• GET /admin/api/watchdog/rollup?hours=N, N in {1, 24, 168}. Per (model_id, provider) with at least one flagged verdict in the window:
  • stalled_sessions = COUNT(DISTINCT session_root) over flagged rows. Not row counts: one row is written per tick.
  • stalled_usd = per flagged session_id, take the MAX cost_since_landed_usd in the window (it is cumulative), then sum per model.
  • traffic_share = the model's share of route_decisions in the same window, its own column; never divide sessions by requests.
  • effective_availability (override if any, else catalog).
• Window picker is radio-style (1h / 24h / 7d), default 24h.
• Each row: model, provider, stalled sessions, $ in stalled sessions, traffic share, availability, and Block model (hidden when already blocked).
• Card sits ABOVE "Model Availability". Empty state: "No stalled sessions in this window".

3. Block from the evidence, with a reason

• Block (from 1 or 2) opens an INLINE confirm row, not a browser dialog: reason input prefilled stalled <N> sessions in <window> (from the Loops panel: looping session <title>), editable, then Confirm / Cancel.
• Confirm posts {availability: "blocked", reason} to the existing endpoint, then refreshes the rollup, the Blocked list and the availability table.
• The existing dropdown path keeps working; when it sets blocked it sends reason set from Models dropdown.

4. Blocked list (models.html, card "Blocked models")

• Rows from admin_model_overrides WHERE availability = 'blocked': model, provider, reason, blocked since (updated_at), Unblock (DELETE .../availability, inline confirm).
• Empty state: "No blocked models". Endpoint: reuse an existing models/overrides endpoint if one returns reason and updated_at; otherwise add GET /admin/api/models/blocked.

5. Alert text carries the evidence

• In src/watchdog.py, the trigger and escalate AlertEvent.summary becomes <title>: <calls> calls, $<cost> since last landed change on <model>. <dashboard_base_url>#loops (omit the on <model> / $ parts when NULL). Keep it one line; notify-send shows title and summary only.

Todos (small; one commit each, by explicit path)

1. Loops endpoint join + resolved + endpoint tests (seeded tables: two ticks for one session so the latest flagged verdict wins; a root with two sessions; NULL model/cost).
2. Loops panel UI + pure formatting helpers (money, relative time, truncation) in marker-delimited functions with node cases in test_admin_js_units.py.
3. Rollup endpoint + tests (distinct roots across ticks; MAX-then-SUM cost; traffic share; window filter).
4. Stalls-by-model card + window radio + node cases for its helpers.
5. Inline Block confirm (shared by Loops and rollup) + reason on the dropdown path + tests that the POST carries the reason.
6. Blocked list card + Unblock + tests.
7. Alert summary text in watchdog.py + a test asserting the summary.
8. Visual QA (below) and the done report.

Visual QA (required; HTTP 200 is not QA)

• On 8081 (scripts/sandbox.sh) against a sqlite-backup copy of the live DB. Seed the COPY with one open alert whose verdicts carry a model, calls and cost, one resolved alert, and one blocked override with a reason. Never write to the live DB.
• Screenshots of Home (Loops) and Models at 1400px and 800px wide, plus zoomed crops of each new card. Check truncation, alignment, empty states, and that Block -> Confirm -> Blocked list -> Unblock round-trips on screen.
• Save screenshots under the scratch/evidence dir, not the repo.

Ground rules (lessons from Lift A)

• Worktree already exists: /home/alee/Sources/6krrt-worktrees/no-progress-lift-a2, branch feat/no-progress-lift-a2 from origin/main 5aaf1e6 (includes #103). /start-work runs with --worktree set to it. Nothing in the main checkout.
• Dispatch to Sisyphus-Junior categories or general. NEVER to oh-my-claudecode:* agents: they are pinned to Anthropic models this opencode cannot reach, and return "completed" with zero work. A subagent session with 0 tool calls did nothing.
• One commit per todo; no squashed "full implementation" commits.
• No test exemptions or skips for labelled cases. If a test cannot pass, stop and report why.
• Lint: pinned uvx ruff@0.16.9, no NEW findings vs origin/main in touched files (recipe in plans/no-progress-lift-a.md Ground rules). Paste the gate's empty output in the done report.
• Full pytest: record the origin/main baseline first; the only known failure is test_incumbent_routing::test_debug_log_emits_incumbent_identity.
• Port 8080 is production: no requests to it, not even GETs.
• Do not push, do not open a PR, do not remove the worktree. The done report lists commits, tests, screenshots and operator steps (router restart).
• ASCII only; no middle dots. No prose paragraphs in the admin UI; mutually exclusive choices are radio-style.
• Stop a worker whose context passes ~150k tokens and start a fresh one.