d5194f43ba
Replaces the Task 31 stub README with complete documentation: quickstart, first-time setup (invite flagging, seeding, smoke), usage examples for all three watch modes, CLI flag reference, env var table, scenario descriptions, error handling summary, test account filtering explanation, and architecture overview. Adds CHECKLIST.md with post-deploy verification, bring-up, scenario, watch mode, failure handling, and staging gate items. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.8 KiB
2.8 KiB
Soak Harness Validation Checklist
Run after significant changes or before calling the harness implementation complete.
Post-deploy schema verification
Run after the server-side changes deploy to each environment.
- Server restarted (docker compose up -d or CI/CD deploy)
- Server logs show
User store schema initializedafter restart \d users_v2showsis_test_accountcolumn with defaultfalse\d invite_codesshowsmarks_as_testcolumn with defaultfalse\d leaderboard_overallshowsis_test_accountcolumn\di idx_users_test_accountshows the partial index- Leaderboard query still works:
curl .../api/stats/leaderboardreturns entries ?include_test=trueparameter is accepted (no 422/500)
Bring-up
- Invite code flagged with
marks_as_test=TRUEon target environment bun run seedcreates/updates accounts in.env.stresstest- All seeded users show
is_test_account=TRUEin the DB
Smoke test
bash scripts/smoke.shexits 0 within 60s
Scenarios
--scenario=populate --rooms=1 --games-per-room=1completes cleanly--scenario=populate --rooms=2 --games-per-room=2runs multiple rooms and multiple games--scenario=stress --games-per-room=3logschaos_injectedevents and completes
Watch modes
--watch=noneproduces JSONL on stdout, nothing else--watch=dashboardopens http://localhost:7777, grid renders, WS showshealthy- Clicking a player tile opens the video modal with live JPEG frames
- Closing the modal (Esc or Close) stops the screencast (check logs for
screencast_stopped) --watch=tiledopens native Chromium windows sized to show the full game table
Failure handling
- Ctrl-C during a run → graceful shutdown, summary printed, exit code 2
- Double Ctrl-C → immediate hard exit (130)
- Health probes detect server down (3 consecutive failures → fatal abort)
- Artifacts directory contains screenshots + state JSON on failure
- Artifacts older than 7 days are pruned on next startup
Server-side filtering
GET /api/stats/leaderboard(default) hides soak accountsGET /api/stats/leaderboard?include_test=trueshows soak accounts- Admin panel user list shows
[Test]badge on soak accounts - Admin panel invite codes tab shows
[Test-seed]badge - "Include test accounts" checkbox toggles visibility in admin
Staging bring-up
5VC2MCCNflagged withmarks_as_test=TRUEon staging DB- 16 accounts seeded via
SOAK_INVITE_CODE=5VC2MCCN bun run seed - Populate run against staging completes with
--watch=dashboard - Staging leaderboard default does NOT show soak accounts
- Staging leaderboard with
?include_test=truedoes show them