diff --git a/docs/soak-harness-bringup.md b/docs/soak-harness-bringup.md new file mode 100644 index 0000000..f3315b4 --- /dev/null +++ b/docs/soak-harness-bringup.md @@ -0,0 +1,125 @@ +# Soak Harness Bring-Up + +One-time setup steps before running `tests/soak` against an environment. + +## Prerequisites + +- An invite code exists with 16+ available uses +- You have psql access to the target DB (or admin SQL access via some other means) + +## 1. Flag the invite code as test-seed + +Any account registered with a `marks_as_test=TRUE` invite code gets +`users_v2.is_test_account=TRUE`, which keeps it out of real-user stats. + +### Staging + +Invite code: `5VC2MCCN` (16 uses, provisioned 2026-04-10). + +```sql +UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN'; +SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN'; +``` + +Expected: `marks_as_test | t`. + +From your workstation: + +```bash +ssh root@129.212.150.189 \ + 'docker compose -f /opt/golfgame/docker-compose.staging.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL' +UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN'; +SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN'; +SQL +``` + +### Production + +Invite code — to be provisioned when production seeding is needed. Same pattern: + +```bash +ssh root@165.245.152.51 \ + 'docker compose -f /opt/golfgame/docker-compose.prod.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL' +UPDATE invite_codes SET marks_as_test = TRUE WHERE code = ''; +SQL +``` + +### Local dev + +The local dev environment uses the `SOAKTEST` invite code. Create it once +(if you wiped the DB since the last run): + +```sql +INSERT INTO invite_codes (code, created_by, expires_at, max_uses, is_active, marks_as_test) +SELECT 'SOAKTEST', id, NOW() + INTERVAL '10 years', 100, TRUE, TRUE +FROM users_v2 LIMIT 1 +ON CONFLICT (code) DO UPDATE SET marks_as_test = TRUE; +``` + +Note: `created_by` references any existing user (the FK doesn't require admin role). +If your dev DB has zero users, register one first via the UI or `curl`. + +Connection string for local dev: + +```bash +PGPASSWORD=devpassword psql -h localhost -U golf -d golf +``` + +## 2. Verify the schema migrations applied + +The server-side changes (columns, matview rebuild, stats filter) run automatically +on server startup via `SCHEMA_SQL` in `server/stores/user_store.py`. After the +server-side changes deploy, verify against each target environment: + +```sql +-- All four should return matching rows +\d users_v2 -- look for is_test_account column +\d invite_codes -- look for marks_as_test column +\d leaderboard_overall -- look for is_test_account column +\di idx_users_test_account +``` + +If `leaderboard_overall` doesn't show the new column, the matview rebuild +didn't run. Check server startup logs for errors around `initialize_schema`. + +## 3. Run the harness + +Once the harness is built (Tasks 9+ of the implementation plan): + +```bash +cd tests/soak +npm install + +# First run only: seed 16 accounts via the invite code +TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run seed + +# Smoke test against local dev +TEST_URL=http://localhost:8000 SOAK_INVITE_CODE=SOAKTEST npm run smoke + +# Populate staging scoreboard +TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:populate + +# Stress test +TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:stress +``` + +See `tests/soak/README.md` for the full flag reference. + +## 4. Verify test account filtering works end-to-end + +After a soak run, the soak-seeded accounts should be visible to admins but +hidden from public stats: + +```bash +# Should return no soak_* usernames (test accounts hidden by default) +curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins" | jq '.entries[] | select(.username | startswith("soak_"))' + +# Should return the soak_* accounts when explicitly requested +curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins&include_test=true" | jq '.entries[] | select(.username | startswith("soak_"))' +``` + +In the admin panel: + +- Users tab shows a `[Test]` badge next to soak accounts +- Invite codes tab shows `[Test-seed]` next to the flagged code +- "Include test accounts" checkbox (default checked) toggles visibility