docs: soak harness bring-up steps
Documents the one-time UPDATE invite_codes SET marks_as_test = TRUE step required before running tests/soak against each environment, schema verification queries, and the expected filter behavior post-run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
125
docs/soak-harness-bringup.md
Normal file
125
docs/soak-harness-bringup.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# Soak Harness Bring-Up
|
||||
|
||||
One-time setup steps before running `tests/soak` against an environment.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An invite code exists with 16+ available uses
|
||||
- You have psql access to the target DB (or admin SQL access via some other means)
|
||||
|
||||
## 1. Flag the invite code as test-seed
|
||||
|
||||
Any account registered with a `marks_as_test=TRUE` invite code gets
|
||||
`users_v2.is_test_account=TRUE`, which keeps it out of real-user stats.
|
||||
|
||||
### Staging
|
||||
|
||||
Invite code: `5VC2MCCN` (16 uses, provisioned 2026-04-10).
|
||||
|
||||
```sql
|
||||
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN';
|
||||
SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN';
|
||||
```
|
||||
|
||||
Expected: `marks_as_test | t`.
|
||||
|
||||
From your workstation:
|
||||
|
||||
```bash
|
||||
ssh root@129.212.150.189 \
|
||||
'docker compose -f /opt/golfgame/docker-compose.staging.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL'
|
||||
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN';
|
||||
SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN';
|
||||
SQL
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
Invite code — to be provisioned when production seeding is needed. Same pattern:
|
||||
|
||||
```bash
|
||||
ssh root@165.245.152.51 \
|
||||
'docker compose -f /opt/golfgame/docker-compose.prod.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL'
|
||||
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '<PROD_INVITE_CODE>';
|
||||
SQL
|
||||
```
|
||||
|
||||
### Local dev
|
||||
|
||||
The local dev environment uses the `SOAKTEST` invite code. Create it once
|
||||
(if you wiped the DB since the last run):
|
||||
|
||||
```sql
|
||||
INSERT INTO invite_codes (code, created_by, expires_at, max_uses, is_active, marks_as_test)
|
||||
SELECT 'SOAKTEST', id, NOW() + INTERVAL '10 years', 100, TRUE, TRUE
|
||||
FROM users_v2 LIMIT 1
|
||||
ON CONFLICT (code) DO UPDATE SET marks_as_test = TRUE;
|
||||
```
|
||||
|
||||
Note: `created_by` references any existing user (the FK doesn't require admin role).
|
||||
If your dev DB has zero users, register one first via the UI or `curl`.
|
||||
|
||||
Connection string for local dev:
|
||||
|
||||
```bash
|
||||
PGPASSWORD=devpassword psql -h localhost -U golf -d golf
|
||||
```
|
||||
|
||||
## 2. Verify the schema migrations applied
|
||||
|
||||
The server-side changes (columns, matview rebuild, stats filter) run automatically
|
||||
on server startup via `SCHEMA_SQL` in `server/stores/user_store.py`. After the
|
||||
server-side changes deploy, verify against each target environment:
|
||||
|
||||
```sql
|
||||
-- All four should return matching rows
|
||||
\d users_v2 -- look for is_test_account column
|
||||
\d invite_codes -- look for marks_as_test column
|
||||
\d leaderboard_overall -- look for is_test_account column
|
||||
\di idx_users_test_account
|
||||
```
|
||||
|
||||
If `leaderboard_overall` doesn't show the new column, the matview rebuild
|
||||
didn't run. Check server startup logs for errors around `initialize_schema`.
|
||||
|
||||
## 3. Run the harness
|
||||
|
||||
Once the harness is built (Tasks 9+ of the implementation plan):
|
||||
|
||||
```bash
|
||||
cd tests/soak
|
||||
npm install
|
||||
|
||||
# First run only: seed 16 accounts via the invite code
|
||||
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run seed
|
||||
|
||||
# Smoke test against local dev
|
||||
TEST_URL=http://localhost:8000 SOAK_INVITE_CODE=SOAKTEST npm run smoke
|
||||
|
||||
# Populate staging scoreboard
|
||||
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:populate
|
||||
|
||||
# Stress test
|
||||
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:stress
|
||||
```
|
||||
|
||||
See `tests/soak/README.md` for the full flag reference.
|
||||
|
||||
## 4. Verify test account filtering works end-to-end
|
||||
|
||||
After a soak run, the soak-seeded accounts should be visible to admins but
|
||||
hidden from public stats:
|
||||
|
||||
```bash
|
||||
# Should return no soak_* usernames (test accounts hidden by default)
|
||||
curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins" | jq '.entries[] | select(.username | startswith("soak_"))'
|
||||
|
||||
# Should return the soak_* accounts when explicitly requested
|
||||
curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins&include_test=true" | jq '.entries[] | select(.username | startswith("soak_"))'
|
||||
```
|
||||
|
||||
In the admin panel:
|
||||
|
||||
- Users tab shows a `[Test]` badge next to soak accounts
|
||||
- Invite codes tab shows `[Test-seed]` next to the flagged code
|
||||
- "Include test accounts" checkbox (default checked) toggles visibility
|
||||
Reference in New Issue
Block a user