835a79cc0f
Documents the one-time UPDATE invite_codes SET marks_as_test = TRUE step required before running tests/soak against each environment, schema verification queries, and the expected filter behavior post-run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
126 lines
4.0 KiB
Markdown
126 lines
4.0 KiB
Markdown
# Soak Harness Bring-Up
|
|
|
|
One-time setup steps before running `tests/soak` against an environment.
|
|
|
|
## Prerequisites
|
|
|
|
- An invite code exists with 16+ available uses
|
|
- You have psql access to the target DB (or admin SQL access via some other means)
|
|
|
|
## 1. Flag the invite code as test-seed
|
|
|
|
Any account registered with a `marks_as_test=TRUE` invite code gets
|
|
`users_v2.is_test_account=TRUE`, which keeps it out of real-user stats.
|
|
|
|
### Staging
|
|
|
|
Invite code: `5VC2MCCN` (16 uses, provisioned 2026-04-10).
|
|
|
|
```sql
|
|
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN';
|
|
SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN';
|
|
```
|
|
|
|
Expected: `marks_as_test | t`.
|
|
|
|
From your workstation:
|
|
|
|
```bash
|
|
ssh root@129.212.150.189 \
|
|
'docker compose -f /opt/golfgame/docker-compose.staging.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL'
|
|
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '5VC2MCCN';
|
|
SELECT code, max_uses, use_count, marks_as_test FROM invite_codes WHERE code = '5VC2MCCN';
|
|
SQL
|
|
```
|
|
|
|
### Production
|
|
|
|
Invite code — to be provisioned when production seeding is needed. Same pattern:
|
|
|
|
```bash
|
|
ssh root@165.245.152.51 \
|
|
'docker compose -f /opt/golfgame/docker-compose.prod.yml exec -T postgres psql -U postgres -d golfgame' <<'SQL'
|
|
UPDATE invite_codes SET marks_as_test = TRUE WHERE code = '<PROD_INVITE_CODE>';
|
|
SQL
|
|
```
|
|
|
|
### Local dev
|
|
|
|
The local dev environment uses the `SOAKTEST` invite code. Create it once
|
|
(if you wiped the DB since the last run):
|
|
|
|
```sql
|
|
INSERT INTO invite_codes (code, created_by, expires_at, max_uses, is_active, marks_as_test)
|
|
SELECT 'SOAKTEST', id, NOW() + INTERVAL '10 years', 100, TRUE, TRUE
|
|
FROM users_v2 LIMIT 1
|
|
ON CONFLICT (code) DO UPDATE SET marks_as_test = TRUE;
|
|
```
|
|
|
|
Note: `created_by` references any existing user (the FK doesn't require admin role).
|
|
If your dev DB has zero users, register one first via the UI or `curl`.
|
|
|
|
Connection string for local dev:
|
|
|
|
```bash
|
|
PGPASSWORD=devpassword psql -h localhost -U golf -d golf
|
|
```
|
|
|
|
## 2. Verify the schema migrations applied
|
|
|
|
The server-side changes (columns, matview rebuild, stats filter) run automatically
|
|
on server startup via `SCHEMA_SQL` in `server/stores/user_store.py`. After the
|
|
server-side changes deploy, verify against each target environment:
|
|
|
|
```sql
|
|
-- All four should return matching rows
|
|
\d users_v2 -- look for is_test_account column
|
|
\d invite_codes -- look for marks_as_test column
|
|
\d leaderboard_overall -- look for is_test_account column
|
|
\di idx_users_test_account
|
|
```
|
|
|
|
If `leaderboard_overall` doesn't show the new column, the matview rebuild
|
|
didn't run. Check server startup logs for errors around `initialize_schema`.
|
|
|
|
## 3. Run the harness
|
|
|
|
Once the harness is built (Tasks 9+ of the implementation plan):
|
|
|
|
```bash
|
|
cd tests/soak
|
|
npm install
|
|
|
|
# First run only: seed 16 accounts via the invite code
|
|
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run seed
|
|
|
|
# Smoke test against local dev
|
|
TEST_URL=http://localhost:8000 SOAK_INVITE_CODE=SOAKTEST npm run smoke
|
|
|
|
# Populate staging scoreboard
|
|
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:populate
|
|
|
|
# Stress test
|
|
TEST_URL=https://staging.adlee.work SOAK_INVITE_CODE=5VC2MCCN npm run soak:stress
|
|
```
|
|
|
|
See `tests/soak/README.md` for the full flag reference.
|
|
|
|
## 4. Verify test account filtering works end-to-end
|
|
|
|
After a soak run, the soak-seeded accounts should be visible to admins but
|
|
hidden from public stats:
|
|
|
|
```bash
|
|
# Should return no soak_* usernames (test accounts hidden by default)
|
|
curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins" | jq '.entries[] | select(.username | startswith("soak_"))'
|
|
|
|
# Should return the soak_* accounts when explicitly requested
|
|
curl -s "https://staging.adlee.work/api/stats/leaderboard?metric=wins&include_test=true" | jq '.entries[] | select(.username | startswith("soak_"))'
|
|
```
|
|
|
|
In the admin panel:
|
|
|
|
- Users tab shows a `[Test]` badge next to soak accounts
|
|
- Invite codes tab shows `[Test-seed]` next to the flagged code
|
|
- "Include test accounts" checkbox (default checked) toggles visibility
|