Files
relicario/docs/superpowers/test-runs/2026-04-24-1c-beta-manual-matrix.md
adlee-was-taken 7b5d36603b docs(test-runs): β₁+β₂ manual test matrix for typed-items
Sections A (β₁ types: Login spot-check + SecureNote/Identity/Card/Key/Totp),
B (β₂ surfaces: custom fields, vault settings, generator popover, ⚙ picker),
C (cross-cutting: field history, icons, search, sync, Firefox parity).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 21:46:27 -04:00

288 lines
15 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Plan 1C-β (β₁ + β₂) — Manual Test Matrix
Walkthrough for validating the typed-item forms (β₁) and the custom-fields editor + vault-settings + generator-popover surfaces (β₂) on Chrome and Firefox.
Branch: `main` @ `783cb7c` (tags `plan-1c-beta1-complete`, `plan-1c-beta2-complete`).
Pre-req: α matrix already validated — this round assumes the foundation works and focuses on the new β surfaces.
---
## Pre-flight
- [ ] **P1.** Bundles built fresh:
```bash
cd /home/alee/Sources/relicario/extension
bun run build:all
```
Expected: "compiled with 2 warnings" (WASM size only) for each of Chrome (`dist/`) and Firefox (`dist-firefox/`).
- [ ] **P2.** Test suites green:
```bash
cd /home/alee/Sources/relicario && cargo test --workspace
cd /home/alee/Sources/relicario/extension && bun run test
```
Expected: 155 Rust + 124 Vitest, all pass.
- [ ] **P3.** Throwaway vault ready (don't pollute your real history). Either reuse the α-validated test vault, or do a fresh `chrome.storage.local` clear and re-init via setup tab.
- [ ] **P4.** Reference JPEG on hand for unlock.
---
## Loading
### Chrome
- [ ] **L1.** `chrome://extensions` → "Load unpacked" → `extension/dist/`. (Or "Update" if already loaded — webpack regenerated everything.)
- [ ] **L2.** Toolbar icon visible. Click → unlock or setup.
### Firefox
- [ ] **L3.** `about:debugging#/runtime/this-firefox` → "Load Temporary Add-on" → `extension/dist-firefox/manifest.json`.
- [ ] **L4.** Toolbar icon visible. Click → unlock or setup.
> Run the entire matrix on Chrome first, then re-run **Section A (β₁ types)** and **Section B (β₂ surfaces)** on Firefox. Section C (cross-cutting) needs to pass on both browsers.
---
## Section A — β₁ typed-item forms
For each new type the matrix checks: **add → list icon → detail render → field round-trip → edit → trash**. Login was validated in α; spot-check it under **A0**.
### A0. Login regression spot-check
- [ ] Open popup → "+ New" → Login.
- [ ] **Expected:** Form has title / url / username / password (with "gen" button) / TOTP secret (optional).
- [ ] Fill and save; verify it appears in the list and detail-view round-trips.
- [ ] **Notes:** ___
### A1. SecureNote
- [ ] **Do:** "+ New" → SecureNote. Title `wifi`. Body `SSID: foo<newline>Password: bar`. Save.
- [ ] **Expected list row:** 📝 (or note icon) + `wifi`.
- [ ] **Expected detail:** Body renders preserving newlines; reveal/copy works on the body.
- [ ] **Edit:** Change body; save; detail reflects new body; modified time bumps.
- [ ] **Trash:** Disappears from list. CLI cross-check: `relicario list --trashed | grep wifi`.
- [ ] **Notes:** ___
### A2. Identity
- [ ] **Do:** "+ New" → Identity. Title `Personal`. Fill at least: full name, email, phone. Leave some fields empty intentionally (e.g. address).
- [ ] **Expected:** Detail view renders only the fields you populated — empty fields should NOT show as blank rows. `core.address === undefined` not `""` (verify via CLI `relicario get Personal --show` if curious).
- [ ] **Edit:** Add a field that was previously empty; save; detail shows the new row.
- [ ] **Trash:** Soft-deletes.
- [ ] **Notes:** ___
### A3. Card
- [ ] **Do:** "+ New" → Card. Title `Visa Test`. Cardholder `J. DOE`. Number `4111111111111111` (the canonical Visa test number — brand should auto-detect to "Visa"). CVV `123`. Expiry `08 / 2029`. PIN `9999`.
- [ ] **Expected during edit:** Brand chip flips to "Visa" once 4+ digits are typed (BIN match on `4`).
- [ ] **Expected detail:** number/cvv/pin are concealed by default; reveal on each works; copy on each puts the value on clipboard. Expiry shows `08/2029`.
- [ ] **Wire-format check (CLI):** `relicario get "Visa Test" --show --json | jq '.core.expiry'` should be `{"month":8,"year":2029}` (numbers, not strings).
- [ ] **Trash:** Soft-deletes.
- [ ] **Notes:** ___
### A4. Key
- [ ] **Do:** "+ New" → Key. Title `gh-deploy`. Algorithm `ed25519` (free-text). Paste a multi-line ASCII key into key_material (any junk is fine — `-----BEGIN OPENSSH PRIVATE KEY-----\nblah\n-----END...`).
- [ ] **Expected:** key_material is concealed/textarea-style; reveal shows full content with line breaks intact; copy puts the multi-line value on clipboard verbatim.
- [ ] **Edit:** Append to algorithm string; save; detail reflects.
- [ ] **Trash:** Soft-deletes.
- [ ] **Notes:** ___
### A5. Totp — TOTP kind (6 digits)
- [ ] **Do:** "+ New" → Totp. Title `GitHub-2FA`. Secret `JBSWY3DPEHPK3PXP` (RFC 6238 vector). Kind: TOTP.
- [ ] **Expected detail signature block:** Big 6-digit code (rotates every 30s); countdown ring shrinks each tick; code refreshes at the rollover without a manual reload.
- [ ] **Cross-check:** `oathtool --totp -b JBSWY3DPEHPK3PXP` (or any TOTP authenticator) → matches what the popup shows for the same wall-clock second.
- [ ] **Copy:** "copy code" button puts current code on clipboard.
- [ ] **Notes:** ___
### A6. Totp — Steam Guard kind (5 alphanumeric)
- [ ] **Do:** "+ New" → Totp. Title `Steam`. Secret `JBSWY3DPEHPK3PXP` (any base32 will do for the test). Toggle kind to **Steam**.
- [ ] **Expected:** Form's `digits` field disappears or locks (Steam is fixed at 5).
- [ ] **Expected detail:** 5-character alphanumeric code (e.g. `H7K2C`). All chars from the Steam alphabet `23456789BCDFGHJKMNPQRTVWXY` (no `0`, `1`, `A`, `E`, `I`, `O`, `S`, `U`, `Z`, `L`).
- [ ] **Edit:** Switch kind to TOTP, save; detail flips to 6-digit decimal. Switch back to Steam; flips back to 5-char.
- [ ] **CRITICAL:** If switching kinds doesn't re-render the detail-view computed code correctly after save, that's a stale-state bug — file before continuing.
- [ ] **Notes:** ___
### A7. Document type — gating
- [ ] **Do:** "+ New" → Document.
- [ ] **Expected:** "Coming soon" placeholder (planned for γ). Back button returns to list. **Should not crash or render a partial form.**
- [ ] **Notes:** ___
---
## Section B — β₂ surfaces
### B1. Custom fields editor — add path
- [ ] **Do:** Open any item form (Login is fine). Scroll to the disclosure labeled "custom fields ▸" (or similar). Click to expand.
- [ ] **Expected:** Disclosure expands; "+ section" / "+ field" controls appear.
- [ ] **Do:** Add a section named `recovery codes`. Add two fields under it: kind=`password` with label `code 1` value `aaaa-bbbb`, and kind=`concealed` with label `code 2` value `cccc-dddd`. Save.
- [ ] **Expected:** Detail view shows the typed Login rows first, then the `recovery codes` section header, then the two custom rows. Each concealed/password row has reveal + copy.
- [ ] **CLI cross-check:** `relicario get <item> --show --json | jq '.sections'` shows the section with both fields.
- [ ] **Notes:** ___
### B2. Custom fields editor — edit path
- [ ] **Do:** Edit the same item. In the disclosure, remove `code 1`, edit `code 2`'s label to `recovery hash`, add a new `text` kind field labeled `notes` value `worked 2024-04`. Save.
- [ ] **Expected:** Detail reflects all three changes (one removed, one renamed, one added).
- [ ] **Edge:** A blank `label` field — does β₂ render as `(unnamed)` or reject save? (Spec says render; verify either is acceptable but consistent.)
- [ ] **Notes:** ___
### B3. Custom fields editor — kind sniff
- [ ] **Do:** On a fresh add of an Identity item (or any type), open custom fields. Add fields of each supported kind (text / password / concealed). For each, verify in detail view: `text` is plain visible; `password` and `concealed` are masked with reveal/copy.
- [ ] **Expected:** No reordering controls (β₂ scope), but adding a new field appends to end.
- [ ] **Notes:** ___
### B4. Vault settings — open path via ⚙ picker
- [ ] **Do:** Click the ⚙ icon in the toolbar. β₂ split this into a picker.
- [ ] **Expected:** A small menu appears with two choices — **device settings** (capture toggle, prompt style, blacklist) and **vault settings** (retention/generator/origin-acks). Pick "vault settings".
- [ ] **Expected:** Vault settings screen renders with a back arrow.
- [ ] **Notes:** ___
### B5. Vault settings — trash retention
- [ ] **Do:** In vault settings, change "trash retention" from default to `7 days`.
- [ ] **Expected:** Save button enables (was disabled because no diff).
- [ ] **Do:** Save; lock; re-unlock; reopen vault settings.
- [ ] **Expected:** Still `7 days` (decrypted from the persisted VaultSettings).
- [ ] **Notes:** ___
### B6. Vault settings — history retention
- [ ] **Do:** Change "field history retention" to `last 5` (or `30 days` if your build offers `last_n` selectors). Save.
- [ ] **Expected:** Persists across lock/unlock.
- [ ] **Notes:** ___
### B7. Vault settings — generator preview + "configure"
- [ ] **Expected by default:** Generator preview line shows current saved default (e.g. `Random, 20 chars, lower+upper+digits+symbols, safe symbols`).
- [ ] **Do:** Click "configure ▾". Popover opens inline (anchored to the preview line).
- [ ] **Do:** Change kind to **BIP39**. Set word count to 8. Set separator to `-`. Set capitalization to `lower`.
- [ ] **Expected:** Preview-string in the popover refreshes per-keystroke (debounced); a sample generated phrase shows.
- [ ] **Do:** Click "save as default". Popover closes. Preview line on the vault-settings screen now reads `BIP39, 8 words, "-" separator, lower`.
- [ ] **Do:** Lock; re-unlock; reopen vault settings.
- [ ] **Expected:** Preview still shows BIP39 default.
- [ ] **Notes:** ___
### B8. Generator popover — open from Login form
- [ ] **Do:** "+ New" → Login. Click the "gen" button next to the password field.
- [ ] **Expected:** Generator popover opens **inheriting the BIP39 default from B7**. Sample phrase visible.
- [ ] **Do:** Click "use this value".
- [ ] **Expected:** The Login form's password field gets the BIP39 phrase. Popover closes.
- [ ] **Edge:** Open popover; toggle kind to **Random**; popover refreshes with random preview; click "use this value" — random string lands in the field. (Toggling shouldn't permanently mutate the saved default.)
- [ ] **Notes:** ___
### B9. Generator popover — kind toggle round-trip
- [ ] **Do:** Open popover from a fresh Login form. Toggle Random ↔ BIP39 several times.
- [ ] **Expected each toggle:** Preview redraws; debounced request shape switches between `generate_password` and `generate_passphrase`.
- [ ] **Smoke:** No console errors on toggle.
- [ ] **Notes:** ___
### B10. Vault settings — origin-ack revoke
- [ ] **Pre-req:** Have at least one acked origin (e.g. github.com from α step 6).
- [ ] **Do:** Vault settings → scroll to "autofill acks". Find the github.com row. Click "revoke".
- [ ] **Expected:** Row disappears (or shows "revoked").
- [ ] **Save** (β₂ batches changes). Lock; re-unlock; reopen.
- [ ] **Expected:** Row stays gone.
- [ ] **Do:** Navigate to github.com/login; click the autofill icon.
- [ ] **Expected:** **TOFU prompt re-fires** — the origin is no longer pre-acked.
- [ ] **CRITICAL:** If autofill silently succeeds without re-prompting, the revoke didn't actually clear `VaultSettings.autofill_origin_acks`.
- [ ] **Notes:** ___
### B11. Vault settings — discard / no-op
- [ ] **Do:** Open vault settings. Don't change anything. Click back arrow.
- [ ] **Expected:** Returns to list with no save attempt (popup didn't network-request).
- [ ] **Do:** Open again; change something; click back without saving.
- [ ] **Expected:** Either a confirm prompt OR silent discard. Reopen; the change is gone (not persisted).
- [ ] **Notes:** ___
### B12. ⚙ picker — device-settings path regression
- [ ] **Do:** ⚙ → "device settings".
- [ ] **Expected:** The α-era device settings screen appears (capture toggle, bar/toast style, blacklist). All controls still functional.
- [ ] **Notes:** ___
---
## Section C — Cross-cutting
### C1. Field history captured for new typed kinds
- [ ] **Do:** Edit the Card item from A3; rotate the cvv. Save.
- [ ] **Do:** Edit the Key item from A4; rotate key_material. Save.
- [ ] **Do:** Edit the Totp item from A5; rotate the secret. Save.
- [ ] **Expected (CLI):** `relicario get <each> --show --json | jq '.field_history'` has an entry for the rotated concealed/password field with old value + timestamp.
- [ ] **Notes:** ___
### C2. List icon parity per type
- [ ] **Do:** Scroll the populated list.
- [ ] **Expected:** Each row's icon matches its type. Login 🔑, SecureNote 📝, Identity 👤, Card 💳, Key 🗝, Totp ⏱ (or whatever the implementation chose — the matrix only checks consistency, not specific glyph).
- [ ] **Notes:** ___
### C3. Search across new types
- [ ] **Do:** Use the search box; type a substring of an item title for each type.
- [ ] **Expected:** Each type-specific item is findable; the type chip/icon is correct in the filtered list.
- [ ] **Notes:** ___
### C4. Sync / git push round-trip
- [ ] **Do:** From your throwaway test vault host, after creating items A1A6 and the custom-field item from B1, run a sync from the popup (sync icon).
- [ ] **Expected:** Push succeeds; `git log` on the test repo shows new commits.
- [ ] **Do:** From CLI in main worktree, `relicario sync` then `relicario list`.
- [ ] **Expected:** Same items visible. (Tests round-trip integrity of the new wire format on a real git host, not just localStorage.)
- [ ] **Notes:** ___
### C5. Firefox parity
- [ ] **Do:** Re-run Section A (A0A7) and Section B (B1B12) on the Firefox-loaded `dist-firefox/`.
- [ ] **Expected:** Behavior identical to Chrome.
- [ ] **Watch for:** WASM-loading drift (FF uses `initDefault(wasmUrl)` not `initSync` because background.js is persistent, not SW). Anything broken on FF that works on Chrome is a WASM-init bug.
- [ ] **Notes:** ___
---
## Final acceptance
- [ ] **A1.** All Section A scenarios pass on Chrome.
- [ ] **A2.** All Section B scenarios pass on Chrome.
- [ ] **A3.** All Section A + B scenarios pass on Firefox.
- [ ] **A4.** Section C cross-cutting all pass.
- [ ] **A5.** Lint sweeps green:
```bash
git grep -n 'idfoto' extension/ # 0
git grep -n '@ts-nocheck' extension/src/ # 0
git grep -n 'innerHTML\|insertAdjacentHTML' extension/src/content/ # 0
git grep -n 'coming-soon\|Coming in' extension/src/popup/components/ # only 'document'
```
---
## Findings / issues
Use this space to log anything weird. For each issue: file path + symptom + repro steps. Bug-fix commits go to main as you find them.
```
(fill in as you go)
```
### Decision
- [ ] All clean — proceed to brainstorm 1C-γ.
- [ ] Bugs found and patched on main; re-run affected sections.
- [ ] Bugs found that warrant a worktree (>3 commits to fix).
---
*Generated 2026-04-24 — sources: spec `2026-04-22-relicario-extension-1c-beta1-design.md` §3.9, spec `2026-04-22-relicario-extension-1c-beta2-design.md` "Manual matrix", α matrix `2026-04-20-1c-alpha-manual-matrix.md`.*