Files
relicario/docs/superpowers/test-runs/2026-04-24-1c-beta-manual-matrix.md
adlee-was-taken 7b5d36603b docs(test-runs): β₁+β₂ manual test matrix for typed-items
Sections A (β₁ types: Login spot-check + SecureNote/Identity/Card/Key/Totp),
B (β₂ surfaces: custom fields, vault settings, generator popover, ⚙ picker),
C (cross-cutting: field history, icons, search, sync, Firefox parity).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 21:46:27 -04:00

15 KiB
Raw Blame History

Plan 1C-β (β₁ + β₂) — Manual Test Matrix

Walkthrough for validating the typed-item forms (β₁) and the custom-fields editor + vault-settings + generator-popover surfaces (β₂) on Chrome and Firefox.

Branch: main @ 783cb7c (tags plan-1c-beta1-complete, plan-1c-beta2-complete). Pre-req: α matrix already validated — this round assumes the foundation works and focuses on the new β surfaces.


Pre-flight

  • P1. Bundles built fresh:

    cd /home/alee/Sources/relicario/extension
    bun run build:all
    

    Expected: "compiled with 2 warnings" (WASM size only) for each of Chrome (dist/) and Firefox (dist-firefox/).

  • P2. Test suites green:

    cd /home/alee/Sources/relicario && cargo test --workspace
    cd /home/alee/Sources/relicario/extension && bun run test
    

    Expected: 155 Rust + 124 Vitest, all pass.

  • P3. Throwaway vault ready (don't pollute your real history). Either reuse the α-validated test vault, or do a fresh chrome.storage.local clear and re-init via setup tab.

  • P4. Reference JPEG on hand for unlock.


Loading

Chrome

  • L1. chrome://extensions → "Load unpacked" → extension/dist/. (Or "Update" if already loaded — webpack regenerated everything.)
  • L2. Toolbar icon visible. Click → unlock or setup.

Firefox

  • L3. about:debugging#/runtime/this-firefox → "Load Temporary Add-on" → extension/dist-firefox/manifest.json.
  • L4. Toolbar icon visible. Click → unlock or setup.

Run the entire matrix on Chrome first, then re-run Section A (β₁ types) and Section B (β₂ surfaces) on Firefox. Section C (cross-cutting) needs to pass on both browsers.


Section A — β₁ typed-item forms

For each new type the matrix checks: add → list icon → detail render → field round-trip → edit → trash. Login was validated in α; spot-check it under A0.

A0. Login regression spot-check

  • Open popup → "+ New" → Login.
  • Expected: Form has title / url / username / password (with "gen" button) / TOTP secret (optional).
  • Fill and save; verify it appears in the list and detail-view round-trips.
  • Notes: ___

A1. SecureNote

  • Do: "+ New" → SecureNote. Title wifi. Body SSID: foo<newline>Password: bar. Save.
  • Expected list row: 📝 (or note icon) + wifi.
  • Expected detail: Body renders preserving newlines; reveal/copy works on the body.
  • Edit: Change body; save; detail reflects new body; modified time bumps.
  • Trash: Disappears from list. CLI cross-check: relicario list --trashed | grep wifi.
  • Notes: ___

A2. Identity

  • Do: "+ New" → Identity. Title Personal. Fill at least: full name, email, phone. Leave some fields empty intentionally (e.g. address).
  • Expected: Detail view renders only the fields you populated — empty fields should NOT show as blank rows. core.address === undefined not "" (verify via CLI relicario get Personal --show if curious).
  • Edit: Add a field that was previously empty; save; detail shows the new row.
  • Trash: Soft-deletes.
  • Notes: ___

A3. Card

  • Do: "+ New" → Card. Title Visa Test. Cardholder J. DOE. Number 4111111111111111 (the canonical Visa test number — brand should auto-detect to "Visa"). CVV 123. Expiry 08 / 2029. PIN 9999.
  • Expected during edit: Brand chip flips to "Visa" once 4+ digits are typed (BIN match on 4).
  • Expected detail: number/cvv/pin are concealed by default; reveal on each works; copy on each puts the value on clipboard. Expiry shows 08/2029.
  • Wire-format check (CLI): relicario get "Visa Test" --show --json | jq '.core.expiry' should be {"month":8,"year":2029} (numbers, not strings).
  • Trash: Soft-deletes.
  • Notes: ___

A4. Key

  • Do: "+ New" → Key. Title gh-deploy. Algorithm ed25519 (free-text). Paste a multi-line ASCII key into key_material (any junk is fine — -----BEGIN OPENSSH PRIVATE KEY-----\nblah\n-----END...).
  • Expected: key_material is concealed/textarea-style; reveal shows full content with line breaks intact; copy puts the multi-line value on clipboard verbatim.
  • Edit: Append to algorithm string; save; detail reflects.
  • Trash: Soft-deletes.
  • Notes: ___

A5. Totp — TOTP kind (6 digits)

  • Do: "+ New" → Totp. Title GitHub-2FA. Secret JBSWY3DPEHPK3PXP (RFC 6238 vector). Kind: TOTP.
  • Expected detail signature block: Big 6-digit code (rotates every 30s); countdown ring shrinks each tick; code refreshes at the rollover without a manual reload.
  • Cross-check: oathtool --totp -b JBSWY3DPEHPK3PXP (or any TOTP authenticator) → matches what the popup shows for the same wall-clock second.
  • Copy: "copy code" button puts current code on clipboard.
  • Notes: ___

A6. Totp — Steam Guard kind (5 alphanumeric)

  • Do: "+ New" → Totp. Title Steam. Secret JBSWY3DPEHPK3PXP (any base32 will do for the test). Toggle kind to Steam.
  • Expected: Form's digits field disappears or locks (Steam is fixed at 5).
  • Expected detail: 5-character alphanumeric code (e.g. H7K2C). All chars from the Steam alphabet 23456789BCDFGHJKMNPQRTVWXY (no 0, 1, A, E, I, O, S, U, Z, L).
  • Edit: Switch kind to TOTP, save; detail flips to 6-digit decimal. Switch back to Steam; flips back to 5-char.
  • CRITICAL: If switching kinds doesn't re-render the detail-view computed code correctly after save, that's a stale-state bug — file before continuing.
  • Notes: ___

A7. Document type — gating

  • Do: "+ New" → Document.
  • Expected: "Coming soon" placeholder (planned for γ). Back button returns to list. Should not crash or render a partial form.
  • Notes: ___

Section B — β₂ surfaces

B1. Custom fields editor — add path

  • Do: Open any item form (Login is fine). Scroll to the disclosure labeled "custom fields ▸" (or similar). Click to expand.
  • Expected: Disclosure expands; "+ section" / "+ field" controls appear.
  • Do: Add a section named recovery codes. Add two fields under it: kind=password with label code 1 value aaaa-bbbb, and kind=concealed with label code 2 value cccc-dddd. Save.
  • Expected: Detail view shows the typed Login rows first, then the recovery codes section header, then the two custom rows. Each concealed/password row has reveal + copy.
  • CLI cross-check: relicario get <item> --show --json | jq '.sections' shows the section with both fields.
  • Notes: ___

B2. Custom fields editor — edit path

  • Do: Edit the same item. In the disclosure, remove code 1, edit code 2's label to recovery hash, add a new text kind field labeled notes value worked 2024-04. Save.
  • Expected: Detail reflects all three changes (one removed, one renamed, one added).
  • Edge: A blank label field — does β₂ render as (unnamed) or reject save? (Spec says render; verify either is acceptable but consistent.)
  • Notes: ___

B3. Custom fields editor — kind sniff

  • Do: On a fresh add of an Identity item (or any type), open custom fields. Add fields of each supported kind (text / password / concealed). For each, verify in detail view: text is plain visible; password and concealed are masked with reveal/copy.
  • Expected: No reordering controls (β₂ scope), but adding a new field appends to end.
  • Notes: ___

B4. Vault settings — open path via ⚙ picker

  • Do: Click the ⚙ icon in the toolbar. β₂ split this into a picker.
  • Expected: A small menu appears with two choices — device settings (capture toggle, prompt style, blacklist) and vault settings (retention/generator/origin-acks). Pick "vault settings".
  • Expected: Vault settings screen renders with a back arrow.
  • Notes: ___

B5. Vault settings — trash retention

  • Do: In vault settings, change "trash retention" from default to 7 days.
  • Expected: Save button enables (was disabled because no diff).
  • Do: Save; lock; re-unlock; reopen vault settings.
  • Expected: Still 7 days (decrypted from the persisted VaultSettings).
  • Notes: ___

B6. Vault settings — history retention

  • Do: Change "field history retention" to last 5 (or 30 days if your build offers last_n selectors). Save.
  • Expected: Persists across lock/unlock.
  • Notes: ___

B7. Vault settings — generator preview + "configure"

  • Expected by default: Generator preview line shows current saved default (e.g. Random, 20 chars, lower+upper+digits+symbols, safe symbols).
  • Do: Click "configure ▾". Popover opens inline (anchored to the preview line).
  • Do: Change kind to BIP39. Set word count to 8. Set separator to -. Set capitalization to lower.
  • Expected: Preview-string in the popover refreshes per-keystroke (debounced); a sample generated phrase shows.
  • Do: Click "save as default". Popover closes. Preview line on the vault-settings screen now reads BIP39, 8 words, "-" separator, lower.
  • Do: Lock; re-unlock; reopen vault settings.
  • Expected: Preview still shows BIP39 default.
  • Notes: ___

B8. Generator popover — open from Login form

  • Do: "+ New" → Login. Click the "gen" button next to the password field.
  • Expected: Generator popover opens inheriting the BIP39 default from B7. Sample phrase visible.
  • Do: Click "use this value".
  • Expected: The Login form's password field gets the BIP39 phrase. Popover closes.
  • Edge: Open popover; toggle kind to Random; popover refreshes with random preview; click "use this value" — random string lands in the field. (Toggling shouldn't permanently mutate the saved default.)
  • Notes: ___

B9. Generator popover — kind toggle round-trip

  • Do: Open popover from a fresh Login form. Toggle Random ↔ BIP39 several times.
  • Expected each toggle: Preview redraws; debounced request shape switches between generate_password and generate_passphrase.
  • Smoke: No console errors on toggle.
  • Notes: ___

B10. Vault settings — origin-ack revoke

  • Pre-req: Have at least one acked origin (e.g. github.com from α step 6).
  • Do: Vault settings → scroll to "autofill acks". Find the github.com row. Click "revoke".
  • Expected: Row disappears (or shows "revoked").
  • Save (β₂ batches changes). Lock; re-unlock; reopen.
  • Expected: Row stays gone.
  • Do: Navigate to github.com/login; click the autofill icon.
  • Expected: TOFU prompt re-fires — the origin is no longer pre-acked.
  • CRITICAL: If autofill silently succeeds without re-prompting, the revoke didn't actually clear VaultSettings.autofill_origin_acks.
  • Notes: ___

B11. Vault settings — discard / no-op

  • Do: Open vault settings. Don't change anything. Click back arrow.
  • Expected: Returns to list with no save attempt (popup didn't network-request).
  • Do: Open again; change something; click back without saving.
  • Expected: Either a confirm prompt OR silent discard. Reopen; the change is gone (not persisted).
  • Notes: ___

B12. ⚙ picker — device-settings path regression

  • Do: ⚙ → "device settings".
  • Expected: The α-era device settings screen appears (capture toggle, bar/toast style, blacklist). All controls still functional.
  • Notes: ___

Section C — Cross-cutting

C1. Field history captured for new typed kinds

  • Do: Edit the Card item from A3; rotate the cvv. Save.
  • Do: Edit the Key item from A4; rotate key_material. Save.
  • Do: Edit the Totp item from A5; rotate the secret. Save.
  • Expected (CLI): relicario get <each> --show --json | jq '.field_history' has an entry for the rotated concealed/password field with old value + timestamp.
  • Notes: ___

C2. List icon parity per type

  • Do: Scroll the populated list.
  • Expected: Each row's icon matches its type. Login 🔑, SecureNote 📝, Identity 👤, Card 💳, Key 🗝, Totp ⏱ (or whatever the implementation chose — the matrix only checks consistency, not specific glyph).
  • Notes: ___

C3. Search across new types

  • Do: Use the search box; type a substring of an item title for each type.
  • Expected: Each type-specific item is findable; the type chip/icon is correct in the filtered list.
  • Notes: ___

C4. Sync / git push round-trip

  • Do: From your throwaway test vault host, after creating items A1A6 and the custom-field item from B1, run a sync from the popup (sync icon).
  • Expected: Push succeeds; git log on the test repo shows new commits.
  • Do: From CLI in main worktree, relicario sync then relicario list.
  • Expected: Same items visible. (Tests round-trip integrity of the new wire format on a real git host, not just localStorage.)
  • Notes: ___

C5. Firefox parity

  • Do: Re-run Section A (A0A7) and Section B (B1B12) on the Firefox-loaded dist-firefox/.
  • Expected: Behavior identical to Chrome.
  • Watch for: WASM-loading drift (FF uses initDefault(wasmUrl) not initSync because background.js is persistent, not SW). Anything broken on FF that works on Chrome is a WASM-init bug.
  • Notes: ___

Final acceptance

  • A1. All Section A scenarios pass on Chrome.
  • A2. All Section B scenarios pass on Chrome.
  • A3. All Section A + B scenarios pass on Firefox.
  • A4. Section C cross-cutting all pass.
  • A5. Lint sweeps green:
    git grep -n 'idfoto' extension/                                       # 0
    git grep -n '@ts-nocheck' extension/src/                              # 0
    git grep -n 'innerHTML\|insertAdjacentHTML' extension/src/content/    # 0
    git grep -n 'coming-soon\|Coming in' extension/src/popup/components/  # only 'document'
    

Findings / issues

Use this space to log anything weird. For each issue: file path + symptom + repro steps. Bug-fix commits go to main as you find them.

(fill in as you go)

Decision

  • All clean — proceed to brainstorm 1C-γ.
  • Bugs found and patched on main; re-run affected sections.
  • Bugs found that warrant a worktree (>3 commits to fix).

Generated 2026-04-24 — sources: spec 2026-04-22-relicario-extension-1c-beta1-design.md §3.9, spec 2026-04-22-relicario-extension-1c-beta2-design.md "Manual matrix", α matrix 2026-04-20-1c-alpha-manual-matrix.md.