Update all documentation for post-consolidation feature set

README.md (608 lines): - Added 11 new feature sections: extract-then-strip EXIF, federation, timestamp anchoring, selective disclosure, evidence packages, cold archives, source drop box, key rotation/recovery, cover mode - Expanded steganography (transport-aware, carrier tracking), attestation (non-image files, investigation namespaces, derivation lineage), fieldkit (forensic scrub, webhook, self-uninstall) - Added Cross-Domain Applications section (human rights, research, elections, supply chain, art, whistleblowing, environment) - Updated CLI reference with chain anchor/disclose/export commands - Updated architecture with all new modules and data directory layout CLAUDE.md (155 lines): - Added metadata.py, evidence.py, archive.py, carrier_tracker.py, anchors.py, exchange.py, dropbox blueprint to architecture tree - Added 7 new design decisions (extract-then-strip, CSRF exemption, client-side hashing, ImageHashes generalization, lazy paths, two-way federation, chain record types) docs/deployment.md (1139 lines): - Added 5 new operational sections: source drop box setup, chain anchoring procedures, cross-org federation, evidence/archive workflows, cover/duress mode - Updated killswitch section with full 10-step destruction sequence - Updated config table with all new fields - Added 5 new troubleshooting entries Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 21:36:58 -04:00
parent fef552b9c1
commit 428750e971
3 changed files with 709 additions and 53 deletions
--- a/docs/deployment.md
+++ b/docs/deployment.md
@@ -207,10 +207,12 @@ This creates the `~/.soosef/` directory structure:
    channel.key         AES-256-GCM channel key
  attestations/         Verisoo attestation log and index
  chain/                Hash chain data
+    anchors/            External timestamp anchors
  auth/                 Web UI user database (SQLite)
  certs/                Self-signed TLS certificates
  fieldkit/             Killswitch, deadman, tamper, USB, geofence state
  temp/                 Ephemeral upload/processing files
+    dropbox/            Source drop box submissions
  instance/             Flask session data
  audit.jsonl           Append-only audit trail
 ```
@@ -319,7 +321,8 @@ panel. All fields have sensible defaults -- you only need to set what you want t
 | `max_upload_mb` | `50` | Maximum file upload size in MB. |
 | `session_timeout_minutes` | `15` | Idle session expiry. Lower is safer. |
 | `login_lockout_attempts` | `5` | Failed logins before lockout. |
-| `login_lockout_minutes` | `15` | Lockout duration. |
+| `login_lockout_minutes` | `15` | Lockout duration after exceeding failed login attempts. |
+| `default_embed_mode` | `auto` | Default steganographic embedding mode for Stegasoo. |
 | `killswitch_enabled` | `false` | Enable software killswitch. |
 | `deadman_enabled` | `false` | Enable dead man's switch. |
 | `deadman_interval_hours` | `24` | Hours between required check-ins. |
@@ -329,7 +332,8 @@ panel. All fields have sensible defaults -- you only need to set what you want t
 | `tamper_monitoring_enabled` | `false` | File integrity monitoring. |
 | `chain_enabled` | `true` | Wrap attestations in a hash chain. |
 | `chain_auto_wrap` | `true` | Automatically chain verisoo attestations. |
-| `backup_reminder_days` | `7` | Warn if no backup in this many days. |
+| `backup_reminder_days` | `7` | Warn if no backup in this many days. `soosef status` reports overdue backups. |
+| `cover_name` | `""` | If set, used as the CN in the self-signed SSL certificate instead of "localhost". See Section 14 (Cover/Duress Mode). |
 | `gpio_killswitch_pin` | `17` | GPIO pin for hardware killswitch button. |
 | `gpio_killswitch_hold_seconds` | `5.0` | Required hold time to trigger hardware killswitch. |

@@ -344,8 +348,10 @@ Example minimal config for a field deployment:
  "deadman_enabled": true,
  "deadman_interval_hours": 12,
  "deadman_grace_hours": 1,
+  "deadman_warning_webhook": "https://hooks.example.com/alert",
  "killswitch_enabled": true,
-  "backup_reminder_days": 3
+  "backup_reminder_days": 3,
+  "cover_name": "Local Inventory Manager"
 }
 ```

@@ -382,8 +388,18 @@ soosef status
 ```

 The dead man's switch enforcement loop runs as a background thread inside `soosef serve`,
-checking every 60 seconds. It will send a webhook warning (if configured) during the grace
-period, then execute a full purge if the grace period expires.
+checking every 60 seconds. It will send a webhook warning (if configured via
+`deadman_warning_webhook`) during the grace period, then execute a full purge if the grace
+period expires. The webhook must be a public URL -- SSRF protection blocks private/internal
+IP ranges.
+
+For cron-based enforcement outside the web server (e.g., on a headless node), use:
+
+```bash
+soosef fieldkit check-deadman
+```
+
+Exit codes: 0 = not armed or not overdue, 1 = unexpected error, 2 = killswitch fired.

 Disarm:

@@ -407,7 +423,7 @@ Coordinates are in decimal degrees, radius in meters.
 Record currently connected USB devices as the trusted baseline:

 ```bash
-soosef fieldkit usb learn
+soosef fieldkit usb snapshot
 ```

 When monitoring is enabled, SooSeF will alert (or trigger killswitch, depending on config)
@@ -423,6 +439,48 @@ soosef fieldkit tamper baseline

 SooSeF monitors for unexpected changes to tracked files when tamper monitoring is enabled.

+### 8.5 Killswitch
+
+The killswitch destroys all key material and data in a deliberate order designed to
+maximize what is gone before any interruption. The destruction sequence is:
+
+1. **Ed25519 identity keys** (most critical -- without these, signed attestations cannot
+   be forged and encrypted data is unrecoverable)
+2. **AES-256-GCM channel key**
+3. **Flask session secret**
+4. **Auth database** (user accounts)
+5. **Attestation log and chain data**
+6. **Temp files and audit log**
+7. **Configuration**
+8. **System journal entries** for the soosef unit
+9. **Deep forensic scrub** (see below)
+10. **Self-uninstall** of the soosef pip package
+
+Trigger manually:
+
+```bash
+soosef fieldkit purge --confirm CONFIRM-PURGE
+```
+
+**Deep forensic scrub.** When the killswitch fires with `ALL` scope, it performs a deep
+forensic scrub that removes traces of SooSeF beyond the `~/.soosef/` directory:
+
+- **Python bytecache**: removes all `__pycache__` directories and `.pyc` files for
+  soosef, stegasoo, and verisoo from site-packages
+- **pip dist-info**: removes package metadata directories that would reveal what was
+  installed
+- **pip download cache**: removes cached wheels and source distributions under
+  `~/.cache/pip/` matching soosef/stegasoo/verisoo
+- **Shell history**: rewrites `~/.bash_history`, `~/.zsh_history`, and fish history to
+  remove all lines containing "soosef"
+- **Self-uninstall**: runs `pip uninstall -y soosef` to remove the package from the
+  virtual environment
+
+After a full purge, the system will show minimal evidence that SooSeF was ever installed.
+Note that this is best-effort -- filesystem journal entries, inode metadata, and flash
+wear-leveling remnants may still exist. For complete deniability, use full-disk encryption
+(LUKS) and physically destroy the storage media.
+
 ---

 ## 9. Key Management
@@ -443,7 +501,7 @@ Back up keys regularly. SooSeF warns if no backup has been taken within the
 `backup_reminder_days` window (default: 7 days).

 ```bash
-soosef keys backup --output /media/usb/soosef-backup.enc
+soosef keys export /media/usb/soosef-backup.enc
 ```

 This creates an encrypted bundle. You will be prompted for a passphrase. Store the USB
@@ -452,25 +510,25 @@ drive physically separate from the Pi.
 ### 9.2 Restore

 ```bash
-soosef keys restore --input /media/usb/soosef-backup.enc
+soosef keys import /media/usb/soosef-backup.enc
 ```

 ### 9.3 Key rotation

-Generate a new channel key (the old one is overwritten):
+Rotate the identity keypair (old key is archived, not destroyed):

 ```bash
-soosef init --no-identity
+soosef keys rotate-identity
 ```

-Generate a new identity (the old one is overwritten -- all previous attestations will
-reference the old fingerprint):
+Rotate the channel key:

 ```bash
-soosef init --no-channel
+soosef keys rotate-channel
 ```

-After rotating keys, take a fresh backup immediately.
+After rotating keys, take a fresh backup immediately. Notify all collaborators of the
+new identity fingerprint so they can update their trusted-key lists.

 ### 9.4 Trusting collaborator keys

@@ -484,7 +542,401 @@ Verify the fingerprint out-of-band (in person, over a secure channel) before tru

 ---

-## 10. Operational Security Notes
+## 10. Source Drop Box
+
+The source drop box provides a SecureDrop-like anonymous file intake that runs inside
+SooSeF. Sources do not need a SooSeF account -- they receive a one-time upload URL and
+submit files through their browser.
+
+### 10.1 Creating tokens
+
+An admin creates a time-limited upload token through the web UI at `/dropbox/admin` or
+through the admin panel. Each token has:
+
+- **Label**: a human-readable name for the source (stored server-side only)
+- **Expiry**: how many hours the link is valid (default: 24)
+- **Max files**: maximum number of uploads allowed on this link (default: 10)
+
+Tokens are 32-byte cryptographically random URL-safe strings. Once created, the admin
+receives a URL of the form `https://<host>:<port>/dropbox/upload/<token>`.
+
+### 10.2 Sharing URLs securely
+
+Share the upload URL with the source over an already-secure channel:
+
+- In-person on paper (best for high-risk sources)
+- Encrypted messaging (Signal, Wire)
+- Verbal dictation over a secure voice call
+
+Never send drop box URLs over unencrypted email or SMS.
+
+### 10.3 What happens on upload
+
+When a source uploads files:
+
+1. The browser computes SHA-256 fingerprints client-side (via SubtleCrypto) before upload
+   so the source has a verifiable record of what they submitted
+2. EXIF metadata is extracted for evidentiary fields (GPS, timestamp) and then stripped
+   from the stored copy to protect source device information
+3. The original bytes are attested (signed) before stripping, so the attestation hash
+   matches what the source actually submitted
+4. The source receives a receipt code (HMAC-derived from the file hash and token) for
+   each file
+5. The token usage counter increments; once max files is reached, the link stops accepting
+
+### 10.4 Receipt verification
+
+Sources can verify their submission was received by posting their receipt code to
+`/dropbox/verify-receipt`. This returns the filename, SHA-256, and reception timestamp if
+the receipt is valid.
+
+### 10.5 Operational security for the drop box
+
+- **No SooSeF branding**: the upload page is a minimal HTML form with no identifying
+  marks, styled generically
+- **No authentication required**: the source never creates an account or reveals
+  identity information
+- **Token self-destruction**: tokens are deleted from the SQLite database after expiry;
+  expired tokens are cleaned up on every admin page load
+- **Revocation**: admins can revoke tokens immediately from `/dropbox/admin`
+- **Tor compatibility**: the upload page is a self-contained HTML page with inline
+  JavaScript (SubtleCrypto only) and no external resources. It works over Tor Browser
+  with JavaScript enabled. No CDN, no fonts, no analytics
+- **No IP logging**: SooSeF does not log source IP addresses. Ensure your reverse proxy
+  (if any) also does not log access. If running behind Tor, the source's real IP is never
+  visible to the server
+- **Receipt codes are deterministic**: the receipt is an HMAC of the file's SHA-256 keyed
+  by the token, so the source can independently verify it corresponds to their file
+
+If operating in a high-risk environment, consider running SooSeF as a Tor hidden service
+(`.onion` address). Configure a torrc hidden service pointing to `127.0.0.1:5000` and
+share the `.onion` URL instead of a LAN address.
+
+### 10.6 Drop box file storage
+
+Uploaded files are stored in `~/.soosef/temp/dropbox/` with filenames derived from the
+SHA-256 prefix. This directory has mode 0700. Token metadata and receipts are stored in a
+SQLite database at `~/.soosef/auth/dropbox.db`.
+
+---
+
+## 11. Chain Anchoring
+
+Chain anchoring externally proves that your attestation chain existed before a given time.
+A single anchor for the chain head implicitly timestamps every record that preceded it,
+because the chain is append-only with hash linkage.
+
+### 11.1 When to anchor
+
+Anchor your chain:
+
+- Before sharing evidence with a third party (proves the chain existed before disclosure)
+- At regular intervals (daily or weekly) to establish a timeline
+- Before and after major investigations or events
+- Before key rotation (locks the existing chain state)
+
+### 11.2 Automated anchoring (RFC 3161 TSA)
+
+If the device has internet access (even temporarily), submit the chain head to a
+Timestamping Authority:
+
+```bash
+soosef chain anchor --tsa https://freetsa.org/tsr
+```
+
+This sends the chain head digest to the TSA, receives a signed timestamp token, and saves
+both the anchor and the TSA response as a JSON file under `~/.soosef/chain/anchors/`.
+
+The TSA token is a cryptographically signed proof from a third party that the hash existed
+at the timestamp. This is legally stronger than a self-asserted timestamp.
+
+### 11.3 Manual anchoring
+
+Without `--tsa`, the command exports the anchor hash for manual external submission:
+
+```bash
+soosef chain anchor
+```
+
+This prints a compact text block containing the chain ID, head index, record count, and
+digest hash. Publish this text to any external witness:
+
+- Tweet or public social media post (timestamped by the platform)
+- Email to a trusted third party (timestamped by the mail server)
+- Newspaper classified advertisement
+- Blockchain transaction (e.g., Bitcoin OP_RETURN)
+- Notarized document
+
+The anchor file is saved locally regardless of whether a TSA was used.
+
+### 11.4 Airgapped anchoring procedure
+
+For fully airgapped deployments:
+
+1. Run `soosef chain anchor` on the airgapped device
+2. Copy the printed anchor text to a USB drive (text file, photograph of screen, or
+   paper transcription)
+3. On an internet-connected device, publish the anchor text to one or more external
+   witnesses
+4. Document the publication (URL, screenshot, transaction ID) and store it alongside
+   the USB key backup
+
+### 11.5 Verifying anchors
+
+To verify that the current chain state matches a previously created anchor:
+
+```bash
+soosef chain verify
+```
+
+This checks all hash linkage and signatures in the chain. If the chain has been tampered
+with since the anchor was created, verification will fail.
+
+---
+
+## 12. Cross-Organization Federation
+
+Federation allows multiple SooSeF instances to exchange attestation records for
+collaborative investigations. Bundles are self-authenticating: each record carries the
+signer's public key, so the importer can verify signatures against their trust store.
+
+### 12.1 Exchanging trust keys
+
+Before two organizations can exchange attestation bundles, they must trust each other's
+identity keys.
+
+On Organization A:
+
+```bash
+# Export public key
+cp ~/.soosef/identity/public.pem /media/usb/org-a-pubkey.pem
+```
+
+On Organization B:
+
+```bash
+# Import Org A's key and verify fingerprint
+soosef keys trust --import /media/usb/org-a-pubkey.pem
+```
+
+Always verify fingerprints out-of-band (in person, over a known-secure voice channel).
+Repeat in both directions so each organization trusts the other.
+
+### 12.2 Exporting attestation bundles
+
+Export a JSON bundle containing attestation records and chain data:
+
+```bash
+soosef chain export --output /media/usb/investigation-bundle.zip
+```
+
+To export only records from a specific index range:
+
+```bash
+soosef chain export --start 100 --end 200 --output /media/usb/partial-bundle.zip
+```
+
+The export includes:
+
+- Attestation records with full signatures
+- Chain records with hash linkage
+- The signer's public key
+- A standalone verification script (requires only Python + `cryptography`)
+- A human-readable README
+
+### 12.3 Importing attestation bundles
+
+On the receiving organization's SooSeF instance:
+
+- Records are imported into the local attestation log with a `federated_from` metadata tag
+- Records signed by untrusted fingerprints are rejected (unless trust-on-first-use is used)
+- Duplicate records (matching SHA-256) are skipped automatically
+
+### 12.4 Delivery acknowledgments
+
+When a bundle is imported and the receiving instance has a chain store and private key,
+SooSeF automatically creates a delivery acknowledgment record in the local chain. This
+records the bundle hash, sender fingerprint, and count of records received. The
+acknowledgment provides a cryptographic receipt that the bundle was delivered and ingested.
+
+### 12.5 Selective disclosure for legal proceedings
+
+To produce evidence for a court order or legal discovery request without revealing the
+entire chain:
+
+```bash
+soosef chain disclose --indices 42,43,44 --output disclosure.json
+```
+
+This exports a proof bundle where the selected records are shown in full and all other
+records appear only as hashes. A third party can verify that the selected records are part
+of an unbroken hash chain without seeing the contents of other records.
+
+### 12.6 Federation on airgapped networks
+
+All federation is designed for offline/sneakernet operation:
+
+1. Export the bundle to a USB drive on the sending instance
+2. Physically carry the USB to the receiving instance
+3. Import the bundle
+4. Export the delivery acknowledgment back on USB if needed
+
+No network connectivity is required at any point.
+
+---
+
+## 13. Evidence Packages and Cold Archives
+
+SooSeF provides two export formats for preserving evidence outside the running instance.
+
+### 13.1 Evidence packages
+
+An evidence package is a self-contained ZIP that bundles everything needed for independent
+verification of specific attested images. Use evidence packages when you need to hand off
+a subset of evidence to a third party (lawyer, court, partner organization).
+
+Contents of an evidence package:
+
+- `images/` -- original image files
+- `manifest.json` -- attestation records and chain data for those images
+- `public_key.pem` -- signer's Ed25519 public key
+- `verify.py` -- standalone verification script (requires only Python 3.11+ and
+  `cryptography`)
+- `README.txt` -- human-readable instructions
+
+The package is self-contained. No SooSeF installation is required to verify the evidence.
+The standalone `verify.py` script checks image SHA-256 hashes against attestation records
+and verifies chain hash linkage.
+
+**When to create evidence packages:**
+
+- Before handing evidence to a legal team
+- When sharing with a partner organization that does not run SooSeF
+- For court submission (the self-contained verifier is the key feature)
+- Before any action that might destroy the running instance (travel through hostile
+  checkpoints, anticipated raids)
+
+### 13.2 Cold archives
+
+A cold archive is a full snapshot of the entire SooSeF evidence store, designed for
+long-term preservation. It follows OAIS (ISO 14721) alignment: the archive is
+self-describing, includes its own verification code, and documents the cryptographic
+algorithms used.
+
+Contents of a cold archive:
+
+- `chain/` -- raw append-only hash chain binary, state checkpoint, and anchor files
+- `attestations/` -- full verisoo attestation log and LMDB index
+- `keys/public.pem` -- signer's public key
+- `keys/bundle.enc` -- encrypted key bundle (optional, password-protected)
+- `keys/trusted/` -- trusted collaborator public keys
+- `manifest.json` -- archive metadata and integrity hashes
+- `verify.py` -- standalone verification script
+- `ALGORITHMS.txt` -- documents all cryptographic algorithms and formats used (Ed25519,
+  SHA-256, AES-256-GCM, Argon2id, CBOR, etc.) so the archive remains verifiable even if
+  SooSeF no longer exists
+- `README.txt` -- human-readable description
+
+To restore a cold archive on a fresh SooSeF instance:
+
+```bash
+soosef archive import <archive.zip>
+```
+
+**When to create cold archives:**
+
+- At regular intervals (weekly or monthly) as part of your backup strategy
+- Before key rotation
+- Before traveling with the device
+- Before anticipated risk events
+- When archiving a completed investigation
+
+### 13.3 Legal discovery workflow
+
+For legal discovery and court proceedings:
+
+1. Use `soosef chain disclose` for selective disclosure (Section 12.5) when you must
+   respond to a specific request without revealing the full chain
+2. Use evidence packages for handing specific images and their attestations to counsel
+3. Use cold archives when full preservation is required
+
+All three formats include standalone verification scripts so that the receiving party does
+not need to install SooSeF.
+
+### 13.4 Long-term archival best practices
+
+- Store cold archives on at least two separate physical media (USB drives, optical discs)
+- Keep one copy offsite (safe deposit box, trusted third party)
+- Include the encrypted key bundle in the archive (set a strong passphrase and store it
+  separately from the archive media)
+- Write the passphrase on paper and store it in a different physical location from the
+  archive media
+- Periodically verify archive integrity: unzip and run `python verify.py`
+- The `ALGORITHMS.txt` file documents everything needed to write a verifier from scratch,
+  even decades from now
+
+---
+
+## 14. Cover/Duress Mode
+
+Cover mode disguises a SooSeF installation so that casual inspection of the device does
+not immediately reveal it as a security toolkit.
+
+### 14.1 Renaming the data directory
+
+By default, SooSeF stores everything under `~/.soosef/`. To use an inconspicuous name,
+set the `SOOSEF_DATA_DIR` environment variable:
+
+```bash
+export SOOSEF_DATA_DIR=~/.local/share/inventory
+soosef init
+```
+
+All SooSeF commands respect this variable. Add it to the soosef user's shell profile or
+the systemd service file:
+
+```ini
+Environment="SOOSEF_DATA_DIR=/home/soosef/.local/share/inventory"
+```
+
+You can also pass `--data-dir` to any command:
+
+```bash
+soosef --data-dir ~/.local/share/inventory serve --host 0.0.0.0
+```
+
+### 14.2 Cover name for SSL certificates
+
+Set `cover_name` in config to change the Common Name (CN) in the self-signed SSL
+certificate. Without this, the certificate CN defaults to "localhost". With a cover name,
+a browser inspector sees a plausible-looking certificate:
+
+```json
+{
+  "cover_name": "Local Inventory Manager"
+}
+```
+
+Delete `~/.soosef/certs/cert.pem` and restart the server to regenerate the certificate
+with the new CN.
+
+### 14.3 Portable USB operation
+
+SooSeF can run entirely from a USB drive:
+
+1. Install SooSeF into a virtualenv on the USB drive
+2. Set `SOOSEF_DATA_DIR` to a directory on the USB
+3. Run `soosef serve` from the USB
+
+When you remove the USB, no trace of SooSeF remains on the host machine (assuming no
+swap, no core dumps, and the host filesystem was not used for temp files).
+
+Combine with the `cover_name` setting and a renamed data directory for maximum
+deniability.
+
+---
+
+## 15. Operational Security Notes

 SooSeF is a tool, not a shield. Understand what it cannot do.

@@ -541,7 +993,7 @@ This is a fundamental limitation of Python-based security tools.

 ---

-## 11. Troubleshooting
+## 16. Troubleshooting

 ### Health check

@@ -561,7 +1013,7 @@ soosef status
 ```

 This checks identity key, channel key, trusted keys, dead man's switch state, geofence,
-chain status, and backup status.
+chain status, and backup status. Use `--json` for machine-readable output.

 ### Common issues

@@ -636,3 +1088,52 @@ Always run SooSeF as the same user.
 ```bash
 ls -la ~/.soosef/
 ```
+
+**Drop box tokens expire immediately**
+
+Token expiry is checked against UTC. If the Pi's system clock is wrong, tokens may appear
+expired as soon as they are created. Verify the clock:
+
+```bash
+date -u
+```
+
+On airgapped systems without NTP, set the clock manually before creating tokens:
+
+```bash
+sudo date -s "2026-04-01 12:00:00"
+```
+
+**Chain anchor TSA submission fails**
+
+TSA submission requires network access. On an airgapped device, use manual anchoring
+instead (`soosef chain anchor` without `--tsa`). If the TSA URL is unreachable, the anchor
+is still saved locally -- only the external timestamp token is missing.
+
+**SSL certificate shows wrong name**
+
+If you set `cover_name` after the certificate was already generated, delete the old
+certificate and restart:
+
+```bash
+rm ~/.soosef/certs/cert.pem ~/.soosef/certs/key.pem
+sudo systemctl restart soosef
+```
+
+**Account lockout after repeated failed logins**
+
+After `login_lockout_attempts` (default: 5) failed login attempts, the account is locked
+for `login_lockout_minutes` (default: 15) minutes. Wait for the lockout to expire, or
+restart the server to clear lockout state.
+
+**Evidence package verify.py fails**
+
+The standalone verification script requires Python 3.11+ and the `cryptography` package.
+Install it in a fresh virtualenv on the verifying machine:
+
+```bash
+python3 -m venv verify-env
+source verify-env/bin/activate
+pip install cryptography
+python verify.py
+```