6325e86873
README.md (700 lines): - Three-tier deployment model with ASCII diagram - Federation blueprint in web UI routes - deploy/ directory in architecture tree - Documentation index linking all guides CLAUDE.md (256 lines): - Updated architecture tree with all new docs and deploy files New guides: - docs/federation.md (317 lines) — gossip protocol mechanics, peer setup, trust filtering, offline bundles, relay deployment, jurisdiction - docs/evidence-guide.md (283 lines) — evidence packages, cold archives, selective disclosure, chain anchoring, legal discovery workflow - docs/source-dropbox.md (220 lines) — token management, client-side hashing, extract-then-strip pipeline, receipt mechanics, opsec - docs/index.md — documentation hub linking all guides Training materials: - docs/training/reporter-quickstart.md (105 lines) — printable one-page card: boot USB, attest photo, encode message, check-in, emergency - docs/training/emergency-card.md (79 lines) — wallet-sized laminated card: three destruction methods, 10-step order, key contacts - docs/training/admin-reference.md (219 lines) — deployment tiers, CLI tables, backup checklist, hardening checklist, troubleshooting Also includes existing architecture docs from the original repos. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
257 lines
14 KiB
Markdown
257 lines
14 KiB
Markdown
# SooSeF -- Claude Code Project Guide
|
|
|
|
SooSeF (Soo Security Fieldkit) is an offline-first security toolkit for journalists, NGOs,
|
|
and at-risk organizations. Monorepo consolidating Stegasoo and Verisoo as subpackages.
|
|
|
|
Version 0.2.0 · Python >=3.11 · MIT License
|
|
|
|
## Quick commands
|
|
|
|
```bash
|
|
# Development install (single command -- stegasoo and verisoo are inlined subpackages)
|
|
pip install -e ".[dev]"
|
|
|
|
pytest # Run tests
|
|
black src/ tests/ frontends/ # Format code
|
|
ruff check src/ tests/ frontends/ --fix # Lint
|
|
mypy src/ # Type check
|
|
```
|
|
|
|
### Pip extras
|
|
|
|
`stego-dct`, `stego-audio`, `stego-compression`, `attest`, `cli`, `web`, `api`,
|
|
`fieldkit`, `federation`, `rpi`, `all`, `dev`
|
|
|
|
## Architecture
|
|
|
|
```
|
|
src/soosef/ Core library
|
|
__init__.py Package init, __version__ (0.2.0)
|
|
_availability.py Runtime checks for optional subpackages (has_stegasoo, has_verisoo)
|
|
api.py Optional unified FastAPI app (uvicorn soosef.api:app)
|
|
audit.py Append-only JSON-lines audit log (~/.soosef/audit.jsonl)
|
|
cli.py Click CLI entry point (soosef command)
|
|
paths.py All ~/.soosef/* path constants (single source of truth, lazy resolution)
|
|
config.py Unified config loader (SoosefConfig dataclass + JSON)
|
|
exceptions.py SoosefError, ChainError, ChainIntegrityError, ChainAppendError, KeystoreError
|
|
metadata.py Extract-then-strip EXIF pipeline with field classification
|
|
evidence.py Self-contained evidence package export (ZIP with verify.py)
|
|
archive.py Cold archive export for long-term preservation (OAIS-aligned)
|
|
|
|
stegasoo/ Steganography engine (inlined from stegasoo v4.3.0)
|
|
encode.py / decode.py Core encode/decode API
|
|
generate.py Cover image generation
|
|
crypto.py AES-256-GCM encryption, channel fingerprints
|
|
channel.py Channel key derivation + management
|
|
steganography.py LSB steganography core
|
|
dct_steganography.py DCT-domain JPEG steganography
|
|
spread_steganography.py MDCT spread-spectrum audio steganography
|
|
audio_steganography.py Audio stego pipeline
|
|
video_steganography.py Video stego pipeline
|
|
backends/ Pluggable stego backend registry (lsb, dct, protocol)
|
|
batch.py Batch operations
|
|
compression.py Payload compression (zstd, lz4)
|
|
steganalysis.py Detection resistance analysis
|
|
validation.py Input validation
|
|
models.py Data models
|
|
constants.py Magic bytes, version constants, AUDIO_ENABLED, VIDEO_ENABLED
|
|
cli.py Stegasoo-specific CLI commands
|
|
api.py / api_auth.py Stegasoo REST API + auth
|
|
carrier_tracker.py Carrier image reuse tracking (warns on reuse)
|
|
platform_presets.py Social-media-aware encoding presets
|
|
image_utils.py / audio_utils.py / video_utils.py
|
|
keygen.py / qr_utils.py / recovery.py / debug.py / utils.py
|
|
|
|
verisoo/ Provenance attestation engine (inlined from verisoo v0.1.0)
|
|
attestation.py Core attestation creation + EXIF extraction
|
|
verification.py Attestation verification
|
|
crypto.py Ed25519 signing
|
|
hashing.py Perceptual + cryptographic hashing (ImageHashes)
|
|
embed.py Attestation embedding in images
|
|
merkle.py Merkle tree + consistency/inclusion proofs
|
|
binlog.py Binary attestation log
|
|
lmdb_store.py LMDB-backed trust store
|
|
storage.py Attestation storage abstraction (LocalStorage)
|
|
federation.py GossipNode, HttpTransport, PeerInfo, SyncStatus
|
|
peer_store.py SQLite-backed peer persistence for federation
|
|
models.py Attestation, AttestationRecord, ImageHashes, Identity
|
|
exceptions.py VerisooError, AttestationError, VerificationError, FederationError
|
|
cli.py Verisoo-specific CLI commands
|
|
api.py Verisoo REST API + federation endpoints
|
|
|
|
federation/ Federated attestation chain system
|
|
chain.py ChainStore -- append-only hash chain with key rotation/recovery/delivery-ack
|
|
entropy.py Entropy source for chain seeds (sys_uptime, fs_snapshot, proc_entropy, boot_id)
|
|
models.py AttestationChainRecord, ChainState, EntropyWitnesses (frozen dataclasses)
|
|
serialization.py CBOR canonical encoding + compute_record_hash + serialize/deserialize
|
|
anchors.py RFC 3161 timestamps + manual chain anchors
|
|
exchange.py Cross-org attestation bundle export/import (JSON bundles)
|
|
|
|
keystore/ Unified key management
|
|
manager.py KeystoreManager -- owns all key material (channel + identity + trust store + backup)
|
|
models.py IdentityInfo, KeystoreStatus, RotationResult dataclasses
|
|
export.py Encrypted key bundle export/import (SOOBNDL format)
|
|
|
|
fieldkit/ Field security features
|
|
killswitch.py Emergency data destruction (PurgeScope.KEYS_ONLY | ALL, deep forensic scrub)
|
|
deadman.py Dead man's switch (webhook warning + auto-purge)
|
|
tamper.py File integrity monitoring (baseline snapshots)
|
|
usb_monitor.py USB device whitelist (Linux/pyudev)
|
|
geofence.py GPS boundary enforcement (gpsd integration)
|
|
|
|
frontends/web/ Unified Flask web UI
|
|
app.py App factory (create_app()), ~36k -- mounts all blueprints
|
|
auth.py SQLite3 multi-user auth with lockout + rate limiting
|
|
temp_storage.py File-based temp storage with expiry
|
|
subprocess_stego.py Crash-safe subprocess isolation for stegasoo
|
|
stego_worker.py Background stego processing
|
|
stego_routes.py Stego route helpers (~87k)
|
|
ssl_utils.py Self-signed HTTPS cert generation (cover_name support)
|
|
blueprints/
|
|
stego.py /encode, /decode, /generate
|
|
attest.py /attest, /verify (~25k -- handles images + arbitrary files)
|
|
fieldkit.py /fieldkit/* (killswitch, deadman, status)
|
|
keys.py /keys/* (unified key management, trust store)
|
|
admin.py /admin/* (user management)
|
|
dropbox.py /dropbox/* (token-gated anonymous source upload, ~13k)
|
|
federation.py /federation/* (peer status dashboard, peer add/remove)
|
|
templates/
|
|
dropbox/admin.html Drop box admin panel
|
|
federation/status.html Federation peer dashboard
|
|
|
|
frontends/cli/ CLI package init (main entry point is src/soosef/cli.py)
|
|
|
|
deploy/ Deployment artifacts
|
|
docker/ Dockerfile (multi-stage: builder, relay, server) + docker-compose.yml
|
|
kubernetes/ namespace.yaml, server-deployment.yaml, relay-deployment.yaml
|
|
live-usb/ build.sh + config/ for Debian Live USB (Tier 1)
|
|
config-presets/ low-threat.json, medium-threat.json, high-threat.json, critical-threat.json
|
|
|
|
docs/ Documentation
|
|
deployment.md Three-tier deployment guide (~1500 lines)
|
|
federation.md Gossip protocol, peer setup, offline bundles, CLI commands
|
|
evidence-guide.md Evidence packages, cold archives, selective disclosure
|
|
source-dropbox.md Source drop box setup: tokens, EXIF pipeline, receipts
|
|
architecture/
|
|
federation.md System architecture overview (threat model, layers, key domains)
|
|
chain-format.md Chain record spec (CBOR, entropy witnesses, serialization)
|
|
export-bundle.md Export bundle spec (SOOSEFX1 binary format, envelope encryption)
|
|
federation-protocol.md Federation server protocol (CT-inspired, gossip, storage tiers)
|
|
training/
|
|
reporter-quickstart.md One-page reporter quick-start for Tier 1 USB (print + laminate)
|
|
reporter-field-guide.md Comprehensive reporter guide: attest, stego, killswitch, evidence
|
|
emergency-card.md Laminated wallet card: emergency destruction reference
|
|
admin-reference.md Admin CLI cheat sheet, hardening checklist, troubleshooting
|
|
admin-operations-guide.md Full admin operations: users, dropbox, federation, incidents
|
|
```
|
|
|
|
## Three-tier deployment model
|
|
|
|
```
|
|
Tier 1: Field Device Tier 2: Org Server Tier 3: Federation Relay
|
|
(Bootable USB + laptop) (Docker on mini PC / VPS) (Docker on VPS)
|
|
Amnesic, LUKS-encrypted Persistent, full features Attestation sync only
|
|
Reporter in the field Newsroom / NGO office Friendly jurisdiction
|
|
```
|
|
|
|
- Tier 1 <-> Tier 2: sneakernet (USB drive) or LAN
|
|
- Tier 2 <-> Tier 3: federation API (port 8000) over internet
|
|
- Tier 2 <-> Tier 2: via Tier 3 relay, or directly via sneakernet
|
|
|
|
## Dependency model
|
|
|
|
Stegasoo and Verisoo are inlined subpackages, not separate pip packages:
|
|
- `from soosef.stegasoo import encode` for steganography
|
|
- `from soosef.verisoo import Attestation` for provenance attestation
|
|
- Never `import stegasoo` or `import verisoo` directly
|
|
- `_availability.py` provides `has_stegasoo()` / `has_verisoo()` for graceful degradation
|
|
when optional extras are not installed
|
|
|
|
## Key design decisions
|
|
|
|
- **Two key domains, never merged**: Stegasoo AES-256-GCM (derived from factors) and
|
|
Verisoo Ed25519 (signing identity) are separate security concerns
|
|
- **Extract-then-strip model**: Stego strips all EXIF (carrier is vessel); attestation
|
|
extracts evidentiary EXIF (GPS, timestamp) then strips dangerous fields (device serial)
|
|
- **subprocess_stego.py copies verbatim** from stegasoo -- it's a crash-safety boundary
|
|
- **All state under ~/.soosef/** -- one directory to back up, one to destroy.
|
|
`SOOSEF_DATA_DIR` env var relocates everything (cover mode, USB mode)
|
|
- **Offline-first**: All static assets vendored, no CDN. pip wheels bundled for airgap install
|
|
- **Flask blueprints**: stego, attest, fieldkit, keys, admin, dropbox, federation
|
|
- **Flask-WTF**: CSRF protection on all form endpoints; drop box is CSRF-exempt (sources
|
|
don't have sessions)
|
|
- **Client-side SHA-256**: Drop box upload page uses SubtleCrypto for pre-upload hashing
|
|
- **Waitress**: Production WSGI server (replaces dev-only Flask server)
|
|
- **FastAPI option**: `soosef.api` provides a REST API alternative to the Flask web UI
|
|
- **Pluggable backends**: Stego backends (LSB, DCT) registered via `backends/registry.py`
|
|
- **ImageHashes generalized**: phash/dhash now optional, enabling non-image attestation
|
|
- **Lazy path resolution**: All paths in paths.py resolve lazily via `__getattr__` from
|
|
`BASE_DIR` so that runtime overrides (--data-dir, SOOSEF_DATA_DIR) propagate correctly
|
|
- **Two-way federation**: Delivery acknowledgment records (`soosef/delivery-ack-v1`)
|
|
enable handshake proof
|
|
- **Chain record types** (in federation/chain.py):
|
|
- `CONTENT_TYPE_KEY_ROTATION = "soosef/key-rotation-v1"` -- signed by OLD key
|
|
- `CONTENT_TYPE_KEY_RECOVERY = "soosef/key-recovery-v1"` -- signed by NEW key
|
|
- `CONTENT_TYPE_DELIVERY_ACK = "soosef/delivery-ack-v1"` -- signed by receiver
|
|
- **Gossip federation** (verisoo/federation.py): GossipNode with async peer sync,
|
|
consistency proofs, HttpTransport over aiohttp. PeerStore for SQLite-backed persistence
|
|
- **Threat level presets**: deploy/config-presets/ with low/medium/high/critical configs
|
|
- **Selective disclosure**: Chain records can be exported with non-selected records
|
|
redacted to hashes only (for legal discovery)
|
|
- **Evidence packages**: Self-contained ZIPs with verify.py that needs only Python + cryptography
|
|
- **Cold archives**: OAIS-aligned full-state export with ALGORITHMS.txt
|
|
- **Transport-aware stego**: --transport whatsapp|signal|telegram auto-selects DCT/JPEG
|
|
and pre-resizes carrier for platform survival
|
|
|
|
## Data directory layout (`~/.soosef/`)
|
|
|
|
```
|
|
~/.soosef/
|
|
config.json Unified configuration (SoosefConfig dataclass)
|
|
audit.jsonl Append-only audit trail (JSON-lines)
|
|
carrier_history.json Carrier reuse tracking database
|
|
identity/ Ed25519 keypair (private.pem, public.pem, identity.meta.json)
|
|
archived/ Timestamped old keypairs from rotations
|
|
stegasoo/ Channel key (channel.key)
|
|
archived/ Timestamped old channel keys from rotations
|
|
attestations/ Verisoo attestation store
|
|
log.bin Binary attestation log
|
|
index/ LMDB index
|
|
peers.json Legacy peer file
|
|
federation/ Federation state
|
|
peers.db SQLite peer + sync history
|
|
chain/ Hash chain (chain.bin, state.cbor)
|
|
anchors/ External timestamp anchors (JSON files)
|
|
auth/ Web UI auth databases
|
|
soosef.db User accounts
|
|
dropbox.db Drop box tokens + receipts
|
|
certs/ Self-signed TLS certificates (cert.pem, key.pem)
|
|
fieldkit/ Fieldkit state
|
|
deadman.json Dead man's switch timer
|
|
tamper/baseline.json File integrity baselines
|
|
usb/whitelist.json USB device whitelist
|
|
geofence.json GPS boundary config
|
|
temp/ Ephemeral file storage
|
|
dropbox/ Source drop box submissions
|
|
instance/ Flask instance (sessions, .secret_key)
|
|
trusted_keys/ Collaborator Ed25519 public keys (trust store)
|
|
<fingerprint>/ Per-key directory (public.pem + meta.json)
|
|
last_backup.json Backup timestamp tracking
|
|
```
|
|
|
|
## Code conventions
|
|
|
|
Black (100-char), Ruff (E, F, I, N, W, UP), mypy (strict, ignore_missing_imports),
|
|
imperative commit messages.
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
pytest # All tests with coverage
|
|
pytest tests/test_chain.py # Chain-specific
|
|
```
|
|
|
|
Test files: `test_chain.py`, `test_chain_security.py`, `test_deadman_enforcement.py`,
|
|
`test_key_rotation.py`, `test_killswitch.py`, `test_serialization.py`,
|
|
`test_stegasoo_audio.py`, `test_stegasoo.py`, `test_verisoo_hashing.py`
|