490f9d4a1d
Complete project rebrand for better positioning in the press freedom and digital security space. FieldWitness communicates both field deployment and evidence testimony — appropriate for the target audience of journalists, NGOs, and human rights organizations. Rename mapping: - soosef → fieldwitness (package, CLI, all imports) - soosef.stegasoo → fieldwitness.stego - soosef.verisoo → fieldwitness.attest - ~/.soosef/ → ~/.fwmetadata/ (innocuous data dir name) - SOOSEF_DATA_DIR → FIELDWITNESS_DATA_DIR - SoosefConfig → FieldWitnessConfig - SoosefError → FieldWitnessError Also includes: - License switch from MIT to GPL-3.0 - C2PA bridge module (Phase 0-2 MVP): cert.py, export.py, vendor_assertions.py - README repositioned to lead with provenance/federation, stego backgrounded - Threat model skeleton at docs/security/threat-model.md - Planning docs: docs/planning/c2pa-integration.md, docs/planning/gtm-feasibility.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
762 lines
32 KiB
Markdown
762 lines
32 KiB
Markdown
# FieldWitness -- FieldWitness
|
|
|
|
**Offline-first provenance attestation with gossip federation for journalists, NGOs, and at-risk organizations.**
|
|
|
|
<!-- badges -->
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
## What is FieldWitness?
|
|
|
|
FieldWitness is a field-deployable evidence integrity system. It lets journalists, human rights
|
|
documenters, and NGOs establish cryptographic chain-of-custody over photos, documents, and
|
|
sensor data -- in airgapped environments, across organizational boundaries, and in adversarial
|
|
conditions where data may need to be destroyed on demand.
|
|
|
|
The core claim: a file attested with FieldWitness can be handed to a court or partner organization
|
|
with proof that it has not been altered, proof of when it was captured (anchored externally
|
|
via RFC 3161), and proof of who held it -- without requiring the verifying party to install
|
|
FieldWitness or trust any central authority.
|
|
|
|
**Three-tier deployment model:**
|
|
|
|
```
|
|
Tier 1: Field Device Tier 2: Org Server Tier 3: Federation Relay
|
|
(Bootable USB + laptop) (Docker on mini PC / VPS) (Docker on VPS)
|
|
|
|
Reporter in the field Newsroom / NGO office Friendly jurisdiction
|
|
Amnesic, LUKS-encrypted Persistent storage Attestation sync only
|
|
Pull USB = zero trace Web UI + federation API Zero knowledge of keys
|
|
\ | /
|
|
\_____ sneakernet ____+____ gossip API ____/
|
|
```
|
|
|
|
Stego (steganography, v4.3.0) and Attest (attestation, v0.1.0) are included as
|
|
subpackages (`import fieldwitness.stego`, `import fieldwitness.attest`). Everything ships as one
|
|
install: `pip install fieldwitness`.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
pip install "fieldwitness[web,cli]"
|
|
fieldwitness init
|
|
fieldwitness serve
|
|
```
|
|
|
|
This creates the `~/.fieldwitness/` directory structure, generates an Ed25519 identity and
|
|
channel key, writes a default config, and starts an HTTPS web UI on
|
|
`https://127.0.0.1:5000`.
|
|
|
|
---
|
|
|
|
## Features
|
|
|
|
### Attestation and Provenance (Attest)
|
|
|
|
- Ed25519 digital signatures for images and arbitrary files (CSV, documents, sensor data)
|
|
- Perceptual hashing (pHash, dHash) for tamper-evident photo attestation -- identifies
|
|
re-uploaded or re-compressed copies. SHA-256-only mode for non-image files
|
|
- Append-only hash chain (CBOR-encoded) with Merkle tree verification -- every attestation
|
|
is chained to all prior attestations, making retroactive tampering detectable
|
|
- LMDB-backed attestation storage
|
|
- Batch attestation for directories
|
|
- **Investigation namespaces** -- tag and filter attestations by case or project
|
|
- **Derived work lineage** -- parent-child attestation tracking for editorial workflows
|
|
- **Chain position proof** -- verification receipts include the record's position in the
|
|
hash chain
|
|
|
|
### Extract-Then-Strip EXIF Pipeline
|
|
|
|
Resolves the tension between protecting sources (strip everything) and proving provenance
|
|
(preserve everything):
|
|
|
|
1. Extract all EXIF metadata from the original image bytes
|
|
2. Classify fields as **evidentiary** (GPS coordinates, timestamp -- valuable for
|
|
provenance) or **dangerous** (device serial number, firmware version -- could identify
|
|
the source)
|
|
3. Preserve evidentiary fields in the attestation record
|
|
4. Strip all metadata from the stored/display copy
|
|
|
|
### Gossip Federation
|
|
|
|
- **Gossip protocol** -- nodes periodically exchange Merkle roots with peers. Divergence
|
|
triggers a consistency proof request and incremental record fetch. No central
|
|
coordinator, no consensus, no leader election -- just append-only logs that converge
|
|
- **Attestation exchange** -- export signed bundles of attestation records and chain data
|
|
for offline transfer (sneakernet) to partner organizations
|
|
- **Delivery acknowledgments** -- when an organization imports a bundle, a
|
|
`fieldwitness/delivery-ack-v1` chain record is signed and can be shared back, creating a
|
|
two-way federation handshake
|
|
- **Trust store** -- import collaborator Ed25519 public keys; only records signed by
|
|
trusted keys are imported during federation
|
|
- **Investigation filtering** -- export/import only records tagged with a specific
|
|
investigation
|
|
|
|
### Evidence Packages
|
|
|
|
Self-contained ZIP bundles for handing evidence to lawyers, courts, or archives:
|
|
|
|
- Original images
|
|
- Attestation records with full Ed25519 signatures
|
|
- Chain segment with hash linkage
|
|
- Signer's public key
|
|
- `verify.py` -- standalone verification script that requires only Python 3.11+ and the
|
|
`cryptography` pip package (no FieldWitness installation needed)
|
|
- Human-readable README
|
|
|
|
### Cold Archive
|
|
|
|
Full-state export for long-term evidence preservation (10+ year horizon), aligned with
|
|
OAIS (ISO 14721):
|
|
|
|
- Raw chain binary and state checkpoint
|
|
- Attestation log and LMDB index
|
|
- External timestamp anchors
|
|
- Public key and trusted collaborator keys
|
|
- Encrypted key bundle (optional, password-protected)
|
|
- `ALGORITHMS.txt` documenting every cryptographic algorithm, parameter, and format used
|
|
- `verify.py` standalone verifier
|
|
- `manifest.json` with SHA-256 integrity hashes of key files
|
|
|
|
### External Timestamp Anchoring
|
|
|
|
Two mechanisms to prove the chain head existed before a given time:
|
|
|
|
- **RFC 3161 TSA** -- automated submission to any RFC 3161 Timestamping Authority (e.g.,
|
|
FreeTSA). The signed timestamp token is saved alongside the chain
|
|
- **Manual anchors** -- export the chain head hash as a compact string for manual
|
|
submission to any external witness (blockchain transaction, newspaper classified, email
|
|
to a TSA)
|
|
|
|
A single anchor for the chain head implicitly timestamps every record that preceded it,
|
|
because the chain is append-only with hash linkage.
|
|
|
|
### Selective Disclosure
|
|
|
|
Produce verifiable proofs for specific chain records while keeping others redacted.
|
|
Selected records are included in full; non-selected records appear only as hashes. A third
|
|
party can verify that the disclosed records are part of an unbroken chain without seeing
|
|
the contents of other records. Designed for legal discovery, court orders, and FOIA
|
|
responses.
|
|
|
|
### Field Security (Fieldkit)
|
|
|
|
- **Killswitch** -- emergency destruction of all data under `~/.fieldwitness/`, ordered by
|
|
sensitivity (keys first, then data, then logs). Includes:
|
|
- **Deep forensic scrub** -- removes `__pycache__`, `.pyc`, pip `dist-info`, pip
|
|
download cache, and scrubs shell history entries containing "fieldwitness"
|
|
- **Self-uninstall** -- runs `pip uninstall -y fieldwitness` as the final step
|
|
- **System log clearing** -- best-effort journald vacuum on Linux
|
|
- **Dead man's switch** -- automated purge if check-in is missed, with a configurable
|
|
grace period. During the grace period, a **webhook warning** is sent (POST to a
|
|
configured URL) and a local warning file is written before the killswitch fires
|
|
- **Tamper detection** -- file integrity monitoring with baseline snapshots
|
|
- **USB whitelist** -- block or alert on unauthorized USB devices (Linux/pyudev)
|
|
- **Geofence** -- GPS boundary enforcement with configurable radius. Supports live GPS via
|
|
**gpsd** (`get_current_location()` connects to `127.0.0.1:2947`)
|
|
- **Hardware killswitch** -- GPIO pin monitoring for Raspberry Pi physical button
|
|
(configurable pin and hold duration)
|
|
|
|
### Source Drop Box
|
|
|
|
SecureDrop-style anonymous intake built into the FieldWitness web UI:
|
|
|
|
- Admin creates a time-limited upload token with a configurable file limit
|
|
- Source opens the token URL (no account or FieldWitness branding -- source safety)
|
|
- **Client-side SHA-256** via SubtleCrypto runs in the browser before upload, so the
|
|
source can independently verify what they submitted
|
|
- Files are run through the extract-then-strip EXIF pipeline and auto-attested on receipt
|
|
- Source receives HMAC-derived receipt codes that prove delivery
|
|
- Tokens and receipts are stored in SQLite; tokens auto-expire
|
|
|
|
### Key Rotation and Recovery
|
|
|
|
- **Key rotation** -- both identity (Ed25519) and channel (AES) keys can be rotated. The
|
|
chain records the rotation as a `fieldwitness/key-rotation-v1` record signed by the OLD key,
|
|
creating a cryptographic trust chain
|
|
- **Identity recovery** -- after device loss, a new key can be introduced via a
|
|
`fieldwitness/key-recovery-v1` chain record. The record carries the old fingerprint and
|
|
optional cosigner fingerprints for audit
|
|
- **Channel key only export** -- share just the channel key (not identity keys) with
|
|
collaborators via encrypted file or QR code (`fieldwitness-channel:` URI scheme)
|
|
- **Backup tracking** -- records when the last backup was taken and warns when overdue
|
|
|
|
### Cover / Duress Mode
|
|
|
|
- **Configurable certificate CN** -- set `cover_name` in config to replace "FieldWitness Local"
|
|
in the self-signed TLS certificate
|
|
- **Portable data directory** -- set `FIELDWITNESS_DATA_DIR` to relocate all state to an
|
|
arbitrary path (e.g., an innocuously named directory on a USB stick). All paths resolve
|
|
lazily from `BASE_DIR`, so runtime overrides propagate correctly
|
|
|
|
### Additional Tools: Steganography (Stego)
|
|
|
|
For situations requiring covert channels -- hiding communications or small payloads inside
|
|
ordinary media files:
|
|
|
|
- **LSB encoding** -- bit-level message hiding in PNG images
|
|
- **DCT encoding** -- frequency-domain hiding in JPEG images (requires `stego-dct` extra)
|
|
- **Audio steganography** -- hide data in WAV/FLAC audio (requires `stego-audio` extra)
|
|
- **Video steganography** -- frame-level encoding
|
|
- **Transport-aware encoding** -- `--transport whatsapp|signal|telegram|discord|email|direct`
|
|
auto-selects the right encoding mode and carrier resolution for lossy messaging
|
|
platforms. WhatsApp/Signal/Telegram force DCT/JPEG mode and pre-resize the carrier to
|
|
survive recompression
|
|
- **Carrier reuse tracking** -- warns when a carrier image has been used before, since
|
|
comparing two versions of the same carrier trivially reveals steganographic modification
|
|
- AES-256-GCM encryption with Argon2id key derivation
|
|
- EXIF stripping on encode to prevent metadata leakage
|
|
- Compression support (zstandard, optional LZ4)
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Basic install (core library only)
|
|
|
|
```bash
|
|
pip install fieldwitness
|
|
```
|
|
|
|
### With extras
|
|
|
|
```bash
|
|
pip install "fieldwitness[web,cli]" # Web UI + CLI (most common)
|
|
pip install "fieldwitness[all]" # Everything except dev tools
|
|
pip install "fieldwitness[dev]" # All + pytest, black, ruff, mypy
|
|
```
|
|
|
|
### Available extras
|
|
|
|
| Extra | What it adds |
|
|
|---|---|
|
|
| `stego-dct` | DCT steganography (numpy, scipy, jpeglib, reedsolo) |
|
|
| `stego-audio` | Audio steganography (pydub, soundfile, reedsolo) |
|
|
| `stego-compression` | LZ4 compression support |
|
|
| `attest` | Attestation features (imagehash, lmdb, exifread) |
|
|
| `cli` | Click CLI with rich output, QR code support, piexif |
|
|
| `web` | Flask web UI with Waitress/Gunicorn, pyzbar QR scanning, includes attest + stego-dct |
|
|
| `api` | FastAPI REST API with uvicorn, includes stego-dct |
|
|
| `fieldkit` | Tamper monitoring (watchdog) and USB whitelist (pyudev) |
|
|
| `federation` | Peer-to-peer attestation federation (aiohttp) |
|
|
| `rpi` | Raspberry Pi deployment (web + cli + fieldkit + gpiozero) |
|
|
| `all` | All of the above |
|
|
| `dev` | All + pytest, pytest-cov, black, ruff, mypy |
|
|
|
|
### Airgapped install
|
|
|
|
Bundle wheels on a networked machine, then install offline:
|
|
|
|
```bash
|
|
# On networked machine
|
|
pip download "fieldwitness[web,cli]" -d ./wheels
|
|
|
|
# Transfer ./wheels to target via USB
|
|
# On airgapped machine
|
|
pip install --no-index --find-links=./wheels "fieldwitness[web,cli]"
|
|
fieldwitness init
|
|
fieldwitness serve --host 0.0.0.0
|
|
```
|
|
|
|
---
|
|
|
|
## Deployment
|
|
|
|
FieldWitness uses a three-tier deployment model designed for field journalism, organizational
|
|
evidence management, and cross-organization federation.
|
|
|
|
**Tier 1 -- Field Device.** A bootable Debian Live USB stick. Boots into a minimal desktop
|
|
with Firefox pointed at the local FieldWitness web UI. LUKS-encrypted persistent partition. Pull
|
|
the USB and the host machine retains nothing.
|
|
|
|
**Tier 2 -- Org Server.** A Docker deployment on a mini PC or trusted VPS. Runs the full
|
|
web UI (port 5000) and federation API (port 8000). Manages keys, attestations, and
|
|
federation sync.
|
|
|
|
**Tier 3 -- Federation Relay.** A lightweight Docker container in a jurisdiction with
|
|
strong press protections. Relays attestation records between organizations. Stores only
|
|
hashes and signatures -- never keys, plaintext, or original media.
|
|
|
|
### Quick deploy
|
|
|
|
```bash
|
|
# Tier 1: Build USB image
|
|
cd deploy/live-usb && sudo ./build.sh
|
|
|
|
# Tier 2: Docker org server
|
|
cd deploy/docker && docker compose up server -d
|
|
|
|
# Tier 3: Docker federation relay
|
|
cd deploy/docker && docker compose up relay -d
|
|
```
|
|
|
|
### Threat level configuration presets
|
|
|
|
FieldWitness ships four configuration presets at `deploy/config-presets/`:
|
|
|
|
| Preset | Session | Killswitch | Dead Man | Cover Name |
|
|
|---|---|---|---|---|
|
|
| `low-threat.json` | 30 min | Off | Off | None |
|
|
| `medium-threat.json` | 15 min | On | 48h / 4h grace | "Office Document Manager" |
|
|
| `high-threat.json` | 5 min | On | 12h / 1h grace | "Local Inventory Tracker" |
|
|
| `critical-threat.json` | 3 min | On | 6h / 1h grace | "System Statistics" |
|
|
|
|
```bash
|
|
cp deploy/config-presets/high-threat.json ~/.fieldwitness/config.json
|
|
```
|
|
|
|
See [docs/deployment.md](docs/deployment.md) for the full deployment guide including
|
|
security hardening, Kubernetes manifests, systemd services, and operational security notes.
|
|
|
|
---
|
|
|
|
## CLI Reference
|
|
|
|
All commands accept `--data-dir PATH` to override the default `~/.fieldwitness` directory,
|
|
and `--json` for machine-readable output.
|
|
|
|
```
|
|
fieldwitness [--data-dir PATH] [--json] COMMAND
|
|
```
|
|
|
|
### Core commands
|
|
|
|
| Command | Description |
|
|
|---|---|
|
|
| `fieldwitness init` | Create directory structure, generate identity + channel key, write default config |
|
|
| `fieldwitness serve` | Start the web UI (default: `https://127.0.0.1:5000`) |
|
|
| `fieldwitness status` | Show instance status: identity, keys, chain, fieldkit, config |
|
|
|
|
### `fieldwitness serve` options
|
|
|
|
| Option | Default | Description |
|
|
|---|---|---|
|
|
| `--host` | `127.0.0.1` | Bind address |
|
|
| `--port` | `5000` | Bind port |
|
|
| `--no-https` | off | Disable HTTPS (use HTTP) |
|
|
| `--debug` | off | Use Flask dev server instead of Waitress |
|
|
| `--workers` | `4` | Number of Waitress/Gunicorn worker threads |
|
|
|
|
### Attestation commands (`fieldwitness attest`)
|
|
|
|
```bash
|
|
# Attest an image (sign with Ed25519 identity)
|
|
fieldwitness attest IMAGE photo.jpg
|
|
fieldwitness attest IMAGE photo.jpg --caption "Field report" --location "Istanbul"
|
|
|
|
# Batch attest a directory
|
|
fieldwitness attest batch ./photos/ --caption "Field report"
|
|
|
|
# Verify an image against the attestation log
|
|
fieldwitness attest verify photo.jpg
|
|
|
|
# View attestation log
|
|
fieldwitness attest log --limit 20
|
|
```
|
|
|
|
### Chain commands (`fieldwitness chain`)
|
|
|
|
```bash
|
|
fieldwitness chain status # Show chain head, length, integrity
|
|
fieldwitness chain verify # Verify entire chain integrity (hashes + signatures)
|
|
fieldwitness chain show INDEX # Show a specific chain record
|
|
fieldwitness chain log --count 20 # Show recent chain entries
|
|
fieldwitness chain backfill # Backfill existing attestations into chain
|
|
|
|
# Evidence export
|
|
fieldwitness chain export --start 0 --end 100 -o chain.zip
|
|
|
|
# Selective disclosure (for legal discovery / court orders)
|
|
fieldwitness chain disclose -i 5,12,47 -o disclosure.json
|
|
|
|
# External timestamp anchoring
|
|
fieldwitness chain anchor # Manual anchor (prints hash for tweet/email/blockchain)
|
|
fieldwitness chain anchor --tsa https://freetsa.org/tsr # RFC 3161 automated anchor
|
|
```
|
|
|
|
### Fieldkit commands (`fieldwitness fieldkit`)
|
|
|
|
```bash
|
|
fieldwitness fieldkit status # Show fieldkit state
|
|
fieldwitness fieldkit checkin # Reset dead man's switch timer
|
|
fieldwitness fieldkit check-deadman # Check if deadman timer has expired
|
|
fieldwitness fieldkit purge --confirm # Activate killswitch (destroys all data)
|
|
fieldwitness fieldkit geofence set --lat 48.8566 --lon 2.3522 --radius 1000
|
|
fieldwitness fieldkit geofence check --lat 48.8600 --lon 2.3500
|
|
fieldwitness fieldkit geofence clear
|
|
fieldwitness fieldkit usb snapshot # Snapshot current USB devices as whitelist
|
|
fieldwitness fieldkit usb check # Check for unauthorized USB devices
|
|
```
|
|
|
|
### Key management commands (`fieldwitness keys`)
|
|
|
|
```bash
|
|
fieldwitness keys show # Display current key info
|
|
fieldwitness keys export -o backup.enc # Export encrypted key bundle
|
|
fieldwitness keys import -b backup.enc # Import key bundle
|
|
fieldwitness keys rotate-identity # Generate new Ed25519 identity (records rotation in chain)
|
|
fieldwitness keys rotate-channel # Generate new channel key
|
|
```
|
|
|
|
### Steganography commands (`fieldwitness stego`)
|
|
|
|
Stego uses multi-factor authentication: a **reference photo** (shared image both
|
|
parties have), a **passphrase** (4+ words), and a **PIN** (6-9 digits). All three are
|
|
required to encode or decode. The passphrase and PIN are prompted interactively
|
|
(hidden input) if not provided via options.
|
|
|
|
```bash
|
|
# Encode a text message into an image
|
|
# CARRIER is the image to hide data in, -r is the shared reference photo
|
|
fieldwitness stego encode cover.png -r shared_photo.jpg -m "Secret message"
|
|
# Passphrase: **** (prompted, hidden)
|
|
# PIN: **** (prompted, hidden)
|
|
# -> writes encoded PNG to current directory
|
|
|
|
# Encode with explicit output path
|
|
fieldwitness stego encode cover.png -r shared_photo.jpg -m "Secret" -o stego_output.png
|
|
|
|
# Encode a file instead of text
|
|
fieldwitness stego encode cover.png -r shared_photo.jpg -f document.pdf
|
|
|
|
# Transport-aware encoding (auto-selects DCT/JPEG and resizes for the platform)
|
|
fieldwitness stego encode cover.jpg -r shared.jpg -m "Secret" --transport whatsapp
|
|
fieldwitness stego encode cover.jpg -r shared.jpg -m "Secret" --transport signal
|
|
fieldwitness stego encode cover.jpg -r shared.jpg -m "Secret" --transport telegram
|
|
fieldwitness stego encode cover.jpg -r shared.jpg -m "Secret" --transport email
|
|
fieldwitness stego encode cover.jpg -r shared.jpg -m "Secret" --transport direct
|
|
|
|
# Dry run -- check capacity without encoding
|
|
fieldwitness stego encode cover.png -r shared_photo.jpg -m "Secret" --dry-run
|
|
|
|
# Decode a message from a stego image (same reference + passphrase + PIN)
|
|
fieldwitness stego decode stego_output.png -r shared_photo.jpg
|
|
# Passphrase: ****
|
|
# PIN: ****
|
|
# -> prints decoded message or saves decoded file
|
|
|
|
# Decode and save file payload to specific path
|
|
fieldwitness stego decode stego_output.png -r shared_photo.jpg -o recovered.pdf
|
|
|
|
# DCT mode for JPEG (survives social media compression)
|
|
fieldwitness stego encode cover.jpg -r shared_photo.jpg -m "Secret" --platform telegram
|
|
|
|
# Audio steganography
|
|
fieldwitness stego audio-encode audio.wav -r shared_photo.jpg -m "Hidden in audio"
|
|
fieldwitness stego audio-decode stego.wav -r shared_photo.jpg
|
|
|
|
# Generate credentials
|
|
fieldwitness stego generate # Generate passphrase + PIN
|
|
fieldwitness stego generate --pin-length 8 # Longer PIN
|
|
|
|
# Channel key management
|
|
fieldwitness stego channel status # Show current channel key
|
|
fieldwitness stego channel generate # Generate new channel key
|
|
|
|
# Image info and capacity
|
|
fieldwitness stego info cover.png # Image details + LSB/DCT capacity
|
|
```
|
|
|
|
---
|
|
|
|
## Web UI
|
|
|
|
Start with `fieldwitness serve`. The web UI provides authenticated access to all features
|
|
through Flask blueprints. Served by **Waitress** (production WSGI server) by default.
|
|
|
|
### Routes
|
|
|
|
| Blueprint | Routes | Description |
|
|
|---|---|---|
|
|
| attest | `/attest`, `/verify` | Attestation signing and verification |
|
|
| federation | `/federation/*` | Federation peer dashboard, peer add/remove |
|
|
| fieldkit | `/fieldkit/*` | Killswitch, dead man's switch, status dashboard |
|
|
| keys | `/keys/*` | Key management, rotation, export/import |
|
|
| admin | `/admin/*` | User management (multi-user auth via SQLite) |
|
|
| dropbox | `/dropbox/admin`, `/dropbox/upload/<token>` | Source drop box: token creation (admin), anonymous upload (source), receipt verification |
|
|
| stego | `/encode`, `/decode`, `/generate` | Steganography operations |
|
|
| health | `/health` | Capability reporting endpoint (see API section) |
|
|
|
|
<!-- TODO: screenshots -->
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
FieldWitness loads configuration from `~/.fieldwitness/config.json`. All fields have sensible defaults.
|
|
`fieldwitness init` writes the default config file.
|
|
|
|
### Config fields
|
|
|
|
| Field | Type | Default | Description |
|
|
|---|---|---|---|
|
|
| `host` | string | `127.0.0.1` | Web UI bind address |
|
|
| `port` | int | `5000` | Web UI bind port |
|
|
| `https_enabled` | bool | `true` | Enable HTTPS with self-signed cert |
|
|
| `auth_enabled` | bool | `true` | Require login for web UI |
|
|
| `max_upload_mb` | int | `50` | Maximum upload size in MB |
|
|
| `session_timeout_minutes` | int | `15` | Session expiry |
|
|
| `login_lockout_attempts` | int | `5` | Failed logins before lockout |
|
|
| `login_lockout_minutes` | int | `15` | Lockout duration |
|
|
| `default_embed_mode` | string | `auto` | Stego encoding mode |
|
|
| `killswitch_enabled` | bool | `false` | Enable killswitch functionality |
|
|
| `deadman_enabled` | bool | `false` | Enable dead man's switch |
|
|
| `deadman_interval_hours` | int | `24` | Check-in interval |
|
|
| `deadman_grace_hours` | int | `2` | Grace period after missed check-in |
|
|
| `deadman_warning_webhook` | string | `""` | URL to POST warning before auto-purge |
|
|
| `usb_monitoring_enabled` | bool | `false` | Enable USB device whitelist enforcement |
|
|
| `tamper_monitoring_enabled` | bool | `false` | Enable file integrity monitoring |
|
|
| `chain_enabled` | bool | `true` | Enable attestation hash chain |
|
|
| `chain_auto_wrap` | bool | `true` | Auto-wrap attestations in chain records |
|
|
| `backup_reminder_days` | int | `7` | Days before backup reminder |
|
|
| `cover_name` | string | `""` | If set, used for SSL cert CN instead of "FieldWitness Local" (cover/duress mode) |
|
|
| `gpio_killswitch_pin` | int | `17` | Raspberry Pi GPIO pin for hardware killswitch |
|
|
| `gpio_killswitch_hold_seconds` | float | `5.0` | Hold duration to trigger hardware killswitch |
|
|
|
|
### Environment variables
|
|
|
|
| Variable | Description |
|
|
|---|---|
|
|
| `FIELDWITNESS_DATA_DIR` | Override the data directory (default: `~/.fieldwitness`). Enables portable USB mode and cover/duress directory naming |
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### Source layout
|
|
|
|
```
|
|
src/fieldwitness/
|
|
__init__.py Package init, __version__
|
|
cli.py Click CLI (entry point: fieldwitness)
|
|
paths.py All path constants (lazy resolution from BASE_DIR)
|
|
config.py Unified config loader (dataclass + JSON)
|
|
exceptions.py FieldWitnessError base exception
|
|
metadata.py Extract-then-strip EXIF pipeline
|
|
evidence.py Self-contained evidence package export
|
|
archive.py Cold archive for long-term preservation (OAIS-aligned)
|
|
attest/ Attestation engine (subpackage)
|
|
models.py ImageHashes (images + arbitrary files), AttestationRecord
|
|
keystore/
|
|
manager.py Key material management (channel + identity + trust store + backup)
|
|
models.py KeyBundle, IdentityBundle dataclasses
|
|
export.py Full bundle export, channel-key-only export, QR code sharing
|
|
federation/
|
|
chain.py Append-only hash chain (key rotation, recovery, delivery ack, selective disclosure)
|
|
anchors.py RFC 3161 timestamps + manual chain anchors
|
|
exchange.py Cross-org attestation bundle export/import
|
|
fieldkit/
|
|
killswitch.py Emergency data destruction (deep forensic scrub, self-uninstall)
|
|
deadman.py Dead man's switch (webhook warnings)
|
|
tamper.py File integrity monitoring
|
|
usb_monitor.py USB device whitelist (Linux/pyudev)
|
|
geofence.py GPS boundary enforcement (gpsd integration)
|
|
stego/ Steganography engine (subpackage)
|
|
encode.py Transport-aware encoding (--transport flag)
|
|
carrier_tracker.py Carrier reuse tracking and warnings
|
|
|
|
frontends/web/
|
|
app.py Flask app factory (create_app())
|
|
auth.py SQLite3 multi-user auth
|
|
temp_storage.py File-based temp storage with expiry
|
|
subprocess_stego.py Crash-safe subprocess isolation
|
|
ssl_utils.py Self-signed HTTPS cert generation
|
|
blueprints/
|
|
attest.py /attest, /verify
|
|
federation.py /federation/* (peer dashboard)
|
|
fieldkit.py /fieldkit/*
|
|
keys.py /keys/*
|
|
admin.py /admin/*
|
|
dropbox.py /dropbox/* (source drop box)
|
|
stego.py /encode, /decode, /generate
|
|
|
|
deploy/ Deployment artifacts
|
|
docker/ Dockerfile (multi-stage: builder, relay, server) + compose
|
|
kubernetes/ Namespace, server, and relay deployments
|
|
live-usb/ Debian Live USB build scripts (Tier 1)
|
|
config-presets/ Threat level presets (low/medium/high/critical)
|
|
```
|
|
|
|
### Data directory (`~/.fieldwitness/`)
|
|
|
|
```
|
|
~/.fieldwitness/
|
|
config.json Unified configuration
|
|
audit.jsonl Append-only audit trail
|
|
carrier_history.json Carrier reuse tracking database
|
|
identity/ Ed25519 keypair (private.pem, public.pem, identity.meta.json)
|
|
stego/ Channel key (channel.key)
|
|
attestations/ Attest attestation store (log.bin, index/, peers.json)
|
|
chain/ Hash chain (chain.bin, state.cbor, anchors/)
|
|
auth/ Web UI auth database (fieldwitness.db, dropbox.db)
|
|
certs/ Self-signed TLS certificates
|
|
fieldkit/ Fieldkit state (deadman.json, tamper/, usb/, geofence.json)
|
|
temp/ Ephemeral file storage (dropbox uploads)
|
|
instance/ Flask instance (sessions, secret key)
|
|
trusted_keys/ Collaborator Ed25519 public keys (trust store)
|
|
```
|
|
|
|
Sensitive directories (`identity/`, `auth/`, `certs/`, and the root) are created with
|
|
`0700` permissions.
|
|
|
|
---
|
|
|
|
## Security Model
|
|
|
|
**Two key domains, never merged.** Stego uses AES-256-GCM with keys derived via
|
|
Argon2id from user-supplied factors. Attest uses Ed25519 for signing. These serve
|
|
different security purposes and are kept strictly separate.
|
|
|
|
**Killswitch priority.** The killswitch destroys all data under `~/.fieldwitness/`, including
|
|
the audit log. This is intentional -- in a field compromise scenario, data destruction
|
|
takes precedence over audit trail preservation. The deep forensic scrub extends beyond
|
|
the data directory to remove Python bytecache, pip metadata, pip download cache, shell
|
|
history entries, and the fieldwitness package itself.
|
|
|
|
**Offline-first.** All static assets are vendored (no CDN calls). Pip wheels can be
|
|
bundled for fully airgapped installation. No network access is required for any core
|
|
functionality. RFC 3161 timestamping and webhook warnings are optional features that
|
|
gracefully degrade when offline.
|
|
|
|
**Web UI hardening:**
|
|
- CSRF protection via Flask-WTF
|
|
- Session timeout (default: 15 minutes)
|
|
- Login rate limiting with lockout (5 attempts, 15-minute lockout)
|
|
- HTTPS by default with auto-generated self-signed certificates
|
|
- EXIF stripping on steganographic encode to prevent metadata leakage
|
|
- Dead man's switch webhook SSRF protection (blocks private/internal targets)
|
|
|
|
**Subprocess isolation.** Steganographic operations run in a subprocess boundary
|
|
(`subprocess_stego.py`) to contain crashes and prevent memory corruption from affecting
|
|
the main web server process.
|
|
|
|
**Chain integrity.** The append-only hash chain uses Ed25519 signatures and SHA-256
|
|
linkage. Key rotation records create a verifiable trust chain; identity recovery records
|
|
are auditable. Selective disclosure uses Merkle-style proofs so third parties can verify
|
|
specific records without accessing the full chain.
|
|
|
|
---
|
|
|
|
## Cross-Domain Applications
|
|
|
|
While FieldWitness was designed for journalist and NGO field security, the attestation chain,
|
|
federation, and evidence packaging capabilities apply to a range of domains:
|
|
|
|
- **Human rights documentation** -- field workers attest photos and videos of incidents
|
|
with GPS and timestamps, federate evidence to international partners, and produce
|
|
court-ready evidence packages
|
|
- **Research integrity** -- researchers attest datasets (CSV, sensor readings) at
|
|
collection time, creating a tamper-evident chain of custody. `ImageHashes.from_file()`
|
|
supports arbitrary file types via SHA-256
|
|
- **Election monitoring** -- observers attest ballot images and tally sheets with location
|
|
metadata, anchor the chain to an RFC 3161 TSA for independent time proof, and use
|
|
selective disclosure for audit requests
|
|
- **Supply chain verification** -- attest inspection photos, sensor data, and certificates
|
|
of origin at each stage. Federation enables multi-party chains across organizations
|
|
- **Art authentication** -- attest high-resolution photographs of artworks with device and
|
|
location metadata, creating provenance records that survive format conversion via
|
|
perceptual hashing
|
|
- **Corporate whistleblowing** -- the source drop box accepts anonymous uploads with
|
|
client-side hashing. Cover mode (`cover_name`, `FIELDWITNESS_DATA_DIR`) disguises the
|
|
installation. The killswitch provides emergency destruction if the instance is
|
|
compromised
|
|
- **Environmental monitoring** -- attest sensor data, satellite imagery, and field
|
|
photographs. Cold archives with `ALGORITHMS.txt` ensure evidence remains verifiable
|
|
decades later
|
|
|
|
---
|
|
|
|
## API
|
|
|
|
### `/health` endpoint
|
|
|
|
The web UI exposes a `/health` endpoint that reports installed capabilities:
|
|
|
|
```json
|
|
{
|
|
"status": "ok",
|
|
"version": "0.2.0",
|
|
"capabilities": ["stego-lsb", "stego-dct", "attest", "fieldkit", "chain"]
|
|
}
|
|
```
|
|
|
|
Useful for monitoring and for clients to discover which extras are installed.
|
|
|
|
### FastAPI (optional)
|
|
|
|
Install the `api` extra for a standalone FastAPI REST interface:
|
|
|
|
```bash
|
|
pip install "fieldwitness[api]"
|
|
```
|
|
|
|
This provides `fieldwitness.api` with a FastAPI application served by uvicorn, suitable for
|
|
programmatic integration.
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
### Setup
|
|
|
|
```bash
|
|
git clone https://github.com/alee/fieldwitness.git
|
|
cd fieldwitness
|
|
pip install -e ".[dev]"
|
|
```
|
|
|
|
### Commands
|
|
|
|
```bash
|
|
pytest # Run tests with coverage
|
|
black src/ tests/ frontends/ # Format (100-char line length)
|
|
ruff check src/ tests/ frontends/ --fix # Lint
|
|
mypy src/ # Type check
|
|
```
|
|
|
|
### Code style
|
|
|
|
- Black with 100-character line length
|
|
- Ruff with E, F, I, N, W, UP rule sets
|
|
- mypy strict mode with missing imports ignored
|
|
- Imperative commit messages (e.g., "Add killswitch purge confirmation")
|
|
|
|
### Python support
|
|
|
|
Python 3.11, 3.12, 3.13, and 3.14.
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
| Document | Audience | Description |
|
|
|---|---|---|
|
|
| [docs/deployment.md](docs/deployment.md) | Field deployers, IT staff | Three-tier deployment guide, security hardening, troubleshooting |
|
|
| [docs/federation.md](docs/federation.md) | System administrators | Gossip protocol, peer setup, offline bundles, federation API |
|
|
| [docs/evidence-guide.md](docs/evidence-guide.md) | Investigators, legal teams | Evidence packages, cold archives, selective disclosure, anchoring |
|
|
| [docs/source-dropbox.md](docs/source-dropbox.md) | Administrators | Source drop box setup, EXIF pipeline, receipt codes |
|
|
| [docs/security/threat-model.md](docs/security/threat-model.md) | Security reviewers, contributors | Threat model, adversary model, trust boundaries, cryptographic primitives |
|
|
| [docs/training/reporter-quickstart.md](docs/training/reporter-quickstart.md) | Field reporters | One-page quick-start card for Tier 1 USB users |
|
|
| [docs/training/emergency-card.md](docs/training/emergency-card.md) | All users | Laminated wallet card: emergency destruction, dead man's switch |
|
|
| [docs/training/admin-reference.md](docs/training/admin-reference.md) | Administrators | CLI cheat sheet, hardening checklist, troubleshooting |
|
|
|
|
Architecture documents (design-level, for contributors):
|
|
|
|
| Document | Description |
|
|
|---|---|
|
|
| [docs/architecture/federation.md](docs/architecture/federation.md) | System architecture overview, threat model, layer design |
|
|
| [docs/architecture/chain-format.md](docs/architecture/chain-format.md) | Chain record spec (CBOR, entropy witnesses) |
|
|
| [docs/architecture/export-bundle.md](docs/architecture/export-bundle.md) | Export bundle spec (binary format, envelope encryption) |
|
|
| [docs/architecture/federation-protocol.md](docs/architecture/federation-protocol.md) | Federation server protocol (CT-inspired, gossip) |
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
GPL-3.0 License. See [LICENSE](LICENSE) for details.
|