fieldwitness/docs/source-dropbox.md
Aaron D. Lee 5b0d90eeaf
Some checks failed
CI / lint (push) Failing after 12s
CI / typecheck (push) Failing after 12s
Fix all power-user review issues (FR-01 through FR-12)
FR-01: Fix data directory default from ~/.fieldwitness to ~/.fwmetadata
FR-02/05/07: Accept all file types for attestation (not just images)
  - Web UI, CLI, and batch now accept PDFs, CSVs, audio, video, etc.
  - Perceptual hashing for images, SHA-256-only for everything else
FR-03: Implement C2PA import path + CLI commands (export/verify/import/show)
FR-04: Fix GPS downsampling bias (math.floor → round)
FR-06: Add HTML/PDF evidence summaries for lawyers
  - Always generates summary.html, optional summary.pdf via xhtml2pdf
FR-08: Fix CLI help text ("FieldWitness -- FieldWitness" artifact)
FR-09: Centralize stray paths (trusted_keys, carrier_history, last_backup)
FR-10: Add 67 C2PA bridge tests (vendor assertions, cert, GPS, export)
FR-12: Add Tor onion service support for source drop box
  - fieldwitness serve --tor flag, persistent/transient modes
  - Killswitch covers hidden service keys

Also: bonus fix for attest/api.py hardcoded path bypassing paths.py

224 tests passing (67 new).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 20:10:37 -04:00

350 lines
13 KiB
Markdown

# Source Drop Box Setup Guide
**Audience**: Administrators setting up FieldWitness's anonymous source intake feature.
**Prerequisites**: A running FieldWitness instance with web UI enabled (`fieldwitness[web]` extra),
an admin account, and HTTPS configured (self-signed is acceptable).
---
## Overview
The source drop box is a SecureDrop-style anonymous file intake built into the FieldWitness web
UI. Admins create time-limited upload tokens, sources open the token URL in a browser and
submit files without creating an account. Files are processed through the extract-then-strip
EXIF pipeline and automatically attested on receipt. Sources receive HMAC-derived receipt
codes that prove delivery.
> **Warning:** The drop box protects source identity through design -- no accounts, no
> branding, no IP logging. However, the security of the system depends on how the upload URL
> is shared. Never send drop box URLs over unencrypted email or SMS.
---
## How It Works
```
Admin Source FieldWitness Server
| | |
|-- Create token ------------->| |
| (label, expiry, max_files) | |
| | |
|-- Share URL (secure channel) | |
| | |
| |-- Open URL in browser --------->|
| | (no login required) |
| | |
| |-- Select files |
| | Browser computes SHA-256 |
| | (SubtleCrypto, client-side) |
| | |
| |-- Upload files ---------------->|
| | |-- Extract EXIF
| | |-- Strip metadata
| | |-- Attest originals
| | |-- Save stripped copy
| | |
| |<-- Receipt codes ---------------|
| | (HMAC of file hash + token) |
```
---
## Setting Up the Drop Box
### Step 1: Ensure HTTPS is enabled
The drop box should always be served over HTTPS. Sources must be able to trust that their
connection is not being intercepted.
```bash
$ fieldwitness serve --host 0.0.0.0
```
FieldWitness auto-generates a self-signed certificate on first HTTPS start. For production use,
place a reverse proxy with a proper TLS certificate in front of FieldWitness.
### Step 2: Create an upload token
Navigate to `/dropbox/admin` in the web UI (admin login required), or use the admin panel.
Each token has:
| Field | Default | Description |
|---|---|---|
| **Label** | "Unnamed source" | Human-readable name for the source (stored server-side only, never shown to the source) |
| **Expiry** | 24 hours | How long the upload link remains valid |
| **Max files** | 10 | Maximum number of uploads allowed on this link |
After creating the token, the admin receives a URL of the form:
```
https://<host>:<port>/dropbox/upload/<token>
```
The token is a 32-byte cryptographically random URL-safe string.
### Step 3: Share the URL with the source
Share the upload URL over an already-secure channel:
- **Best**: in person, on paper
- **Good**: encrypted messaging (Signal, Wire)
- **Acceptable**: verbal dictation over a secure voice call
- **Never**: unencrypted email, SMS, or any channel that could be intercepted
### Step 4: Source uploads files
The source opens the URL in their browser. The upload page is minimal -- no FieldWitness branding,
no identifying marks, generic styling. The page works over Tor Browser with JavaScript
enabled (no external resources, no CDN, no fonts, no analytics).
When files are selected:
1. The browser computes SHA-256 fingerprints client-side using SubtleCrypto
2. The source sees the fingerprints and is prompted to save them before uploading
3. On upload, the server processes each file through the extract-then-strip pipeline
4. The source receives receipt codes for each file
### Step 5: Monitor submissions
The admin panel at `/dropbox/admin` shows:
- Active tokens with their usage counts
- Token expiry times
- Ability to revoke tokens immediately
---
## The Extract-Then-Strip Pipeline
Every file uploaded through the drop box is processed through FieldWitness's EXIF pipeline:
1. **Extract**: all EXIF metadata is read from the original image bytes
2. **Classify**: fields are split into evidentiary (GPS coordinates, capture timestamp --
valuable for provenance) and dangerous (device serial number, firmware version -- could
identify the source's device)
3. **Attest**: the original bytes are attested (Ed25519 signed) with evidentiary metadata
included in the attestation record. The attestation hash matches what the source actually
submitted.
4. **Strip**: all metadata is removed from the stored copy. The stripped copy is saved to
disk. No device fingerprint persists on the server's storage.
This resolves the tension between protecting the source (strip device-identifying metadata)
and preserving evidence (retain GPS and timestamp for provenance).
---
## Receipt Codes
Each uploaded file generates an HMAC-derived receipt code:
```
receipt_code = HMAC-SHA256(token, file_sha256)[:16]
```
The receipt code proves:
- The server received the specific file (tied to the file's SHA-256)
- The file was received under the specific token (tied to the token value)
Sources can verify their receipt by posting it to `/dropbox/verify-receipt`. This returns
the filename, SHA-256, and reception timestamp if the receipt is valid.
> **Note:** Receipt codes are deterministic. The source can compute the expected receipt
> themselves if they know the token value and the file's SHA-256 hash, providing
> independent verification.
---
## Client-Side SHA-256
The upload page computes SHA-256 fingerprints in the browser before upload using the
SubtleCrypto Web API. This gives the source a verifiable record of exactly what they
submitted -- the hash is computed on their device, not the server.
The source should save these fingerprints before uploading. If the server later claims to
have received different content, the source can prove what they actually submitted by
comparing their locally computed hash with the server's receipt.
---
## Storage
| What | Where |
|---|---|
| Uploaded files (stripped) | `~/.fwmetadata/temp/dropbox/` (mode 0700) |
| Token metadata | `~/.fwmetadata/auth/dropbox.db` (SQLite) |
| Receipt codes | `~/.fwmetadata/auth/dropbox.db` (SQLite) |
| Attestation records | `~/.fwmetadata/attestations/` (standard attestation log) |
Expired tokens are cleaned up automatically on every admin page load.
---
## Operational Security
### Source safety
- **No FieldWitness branding** on the upload page. Generic "Secure File Upload" title.
- **No authentication required** -- sources never create accounts or reveal identity.
- **No IP logging** -- FieldWitness does not log source IP addresses. Ensure your reverse proxy
(if any) also does not log access requests to `/dropbox/upload/` paths.
- **Self-contained page** -- inline CSS and JavaScript only. No external resources, CDN
calls, web fonts, or analytics. Works with Tor Browser.
- **CSRF exempt** -- the upload endpoint does not require CSRF tokens because sources do
not have sessions.
### Token management
- **Short expiry** -- set token expiry as short as practical. 24 hours is the default; for
high-risk sources, consider 1-4 hours.
- **Low file limits** -- set `max_files` to the expected number of submissions.
Once reached, the link stops accepting uploads.
- **Revoke immediately** -- if a token is compromised or no longer needed, revoke it from
the admin panel. This deletes the token and all associated receipt records from SQLite.
- **Audit trail** -- token creation events are logged to `~/.fwmetadata/audit.jsonl` with the
action `dropbox.token_created`.
### Running as a Tor hidden service
FieldWitness has built-in support for exposing the drop box as a Tor hidden service
(a `.onion` address). When a source accesses the drop box over Tor, the server never
sees their real IP address -- Tor's onion routing ensures that only the Tor network knows
both the source and the destination.
#### Step 1: Install and configure Tor
```bash
# Debian / Ubuntu
sudo apt install tor
# macOS (Homebrew)
brew install tor
# Fedora / RHEL
sudo dnf install tor
```
Enable the control port so FieldWitness can manage the hidden service. Add these lines
to `/etc/tor/torrc`:
```
ControlPort 9051
CookieAuthentication 1
```
Then restart Tor:
```bash
sudo systemctl restart tor
```
**Authentication note:** `CookieAuthentication 1` lets FieldWitness authenticate using the
cookie file that Tor creates automatically. Alternatively, use a password:
```
ControlPort 9051
HashedControlPassword <hash produced by: tor --hash-password yourpassword>
```
#### Step 2: Install the stem library
stem is an optional FieldWitness dependency. Install it alongside the fieldkit extra:
```bash
pip install 'fieldwitness[tor]'
# or, if already installed:
pip install stem>=1.8.0
```
#### Step 3: Start FieldWitness with --tor
```bash
fieldwitness serve --host 127.0.0.1 --port 5000 --tor
```
With a custom control port or password:
```bash
fieldwitness serve --tor --tor-control-port 9051 --tor-password yourpassword
```
For a one-off intake session where a fixed address is not needed:
```bash
fieldwitness serve --tor --tor-transient
```
On startup, FieldWitness will print the `.onion` address:
```
============================================================
TOR HIDDEN SERVICE ACTIVE
============================================================
.onion address : abc123def456ghi789jkl012mno345pq.onion
Drop box URL : http://abc123def456ghi789jkl012mno345pq.onion/dropbox/upload/<token>
Persistent : yes (key saved to ~/.fwmetadata/fieldkit/tor/)
============================================================
Sources must use Tor Browser to access the .onion URL.
Share the drop box upload URL over a secure channel (Signal, in person).
============================================================
```
#### Step 4: Share the .onion drop box URL
Create a drop box token as usual (see Step 2 above), then construct the `.onion` upload URL:
```
http://<onion-address>/dropbox/upload/<token>
```
Share this URL with the source over Signal or in person. The source opens it in
**Tor Browser** -- not in a regular browser.
#### Persistent vs. transient hidden services
| Mode | Command | Behaviour |
|---|---|---|
| **Persistent** (default) | `--tor` | Same `.onion` address on every restart. Key stored at `~/.fwmetadata/fieldkit/tor/hidden_service/`. |
| **Transient** | `--tor --tor-transient` | New `.onion` address each run. No key written to disk. |
Use persistent mode when you want sources to bookmark the address or share it in advance.
Use transient mode for single-session intake where address continuity does not matter.
#### Source instructions
Tell sources:
1. Install **Tor Browser** from [torproject.org](https://www.torproject.org/download/).
2. Open Tor Browser and paste the full `.onion` URL into the address bar.
3. Do not open the `.onion` URL in a regular browser -- your real IP will be visible.
4. JavaScript must be enabled for the SHA-256 fingerprint feature to work.
Set the Security Level to "Standard" in Tor Browser (shield icon in the toolbar).
5. Save the SHA-256 fingerprints shown before clicking Upload.
6. Save the receipt codes shown after upload.
#### Logging and access logs
Even with Tor, access logs on the server can record that *someone* connected -- the
connection appears to come from a Tor exit relay (for regular Tor) or from a Tor internal
address (for hidden services). For hidden services, the server sees no IP at all; the
connection comes from the Tor daemon on localhost.
**Recommendation:** Set the web server or reverse proxy to not log access requests to
`/dropbox/upload/` paths. If using FieldWitness directly (Waitress), access log output
goes to stdout; redirect it to `/dev/null` or omit the log handler.
#### Killswitch and the hidden service key
The persistent hidden service key is stored under `~/.fwmetadata/fieldkit/tor/hidden_service/`
and is treated as key material. The FieldWitness killswitch (`fieldwitness fieldkit purge`)
destroys this directory during both `KEYS_ONLY` and `ALL` purge scopes. After a purge,
the `.onion` address cannot be linked to the operator even if the server hardware is seized.
> **Warning:** Even with Tor, timing analysis and traffic correlation attacks are possible
> at the network level. The drop box protects source identity at the application layer;
> network-layer protection requires operational discipline beyond what software can provide.
> Tor is not a silver bullet -- but it removes the most direct risk (IP address exposure)
> that limitation L7 in the threat model describes.