deploy slice G (docs + ADR-0011): docs/deploy/ tree + ADR-0011 Proposed (#23)
Co-authored-by: Aaron D. Lee <himself@adlee.work> Co-committed-by: Aaron D. Lee <himself@adlee.work>
This commit was merged in pull request #23.
This commit is contained in:
132
docs/deploy/homelab.md
Normal file
132
docs/deploy/homelab.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Homelab & CGNAT — the honest story
|
||||
|
||||
**The truth first: no tunnel carries inbound UDP. WebRTC callers are unreachable behind any
|
||||
tunnel, so a homelab behind CGNAT is PSTN-only, period.** The engine's WebRTC media is
|
||||
UDP-direct to an advertised IP (no STUN, no TURN); the CPaaS side offers no relay or
|
||||
rendezvous either, and intermediaries in the media path are a documented frame-dropping hazard
|
||||
([livekit/agents#3379](https://github.com/livekit/agents/issues/3379)). Nothing on this page
|
||||
makes CGNAT production-grade for free. Three tiers, worst to best:
|
||||
|
||||
## Tier 1 — dev/demo: ngrok (the blessed 5-minute path)
|
||||
|
||||
ngrok is the **only** tunnel with a proven Twilio Media Streams record. Do **not** use
|
||||
Cloudflare Tunnel even for dev: cloudflared has an open Twilio 31920 handshake bug
|
||||
([cloudflared#1465](https://github.com/cloudflare/cloudflared/issues/1465)), recurring 1006
|
||||
closures ([cloudflared#1282](https://github.com/cloudflare/cloudflared/issues/1282)), and
|
||||
Cloudflare documents killing WebSockets on edge code releases.
|
||||
|
||||
```bash
|
||||
# 1. Run the engine, plaintext :8080 (no Caddy needed — ngrok terminates TLS):
|
||||
docker run -d --name rutster-engine --network host \
|
||||
-e RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
|
||||
-e RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token \
|
||||
-e RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081 \
|
||||
-e RUTSTER_TWILIO_WEBHOOK_BASE=https://REPLACE-ME.ngrok-free.app \
|
||||
-e RUTSTER_TRUSTED_PROXIES=127.0.0.1/32 \
|
||||
git.adlee.work/alee/rutster-engine:latest
|
||||
# (or from source: cargo run — see docs/QUICKSTART.md)
|
||||
|
||||
# 2. Tunnel it:
|
||||
ngrok http 8080
|
||||
# 3. Put the printed https://xxxx.ngrok-free.app URL into RUTSTER_TWILIO_WEBHOOK_BASE
|
||||
# (restart the container), and into the Twilio number's webhook:
|
||||
# POST https://xxxx.ngrok-free.app/v1/trunk/webhook
|
||||
# 4. Dial your Twilio number.
|
||||
```
|
||||
|
||||
**Free-tier arithmetic** (why this is a demo, not a deployment — research: [TLS brief §3(d)](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)): a Media Streams call is
|
||||
8 kHz µ-law — 8 kB/s of audio per direction, base64-encoded inside a JSON envelope at 50
|
||||
messages/s, both directions ≈ **~140 MB per call-hour** on the wire. ngrok's free 1 GB/month
|
||||
data cap ([ngrok pricing](https://ngrok.com/pricing)) therefore buys roughly **seven call-hours a month**. And the audio transits ngrok's
|
||||
edge **in plaintext** (they terminate TLS) — an unconsented subprocessor, which is a
|
||||
DPA/BAA/PCI failure. Dev only. Never production.
|
||||
|
||||
## Tier 2 — single-user demo: Tailscale Funnel
|
||||
|
||||
Privacy-clean variant: TLS terminates **on your node**, so Funnel relays ciphertext it cannot
|
||||
read.
|
||||
|
||||
```bash
|
||||
tailscale funnel 8080
|
||||
# webhook base = https://<your-node>.<tailnet>.ts.net
|
||||
```
|
||||
|
||||
Still PSTN-only (no inbound UDP), bandwidth cap undisclosed, one user. Unsizable beyond a
|
||||
personal demo.
|
||||
|
||||
## Tier 3 — production graduation: cheap VPS + WireGuard, TLS at home
|
||||
|
||||
The recommended path. The VPS is a **dumb layer-4 forwarder**: TLS terminates at home (your
|
||||
Caddy, DNS-01 certs), so the forwarder **physically cannot read the audio** — the strongest
|
||||
privacy topology available. Bonus: forwarding the media UDP range over the same tunnel
|
||||
restores WebRTC, which no tunnel product can do. Cost: one small VPS (~$5/mo) and one extra
|
||||
network hop of media latency — pick a VPS region near home.
|
||||
|
||||
Point DNS at the VPS: `pbx.example.com A <VPS-public-IP>`.
|
||||
|
||||
**VPS `/etc/wireguard/wg0.conf`:**
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
Address = 10.88.0.1/24
|
||||
ListenPort = 51820
|
||||
PrivateKey = <vps-private-key>
|
||||
|
||||
[Peer]
|
||||
PublicKey = <home-public-key>
|
||||
AllowedIPs = 10.88.0.2/32
|
||||
```
|
||||
|
||||
**Home box `/etc/wireguard/wg0.conf`** (home initiates — CGNAT-friendly; the keepalive holds
|
||||
the NAT mapping):
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
Address = 10.88.0.2/24
|
||||
PrivateKey = <home-private-key>
|
||||
|
||||
[Peer]
|
||||
PublicKey = <vps-public-key>
|
||||
Endpoint = <VPS-public-IP>:51820
|
||||
AllowedIPs = 10.88.0.0/24
|
||||
PersistentKeepalive = 25
|
||||
```
|
||||
|
||||
**VPS forwarding** (`sysctl -w net.ipv4.ip_forward=1`, persist it, then nftables):
|
||||
|
||||
```nft
|
||||
table ip rutster-fwd {
|
||||
chain prerouting {
|
||||
type nat hook prerouting priority dstnat;
|
||||
iifname "eth0" tcp dport { 80, 443 } dnat to 10.88.0.2
|
||||
iifname "eth0" udp dport 49152-49407 dnat to 10.88.0.2
|
||||
}
|
||||
chain postrouting {
|
||||
type nat hook postrouting priority srcnat;
|
||||
oifname "wg0" masquerade
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Home side:** run T1 or T2 exactly per [quickstart-docker.md](quickstart-docker.md), with:
|
||||
|
||||
- `RUTSTER_MEDIA_ADVERTISED_IP=<VPS-public-IP>` — callers send media UDP to the VPS; the DNAT
|
||||
delivers it home through the tunnel.
|
||||
- `RUTSTER_MEDIA_PORT_RANGE=49152-49407` (must match the nftables rule).
|
||||
- Certificates: DNS-01 is the robust choice here ([certificates.md](certificates.md));
|
||||
HTTP-01 also works since `:80` is forwarded.
|
||||
|
||||
First call: same two paths as [quickstart-docker.md](quickstart-docker.md) — browser at
|
||||
`https://pbx.example.com/` (WebRTC now works — the UDP range rides the tunnel) and the Twilio
|
||||
webhook at `https://pbx.example.com/v1/trunk/webhook`.
|
||||
|
||||
**Or skip the tunnel entirely:** run the engine *on* the VPS (that is just
|
||||
[T1](quickstart-docker.md) on rented hardware). You trade at-home media for zero forwarding
|
||||
complexity.
|
||||
|
||||
## Explicitly unsupported for production
|
||||
|
||||
Cloudflare Tunnel or ngrok in the live audio path: plaintext audio at the vendor edge
|
||||
(unconsented subprocessor — DPA/BAA/PCI failure), documented mid-call WS terminations, zero
|
||||
SLA, and Cloudflare's discretionary "disproportionate audio" ToS clause aimed at exactly this
|
||||
traffic profile (research basis: [TLS brief §4](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)). Ratified in [ADR-0011](../adr/0011-deployment-topology.md).
|
||||
Reference in New Issue
Block a user