docs(deploy): homelab.md — CGNAT truth: PSTN-only behind tunnels, VPS+WireGuard graduation (slice-G)
Acceptance bar honored by honesty: the doc leads with what cannot work (no tunnel carries inbound UDP -> WebRTC unreachable behind CGNAT). Three tiers: ngrok blessed dev path with the free-tier arithmetic (~140 MB/ call-hour -> ~7 call-hours/month), Tailscale Funnel single-user demo (ciphertext-only), and the VPS+WireGuard graduation with TLS terminating at home (forwarder physically cannot read audio; media UDP DNAT restores WebRTC). cloudflared explicitly excluded even for dev (#1465, #1282). Signed-off-by: Aaron D. Lee <himself@adlee.work>
This commit is contained in:
132
docs/deploy/homelab.md
Normal file
132
docs/deploy/homelab.md
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
# Homelab & CGNAT — the honest story
|
||||||
|
|
||||||
|
**The truth first: no tunnel carries inbound UDP. WebRTC callers are unreachable behind any
|
||||||
|
tunnel, so a homelab behind CGNAT is PSTN-only, period.** The engine's WebRTC media is
|
||||||
|
UDP-direct to an advertised IP (no STUN, no TURN); the CPaaS side offers no relay or
|
||||||
|
rendezvous either, and intermediaries in the media path are a documented frame-dropping hazard
|
||||||
|
([livekit/agents#3379](https://github.com/livekit/agents/issues/3379)). Nothing on this page
|
||||||
|
makes CGNAT production-grade for free. Three tiers, worst to best:
|
||||||
|
|
||||||
|
## Tier 1 — dev/demo: ngrok (the blessed 5-minute path)
|
||||||
|
|
||||||
|
ngrok is the **only** tunnel with a proven Twilio Media Streams record. Do **not** use
|
||||||
|
Cloudflare Tunnel even for dev: cloudflared has an open Twilio 31920 handshake bug
|
||||||
|
([cloudflared#1465](https://github.com/cloudflare/cloudflared/issues/1465)), recurring 1006
|
||||||
|
closures ([cloudflared#1282](https://github.com/cloudflare/cloudflared/issues/1282)), and
|
||||||
|
Cloudflare documents killing WebSockets on edge code releases.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Run the engine, plaintext :8080 (no Caddy needed — ngrok terminates TLS):
|
||||||
|
docker run -d --name rutster-engine --network host \
|
||||||
|
-e RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
|
||||||
|
-e RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token \
|
||||||
|
-e RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081 \
|
||||||
|
-e RUTSTER_TWILIO_WEBHOOK_BASE=https://REPLACE-ME.ngrok-free.app \
|
||||||
|
-e RUTSTER_TRUSTED_PROXIES=127.0.0.1/32 \
|
||||||
|
git.adlee.work/alee/rutster-engine:latest
|
||||||
|
# (or from source: cargo run — see docs/QUICKSTART.md)
|
||||||
|
|
||||||
|
# 2. Tunnel it:
|
||||||
|
ngrok http 8080
|
||||||
|
# 3. Put the printed https://xxxx.ngrok-free.app URL into RUTSTER_TWILIO_WEBHOOK_BASE
|
||||||
|
# (restart the container), and into the Twilio number's webhook:
|
||||||
|
# POST https://xxxx.ngrok-free.app/v1/trunk/webhook
|
||||||
|
# 4. Dial your Twilio number.
|
||||||
|
```
|
||||||
|
|
||||||
|
**Free-tier arithmetic** (why this is a demo, not a deployment — research: [TLS brief §3(d)](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)): a Media Streams call is
|
||||||
|
8 kHz µ-law — 8 kB/s of audio per direction, base64-encoded inside a JSON envelope at 50
|
||||||
|
messages/s, both directions ≈ **~140 MB per call-hour** on the wire. ngrok's free 1 GB/month
|
||||||
|
data cap ([ngrok pricing](https://ngrok.com/pricing)) therefore buys roughly **seven call-hours a month**. And the audio transits ngrok's
|
||||||
|
edge **in plaintext** (they terminate TLS) — an unconsented subprocessor, which is a
|
||||||
|
DPA/BAA/PCI failure. Dev only. Never production.
|
||||||
|
|
||||||
|
## Tier 2 — single-user demo: Tailscale Funnel
|
||||||
|
|
||||||
|
Privacy-clean variant: TLS terminates **on your node**, so Funnel relays ciphertext it cannot
|
||||||
|
read.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tailscale funnel 8080
|
||||||
|
# webhook base = https://<your-node>.<tailnet>.ts.net
|
||||||
|
```
|
||||||
|
|
||||||
|
Still PSTN-only (no inbound UDP), bandwidth cap undisclosed, one user. Unsizable beyond a
|
||||||
|
personal demo.
|
||||||
|
|
||||||
|
## Tier 3 — production graduation: cheap VPS + WireGuard, TLS at home
|
||||||
|
|
||||||
|
The recommended path. The VPS is a **dumb layer-4 forwarder**: TLS terminates at home (your
|
||||||
|
Caddy, DNS-01 certs), so the forwarder **physically cannot read the audio** — the strongest
|
||||||
|
privacy topology available. Bonus: forwarding the media UDP range over the same tunnel
|
||||||
|
restores WebRTC, which no tunnel product can do. Cost: one small VPS (~$5/mo) and one extra
|
||||||
|
network hop of media latency — pick a VPS region near home.
|
||||||
|
|
||||||
|
Point DNS at the VPS: `pbx.example.com A <VPS-public-IP>`.
|
||||||
|
|
||||||
|
**VPS `/etc/wireguard/wg0.conf`:**
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
Address = 10.88.0.1/24
|
||||||
|
ListenPort = 51820
|
||||||
|
PrivateKey = <vps-private-key>
|
||||||
|
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <home-public-key>
|
||||||
|
AllowedIPs = 10.88.0.2/32
|
||||||
|
```
|
||||||
|
|
||||||
|
**Home box `/etc/wireguard/wg0.conf`** (home initiates — CGNAT-friendly; the keepalive holds
|
||||||
|
the NAT mapping):
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
Address = 10.88.0.2/24
|
||||||
|
PrivateKey = <home-private-key>
|
||||||
|
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <vps-public-key>
|
||||||
|
Endpoint = <VPS-public-IP>:51820
|
||||||
|
AllowedIPs = 10.88.0.0/24
|
||||||
|
PersistentKeepalive = 25
|
||||||
|
```
|
||||||
|
|
||||||
|
**VPS forwarding** (`sysctl -w net.ipv4.ip_forward=1`, persist it, then nftables):
|
||||||
|
|
||||||
|
```nft
|
||||||
|
table ip rutster-fwd {
|
||||||
|
chain prerouting {
|
||||||
|
type nat hook prerouting priority dstnat;
|
||||||
|
iifname "eth0" tcp dport { 80, 443 } dnat to 10.88.0.2
|
||||||
|
iifname "eth0" udp dport 49152-49407 dnat to 10.88.0.2
|
||||||
|
}
|
||||||
|
chain postrouting {
|
||||||
|
type nat hook postrouting priority srcnat;
|
||||||
|
oifname "wg0" masquerade
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Home side:** run T1 or T2 exactly per [quickstart-docker.md](quickstart-docker.md), with:
|
||||||
|
|
||||||
|
- `RUTSTER_MEDIA_ADVERTISED_IP=<VPS-public-IP>` — callers send media UDP to the VPS; the DNAT
|
||||||
|
delivers it home through the tunnel.
|
||||||
|
- `RUTSTER_MEDIA_PORT_RANGE=49152-49407` (must match the nftables rule).
|
||||||
|
- Certificates: DNS-01 is the robust choice here ([certificates.md](certificates.md));
|
||||||
|
HTTP-01 also works since `:80` is forwarded.
|
||||||
|
|
||||||
|
First call: same two paths as [quickstart-docker.md](quickstart-docker.md) — browser at
|
||||||
|
`https://pbx.example.com/` (WebRTC now works — the UDP range rides the tunnel) and the Twilio
|
||||||
|
webhook at `https://pbx.example.com/v1/trunk/webhook`.
|
||||||
|
|
||||||
|
**Or skip the tunnel entirely:** run the engine *on* the VPS (that is just
|
||||||
|
[T1](quickstart-docker.md) on rented hardware). You trade at-home media for zero forwarding
|
||||||
|
complexity.
|
||||||
|
|
||||||
|
## Explicitly unsupported for production
|
||||||
|
|
||||||
|
Cloudflare Tunnel or ngrok in the live audio path: plaintext audio at the vendor edge
|
||||||
|
(unconsented subprocessor — DPA/BAA/PCI failure), documented mid-call WS terminations, zero
|
||||||
|
SLA, and Cloudflare's discretionary "disproportionate audio" ToS clause aimed at exactly this
|
||||||
|
traffic profile (research basis: [TLS brief §4](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)). Ratified in [ADR-0011](../adr/0011-deployment-topology.md).
|
||||||
Reference in New Issue
Block a user