From 33c3a145b060bdebecfe2ffb00ee194c3dd4517a Mon Sep 17 00:00:00 2001 From: "Aaron D. Lee" Date: Sun, 5 Jul 2026 22:40:34 -0400 Subject: [PATCH] =?UTF-8?q?docs(deploy):=20homelab.md=20=E2=80=94=20CGNAT?= =?UTF-8?q?=20truth:=20PSTN-only=20behind=20tunnels,=20VPS+WireGuard=20gra?= =?UTF-8?q?duation=20(slice-G)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Acceptance bar honored by honesty: the doc leads with what cannot work (no tunnel carries inbound UDP -> WebRTC unreachable behind CGNAT). Three tiers: ngrok blessed dev path with the free-tier arithmetic (~140 MB/ call-hour -> ~7 call-hours/month), Tailscale Funnel single-user demo (ciphertext-only), and the VPS+WireGuard graduation with TLS terminating at home (forwarder physically cannot read audio; media UDP DNAT restores WebRTC). cloudflared explicitly excluded even for dev (#1465, #1282). Signed-off-by: Aaron D. Lee --- docs/deploy/homelab.md | 132 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100644 docs/deploy/homelab.md diff --git a/docs/deploy/homelab.md b/docs/deploy/homelab.md new file mode 100644 index 0000000..b56cc29 --- /dev/null +++ b/docs/deploy/homelab.md @@ -0,0 +1,132 @@ +# Homelab & CGNAT — the honest story + +**The truth first: no tunnel carries inbound UDP. WebRTC callers are unreachable behind any + tunnel, so a homelab behind CGNAT is PSTN-only, period.** The engine's WebRTC media is +UDP-direct to an advertised IP (no STUN, no TURN); the CPaaS side offers no relay or +rendezvous either, and intermediaries in the media path are a documented frame-dropping hazard +([livekit/agents#3379](https://github.com/livekit/agents/issues/3379)). Nothing on this page +makes CGNAT production-grade for free. Three tiers, worst to best: + +## Tier 1 — dev/demo: ngrok (the blessed 5-minute path) + +ngrok is the **only** tunnel with a proven Twilio Media Streams record. Do **not** use +Cloudflare Tunnel even for dev: cloudflared has an open Twilio 31920 handshake bug +([cloudflared#1465](https://github.com/cloudflare/cloudflared/issues/1465)), recurring 1006 +closures ([cloudflared#1282](https://github.com/cloudflare/cloudflared/issues/1282)), and +Cloudflare documents killing WebSockets on edge code releases. + +```bash +# 1. Run the engine, plaintext :8080 (no Caddy needed — ngrok terminates TLS): +docker run -d --name rutster-engine --network host \ + -e RUTSTER_TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \ + -e RUTSTER_TWILIO_AUTH_TOKEN=your_auth_token \ + -e RUTSTER_TWILIO_MEDIA_BIND=0.0.0.0:8081 \ + -e RUTSTER_TWILIO_WEBHOOK_BASE=https://REPLACE-ME.ngrok-free.app \ + -e RUTSTER_TRUSTED_PROXIES=127.0.0.1/32 \ + git.adlee.work/alee/rutster-engine:latest +# (or from source: cargo run — see docs/QUICKSTART.md) + +# 2. Tunnel it: +ngrok http 8080 +# 3. Put the printed https://xxxx.ngrok-free.app URL into RUTSTER_TWILIO_WEBHOOK_BASE +# (restart the container), and into the Twilio number's webhook: +# POST https://xxxx.ngrok-free.app/v1/trunk/webhook +# 4. Dial your Twilio number. +``` + +**Free-tier arithmetic** (why this is a demo, not a deployment — research: [TLS brief §3(d)](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)): a Media Streams call is +8 kHz µ-law — 8 kB/s of audio per direction, base64-encoded inside a JSON envelope at 50 +messages/s, both directions ≈ **~140 MB per call-hour** on the wire. ngrok's free 1 GB/month +data cap ([ngrok pricing](https://ngrok.com/pricing)) therefore buys roughly **seven call-hours a month**. And the audio transits ngrok's +edge **in plaintext** (they terminate TLS) — an unconsented subprocessor, which is a +DPA/BAA/PCI failure. Dev only. Never production. + +## Tier 2 — single-user demo: Tailscale Funnel + +Privacy-clean variant: TLS terminates **on your node**, so Funnel relays ciphertext it cannot +read. + +```bash +tailscale funnel 8080 +# webhook base = https://..ts.net +``` + +Still PSTN-only (no inbound UDP), bandwidth cap undisclosed, one user. Unsizable beyond a +personal demo. + +## Tier 3 — production graduation: cheap VPS + WireGuard, TLS at home + +The recommended path. The VPS is a **dumb layer-4 forwarder**: TLS terminates at home (your +Caddy, DNS-01 certs), so the forwarder **physically cannot read the audio** — the strongest +privacy topology available. Bonus: forwarding the media UDP range over the same tunnel +restores WebRTC, which no tunnel product can do. Cost: one small VPS (~$5/mo) and one extra +network hop of media latency — pick a VPS region near home. + +Point DNS at the VPS: `pbx.example.com A `. + +**VPS `/etc/wireguard/wg0.conf`:** + +```ini +[Interface] +Address = 10.88.0.1/24 +ListenPort = 51820 +PrivateKey = + +[Peer] +PublicKey = +AllowedIPs = 10.88.0.2/32 +``` + +**Home box `/etc/wireguard/wg0.conf`** (home initiates — CGNAT-friendly; the keepalive holds +the NAT mapping): + +```ini +[Interface] +Address = 10.88.0.2/24 +PrivateKey = + +[Peer] +PublicKey = +Endpoint = :51820 +AllowedIPs = 10.88.0.0/24 +PersistentKeepalive = 25 +``` + +**VPS forwarding** (`sysctl -w net.ipv4.ip_forward=1`, persist it, then nftables): + +```nft +table ip rutster-fwd { + chain prerouting { + type nat hook prerouting priority dstnat; + iifname "eth0" tcp dport { 80, 443 } dnat to 10.88.0.2 + iifname "eth0" udp dport 49152-49407 dnat to 10.88.0.2 + } + chain postrouting { + type nat hook postrouting priority srcnat; + oifname "wg0" masquerade + } +} +``` + +**Home side:** run T1 or T2 exactly per [quickstart-docker.md](quickstart-docker.md), with: + +- `RUTSTER_MEDIA_ADVERTISED_IP=` — callers send media UDP to the VPS; the DNAT + delivers it home through the tunnel. +- `RUTSTER_MEDIA_PORT_RANGE=49152-49407` (must match the nftables rule). +- Certificates: DNS-01 is the robust choice here ([certificates.md](certificates.md)); + HTTP-01 also works since `:80` is forwarded. + +First call: same two paths as [quickstart-docker.md](quickstart-docker.md) — browser at +`https://pbx.example.com/` (WebRTC now works — the UDP range rides the tunnel) and the Twilio +webhook at `https://pbx.example.com/v1/trunk/webhook`. + +**Or skip the tunnel entirely:** run the engine *on* the VPS (that is just +[T1](quickstart-docker.md) on rented hardware). You trade at-home media for zero forwarding +complexity. + +## Explicitly unsupported for production + +Cloudflare Tunnel or ngrok in the live audio path: plaintext audio at the vendor edge +(unconsented subprocessor — DPA/BAA/PCI failure), documented mid-call WS terminations, zero +SLA, and Cloudflare's discretionary "disproportionate audio" ToS clause aimed at exactly this +traffic profile (research basis: [TLS brief §4](../superpowers/specs/2026-07-05-tls-edge-decision-brief.md)). Ratified in [ADR-0011](../adr/0011-deployment-topology.md).