9dfe00acd9
Direct response to docs/incidents.md #5, where `git clean -fdx` truncated router.db to 0 bytes and deleted .env, .venv and config/config.local.yaml. Recovery was luck -- a QA copy happened to exist in /tmp from 25 seconds earlier. There was no backup policy at all. llm-router-backup.sh + .service + .timer: hourly, keeps 24. Uses sqlite3 .backup rather than cp, because copying a live database with an open writer can capture a torn page set that passes a size check and fails integrity_check. It verifies the new snapshot with integrity_check BEFORE rotating, so a failing run never leaves fewer copies than it started with. workstation-backup.sh: tarballs what git does not have -- router.db, .env, config/config.local.yaml, .omo/ (plans and evidence ledger), tuned systemd units, opencode plugins, and this project's Claude memory -- plus a RESTORE.md with the clone -> venv -> restore sequence. Excludes .venv and node_modules (rebuildable from pinned requirements) and Ollama models (~30GB, re-pullable, recipes in docs/local-models.md). Backups write OUTSIDE the repository by design. A backup kept inside it, even gitignored, would have been destroyed by the same command that caused the incident. Test-restored before committing: 22,821 observations with integrity=ok, API key present, tariff intact, 7 systemd units, 2 opencode plugins, 11 memory files. That test caught a second loss nobody had noticed -- .omo/plans had also been destroyed by the same clean, taking 35 plan artifacts and 127 evidence files, since .omo/ is gitignored too. Recovered from the same QA copy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRQXz5SYZYVWscxS1QqF6U
15 lines
416 B
SYSTEMD
15 lines
416 B
SYSTEMD
# Hourly, with 24 kept by default -- roughly a day of hourly granularity.
|
|
# Deliberately more frequent than the 2h poller: the poller refetches a remote
|
|
# catalog that can always be refetched, while energy_observations and
|
|
# route_decisions exist nowhere else.
|
|
[Unit]
|
|
Description=Hourly snapshot of the LLM router database
|
|
|
|
[Timer]
|
|
OnBootSec=5min
|
|
OnUnitActiveSec=1h
|
|
Persistent=true
|
|
|
|
[Install]
|
|
WantedBy=timers.target
|