18 tasks covering README, home user guide, operator guide, architecture overview + conventions, and 12 per-subsystem reference docs. Each task is grounded in reading real source to avoid invented facts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1721 lines
55 KiB
Markdown
1721 lines
55 KiB
Markdown
# Project Documentation Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Create a polished top-level `README.md`, a linear home user guide, a reference operator guide, and a split architectural reference (overview + conventions + 12 per-subsystem files) for Vigilar.
|
||
|
||
**Architecture:** Approach 3 (Hybrid) from the spec — monolithic linear guides for users, split reference docs for contributors. No tooling added, no code changes, no image assets. Plain Markdown organized as if MkDocs-ready.
|
||
|
||
**Tech Stack:** Markdown only. No build step. All claims must be grounded in the actual code under `/home/alee/Sources/vigilar/` at write-time.
|
||
|
||
**Spec:** `docs/superpowers/specs/2026-04-05-project-documentation-design.md` — read it before starting.
|
||
|
||
---
|
||
|
||
## Universal rules for every task in this plan
|
||
|
||
1. **Do not guess.** Every file path, command, TOML key, MQTT topic, DB table, CLI subcommand, blueprint route, and function/file reference must be verified against the actual repo before writing. If you cannot verify something, omit it — or, for subsystem docs, write `"no publishers found at time of writing"` (or equivalent).
|
||
2. **Do not create code, scripts, unit files, or images.** This effort is docs-only. The only exception is creating the `docs/architecture/` and `docs/architecture/subsystems/` directories implicitly by writing files into them.
|
||
3. **Do not modify `docs/camera-hardware-guide.md` or anything under `docs/superpowers/`** other than this plan file.
|
||
4. **Commit after each task.** One commit per task unless a task says otherwise.
|
||
5. **Tone:** plain, direct, no marketing fluff, no emoji, no exclamation marks. Match the tone of `CLAUDE.md` and `scripts/backup.sh` comments.
|
||
6. **Line wrap:** keep prose at ~100 columns for readability in terminals and diffs.
|
||
7. **Relative links:** docs inside `docs/` link to each other with relative paths (e.g. `architecture/overview.md`). The top-level README links use `docs/...` paths.
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
Files to be created in this plan (17 total):
|
||
|
||
```
|
||
README.md # new
|
||
docs/home-user-guide.md # new
|
||
docs/operator-guide.md # new
|
||
docs/architecture/overview.md # new
|
||
docs/architecture/conventions.md # new
|
||
docs/architecture/subsystems/camera.md # new
|
||
docs/architecture/subsystems/detection.md # new
|
||
docs/architecture/subsystems/events.md # new
|
||
docs/architecture/subsystems/alerts.md # new
|
||
docs/architecture/subsystems/sensors.md # new
|
||
docs/architecture/subsystems/ups.md # new
|
||
docs/architecture/subsystems/storage.md # new
|
||
docs/architecture/subsystems/highlights.md # new
|
||
docs/architecture/subsystems/presence.md # new
|
||
docs/architecture/subsystems/pets.md # new
|
||
docs/architecture/subsystems/health.md # new
|
||
docs/architecture/subsystems/web.md # new
|
||
```
|
||
|
||
Tasks are ordered so that later tasks can reference and link to earlier ones.
|
||
|
||
---
|
||
|
||
## Task 1: Architecture overview
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/overview.md`
|
||
|
||
- [ ] **Step 1: Gather grounding facts by reading source**
|
||
|
||
Read the following files and take notes. Do not write prose yet.
|
||
|
||
```bash
|
||
ls vigilar/
|
||
ls vigilar/camera vigilar/events vigilar/alerts vigilar/storage vigilar/web/blueprints
|
||
```
|
||
|
||
Read these files in full:
|
||
- `CLAUDE.md`
|
||
- `config/vigilar.toml`
|
||
- `vigilar/camera/worker.py`
|
||
- `vigilar/camera/motion.py`
|
||
- `vigilar/camera/recorder.py`
|
||
- `vigilar/camera/ring_buffer.py`
|
||
- `vigilar/events/processor.py`
|
||
- `vigilar/events/rules.py`
|
||
- `vigilar/storage/db.py`
|
||
- `vigilar/storage/schema.py`
|
||
- `vigilar/storage/encryption.py`
|
||
- `vigilar/web/__init__.py`
|
||
- `systemd/vigilar.service`
|
||
- `systemd/vigilar-mosquitto.conf`
|
||
|
||
From these, write down on scratchpad:
|
||
- The exact MQTT topic format used (look for `mqtt.publish` / `client.publish` calls and topic string literals).
|
||
- The actual table names in `vigilar/storage/schema.py`.
|
||
- The file+function name where RTSP frames are captured.
|
||
- The file+function name where motion is detected.
|
||
- The file+function name where a recording is started.
|
||
- The file+function name where an event row is written.
|
||
- The file+function name where a push notification is sent.
|
||
- Whether the process model is actually `multiprocessing.Process` (check `vigilar/cli/cmd_start.py` or wherever subsystems are spawned).
|
||
|
||
If any of these cannot be found, note them as "unverified" and they will be omitted from the doc.
|
||
|
||
- [ ] **Step 2: Create `docs/architecture/overview.md`**
|
||
|
||
Write the file with these sections, in this order. Each section's content must reflect what you actually found in Step 1, not generic text.
|
||
|
||
```markdown
|
||
# Vigilar Architecture Overview
|
||
|
||
This document explains how Vigilar is put together for someone reading the
|
||
codebase for the first time. It is short on purpose — per-subsystem details
|
||
live under `subsystems/`.
|
||
|
||
## Design principles
|
||
|
||
- **Offline-first.** No external calls in the critical path. Cloud integrations,
|
||
if any, are opt-in and off the hot path.
|
||
- **Subsystem isolation.** Each subsystem runs in its own process. A crash in
|
||
one subsystem cannot take down another.
|
||
- **Loose coupling via MQTT.** Subsystems do not call each other directly.
|
||
They publish and subscribe to a local Mosquitto broker on
|
||
`127.0.0.1:1883`.
|
||
- **SQLite (WAL) is the single durable store.** Access goes through
|
||
SQLAlchemy Core expressions, not ORM mapped classes.
|
||
- **Adaptive cost.** Cameras idle at 2 FPS and jump to 30 FPS on motion, with
|
||
a 5-second ring buffer so the moment leading up to the trigger is kept.
|
||
- **Configuration is typed.** `config/vigilar.toml` is loaded and validated
|
||
by Pydantic v2. Secrets are never inline — they are file paths under
|
||
`/etc/vigilar/secrets/`.
|
||
|
||
## Process topology
|
||
|
||
<ASCII diagram — draw this based on what you actually find in cmd_start.py.
|
||
It should show: supervisor → N subsystem processes → mosquitto broker →
|
||
flask web process. Example shape:>
|
||
|
||
```
|
||
┌────────────────┐
|
||
│ vigilar start │ (supervisor)
|
||
└───────┬────────┘
|
||
│ spawns
|
||
┌────────┬────────┬──────┼──────┬────────┬────────┐
|
||
▼ ▼ ▼ ▼ ▼ ▼ ▼
|
||
camera sensors events ups alerts health web
|
||
↑ ↑ ↑ ↑ ↑ ↑ ↑
|
||
└─────────┴────────┴──────┴──────┴───────┴────────┘
|
||
MQTT (127.0.0.1:1883)
|
||
```
|
||
|
||
## The MQTT bus
|
||
|
||
- Broker: Mosquitto, bound to loopback only. Config:
|
||
`systemd/vigilar-mosquitto.conf`.
|
||
- Topic convention: `<convention you actually found>` — quote one or two
|
||
real topics from the codebase.
|
||
- Why MQTT rather than an in-process queue: crash isolation, introspection
|
||
with `mosquitto_sub`, and the option to move subsystems to separate hosts
|
||
later without changing the wire format.
|
||
|
||
## Data flow: from motion to phone notification
|
||
|
||
Numbered sequence. Each step names the real file and function where the
|
||
work happens.
|
||
|
||
1. `vigilar/camera/worker.py:<func>` — RTSP capture at 2 FPS.
|
||
2. `vigilar/camera/motion.py:<func>` — OpenCV MOG2 detects movement and
|
||
publishes `<real topic>`.
|
||
3. `vigilar/camera/recorder.py:<func>` — flips to 30 FPS, flushes the ring
|
||
buffer, begins writing a `.vge` file.
|
||
4. `vigilar/events/processor.py:<func>` — inserts a row into `<table>` and
|
||
publishes `<real topic>`.
|
||
5. `vigilar/highlights/...` — scores the event.
|
||
6. `vigilar/alerts/...` — sends a VAPID web-push notification.
|
||
7. Web UI — updates the timeline (mechanism: <SSE / polling / whatever you
|
||
find>).
|
||
|
||
## Storage layout
|
||
|
||
- `vigilar.db` (SQLite, WAL mode). Tables: <list from schema.py>.
|
||
- Recordings: `.vge` files, AES-256-GCM. Key at
|
||
`/etc/vigilar/secrets/storage.key`. Losing the key means losing the
|
||
recordings.
|
||
- HLS: rolling segments under `[system] hls_dir` (default
|
||
`/var/vigilar/hls`).
|
||
- Backups: DB + `/etc/vigilar` tarball at `[system]`-adjacent path (see
|
||
`scripts/backup.sh`).
|
||
|
||
## Configuration and secrets
|
||
|
||
- `config/vigilar.toml` is the only configuration file the app reads.
|
||
- It is validated by Pydantic v2 at startup — invalid config stops the
|
||
process before any subsystem spawns.
|
||
- Secrets never live in the TOML. They live as files under
|
||
`/etc/vigilar/secrets/` and the TOML only references paths.
|
||
- The arm PIN and admin password are hashed; comparisons are
|
||
constant-time.
|
||
|
||
## The web tier
|
||
|
||
- Flask with Blueprints, one per feature area
|
||
(`vigilar/web/blueprints/*.py`).
|
||
- Jinja2 templates, Bootstrap 5 dark theme.
|
||
- Live view: `hls.js` grid for bandwidth efficiency, MJPEG single view for
|
||
low latency.
|
||
- PWA with VAPID push — no Firebase, no Google Cloud Messaging.
|
||
|
||
## What is NOT in the critical path
|
||
|
||
- Remote access (`[remote]` section) — optional, bandwidth-shaped.
|
||
- Email alerts (`[alerts.email]`) — optional.
|
||
- Any cloud service — never.
|
||
|
||
## Where to go next
|
||
|
||
- Conventions: `conventions.md`
|
||
- Per-subsystem details: `subsystems/`
|
||
```
|
||
|
||
Replace every `<placeholder>` with the real value from Step 1. If a fact
|
||
cannot be verified, delete the bullet or sentence entirely rather than
|
||
guessing.
|
||
|
||
- [ ] **Step 3: Verify all claims against source**
|
||
|
||
For every file path referenced in the overview, run:
|
||
|
||
```bash
|
||
ls <path>
|
||
```
|
||
|
||
For every topic string quoted, run:
|
||
|
||
```
|
||
(use Grep tool, NOT bash grep)
|
||
Grep for the topic string across vigilar/ — must return at least one hit.
|
||
```
|
||
|
||
For every table name quoted, confirm it appears in `vigilar/storage/schema.py`.
|
||
|
||
Fix any that fail.
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
git add docs/architecture/overview.md
|
||
git commit -m "docs: add architecture overview"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 2: Architecture conventions
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/conventions.md`
|
||
|
||
- [ ] **Step 1: Re-read `CLAUDE.md` and `vigilar/constants.py`**
|
||
|
||
```bash
|
||
ls vigilar/constants.py # confirm it exists
|
||
```
|
||
|
||
Read `vigilar/constants.py` in full and note the `StrEnum` style actually
|
||
used.
|
||
|
||
- [ ] **Step 2: Create `docs/architecture/conventions.md`**
|
||
|
||
Target length ~400 words. Write the file:
|
||
|
||
```markdown
|
||
# Coding Conventions
|
||
|
||
These are the rules a contributor needs in their head before touching
|
||
Vigilar. They are distilled from `CLAUDE.md` and from the patterns already
|
||
in the codebase.
|
||
|
||
## Language and style
|
||
|
||
- Python 3.11+.
|
||
- Ruff for linting. Line length 100.
|
||
- Type hints on all public functions. Internal helpers may omit them when
|
||
obvious.
|
||
- No docstrings unless the logic is non-obvious. Let short, well-named
|
||
functions speak for themselves.
|
||
|
||
## String constants
|
||
|
||
All string constants that are referenced in more than one place live in
|
||
`vigilar/constants.py` as `StrEnum` members. This includes MQTT topics,
|
||
event kinds, alert channels, and anything else that would otherwise be a
|
||
magic string. If you are about to write `"motion_detected"` in two places,
|
||
add it to the enum instead.
|
||
|
||
## Database access
|
||
|
||
- SQLite in WAL mode.
|
||
- SQLAlchemy Core expressions only — no mapped classes, no ORM session.
|
||
- Schema lives in `vigilar/storage/schema.py`. Queries live in
|
||
`vigilar/storage/queries.py`.
|
||
- New tables go in `schema.py`. New query helpers go in `queries.py`. Do
|
||
not scatter ad-hoc SQL across subsystem code.
|
||
|
||
## Processes and the MQTT bus
|
||
|
||
- Each subsystem runs in its own `multiprocessing.Process`.
|
||
- Subsystems communicate only through the local MQTT broker. They do not
|
||
share memory or call each other's functions across process boundaries.
|
||
- Topic naming: <quote the convention you found in Task 1, e.g.
|
||
`vigilar/<subsystem>/<entity>/<event>`>.
|
||
- If you are tempted to reach into another subsystem directly, publish a
|
||
topic instead.
|
||
|
||
## Configuration
|
||
|
||
- `config/vigilar.toml` is the only config file the app reads at runtime.
|
||
- It is validated by Pydantic v2. Add new fields to the Pydantic models,
|
||
not just to the TOML.
|
||
- Secrets are file paths. Never put a key, password, or token directly in
|
||
the TOML.
|
||
|
||
## Testing
|
||
|
||
- `pytest` from the repo root.
|
||
- Tests live under `tests/`.
|
||
|
||
## Committing
|
||
|
||
- `ruff check vigilar/` must pass.
|
||
- `pytest` must pass.
|
||
- One logical change per commit. Commit often.
|
||
```
|
||
|
||
Replace `<quote the convention you found in Task 1>` with the actual
|
||
topic convention observed.
|
||
|
||
- [ ] **Step 3: Verify**
|
||
|
||
```bash
|
||
ls vigilar/constants.py vigilar/storage/schema.py vigilar/storage/queries.py
|
||
```
|
||
|
||
All three must exist. If `queries.py` does not exist, remove the reference
|
||
to it from the doc.
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
git add docs/architecture/conventions.md
|
||
git commit -m "docs: add coding conventions reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 3: Subsystem doc template and the `camera` subsystem
|
||
|
||
The next 12 tasks (3–14) all follow the same template. Task 3 is written
|
||
in full. Tasks 4–14 reuse the same template verbatim and only change which
|
||
subsystem is being documented.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/camera.md`
|
||
|
||
- [ ] **Step 1: Enumerate the subsystem's files**
|
||
|
||
```bash
|
||
ls vigilar/camera/
|
||
```
|
||
|
||
Read every `.py` file in the directory. For each one, note in 1 sentence
|
||
what it is responsible for.
|
||
|
||
- [ ] **Step 2: Find MQTT topics**
|
||
|
||
Use the Grep tool (NOT bash grep) to find what this subsystem publishes
|
||
and subscribes to:
|
||
|
||
```
|
||
Grep pattern: publish\(|subscribe\( path: vigilar/camera output_mode: content
|
||
```
|
||
|
||
Write down every topic string literal you see, and whether it is being
|
||
published or subscribed to. If a topic comes from `vigilar/constants.py`,
|
||
resolve it.
|
||
|
||
- [ ] **Step 3: Find DB tables touched**
|
||
|
||
```
|
||
Grep pattern: from vigilar.storage path: vigilar/camera output_mode: content
|
||
Grep pattern: \.insert\(|\.select\(|\.update\(|\.delete\( path: vigilar/camera output_mode: content
|
||
```
|
||
|
||
Note any table names referenced. If none, the Database section will say
|
||
"none".
|
||
|
||
- [ ] **Step 4: Write the file**
|
||
|
||
Create `docs/architecture/subsystems/camera.md` using this exact template.
|
||
Fill every section from the evidence gathered in Steps 1–3.
|
||
|
||
```markdown
|
||
# camera
|
||
|
||
## Purpose
|
||
|
||
<One paragraph — what this subsystem owns. For the camera subsystem: RTSP
|
||
capture, adaptive FPS, motion detection, HLS segment production, recording
|
||
to encrypted .vge files, ring buffer for pre-motion context.>
|
||
|
||
## Key files
|
||
|
||
- `vigilar/camera/worker.py` — <role>
|
||
- `vigilar/camera/manager.py` — <role>
|
||
- `vigilar/camera/motion.py` — <role>
|
||
- `vigilar/camera/recorder.py` — <role>
|
||
- `vigilar/camera/ring_buffer.py` — <role>
|
||
- `vigilar/camera/hls.py` — <role>
|
||
|
||
## MQTT topics
|
||
|
||
**Subscribes:** <list, or "none found">
|
||
**Publishes:** <list, or "none found">
|
||
|
||
## Database tables
|
||
|
||
<Table name and what it holds, or "none">
|
||
|
||
## Depends on
|
||
|
||
- <sister subsystem> — <why / how>
|
||
- ... or "none"
|
||
|
||
## Consumed by
|
||
|
||
- <sister subsystem> — <why / how>
|
||
- ... or "none"
|
||
|
||
## Notes
|
||
|
||
<Only include if something is genuinely non-obvious. Examples for camera:
|
||
why MOG2 over background subtraction, why the ring buffer is 5 seconds,
|
||
why HLS is used for the grid and MJPEG for single view. Otherwise omit
|
||
this section entirely.>
|
||
```
|
||
|
||
If you genuinely cannot find any MQTT publishers in this subsystem, write
|
||
the literal text `"No MQTT publishers found at time of writing."` under
|
||
**Publishes:**. Same for subscribes. **Do not invent topics.**
|
||
|
||
- [ ] **Step 5: Verify**
|
||
|
||
```bash
|
||
ls vigilar/camera/worker.py vigilar/camera/manager.py vigilar/camera/motion.py \
|
||
vigilar/camera/recorder.py vigilar/camera/ring_buffer.py vigilar/camera/hls.py
|
||
```
|
||
|
||
All must exist. If any do not, remove them from the Key Files list.
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/camera.md
|
||
git commit -m "docs: add camera subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 4: `detection` subsystem
|
||
|
||
Follow the exact same 6-step procedure as Task 3, but for the `detection`
|
||
subsystem.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/detection.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/detection/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` and `subscribe(` in `vigilar/detection`.
|
||
- [ ] **Step 3:** Grep for DB access in `vigilar/detection`.
|
||
- [ ] **Step 4:** Write `docs/architecture/subsystems/detection.md` using
|
||
the template from Task 3. Fill from evidence. No guessing.
|
||
- [ ] **Step 5:** Verify every file path in the Key Files list exists.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/detection.md
|
||
git commit -m "docs: add detection subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 5: `events` subsystem
|
||
|
||
Follow the Task 3 procedure for `events`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/events.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/events/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` and `subscribe(` in `vigilar/events`.
|
||
- [ ] **Step 3:** Grep for DB access in `vigilar/events`. This subsystem
|
||
almost certainly writes to an events table — name it.
|
||
- [ ] **Step 4:** Write `docs/architecture/subsystems/events.md` using
|
||
the template from Task 3. Fill from evidence.
|
||
- [ ] **Step 5:** Verify Key Files paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/events.md
|
||
git commit -m "docs: add events subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 6: `alerts` subsystem
|
||
|
||
Follow the Task 3 procedure for `alerts`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/alerts.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/alerts/` and read every `.py` file. Expect
|
||
sub-channels for local (syslog/desktop), web push (VAPID), and
|
||
email.
|
||
- [ ] **Step 2:** Grep for `publish(` and `subscribe(` in `vigilar/alerts`.
|
||
Alerts almost certainly subscribe to event topics.
|
||
- [ ] **Step 3:** Grep for DB access in `vigilar/alerts`.
|
||
- [ ] **Step 4:** Write the doc. In **Notes**, mention the VAPID / PWA
|
||
push path if and only if you find the code for it (do not guess).
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/alerts.md
|
||
git commit -m "docs: add alerts subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 7: `sensors` subsystem
|
||
|
||
Follow the Task 3 procedure for `sensors`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/sensors.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/sensors/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(` in `vigilar/sensors`.
|
||
Sensors likely integrate with Zigbee2MQTT — look for
|
||
`zigbee2mqtt` topic prefixes too.
|
||
- [ ] **Step 3:** Grep for DB access in `vigilar/sensors`.
|
||
- [ ] **Step 4:** Write the doc. Mention Zigbee2MQTT in **Depends on** if
|
||
the code actually subscribes to its topics.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/sensors.md
|
||
git commit -m "docs: add sensors subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 8: `ups` subsystem
|
||
|
||
Follow the Task 3 procedure for `ups`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/ups.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/ups/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(` in `vigilar/ups`.
|
||
- [ ] **Step 3:** Grep for DB access in `vigilar/ups`.
|
||
- [ ] **Step 4:** Write the doc. In **Notes**, mention that this subsystem
|
||
polls NUT (`127.0.0.1:3493` by default, from `[ups]` in the TOML)
|
||
and triggers graceful shutdown on critical battery — but only if
|
||
you can point to the code that does each of those things.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/ups.md
|
||
git commit -m "docs: add ups subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 9: `storage` subsystem
|
||
|
||
Follow the Task 3 procedure for `storage`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/storage.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/storage/` and read `db.py`, `schema.py`,
|
||
`encryption.py`, `queries.py`, `__init__.py`.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(` in `vigilar/storage`
|
||
— this subsystem may have none, which is fine.
|
||
- [ ] **Step 3:** This IS the DB layer, so the **Database tables**
|
||
section should list every table defined in `schema.py`, with a one
|
||
phrase description of each.
|
||
- [ ] **Step 4:** Write the doc. In **Notes**, explain the `.vge`
|
||
encryption format (AES-256-GCM) and where the key lives
|
||
(`/etc/vigilar/secrets/storage.key`) — but only if `encryption.py`
|
||
actually implements that. Quote the function name.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/storage.md
|
||
git commit -m "docs: add storage subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 10: `highlights` subsystem
|
||
|
||
Follow the Task 3 procedure for `highlights`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/highlights.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/highlights/` and read every `.py` file.
|
||
Expect scoring, reel assembly, and possibly timelapse code.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(`.
|
||
- [ ] **Step 3:** Grep for DB access.
|
||
- [ ] **Step 4:** Write the doc. Mention in **Notes** how scoring works
|
||
if the code makes it obvious (e.g. FFmpeg clip assembly, event
|
||
score thresholds). Do not invent scoring weights.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/highlights.md
|
||
git commit -m "docs: add highlights subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 11: `presence` subsystem
|
||
|
||
Follow the Task 3 procedure for `presence`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/presence.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/presence/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(`.
|
||
- [ ] **Step 3:** Grep for DB access.
|
||
- [ ] **Step 4:** Write the doc. If you cannot figure out what this
|
||
subsystem does from its code, say so honestly in **Purpose** —
|
||
e.g. "Tracks resident presence state. Mechanism not documented at
|
||
time of writing; see source for details."
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/presence.md
|
||
git commit -m "docs: add presence subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 12: `pets` subsystem
|
||
|
||
Follow the Task 3 procedure for `pets`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/pets.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/pets/` and read every `.py` file.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(`.
|
||
- [ ] **Step 3:** Grep for DB access.
|
||
- [ ] **Step 4:** Write the doc.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/pets.md
|
||
git commit -m "docs: add pets subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 13: `health` subsystem
|
||
|
||
Follow the Task 3 procedure for `health`.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/health.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/health/` — expected files: `digest.py`,
|
||
`monitor.py`, `pruner.py`, `__init__.py`. Read all four.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(`.
|
||
- [ ] **Step 3:** Grep for DB access. The pruner likely deletes old rows
|
||
and old recordings — note what it prunes.
|
||
- [ ] **Step 4:** Write the doc. In **Notes**, describe what the pruner
|
||
enforces (e.g. `max_disk_usage_gb`, `free_space_floor_gb` from the
|
||
`[storage]` TOML section) — but only if the code actually reads
|
||
those settings. Quote the variable name.
|
||
- [ ] **Step 5:** Verify paths.
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/health.md
|
||
git commit -m "docs: add health subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 14: `web` subsystem
|
||
|
||
Follow the Task 3 procedure for `web`. This one is slightly different:
|
||
the **Key files** list should enumerate every blueprint, and the
|
||
**Purpose** should name the HTTP-facing surface.
|
||
|
||
**Files:**
|
||
- Create: `docs/architecture/subsystems/web.md`
|
||
|
||
- [ ] **Step 1:** `ls vigilar/web/blueprints/` and read each blueprint's
|
||
top-of-file routes. Also read `vigilar/web/__init__.py` to find the
|
||
Flask app factory.
|
||
- [ ] **Step 2:** Grep for `publish(` / `subscribe(` in `vigilar/web`.
|
||
The web tier likely subscribes to event topics for live updates —
|
||
verify.
|
||
- [ ] **Step 3:** Grep for DB access. Blueprints likely use
|
||
`vigilar.storage.queries`.
|
||
- [ ] **Step 4:** Write `docs/architecture/subsystems/web.md`. Under
|
||
**Key files**, list each blueprint from
|
||
`vigilar/web/blueprints/` with a one-sentence description of what
|
||
it serves:
|
||
|
||
```
|
||
- `vigilar/web/blueprints/cameras.py` — live view and camera management UI
|
||
- `vigilar/web/blueprints/events.py` — event timeline
|
||
- `vigilar/web/blueprints/recordings.py` — recording playback
|
||
- `vigilar/web/blueprints/sensors.py` — sensor status
|
||
- `vigilar/web/blueprints/system.py` — system health, settings, PIN
|
||
- `vigilar/web/blueprints/kiosk.py` — wall display mode
|
||
- `vigilar/web/blueprints/pets.py` — pet tracking UI
|
||
- `vigilar/web/blueprints/visitors.py` — visitor tracking UI
|
||
- `vigilar/web/blueprints/wildlife.py` — wildlife tracking UI
|
||
```
|
||
|
||
Adjust each description based on the actual routes you read.
|
||
|
||
In **Notes**, mention: Jinja2 + Bootstrap 5 dark, `hls.js` for the grid,
|
||
MJPEG for single-camera low-latency, and PWA + VAPID push — but only
|
||
the items you can locate in the codebase.
|
||
|
||
- [ ] **Step 5:** Verify paths.
|
||
|
||
```bash
|
||
ls vigilar/web/blueprints/cameras.py vigilar/web/blueprints/events.py \
|
||
vigilar/web/blueprints/recordings.py vigilar/web/blueprints/sensors.py \
|
||
vigilar/web/blueprints/system.py vigilar/web/blueprints/kiosk.py \
|
||
vigilar/web/blueprints/pets.py vigilar/web/blueprints/visitors.py \
|
||
vigilar/web/blueprints/wildlife.py
|
||
```
|
||
|
||
- [ ] **Step 6:** Commit.
|
||
|
||
```bash
|
||
git add docs/architecture/subsystems/web.md
|
||
git commit -m "docs: add web subsystem reference"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 15: Home user guide
|
||
|
||
**Files:**
|
||
- Create: `docs/home-user-guide.md`
|
||
|
||
- [ ] **Step 1: Re-read the grounding sources**
|
||
|
||
```bash
|
||
ls scripts/install.sh scripts/gen_vapid_keys.sh scripts/gen_cert.sh \
|
||
scripts/backup.sh systemd/vigilar.service
|
||
```
|
||
|
||
Read `scripts/install.sh` in full so you can accurately describe in 3
|
||
bullets what it does (install system deps, create vigilar user, set up
|
||
venv, install the package, generate VAPID keys, install the systemd unit
|
||
— confirm each against the actual script).
|
||
|
||
Read `config/vigilar.toml` and confirm the web port default is `49735`
|
||
under `[web]`. If it has changed, use the actual value.
|
||
|
||
- [ ] **Step 2: Create `docs/home-user-guide.md`**
|
||
|
||
Write the file with exactly these sections, in this order. Keep the tone
|
||
direct and practical. Target length 1500–2500 words.
|
||
|
||
```markdown
|
||
# Vigilar Home User Guide
|
||
|
||
This guide takes you from a bare mini PC to working cameras on your phone.
|
||
It assumes you are comfortable with a Linux command line and can SSH into
|
||
a machine on your network.
|
||
|
||
If you are running Vigilar on a server you already administer, see the
|
||
[Operator Guide](operator-guide.md) instead.
|
||
|
||
## What you will end up with
|
||
|
||
- A small always-on box in your house that records from your IP cameras.
|
||
- A web UI at `http://<mini-pc-ip>:49735` on your LAN.
|
||
- Push notifications on your phone when motion is detected.
|
||
- Optional nightly config + database backup to a NAS share.
|
||
|
||
```
|
||
[IP cameras] ──RTSP──▶ [mini PC running Vigilar] ──LAN──▶ [your phone/browser]
|
||
│
|
||
└── optional nightly ──▶ [NAS]
|
||
```
|
||
|
||
## What you need
|
||
|
||
- A mini PC. x86_64, 4 GB RAM minimum, 128 GB SSD minimum. Any modern
|
||
Intel NUC, Beelink, MinisForum, or similar will do.
|
||
- A USB stick (8 GB or larger) for installing Linux.
|
||
- One or more RTSP IP cameras on your LAN. For picks, see the
|
||
[Camera Hardware Guide](camera-hardware-guide.md).
|
||
- A phone for push notifications.
|
||
- Optional: a NAS on your LAN for backups.
|
||
|
||
## Step 1 — Install Linux on the mini PC
|
||
|
||
Install Debian 12 or Ubuntu Server 24.04 (or later) on the mini PC. The
|
||
official installers walk you through it. During install:
|
||
|
||
- Choose **OpenSSH server** when asked for tasks/components.
|
||
- Set a hostname you will remember (e.g. `vigilar`).
|
||
- Create a user for yourself. You will need `sudo`.
|
||
|
||
After install, note the machine's LAN IP address (`ip addr` on the box)
|
||
and make sure you can SSH in from your laptop.
|
||
|
||
## Step 2 — Install Vigilar
|
||
|
||
SSH into the mini PC and run:
|
||
|
||
```bash
|
||
git clone <repo URL> vigilar
|
||
cd vigilar
|
||
sudo ./scripts/install.sh
|
||
```
|
||
|
||
`install.sh` will:
|
||
|
||
- Install system dependencies (ffmpeg, mosquitto, Python 3, NUT client).
|
||
- Create a dedicated `vigilar` system user.
|
||
- Set up a Python virtualenv at `/opt/vigilar/venv`.
|
||
- Generate VAPID keys for web push notifications.
|
||
- Install the systemd unit and Mosquitto broker config.
|
||
|
||
Review the `[INFO]` output as it runs. Anything marked `[FAIL]` needs
|
||
attention before you continue.
|
||
|
||
## Step 3 — Start it up
|
||
|
||
```bash
|
||
sudo systemctl enable --now vigilar
|
||
sudo systemctl status vigilar
|
||
```
|
||
|
||
You should see `active (running)`. If it is not, run
|
||
`journalctl -u vigilar -e` and look for the first `ERROR` line.
|
||
|
||
Open a browser on any device on the same LAN:
|
||
|
||
```
|
||
http://<mini-pc-ip>:49735
|
||
```
|
||
|
||
The port is configurable under `[web] port` in
|
||
`/etc/vigilar/vigilar.toml` if 49735 conflicts with something else you
|
||
run.
|
||
|
||
## Step 4 — Set your PIN
|
||
|
||
On the first visit you will be asked to set an arm/disarm PIN. Choose
|
||
something you will remember. The PIN is stored as a hash — there is no
|
||
recovery. If you lose it you will need to reset it from the command line
|
||
with `vigilar config set-pin` on the mini PC.
|
||
|
||
> **Screenshot placeholder:** `docs/images/first-run-pin.png`
|
||
|
||
## Step 5 — Add your first camera
|
||
|
||
From the web UI, go to **Cameras → Add camera**. You will need:
|
||
|
||
- A name for the camera (e.g. "Front door").
|
||
- The camera's RTSP URL. Format varies by vendor — see the
|
||
[Camera Hardware Guide](camera-hardware-guide.md) for the common ones.
|
||
- Username and password for the camera's RTSP stream (not the camera's
|
||
web UI login, if they are different).
|
||
|
||
Click **Test** to confirm Vigilar can pull a frame, then **Save**.
|
||
|
||
The camera will appear in the live grid. It idles at 2 FPS to save CPU
|
||
and jumps to 30 FPS when motion is detected.
|
||
|
||
> **Screenshot placeholder:** `docs/images/add-camera.png`
|
||
|
||
## Step 6 — Phone push notifications
|
||
|
||
Vigilar runs as a Progressive Web App. To get notifications on your
|
||
phone:
|
||
|
||
1. Open `http://<mini-pc-ip>:49735` in your phone's browser on the same
|
||
Wi-Fi.
|
||
2. Use your browser's "Add to Home Screen" option.
|
||
3. Open the installed app from your home screen.
|
||
4. When prompted, allow notifications.
|
||
|
||
Under the hood, `install.sh` already generated the VAPID keys at
|
||
`/etc/vigilar/secrets/vapid_private.pem`. You do not need to do anything
|
||
else.
|
||
|
||
## Step 7 — Optional: NAS backup
|
||
|
||
This step backs up your SQLite database and `/etc/vigilar` (config and
|
||
secrets) to a NAS share once a day. It does **not** back up recordings —
|
||
recordings stay on the mini PC's local disk. Recording backup is planned
|
||
as a later improvement.
|
||
|
||
### Mount the NAS share
|
||
|
||
If your NAS offers NFS:
|
||
|
||
```bash
|
||
sudo apt-get install -y nfs-common
|
||
sudo mkdir -p /mnt/nas/vigilar-backups
|
||
echo "nas.lan:/volume1/vigilar-backups /mnt/nas/vigilar-backups nfs defaults,_netdev 0 0" | sudo tee -a /etc/fstab
|
||
sudo mount -a
|
||
```
|
||
|
||
Replace `nas.lan:/volume1/vigilar-backups` with your NAS's actual export
|
||
path.
|
||
|
||
### Point `backup.sh` at the mount and schedule it
|
||
|
||
Create `/etc/systemd/system/vigilar-backup.service`:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=Vigilar nightly backup
|
||
After=vigilar.service
|
||
|
||
[Service]
|
||
Type=oneshot
|
||
Environment=VIGILAR_BACKUP_DIR=/mnt/nas/vigilar-backups
|
||
Environment=VIGILAR_BACKUP_RETENTION_DAYS=30
|
||
ExecStart=/opt/vigilar/scripts/backup.sh
|
||
```
|
||
|
||
Create `/etc/systemd/system/vigilar-backup.timer`:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=Run vigilar backup nightly
|
||
|
||
[Timer]
|
||
OnCalendar=daily
|
||
Persistent=true
|
||
|
||
[Install]
|
||
WantedBy=timers.target
|
||
```
|
||
|
||
Enable it:
|
||
|
||
```bash
|
||
sudo systemctl daemon-reload
|
||
sudo systemctl enable --now vigilar-backup.timer
|
||
sudo systemctl list-timers | grep vigilar
|
||
```
|
||
|
||
The first run happens on the next daily rollover. To test it now:
|
||
|
||
```bash
|
||
sudo systemctl start vigilar-backup.service
|
||
ls -lh /mnt/nas/vigilar-backups/
|
||
```
|
||
|
||
You should see a file like `vigilar-backup-20260405-030000.tar.gz`.
|
||
|
||
## Troubleshooting
|
||
|
||
### The web UI will not load
|
||
|
||
- `sudo systemctl status vigilar` — is it running?
|
||
- `journalctl -u vigilar -e` — what does the last error say?
|
||
- Is port 49735 blocked by a firewall? `sudo ss -tlnp | grep 49735`
|
||
should show the Vigilar process listening.
|
||
|
||
### A camera will not connect
|
||
|
||
- Can you open the RTSP URL in VLC from your laptop? If not, it is a
|
||
camera/network problem, not a Vigilar problem.
|
||
- Double-check username/password — many cameras have a separate RTSP
|
||
credential.
|
||
- Some cameras need a specific stream path (e.g. `/stream1`). See the
|
||
[Camera Hardware Guide](camera-hardware-guide.md).
|
||
|
||
### No push notifications on my phone
|
||
|
||
- Did you open the PWA from your home screen, not from the browser tab?
|
||
- Check the browser's notification permission for the site.
|
||
- Safari on iOS only supports web push from installed PWAs, not from a
|
||
tab.
|
||
|
||
### Service keeps restarting
|
||
|
||
- `journalctl -u vigilar -n 100` will show the last 100 log lines.
|
||
- A common cause is an invalid `vigilar.toml`. Run
|
||
`vigilar config validate` to check.
|
||
|
||
### I forgot my PIN
|
||
|
||
```bash
|
||
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-pin
|
||
```
|
||
|
||
### Motion detection is too sensitive
|
||
|
||
See the `[camera]` and motion-related sections in the
|
||
[Operator Guide](operator-guide.md) for tuning.
|
||
|
||
## Where to go next
|
||
|
||
- [Operator Guide](operator-guide.md) — config reference, backups,
|
||
upgrades, security hardening.
|
||
- [Architecture Overview](architecture/overview.md) — how Vigilar is put
|
||
together, if you are curious.
|
||
```
|
||
|
||
**Important:** every CLI command (`vigilar config set-pin`,
|
||
`vigilar config validate`) must be verified against `vigilar/cli/` before
|
||
you write it. If a command does not exist, remove that part of the doc or
|
||
reword it around what does exist.
|
||
|
||
- [ ] **Step 3: Verify CLI commands exist**
|
||
|
||
Use Grep to check:
|
||
|
||
```
|
||
Grep pattern: set-pin path: vigilar/cli/ output_mode: content
|
||
Grep pattern: set-password path: vigilar/cli/ output_mode: content
|
||
Grep pattern: validate path: vigilar/cli/ output_mode: content
|
||
```
|
||
|
||
If `set-pin` is not found, the "I forgot my PIN" and Step 4 sections must
|
||
be rewritten to match what actually exists.
|
||
|
||
- [ ] **Step 4: Verify file paths referenced**
|
||
|
||
```bash
|
||
ls scripts/install.sh scripts/backup.sh config/vigilar.toml
|
||
```
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add docs/home-user-guide.md
|
||
git commit -m "docs: add home user setup guide"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 16: Operator guide
|
||
|
||
**Files:**
|
||
- Create: `docs/operator-guide.md`
|
||
|
||
- [ ] **Step 1: Gather grounding facts**
|
||
|
||
Read, in full:
|
||
- `config/vigilar.toml` — note every `[section]` and every key.
|
||
- `vigilar/cli/main.py` — note every top-level Click command.
|
||
- `vigilar/cli/cmd_config.py` — note every `config` subcommand.
|
||
- `vigilar/cli/cmd_start.py` — note the `start` command's behavior.
|
||
- `scripts/install.sh`
|
||
- `scripts/backup.sh`
|
||
- `scripts/gen_cert.sh`
|
||
- `scripts/gen_vapid_keys.sh`
|
||
- `scripts/setup_nut.sh`
|
||
- `scripts/uninstall.sh`
|
||
- `systemd/vigilar.service`
|
||
- `systemd/vigilar-mosquitto.conf`
|
||
|
||
For each TOML section, write a one-line purpose. For each CLI command,
|
||
write a one-line purpose.
|
||
|
||
- [ ] **Step 2: Find health endpoints**
|
||
|
||
```
|
||
Grep pattern: @.*route path: vigilar/web/blueprints/system.py output_mode: content
|
||
```
|
||
|
||
Also check `vigilar/health/` for any HTTP-facing code. Note any `/health`,
|
||
`/status`, `/metrics`, etc. endpoints that actually exist.
|
||
|
||
- [ ] **Step 3: Create `docs/operator-guide.md`**
|
||
|
||
Write the file with these sections, in this order. Target length 2500–4000
|
||
words. Be concrete — every key gets a real description, every command
|
||
gets a real usage line.
|
||
|
||
```markdown
|
||
# Vigilar Operator Guide
|
||
|
||
This guide is a reference for running Vigilar on a server you administer.
|
||
It assumes comfort with Linux, systemd, and the CLI. For first-time home
|
||
setup instead, see the [Home User Guide](home-user-guide.md).
|
||
|
||
## Layout on disk
|
||
|
||
| Path | Purpose |
|
||
|----------------------------------------|----------------------------------|
|
||
| `/opt/vigilar/` | Source tree and Python venv. |
|
||
| `/opt/vigilar/venv/bin/vigilar` | CLI entry point. |
|
||
| `/etc/vigilar/vigilar.toml` | Main configuration file. |
|
||
| `/etc/vigilar/certs/` | TLS cert and key (if enabled). |
|
||
| `/etc/vigilar/secrets/storage.key` | AES-256 key for recordings. |
|
||
| `/etc/vigilar/secrets/vapid_*.pem` | VAPID keys for web push. |
|
||
| `/var/vigilar/data/vigilar.db` | SQLite database (WAL mode). |
|
||
| `/var/vigilar/recordings/` | Encrypted .vge recording files. |
|
||
| `/var/vigilar/hls/` | Live HLS segments (rolling). |
|
||
| `/var/vigilar/backups/` | Default backup destination. |
|
||
|
||
## Installation
|
||
|
||
`scripts/install.sh` is the blessed install path. It detects the package
|
||
manager (apt or pacman), installs system dependencies, creates a
|
||
`vigilar` system user, builds a virtualenv at `/opt/vigilar/venv`,
|
||
installs the Python package, generates VAPID keys if missing, and drops
|
||
the systemd unit and Mosquitto broker config into place.
|
||
|
||
### System dependencies
|
||
|
||
On Debian/Ubuntu: `ffmpeg mosquitto python3 python3-venv python3-pip nut-client`.
|
||
On Arch: `ffmpeg mosquitto python python-virtualenv nut`.
|
||
|
||
### Systemd unit
|
||
|
||
`systemd/vigilar.service` runs `vigilar start` as the `vigilar` user.
|
||
Mosquitto runs as a separate unit, configured by
|
||
`systemd/vigilar-mosquitto.conf` to bind to loopback only.
|
||
|
||
## Configuration reference (`/etc/vigilar/vigilar.toml`)
|
||
|
||
<One subsection per [section] you found in Step 1. For each:
|
||
- Bullet list of keys with their default values from the shipped TOML.
|
||
- A short "when to change" sentence for each key.
|
||
>
|
||
|
||
### `[system]`
|
||
- `name` (default: `"Vigilar Home Security"`) — display name shown in
|
||
the web UI. Cosmetic.
|
||
- `timezone` (default: `"America/New_York"`) — used for timestamps in
|
||
logs and the UI. Set this to your local timezone.
|
||
- `data_dir` (default: `"/var/vigilar/data"`) — SQLite database
|
||
location. Change only if you are redirecting state to a different
|
||
disk.
|
||
- `recordings_dir` (default: `"/var/vigilar/recordings"`) — where
|
||
encrypted recordings live. May be pointed at a larger disk.
|
||
- `hls_dir` (default: `"/var/vigilar/hls"`) — live HLS segments.
|
||
Consider tmpfs for reduced SSD wear.
|
||
- `log_level` (default: `"INFO"`) — one of `DEBUG`, `INFO`, `WARNING`,
|
||
`ERROR`.
|
||
|
||
### `[mqtt]`
|
||
- `host` (default: `"127.0.0.1"`)
|
||
- `port` (default: `1883`)
|
||
|
||
Leave as-is unless you are running an external broker.
|
||
|
||
### `[web]`
|
||
- `host` (default: `"0.0.0.0"`) — bind address. Leave as-is for LAN
|
||
access; set to `127.0.0.1` to expose only via a reverse proxy.
|
||
- `port` (default: `49735`) — HTTP port.
|
||
- `tls_cert`, `tls_key` — PEM paths for HTTPS. Generate with
|
||
`scripts/gen_cert.sh`.
|
||
- `username` (default: `"admin"`) — admin username.
|
||
- `password_hash` — set via `vigilar config set-password`.
|
||
|
||
<Continue in the same shape for [zigbee2mqtt], [ups], [storage],
|
||
[remote], [alerts.local], [alerts.web_push], [alerts.email], and any
|
||
other section you found. If the shipped TOML has a section not listed
|
||
here, add it. If a section listed here is not in the TOML, remove it.>
|
||
|
||
## CLI reference
|
||
|
||
The CLI is `vigilar`, installed at `/opt/vigilar/venv/bin/vigilar`. Run
|
||
commands as the `vigilar` user:
|
||
|
||
```bash
|
||
sudo -u vigilar /opt/vigilar/venv/bin/vigilar <command>
|
||
```
|
||
|
||
<One subsection per top-level Click command you found in
|
||
`vigilar/cli/main.py`. For each: the command name, a one-line purpose,
|
||
and a usage example. Do not guess command names — only list what exists.>
|
||
|
||
### `vigilar start`
|
||
<purpose — from cmd_start.py>
|
||
|
||
Usage:
|
||
```bash
|
||
sudo -u vigilar /opt/vigilar/venv/bin/vigilar start
|
||
```
|
||
|
||
Normally you run this via systemd (`systemctl start vigilar`) rather
|
||
than directly.
|
||
|
||
### `vigilar config <subcommand>`
|
||
<List every subcommand you found in cmd_config.py. Examples if they
|
||
exist: `validate`, `set-pin`, `set-password`, `show`. Do not invent.>
|
||
|
||
## Secrets and security
|
||
|
||
### `/etc/vigilar/secrets/` layout
|
||
|
||
- `storage.key` — AES-256 key used to encrypt `.vge` recording files.
|
||
**If you lose this file, every existing recording becomes
|
||
unrecoverable.** Include it in your backups (`scripts/backup.sh` does
|
||
this by default because it tars `/etc/vigilar`).
|
||
- `vapid_private.pem`, `vapid_public.pem` — VAPID keys for web push
|
||
notifications. Generated by `scripts/gen_vapid_keys.sh`.
|
||
- Any other files in `secrets/` discovered at runtime.
|
||
|
||
File permissions should be `0600 root:root` or `0600 vigilar:vigilar`
|
||
for anything containing a key. `install.sh` sets these.
|
||
|
||
### Arm/disarm PIN
|
||
|
||
```bash
|
||
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-pin
|
||
```
|
||
|
||
PINs are stored as hashes. Comparisons are constant-time.
|
||
|
||
### Admin password
|
||
|
||
```bash
|
||
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-password
|
||
```
|
||
|
||
### TLS
|
||
|
||
```bash
|
||
sudo ./scripts/gen_cert.sh
|
||
```
|
||
|
||
Then set `[web] tls_cert` and `[web] tls_key` in the TOML and restart
|
||
Vigilar.
|
||
|
||
### Firewall stance
|
||
|
||
Vigilar is LAN-only by default — the Mosquitto broker binds to
|
||
`127.0.0.1` and the web server binds to `0.0.0.0` on port 49735. If you
|
||
need remote access, use the `[remote]` tunnel feature rather than
|
||
exposing port 49735 to the internet.
|
||
|
||
## UPS / NUT integration
|
||
|
||
`scripts/setup_nut.sh` configures a local NUT server for a USB-connected
|
||
UPS. After running it, set `[ups] enabled = true` and adjust:
|
||
|
||
- `nut_host`, `nut_port`, `ups_name` — NUT connection.
|
||
- `poll_interval_s` — how often Vigilar polls the UPS.
|
||
- `low_battery_threshold_pct` — triggers a warning alert.
|
||
- `critical_runtime_threshold_s` — triggers a graceful shutdown.
|
||
- `shutdown_delay_s` — seconds to wait before issuing `shutdown -h`.
|
||
|
||
## Backups
|
||
|
||
`scripts/backup.sh` captures:
|
||
|
||
- The SQLite database (via `sqlite3 .backup` for a consistent snapshot
|
||
if `sqlite3` is installed, otherwise a raw file copy).
|
||
- Any WAL/SHM sidecars next to the DB.
|
||
- The entire `/etc/vigilar/` directory (config + certs + secrets).
|
||
|
||
It does **not** back up recordings under `/var/vigilar/recordings/`.
|
||
|
||
### Environment variables
|
||
|
||
- `VIGILAR_BACKUP_DIR` (default: `/var/vigilar/backups`) — destination.
|
||
Set this to a NAS mount if you want off-box backups.
|
||
- `VIGILAR_BACKUP_RETENTION_DAYS` (default: `30`) — older archives are
|
||
pruned.
|
||
|
||
### Suggested systemd timer
|
||
|
||
Create `/etc/systemd/system/vigilar-backup.service`:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=Vigilar nightly backup
|
||
After=vigilar.service
|
||
|
||
[Service]
|
||
Type=oneshot
|
||
Environment=VIGILAR_BACKUP_DIR=/var/vigilar/backups
|
||
Environment=VIGILAR_BACKUP_RETENTION_DAYS=30
|
||
ExecStart=/opt/vigilar/scripts/backup.sh
|
||
```
|
||
|
||
Create `/etc/systemd/system/vigilar-backup.timer`:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=Run vigilar backup nightly
|
||
|
||
[Timer]
|
||
OnCalendar=daily
|
||
Persistent=true
|
||
|
||
[Install]
|
||
WantedBy=timers.target
|
||
```
|
||
|
||
Enable:
|
||
|
||
```bash
|
||
sudo systemctl daemon-reload
|
||
sudo systemctl enable --now vigilar-backup.timer
|
||
```
|
||
|
||
### Restore procedure
|
||
|
||
```bash
|
||
sudo systemctl stop vigilar
|
||
sudo tar -C / -xzpf /path/to/vigilar-backup-YYYYMMDD-HHMMSS.tar.gz
|
||
sudo chown -R vigilar:vigilar /var/vigilar/data /etc/vigilar
|
||
sudo systemctl start vigilar
|
||
```
|
||
|
||
## Upgrades
|
||
|
||
```bash
|
||
cd /opt/vigilar
|
||
sudo -u vigilar git pull
|
||
sudo -u vigilar /opt/vigilar/venv/bin/pip install -e .
|
||
sudo systemctl restart vigilar
|
||
```
|
||
|
||
<If the project has a migrations system, document how to run it here.
|
||
If not, note: "No automated database migrations at time of writing. Do
|
||
not downgrade — schema changes are forward-only.">
|
||
|
||
### Rolling back
|
||
|
||
```bash
|
||
sudo systemctl stop vigilar
|
||
# Check out the previous tag or commit
|
||
sudo -u vigilar git -C /opt/vigilar checkout <previous-rev>
|
||
sudo -u vigilar /opt/vigilar/venv/bin/pip install -e /opt/vigilar
|
||
# Restore the pre-upgrade backup if schema changed
|
||
sudo systemctl start vigilar
|
||
```
|
||
|
||
## Logs and health
|
||
|
||
### Logs
|
||
|
||
```bash
|
||
sudo journalctl -u vigilar -f # follow
|
||
sudo journalctl -u vigilar -n 200 # last 200 lines
|
||
sudo journalctl -u vigilar --since "1 hour ago"
|
||
```
|
||
|
||
Raise verbosity by setting `[system] log_level = "DEBUG"` in the TOML
|
||
and restarting.
|
||
|
||
### Health endpoints
|
||
|
||
<List any routes you found in Step 2. Example shape:>
|
||
|
||
- `GET /system/health` — JSON summary of subsystem status.
|
||
- `GET /system/logs` — recent log entries from the web UI.
|
||
|
||
If none exist, write: "There are no dedicated health HTTP endpoints at
|
||
time of writing. Use `journalctl -u vigilar` and the system status page
|
||
in the web UI."
|
||
|
||
## Remote access
|
||
|
||
`[remote] enabled = true` turns on the tunnel-based remote viewing
|
||
feature. It serves a downscaled HLS stream shaped to
|
||
`upload_bandwidth_mbps` over a tunnel on `tunnel_ip`. This is **not** a
|
||
cloud service — you are still the one running the tunnel endpoint.
|
||
|
||
Key knobs:
|
||
|
||
- `upload_bandwidth_mbps` — your upload ceiling.
|
||
- `remote_hls_resolution`, `remote_hls_fps`, `remote_hls_bitrate_kbps` —
|
||
shape the remote stream.
|
||
- `max_remote_viewers` — concurrency cap.
|
||
|
||
## Troubleshooting
|
||
|
||
### Service is in a crash loop
|
||
|
||
`journalctl -u vigilar -n 200` and look for the first `ERROR`. Common
|
||
causes:
|
||
|
||
- Invalid TOML — run `vigilar config validate` (if that command exists).
|
||
- Missing secrets — `storage.key` or `vapid_private.pem` not readable by
|
||
the `vigilar` user.
|
||
- Port 49735 already in use by something else.
|
||
|
||
### Mosquitto broker will not start
|
||
|
||
- `sudo systemctl status mosquitto`
|
||
- Check `/etc/mosquitto/conf.d/` for a conflicting config file on the
|
||
same port.
|
||
- Run `mosquitto -v -c /etc/mosquitto/conf.d/vigilar.conf` by hand to
|
||
see the error directly.
|
||
|
||
### A camera worker is thrashing
|
||
|
||
- Check `journalctl -u vigilar | grep <camera name>`.
|
||
- Verify the RTSP URL with `ffprobe <rtsp url>`.
|
||
- Reduce expected FPS in the camera's entry if the stream is unstable.
|
||
|
||
### Disk is full / `free_space_floor_gb` triggered
|
||
|
||
The health pruner is supposed to delete the oldest recordings when free
|
||
space drops below `[storage] free_space_floor_gb`. If it is not keeping
|
||
up:
|
||
|
||
- Lower `[storage] max_disk_usage_gb`.
|
||
- Move `recordings_dir` to a larger disk.
|
||
- Check `journalctl -u vigilar | grep -i prun` for pruner errors.
|
||
|
||
### HLS grid is stalling
|
||
|
||
- The mini PC's CPU may be saturated. Check `top`.
|
||
- Drop the grid to fewer cameras or lower resolution.
|
||
- Single-camera view uses MJPEG, which is cheaper to serve than HLS.
|
||
```
|
||
|
||
Every TOML key, every CLI command, every endpoint, every file path in
|
||
this document must be backed by something you actually read in Step 1 or
|
||
Step 2. Delete any bullet you cannot verify.
|
||
|
||
- [ ] **Step 4: Verify CLI commands exist**
|
||
|
||
```
|
||
Grep pattern: @.*\.command path: vigilar/cli/ output_mode: content
|
||
```
|
||
|
||
Every `vigilar <subcommand>` mentioned in the doc must be present here.
|
||
Remove any that are not.
|
||
|
||
- [ ] **Step 5: Verify TOML coverage**
|
||
|
||
Every `[section]` in `config/vigilar.toml` must have a subsection in
|
||
"Configuration reference". Every subsection in the doc must correspond to
|
||
a real `[section]`.
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add docs/operator-guide.md
|
||
git commit -m "docs: add operator guide"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 17: Top-level README
|
||
|
||
**Files:**
|
||
- Create: `README.md`
|
||
|
||
- [ ] **Step 1: Confirm no existing README**
|
||
|
||
```bash
|
||
ls README.md 2>/dev/null || echo "no existing README — good"
|
||
```
|
||
|
||
- [ ] **Step 2: Confirm all linked targets exist**
|
||
|
||
Every relative link in this README will point to a file created in
|
||
Tasks 1–16. Verify:
|
||
|
||
```bash
|
||
ls docs/home-user-guide.md docs/operator-guide.md \
|
||
docs/architecture/overview.md docs/architecture/conventions.md \
|
||
docs/camera-hardware-guide.md
|
||
ls docs/architecture/subsystems/*.md | wc -l # should be 12
|
||
```
|
||
|
||
- [ ] **Step 3: Create `README.md`**
|
||
|
||
```markdown
|
||
# Vigilar
|
||
|
||
> DIY offline-first home security. Your cameras, your house, your data.
|
||
|
||
Vigilar runs on a small mini PC in your home, records from your IP
|
||
cameras, detects motion, and pushes notifications to your phone — with
|
||
no cloud, no account, and no external service in the critical path.
|
||
|
||
<!-- Screenshot placeholder: replace when a real screenshot is available. -->
|
||

|
||
|
||
## Why Vigilar
|
||
|
||
- **Offline-first.** No cloud dependency. No external calls in the hot
|
||
path.
|
||
- **Multi-camera live view.** HLS grid for bandwidth efficiency, MJPEG
|
||
single view for low-latency focus.
|
||
- **On-device motion detection.** OpenCV MOG2 with a 5-second pre-motion
|
||
ring buffer so you never miss the moment leading up to a trigger.
|
||
- **Event timeline, highlight reels, timelapses.** Visitor, pet, and
|
||
wildlife tracking as separate views.
|
||
- **Phone push notifications.** PWA + VAPID web push. No Firebase, no
|
||
Google Cloud Messaging.
|
||
- **Encrypted recordings.** AES-256-GCM `.vge` files. Key stays on your
|
||
box.
|
||
- **UPS aware.** NUT integration, graceful shutdown on critical
|
||
battery.
|
||
- **Runs on a cheap mini PC.** 4 GB RAM, 128 GB SSD, any modern x86_64
|
||
box.
|
||
|
||
## Quick paths
|
||
|
||
| I want to… | Start here |
|
||
|----------------------------------------------|--------------------------------------------------------------|
|
||
| Set it up at home on a mini PC | [Home User Guide](docs/home-user-guide.md) |
|
||
| Run it as a self-hoster / sysadmin | [Operator Guide](docs/operator-guide.md) |
|
||
| Understand how it works internally | [Architecture Overview](docs/architecture/overview.md) |
|
||
| Pick cameras that work well with it | [Camera Hardware Guide](docs/camera-hardware-guide.md) |
|
||
|
||
## 60-second overview
|
||
|
||
```
|
||
[IP cameras] ──RTSP──▶ [mini PC running Vigilar] ──LAN──▶ [phone / browser]
|
||
│
|
||
└── optional nightly ──▶ [NAS]
|
||
```
|
||
|
||
- **Language:** Python 3.11+
|
||
- **Web:** Flask + Bootstrap 5 dark theme + `hls.js`
|
||
- **Storage:** SQLite (WAL) via SQLAlchemy Core
|
||
- **Bus:** Local Mosquitto on `127.0.0.1:1883`
|
||
- **Video:** OpenCV (capture + motion), FFmpeg subprocess (recording,
|
||
HLS)
|
||
|
||
## Status
|
||
|
||
**Alpha.** Works on the author's hardware. Expect rough edges and
|
||
breaking changes. Not recommended for anyone who cannot read a stack
|
||
trace.
|
||
|
||
## Installation (TL;DR)
|
||
|
||
```bash
|
||
git clone <repo URL> vigilar
|
||
cd vigilar
|
||
sudo ./scripts/install.sh
|
||
sudo systemctl enable --now vigilar
|
||
```
|
||
|
||
Then open `http://<mini-pc-ip>:49735` on your LAN.
|
||
|
||
For the full walkthrough (OS install, camera setup, PIN, push
|
||
notifications, NAS backup), see the
|
||
[Home User Guide](docs/home-user-guide.md).
|
||
|
||
For configuration, CLI, secrets, backups, and upgrades, see the
|
||
[Operator Guide](docs/operator-guide.md).
|
||
|
||
## Documentation
|
||
|
||
- **User guides**
|
||
- [Home User Guide](docs/home-user-guide.md) — mini PC to working
|
||
cameras, step by step.
|
||
- [Operator Guide](docs/operator-guide.md) — configuration, CLI,
|
||
backups, security, upgrades.
|
||
- [Camera Hardware Guide](docs/camera-hardware-guide.md) — picking and
|
||
wiring up cameras.
|
||
- **Architecture**
|
||
- [Overview](docs/architecture/overview.md) — process model, MQTT bus,
|
||
data flow, storage.
|
||
- [Conventions](docs/architecture/conventions.md) — coding rules for
|
||
contributors.
|
||
- [Subsystems](docs/architecture/subsystems/) — one short reference
|
||
per subsystem: `camera`, `detection`, `events`, `alerts`, `sensors`,
|
||
`ups`, `storage`, `highlights`, `presence`, `pets`, `health`,
|
||
`web`.
|
||
|
||
## License
|
||
|
||
Vigilar is distributed under the **GNU General Public License v3.0**.
|
||
See `LICENSE` for the full text. (If the `LICENSE` file is not yet in
|
||
the tree, it will be added in a follow-up.)
|
||
|
||
## Contributing
|
||
|
||
Issues and pull requests are welcome. Before sending a PR:
|
||
|
||
- Read `CLAUDE.md` for the project's code conventions.
|
||
- Run `ruff check vigilar/` and `pytest` — both must pass.
|
||
- Keep commits small and focused.
|
||
```
|
||
|
||
- [ ] **Step 4: Verify every link in the README resolves**
|
||
|
||
```bash
|
||
ls README.md docs/home-user-guide.md docs/operator-guide.md \
|
||
docs/architecture/overview.md docs/architecture/conventions.md \
|
||
docs/architecture/subsystems docs/camera-hardware-guide.md CLAUDE.md
|
||
```
|
||
|
||
All must exist. If any do not, that means an earlier task was skipped —
|
||
go back and do it.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add README.md
|
||
git commit -m "docs: add top-level README"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 18: Final verification pass
|
||
|
||
**Files:**
|
||
- Modify: any of the above, if issues are found
|
||
|
||
- [ ] **Step 1: Link check**
|
||
|
||
```
|
||
Grep pattern: \]\( path: README.md output_mode: content
|
||
Grep pattern: \]\( path: docs/ output_mode: content
|
||
```
|
||
|
||
For each relative link discovered, confirm the target exists with
|
||
`ls <target>`. Fix any broken links inline.
|
||
|
||
- [ ] **Step 2: Command check**
|
||
|
||
For every shell command in `docs/home-user-guide.md` and
|
||
`docs/operator-guide.md`, confirm either:
|
||
- the script exists (`ls scripts/<name>.sh`), or
|
||
- the `vigilar` subcommand exists (Grep in `vigilar/cli/`), or
|
||
- it is a standard Linux command (`systemctl`, `journalctl`, `git`,
|
||
`pip`, `ls`, etc.).
|
||
|
||
Remove or rewrite anything that fails.
|
||
|
||
- [ ] **Step 3: TOML coverage check**
|
||
|
||
Open `config/vigilar.toml` and `docs/operator-guide.md` side by side.
|
||
Every `[section]` in the TOML must appear in the operator guide's
|
||
Configuration Reference. Every section in the guide must appear in the
|
||
TOML. Fix any asymmetry.
|
||
|
||
- [ ] **Step 4: Subsystem coverage check**
|
||
|
||
```bash
|
||
ls vigilar/
|
||
ls docs/architecture/subsystems/
|
||
```
|
||
|
||
Every subsystem directory under `vigilar/` that is listed in the spec
|
||
(`camera`, `detection`, `events`, `alerts`, `sensors`, `ups`, `storage`,
|
||
`highlights`, `presence`, `pets`, `health`, `web`) must have a matching
|
||
`.md` file in `docs/architecture/subsystems/`. If a subsystem directory
|
||
exists in `vigilar/` that is NOT in the list (because something was added
|
||
to the codebase since the spec was written), note it but do not
|
||
necessarily document it — flag it at the end.
|
||
|
||
- [ ] **Step 5: Read-through**
|
||
|
||
Read every new `.md` file top to bottom in this order: README,
|
||
home-user-guide, operator-guide, architecture/overview,
|
||
architecture/conventions, then each subsystem doc alphabetically.
|
||
|
||
Check for:
|
||
- Tone drift (the README should not shift register halfway through).
|
||
- Terminology drift (if one doc says "motion event" and another says
|
||
"movement trigger" for the same thing, unify).
|
||
- Stale references (e.g. a doc saying "see the X section" where X was
|
||
removed).
|
||
|
||
Fix inline.
|
||
|
||
- [ ] **Step 6: Final commit**
|
||
|
||
If Steps 1–5 surfaced any fixes:
|
||
|
||
```bash
|
||
git add -A
|
||
git commit -m "docs: final verification pass fixes"
|
||
```
|
||
|
||
If no fixes were needed, skip the commit and report "no issues found in
|
||
final verification pass" in the task output.
|
||
|
||
---
|
||
|
||
## Self-review notes (for the plan author)
|
||
|
||
- **Spec coverage:** Each deliverable in §§3-9 of the spec maps to a
|
||
task: README → Task 17, home-user → Task 15, operator → Task 16,
|
||
overview → Task 1, conventions → Task 2, 12 subsystems → Tasks 3–14.
|
||
Verification checklist from spec §10 → Task 18. Out-of-scope items
|
||
from spec §11 are explicitly excluded in the "Universal rules".
|
||
- **Placeholder scan:** The plan contains `<angle bracket>` placeholders
|
||
**inside task output templates**. These are **instructions to the
|
||
executor** ("replace this with the value you found"), not deferred
|
||
work in the plan itself. Each template placeholder is accompanied by
|
||
a specific grep or file to read that tells the executor how to fill
|
||
it. This is intentional — the alternative would be to pre-populate
|
||
architecture facts in the plan without ever reading the code, which
|
||
would make the plan itself guess.
|
||
- **Type/signature consistency:** N/A — docs, not code.
|
||
- **Order:** Task 1 (overview) is first so the MQTT topic convention is
|
||
established once and can be referenced consistently by subsystem docs.
|
||
README is last so every link it makes already exists.
|