40 Commits

Author SHA1 Message Date
adlee-was-taken
5745388880 fix: address final-review items (status endpoint, docs, tests)
Follow-up to the holistic review of the PIN-unification branch:

- /system/status now reads the real arm state from the arm_state_log
  table via get_current_arm_state, instead of returning a hardcoded
  'DISARMED' stub. Without this, polling after the new async 202
  arm/disarm flow was a UX dead-end — clients never saw the state
  change they just requested. DB read failures degrade gracefully.

- Operator guide: correct the claim that 'vigilar config set-pin'
  populates recovery_passphrase_hash. It doesn't. recovery_passphrase
  _hash has no CLI helper today; it must be set manually.

- Tests: add a fail-closed regression for verify_pin on malformed
  stored hashes, and a companion test confirming the deprecation
  warning stays silent on a fully migrated config.

All address specific review comments on the branch; no scope creep.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 12:58:09 -04:00
adlee-was-taken
eb281ad058 docs(operator-guide): PIN hashing is unified (issue #2)
Describes the canonical [security] pin_hash key, the PBKDF2 format
emitted by 'vigilar config set-pin', and the deprecation warning for
the legacy [system] arm_pin_hash. Drops the three-way mismatch
known-limitation.
2026-04-05 12:58:09 -04:00
adlee-was-taken
385bafc73f test: end-to-end PIN unification regression guard (issue #2) 2026-04-05 12:58:09 -04:00
adlee-was-taken
12821648ca fix(web): raise on MQTT connect timeout in _publish_arm_request
Code review on 9f203d8 caught a silent-failure mode: MessageBus.connect
logs and returns without raising when the MQTT handshake times out, so
an overloaded broker would let bus.publish() enqueue into paho's outbox
only to be discarded by the immediate disconnect(). The web endpoint
would return 202 even though the FSM never received the request.

Guard with 'if not bus.connected: raise RuntimeError'. The existing
try/except in arm_system/disarm_system catches the exception and turns
it into a 503 with the same log message as other bus failures.
2026-04-05 12:58:09 -04:00
adlee-was-taken
7b33cb7bb4 fix(web): align arm/disarm 202 response shape with {"ok": true} convention
Follow-up to efd5c4a. The plan invented {"accepted": True, ...} for
the new 202 responses, but every other 2xx endpoint in the Flask app
returns {"ok": True, ...} — including cameras.py:108 which is direct
prior art for a 202 with the same convention. The shared JS helper
at static/js/settings.js:54 does 'if (resp.ok && result.ok)' and was
falling into the error branch on our success responses, showing a
bogus "Save failed" toast after every arm/disarm click.

Keep the 202 status. Swap the body key from 'accepted' to 'ok'.
No JS change needed.
2026-04-05 12:58:09 -04:00
adlee-was-taken
4b0d547322 fix(web): arm/disarm actually transition the FSM via MQTT (issue #2)
Was: /system/api/arm verified the PIN against [security] pin_hash and
returned {ok: true} without ever calling the FSM. State never changed.
Now: the endpoint publishes a SYSTEM_ARM_REQUEST message to the local
MQTT broker. The event processor (see previous commit) picks it up,
ArmStateFSM verifies the PIN via alerts.pin.verify_pin and performs
the transition. Response is 202 Accepted; clients poll /system/status
for the new state.

Design: PIN travels over localhost-only MQTT, which matches the
existing trust boundary for the internal bus.
2026-04-05 12:58:09 -04:00
adlee-was-taken
e6069a68fc refactor(events): drop forward-ref quote and test triggered_by default
Code review follow-up on f4d66dd:
- _handle_arm_request signature used "ArmStateFSM" as a string forward
  reference even though the type is imported at module top.
  _handle_event uses the bare form; match it for consistency.
- Add a test asserting that omitting triggered_by in an arm-request
  payload defaults to "unknown". That value feeds the audit log, so
  it deserves explicit regression coverage.

No behavior change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 12:58:09 -04:00
adlee-was-taken
82ff7fb276 feat(events): processor handles SYSTEM_ARM_REQUEST over MQTT
Adds _handle_arm_request and a dedicated bus.subscribe on
Topics.SYSTEM_ARM_REQUEST. Payload {mode, pin, triggered_by} is
dispatched to ArmStateFSM.transition, which verifies the PIN via
alerts.pin.verify_pin and performs the state change.

This is the missing link for web /system/api/arm to actually move
the system into an armed state. Part of issue #2.
2026-04-05 12:58:09 -04:00
adlee-was-taken
17721eeaa7 style(config): move log handle below import block
Code review follow-up on c9dd348 — the log = logging.getLogger(__name__)
assignment was interleaved between 'import X' and 'from X import Y'
statements. Move it below all imports per standard ordering.

No behavior change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 12:57:20 -04:00
adlee-was-taken
e568f20871 feat(config): deprecation warning for [system] arm_pin_hash
If a config still has the legacy [system] arm_pin_hash set but no
[security] pin_hash, load_config logs a WARNING telling the operator
to re-run 'vigilar config set-pin'. The legacy field is still parsed
(so old configs don't fail validation) but ignored at runtime.

Part of issue #2 PIN hashing unification.
2026-04-05 12:57:20 -04:00
adlee-was-taken
2032fac227 fix(cli): redact security.recovery_passphrase_hash in show_cmd
Adjacent secret leak in show_cmd noticed during Task 3 code review.
SecurityConfig has two sensitive fields and the redaction block only
covered pin_hash. vigilar config show would print the recovery
passphrase hash verbatim whenever one was configured.

One-line fix; same redaction pattern as the surrounding secrets.
Part of issue #2.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 12:57:20 -04:00
adlee-was-taken
c2976876ed fix(cli): set-pin emits PBKDF2 under [security] pin_hash (issue #2)
Was: HMAC-SHA256(random, pin) written to [system] arm_pin_hash —
no verifier in the codebase accepted this output.
Now: PBKDF2-SHA256 via alerts.pin.hash_pin written to [security]
pin_hash, matching what the web and FSM paths verify against.

Also fixes show_cmd to redact the new location.
2026-04-05 12:57:20 -04:00
adlee-was-taken
54ad58c870 refactor(events): drop verify_pin alias and clarify audit-log comment
Code review feedback on the Task 2 commit (7fda351):
- The 'verify_pin as _verify_pin_hash' alias was unnecessary — the
  method self.verify_pin and the module-level verify_pin do not
  collide (one is accessed via self, the other via the bare name).
  Removing the alias matches how web/blueprints/system.py already
  imports verify_pin and makes the call site read cleanly.
- The comment on the insert_arm_state None argument now explains
  WHY (PBKDF2 salt is fresh per call, so re-hashing is worthless for
  audit correlation) instead of only referencing the issue.

No behavior change. Part of issue #2.
2026-04-05 12:57:20 -04:00
adlee-was-taken
efa3ce4b1b fix(events): ArmStateFSM uses PBKDF2 via alerts.pin (issue #2)
Was: unsalted SHA-256 read from [system] arm_pin_hash.
Now: PBKDF2-SHA256 600k iterations read from [security] pin_hash,
matching the web arm/disarm path and the alerts/pin module.

Also drops the redundant pin re-hash on the arm_state_log audit row
(a fresh PBKDF2 salt made the column valueless for traceability).

Part of issue #2 PIN hashing unification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 12:57:20 -04:00
adlee-was-taken
c64f863741 feat(constants): add Topics.SYSTEM_ARM_REQUEST
Topic for web-originated arm/disarm requests that the event processor
will subscribe to and dispatch to ArmStateFSM.transition. Part of the
PIN unification work (issue #2).
2026-04-05 12:57:20 -04:00
adlee-was-taken
e048eb955e docs(plan): implementation plan for PIN hashing unification (issue #2)
Plan document for issue #2 — the three-way PIN hash mismatch across
CLI, events FSM, and web arm/disarm. Proposes canonicalizing on
PBKDF2-SHA256 via alerts/pin and [security] pin_hash, deprecating
[system] arm_pin_hash, and wiring web arm/disarm through MQTT to the
FSM so the web buttons actually transition state.

Nine tasks, TDD throughout. No code changes in this commit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 12:57:20 -04:00
adlee-was-taken
09b59e3bb5 feat: wire MQTT → SSE bridge so the event timeline updates live
Closes #1.

The Flask event-timeline was dead: `broadcast_sse_event` existed in
`vigilar/web/blueprints/events.py` but had zero call sites. Clients
subscribed to `/events/stream`, received the initial "connected"
message, and then only keepalives — a page refresh was required to
see new events. (Web Push via VAPID was independent and already worked.)

The root cause was a process-boundary gap: the events subsystem runs
in its own OS process and emits to MQTT, while the Flask app runs in a
separate process with no MQTT client of its own.

This change adds a thin bridge:

- EventProcessor._handle_event now publishes a classified summary
  (id, ts, type, severity, source_id, payload) to a new topic
  `Topics.EVENTS_PUBLISHED = "vigilar/events/published"` right after
  `insert_event()`. Classification logic stays in one place.

- A new module `vigilar/web/sse_bridge.py` provides `forward_event`
  (MQTT handler) and `start_sse_bridge(cfg)` (creates a MessageBus,
  subscribes forward_event to EVENTS_PUBLISHED, connects, returns the
  bus).

- `vigilar/main.py:_run_web` starts the bridge after `create_app(cfg)`
  and disconnects it on shutdown. Bridge failure is logged but does
  not kill the web process — the UI still works without live updates.

- `create_app` is deliberately NOT changed. Keeping the bridge out of
  the app factory means no existing test triggers a real MQTT
  connection, and the bridge stays a production-only concern wired by
  the supervisor.

Tests (all added with TDD, RED verified before GREEN):

- tests/unit/test_events.py::TestEventsPublishedBroadcast — asserts
  `_handle_event` publishes the classified payload for a motion event
  and does NOT publish for unclassified topics (heartbeats).
- tests/unit/test_sse_bridge.py — asserts `forward_event` reaches SSE
  subscribers, and `start_sse_bridge` wires the handler to
  `Topics.EVENTS_PUBLISHED` on a connected bus (fake bus, no real
  MQTT in tests).

Also refreshes the docs that previously flagged the dead SSE as a
known limitation (operator guide, web architecture doc).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 16:55:27 +00:00
adlee-was-taken
9f959f8c78 test: isolate VIGILAR_CONFIG via autouse session fixture
Some web endpoint handlers call _save_and_reload(), which resolves the
target path via VIGILAR_CONFIG env var with a fallback to the relative
"config/vigilar.toml". Any test exercising such an endpoint without
setting the env var rewrites the repo's committed config file via a
Pydantic model_dump round-trip, stripping comments and non-default
fields. The culprit discovered was test_reset_pin_correct_passphrase
in tests/unit/test_system_pin.py.

Add an autouse session-scoped fixture in tests/conftest.py that points
VIGILAR_CONFIG at a path inside pytest's session tmp dir so no test
can touch the real file. Restore the previous env var value on teardown.

Fixes #3.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 16:54:12 +00:00
adlee-was-taken
17bd403217 fix: correct set-password docstring (scrypt, not bcrypt)
The set_password_cmd docstring and inline comment claimed bcrypt /
SHA-256, but the implementation actually uses scrypt via
cryptography.hazmat.primitives.kdf.scrypt. Correct the docstring,
drop the misleading comment, and remove the now-unused hashlib import.

No behavior change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 10:41:10 -04:00
adlee-was-taken
1633e8b34e docs: final verification pass fixes
Convert the "Where to go next" items in the architecture overview from
plain text to proper Markdown links. This was the only finding from the
Task 18 verification pass; everything else (links, commands, TOML
coverage, subsystem coverage, terminology) is self-consistent across
the 17 new doc files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 10:26:02 -04:00
adlee-was-taken
f02dbaad8c docs: add top-level README
Front-door for the repo. Pitches the project, surfaces known limitations
honestly (timeline not live, CTR not authenticated, no migrations), and
links into the three doc trees: home user guide, operator guide, and
architecture overview + 12 subsystem references.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 10:23:29 -04:00
adlee-was-taken
e38a0dc174 docs: add operator guide 2026-04-05 10:21:58 -04:00
adlee-was-taken
68f9454a7c docs: add home user setup guide
Linear walkthrough from bare mini PC to working cameras on phone, with
optional NAS backup. Verified against real install.sh, backup.sh, and
CLI subcommands; honest about the in-browser event timeline not being
wired to SSE yet (push notifications do work).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 10:12:43 -04:00
adlee-was-taken
5e0d6c8320 docs: add web subsystem reference 2026-04-05 09:50:27 -04:00
adlee-was-taken
c4d119310a docs: add health subsystem reference 2026-04-05 09:48:54 -04:00
adlee-was-taken
07f5f341e6 docs: add pets subsystem reference 2026-04-05 09:48:12 -04:00
adlee-was-taken
58622722c7 docs: add presence subsystem reference 2026-04-05 09:47:35 -04:00
adlee-was-taken
843daf9c0b docs: add highlights subsystem reference 2026-04-05 09:47:07 -04:00
adlee-was-taken
d3db384c35 docs: add storage subsystem reference 2026-04-05 09:46:39 -04:00
adlee-was-taken
62696e919c docs: add ups subsystem reference 2026-04-05 09:46:06 -04:00
adlee-was-taken
87d2df1446 docs: add sensors subsystem reference 2026-04-05 09:45:41 -04:00
adlee-was-taken
c1779dfdb8 docs: add alerts subsystem reference 2026-04-05 09:45:12 -04:00
adlee-was-taken
226a473d4d docs: add events subsystem reference 2026-04-05 09:44:41 -04:00
adlee-was-taken
67b8dd672c docs: add detection subsystem reference 2026-04-05 09:44:06 -04:00
adlee-was-taken
c8d8421112 docs: add camera subsystem reference 2026-04-05 09:42:29 -04:00
adlee-was-taken
484235f74c docs: add coding conventions reference 2026-04-05 09:39:47 -04:00
adlee-was-taken
d38b0c4e25 docs: add architecture overview
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:36:37 -04:00
adlee-was-taken
0e4e2c1ca7 docs: add implementation plan for project documentation
18 tasks covering README, home user guide, operator guide, architecture
overview + conventions, and 12 per-subsystem reference docs. Each task
is grounded in reading real source to avoid invented facts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:29:57 -04:00
adlee-was-taken
1fd80ad31c docs: clarify NAS backup steps in documentation spec
Specify that backup timer snippets are inline in the guides, not
shipped as new unit files, to match the no-code-changes scope.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:17:58 -04:00
adlee-was-taken
4dc2db00e0 docs: add design spec for project documentation effort
Captures scope and structure for top-level README, home user guide,
operator guide, and architecture docs (overview + conventions + 12
per-subsystem files). Approach 3 (hybrid): monolithic user guides,
split architecture reference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:17:07 -04:00
37 changed files with 5436 additions and 61 deletions

131
README.md Normal file
View File

@@ -0,0 +1,131 @@
# Vigilar
> DIY offline-first home security. Your cameras, your house, your data.
Vigilar runs on a small mini PC in your home, records from your IP cameras,
detects motion, and pushes notifications to your phone — with no cloud, no
account, and no external service in the critical path.
<!-- Screenshot placeholder: replace with a real UI grid shot when available. -->
![Vigilar web UI grid view](docs/images/grid.png)
## Why Vigilar
- **Offline-first.** No cloud dependency. No external calls in the hot path.
- **Multi-camera live view.** HLS grid for bandwidth efficiency, MJPEG single
view for low-latency focus.
- **On-device motion detection.** OpenCV MOG2 with a 5-second pre-motion ring
buffer, so the run-up to a trigger is captured.
- **Adaptive FPS.** Cameras idle at 2 FPS and jump to 30 FPS on motion.
- **Event timeline and highlight reels.** Plus timelapses and visitor, pet,
and wildlife tracking as separate views.
- **Phone push notifications.** PWA + VAPID web push. No Firebase, no
Google Cloud Messaging.
- **Encrypted recordings at rest.** AES-256-CTR `.vge` files, key stays on
your box. (Confidential, not tamper-evident — see the
[Operator Guide](docs/operator-guide.md) for the details of the threat
model.)
- **UPS aware.** NUT integration, graceful shutdown on critical battery.
- **Runs on a cheap mini PC.** 4 GB RAM, 128 GB SSD, any modern x86_64 box.
## Quick paths
| I want to… | Start here |
|----------------------------------------------|--------------------------------------------------------------|
| Set it up at home on a mini PC | [Home User Guide](docs/home-user-guide.md) |
| Run it as a self-hoster / sysadmin | [Operator Guide](docs/operator-guide.md) |
| Understand how it works internally | [Architecture Overview](docs/architecture/overview.md) |
| Pick cameras that work well with it | [Camera Hardware Guide](docs/camera-hardware-guide.md) |
## 60-second overview
```
[IP cameras] ──RTSP──▶ [mini PC running Vigilar] ──LAN──▶ [phone / browser]
└── optional nightly ──▶ [NAS]
```
- **Language:** Python 3.11+
- **Web:** Flask + Bootstrap 5 dark + `hls.js`
- **Storage:** SQLite (WAL) via SQLAlchemy Core
- **Bus:** Local Mosquitto on `127.0.0.1:1883`
- **Video:** OpenCV (capture + motion), FFmpeg subprocess (recording, HLS)
## Status
**Alpha.** Works on the author's hardware. Expect rough edges and breaking
changes. Not recommended for anyone who cannot read a stack trace.
Known limitations worth calling out up front:
- The in-browser event timeline does not yet update live. Push
notifications to your phone work in real time; the timeline itself
needs a page refresh.
- Recording encryption is AES-256-CTR: confidential at rest, but not
tamper-evident.
- There is no automated database migration framework yet. Schema changes
are forward-only.
See the [Operator Guide](docs/operator-guide.md) for the full list.
## Installation (TL;DR)
```bash
git clone <repo URL> vigilar
cd vigilar
sudo ./scripts/install.sh
sudo systemctl enable --now vigilar
```
Then open `http://<mini-pc-ip>:49735` on your LAN.
For the full walkthrough (OS install, camera setup, PIN, push
notifications, NAS backup), see the
[Home User Guide](docs/home-user-guide.md).
For configuration, CLI, secrets, backups, and upgrades, see the
[Operator Guide](docs/operator-guide.md).
## Documentation
- **User guides**
- [Home User Guide](docs/home-user-guide.md) — mini PC to working cameras,
step by step.
- [Operator Guide](docs/operator-guide.md) — configuration, CLI, backups,
security, upgrades.
- [Camera Hardware Guide](docs/camera-hardware-guide.md) — picking and
wiring up cameras.
- **Architecture**
- [Overview](docs/architecture/overview.md) — process model, MQTT bus,
data flow, storage layout.
- [Conventions](docs/architecture/conventions.md) — coding rules for
contributors.
- [Subsystems](docs/architecture/subsystems/) — one short reference per
subsystem: [camera](docs/architecture/subsystems/camera.md),
[detection](docs/architecture/subsystems/detection.md),
[events](docs/architecture/subsystems/events.md),
[alerts](docs/architecture/subsystems/alerts.md),
[sensors](docs/architecture/subsystems/sensors.md),
[ups](docs/architecture/subsystems/ups.md),
[storage](docs/architecture/subsystems/storage.md),
[highlights](docs/architecture/subsystems/highlights.md),
[presence](docs/architecture/subsystems/presence.md),
[pets](docs/architecture/subsystems/pets.md),
[health](docs/architecture/subsystems/health.md),
[web](docs/architecture/subsystems/web.md).
## License
Vigilar is distributed under the **GNU General Public License v3.0**. See
`LICENSE` for the full text. (If the `LICENSE` file is not yet in the tree,
it will be added in a follow-up.)
## Contributing
Issues and pull requests are welcome. Before sending a PR:
- Read [`CLAUDE.md`](CLAUDE.md) for the project's code conventions, or the
human-friendly distillation at
[`docs/architecture/conventions.md`](docs/architecture/conventions.md).
- Run `ruff check vigilar/` and `pytest` — both must pass.
- Keep commits small and focused.

View File

@@ -0,0 +1,67 @@
# Coding Conventions
These are the rules a contributor needs in their head before touching Vigilar.
They are distilled from `CLAUDE.md` and from the patterns already in the codebase.
## Language and style
- Python 3.11+.
- Ruff for linting. Line length 100 (see `pyproject.toml`).
- Type hints on all public functions. Internal helpers may omit them when the
types are obvious from context.
- No docstrings unless the logic is non-obvious. Short, well-named functions
should speak for themselves.
## String constants
All string constants referenced in more than one place live in
`vigilar/constants.py`. The file uses `StrEnum` for enumerated values —
`ArmState`, `Severity`, `EventType`, `SensorType`, `AlertChannel`,
`RecordingTrigger`, and friends — and a `Topics` class for MQTT topic builders.
If you are about to write `"MOTION_START"` or `"vigilar/camera/..."` in a second
place, stop and add it to the appropriate enum or the `Topics` class instead.
Defaults like `DEFAULT_IDLE_FPS` also live in this module.
## Database access
- SQLite in WAL mode, opened through `vigilar/storage/db.py`.
- SQLAlchemy Core expressions only — no mapped classes, no ORM session.
- Schema lives in `vigilar/storage/schema.py`. Named query helpers live in
`vigilar/storage/queries.py`.
- New tables go in `schema.py`. New query helpers go in `queries.py`. Do not
scatter ad-hoc SQL across subsystem code.
## Processes and the MQTT bus
- Each subsystem runs in its own `multiprocessing.Process`, spawned by the
supervisor in `vigilar/main.py` via the `SubsystemProcess` wrapper. Cameras
are the exception: `CameraManager` owns one child process per camera
directly.
- Subsystems communicate only through the local MQTT broker at
`127.0.0.1:1883`. No direct imports or function calls across subsystem
boundaries.
- Topic naming: `vigilar/<domain>/<id>/<event>`. Every topic comes from the
`Topics` class in `vigilar/constants.py` — do not construct topic strings
ad hoc.
- If you are tempted to reach into another subsystem directly, publish a
topic instead.
## Configuration
- `config/vigilar.toml` is the only config file the app reads at runtime.
- It is validated by Pydantic v2 models in `vigilar/config.py`. Add new fields
to the Pydantic models, not just to the TOML.
- Secrets are file paths, not inline values. Never put a key, password, or
token directly in the TOML.
## Testing
- `pytest` from the repo root.
- Tests live under `tests/`.
## Committing
- `ruff check vigilar/` must pass.
- `pytest` must pass.
- One logical change per commit. Commit often.

View File

@@ -0,0 +1,170 @@
# Vigilar Architecture Overview
This document explains how Vigilar is put together for someone reading the
codebase for the first time. It is short on purpose — per-subsystem details
live under `subsystems/`.
## Design principles
- **Offline-first.** No external calls in the critical path. Cloud integrations,
if any, are opt-in and off the hot path.
- **Subsystem isolation.** Each subsystem runs in its own process. A crash in
one subsystem cannot take down another — the supervisor in `vigilar/main.py`
restarts crashed children with exponential backoff.
- **Loose coupling via MQTT.** Subsystems do not call each other directly.
They publish and subscribe to a local Mosquitto broker on `127.0.0.1:1883`.
- **SQLite (WAL) is the single durable store.** Access goes through
SQLAlchemy Core expressions (`vigilar/storage/schema.py`), not ORM mapped
classes. WAL mode and `synchronous=NORMAL` are set on every connection in
`vigilar/storage/db.py`.
- **Adaptive cost.** Cameras idle at 2 FPS and jump to 30 FPS on motion, with
a 5-second ring buffer so the moment leading up to the trigger is kept.
- **Configuration is typed.** `config/vigilar.toml` is loaded and validated
by Pydantic v2. Secrets are never inline — they are file paths under
`/etc/vigilar/secrets/`.
## Process topology
`vigilar start` loads config and calls `run_supervisor()` in `vigilar/main.py`.
The supervisor spawns every subsystem as a `multiprocessing.Process` (via the
`SubsystemProcess` wrapper) and monitors them in a 2-second restart loop.
Cameras are managed separately by `CameraManager`, which owns one child
process per configured camera.
```
systemd (vigilar.service)
|
v
vigilar start (supervisor, main.py)
|
+--------+-----------+-----------+-----------+----------+
| | | | | |
v v v v v v
web event- sensor- ups- presence- health-
(Flask) processor bridge monitor monitor monitor
CameraManager --> camera worker (front_door)
--> camera worker (backyard)
--> camera worker (side_yard)
--> camera worker (garage)
^ ^
| MQTT |
v v
mosquitto (127.0.0.1:1883, loopback only)
```
Every arrow touching the broker is a local TCP connection to loopback. The
`web` process is a Flask server (`vigilar/web/app.py:create_app`) with one
Blueprint per feature area under `vigilar/web/blueprints/`.
## The MQTT bus
- Broker: Mosquitto, bound to loopback only (see `systemd/vigilar-mosquitto.conf`:
`listener 1883 127.0.0.1`, `allow_anonymous true`, `persistence false`).
- Topic convention: every topic starts with `vigilar/` and is defined in
`vigilar/constants.py` via the `Topics` class (either static strings or
builder functions taking an ID). Real examples:
- `vigilar/camera/{camera_id}/motion/start`
- `vigilar/camera/{camera_id}/motion/end`
- `vigilar/camera/{camera_id}/heartbeat`
- `vigilar/sensor/{sensor_id}/{event_type}`
- `vigilar/ups/status`, `vigilar/ups/power_loss`, `vigilar/ups/low_battery`
- `vigilar/system/arm_state`, `vigilar/system/alert`
- Wildcard subscriptions: `vigilar/#`, `vigilar/camera/#`, `vigilar/sensor/#`
- Payloads are JSON dicts. Publishers use `bus.publish_event(topic, **kwargs)`
from `vigilar/bus.py`; new fields are callers' responsibility.
- Why MQTT rather than an in-process queue: crash isolation, introspection with
`mosquitto_sub`, and the option to move subsystems to separate hosts later
without changing the wire format.
## Data flow: from motion to phone notification
1. `vigilar/camera/worker.py:run_camera_worker` — opens the RTSP stream via
`cv2.VideoCapture(..., cv2.CAP_FFMPEG)` with reconnect/backoff, pushes every
frame into a ring buffer, and drives the capture loop.
2. `vigilar/camera/motion.py:MotionDetector.detect` — MOG2 background
subtraction on a downscaled frame; when a new motion edge is found,
`worker.py` publishes `vigilar/camera/{camera_id}/motion/start` with
confidence and zone count.
3. `vigilar/camera/recorder.py:AdaptiveRecorder.start_motion_recording`
stops any idle recording, launches a fresh FFmpeg subprocess at
`motion_fps` (default 30), and writes the flushed ring-buffer frames
(default 5s of pre-roll) before the live frames. On stop, if
`VIGILAR_ENCRYPTION_KEY` is set, the MP4 is re-encrypted in place to
`.vge` via `vigilar/storage/encryption.py:encrypt_file` (AES-256-CTR).
4. `vigilar/events/processor.py:EventProcessor._handle_event` — subscribes
to `vigilar/#`, classifies the topic into an `EventType`/`Severity`, and
writes a row to the `events` table via
`vigilar/storage/queries.py:insert_event`. Wildlife and pet sightings also
get rows in `wildlife_sightings` / `pet_sightings`.
5. `vigilar/events/rules.py:RuleEngine.evaluate` — matches the event against
configured `[[rules]]` from `vigilar.toml` (AND/OR on arm state, sensor
event, camera motion, time window), honours per-rule cooldowns, and returns
a list of actions.
6. `vigilar/alerts/sender.py:send_alert` — for `alert_all` / `push_and_record`
actions, builds a notification from the `_CONTENT_MAP` table, loads the
VAPID key from `[alerts.web_push].vapid_private_key_file`, and calls
`pywebpush.webpush` for every row in `push_subscriptions`. Successes and
failures are recorded in `alert_log`; endpoints returning `410 Gone` are
pruned.
7. Web UI — the browser holds an open SSE connection to a handler in
`vigilar/web/blueprints/events.py` (`mimetype="text/event-stream"`), which
tails new event rows and pushes them to the timeline live.
## Storage layout
- `vigilar.db` under `[system] data_dir` (default `/var/vigilar/data`), SQLite
in WAL mode. Tables defined in `vigilar/storage/schema.py`:
`cameras`, `sensors`, `sensor_states`, `events`, `recordings`,
`system_events`, `arm_state_log`, `alert_log`, `push_subscriptions`,
`pets`, `pet_sightings`, `wildlife_sightings`, `package_events`,
`pet_training_images`, `pet_rules`, `face_profiles`, `face_embeddings`,
`visits`, `timelapse_schedules`.
- Recordings: `.vge` files under `[system] recordings_dir` (default
`/var/vigilar/recordings`), AES-256-CTR with a random 16-byte IV prefixed
to each file. Key at `/etc/vigilar/secrets/storage.key`. **Losing the key
means losing the recordings** — there is no recovery path.
- HLS: rolling segments under `[system] hls_dir` (default `/var/vigilar/hls`),
written by the per-camera `HLSStreamer` in `vigilar/camera/hls.py`.
- Backups: DB + `/etc/vigilar` tarball via `scripts/backup.sh`.
## Configuration and secrets
- `config/vigilar.toml` is the only configuration file the app reads
(systemd points `VIGILAR_CONFIG` at `/etc/vigilar/vigilar.toml` in
production).
- Validated by Pydantic v2 at startup (`vigilar/config.py`).
- Secrets never live in the TOML; they are file paths under
`/etc/vigilar/secrets/` (`storage.key`, `vapid_private.pem`).
- The arm PIN and admin password are hashed; comparisons are constant-time
(see `vigilar/alerts/pin.py`). The PIN hash is written into the TOML via
`vigilar config set-pin`, never typed by hand.
## The web tier
- Flask with Blueprints, one per feature area under
`vigilar/web/blueprints/`: `cameras`, `events`, `kiosk`, `pets`,
`recordings`, `sensors`, `system`, `visitors`, `wildlife`. All registered
in `vigilar/web/app.py:create_app`.
- Jinja2 templates under `vigilar/web/templates/`, Bootstrap 5 dark theme,
static assets under `vigilar/web/static/`.
- Live view: `hls.js` grid for bandwidth efficiency, MJPEG single view for
low latency.
- Live timeline updates via Server-Sent Events from
`vigilar/web/blueprints/events.py`.
- PWA with VAPID web push — no Firebase, no Google Cloud Messaging. Service
worker at `vigilar/web/static/sw.js`.
## What is NOT in the critical path
- Remote access (`[remote]` section) — optional, bandwidth-shaped HLS over
a WireGuard tunnel.
- Email alerts (`[alerts.email]`) and webhook alerts (`[alerts.webhook]`)
— optional, off by default.
- Any cloud service — never.
## Where to go next
- [Conventions](conventions.md) — coding rules distilled for contributors.
- [Subsystems](subsystems/) — per-subsystem references.

View File

@@ -0,0 +1,35 @@
# alerts
## Purpose
Delivers user-facing notifications for events the rule engine has flagged. Provides Web Push (VAPID) delivery to registered browsers and phones, PIN hashing/verification for arm/disarm flows, and a smart-profile matcher that selects recipients based on household state and time-of-day windows. Every alert attempt is logged through `vigilar.alerts` logger; the main supervisor attaches a dedicated syslog handler to that logger so alerts leave an OS-level audit trail.
## Key files
- `vigilar/alerts/sender.py``send_alert`, `build_notification`, VAPID Web Push dispatcher; reads `push_subscriptions` and writes `alert_log`
- `vigilar/alerts/profiles.py` — smart alert profile matcher (household state + time windows)
- `vigilar/alerts/pin.py` — PBKDF2-SHA256 PIN hashing and constant-time verification
## MQTT topics
**Subscribes:** none directly. `alerts` is not a standalone subsystem process; it is invoked synchronously from the events processor when a rule action fires.
**Publishes:** No MQTT publishers found at time of writing. The events processor publishes `vigilar/system/alert` on behalf of alert actions.
## Database tables
- `alert_log` — one row per delivery attempt (channel + status + error)
- `push_subscriptions` — VAPID Web Push subscriptions, read to fan out a notification; stale entries are removed via `delete_push_subscription`
## Depends on
- `events``send_alert` is called from `EventProcessor._execute_action` for `alert_all` / `push_and_record`
- `storage` — reads `push_subscriptions`, writes `alert_log`
## Consumed by
- Phones and browsers — receive Web Push notifications via VAPID
- `web` — the `system` blueprint manages push subscription registration and PIN verification
## Notes
The `vigilar.alerts` logger gets a syslog handler installed by the supervisor (`vigilar/main.py`) so that every alert emission is mirrored into the host's syslog as an out-of-band audit trail, independent of SQLite. PIN verification uses `hmac.compare_digest` to avoid timing leaks. The smart-profile matcher honours `household_state` and HH:MM time windows (including windows that wrap past midnight).

View File

@@ -0,0 +1,44 @@
# camera
## Purpose
Owns all per-camera capture, motion detection, recording, and live streaming. One worker process per configured camera pulls RTSP frames with OpenCV, runs MOG2 background-subtraction motion detection plus optional YOLO object detection, writes H.264 segments via FFmpeg, and exposes HLS for the web grid.
## Key files
- `vigilar/camera/manager.py``CameraManager`, spawns and supervises one `multiprocessing.Process` per camera; polled by main supervisor via `check_and_restart()`
- `vigilar/camera/worker.py` — per-camera capture loop: RTSP read, ring buffer push, motion detect, adaptive-FPS record, YOLO classification, MQTT publish
- `vigilar/camera/motion.py` — MOG2-based `MotionDetector`
- `vigilar/camera/recorder.py` — FFmpeg-subprocess recorder with idle and motion modes; calls `vigilar.storage.encryption.encrypt_file` to produce `.vge` output
- `vigilar/camera/ring_buffer.py` — 5-second pre-motion frame buffer
- `vigilar/camera/hls.py` — HLS segment writer for the web grid view
## MQTT topics
**Subscribes:** `vigilar/camera/{id}/config` (runtime threshold updates)
**Publishes:**
- `vigilar/camera/{id}/motion/start` (MOG2 trigger, and YOLO person/vehicle detections)
- `vigilar/camera/{id}/motion/end`
- `vigilar/camera/{id}/heartbeat`
- `vigilar/camera/{id}/error`
- `vigilar/camera/{id}/pet/detected`
- `vigilar/camera/{id}/wildlife/detected`
- `vigilar/pets/{pet_name}/location` (when a known pet is identified)
## Database tables
The camera subsystem does not write to SQLite directly; recording metadata and events are persisted by the `events` subsystem after consuming motion topics. Encrypted clips are written to disk by `recorder.py`.
## Depends on
- `storage` — uses `vigilar.storage.encryption.encrypt_file` to seal recordings as `.vge`
- `detection` — optional YOLO detector, pet classifier, and wildlife threat classifier are loaded into the worker process
## Consumed by
- `events` — subscribes to the whole bus and classifies motion/heartbeat/error topics into rows in `events` and `recordings`
- `web` — reads HLS segments for the camera grid and decrypts recordings for playback
## Notes
`CameraManager` is the one subsystem NOT wrapped in `SubsystemProcess`; it owns its own child processes and is polled directly from the main supervisor loop. Adaptive FPS runs at 2 FPS idle and 30 FPS while motion is active, with a 5-second ring buffer flushed into the front of each motion clip for pre-event context. YOLO person/vehicle detections are published as extra `motion/start` messages with an additional `detection` field rather than on dedicated topics.

View File

@@ -0,0 +1,44 @@
# detection
## Purpose
Library of computer-vision models and classifiers that the camera worker loads in-process. Provides YOLO/SSD object detection, pet identification, face recognition, wildlife threat scoring, package-delivery state machine, vehicle fingerprinting, activity heatmaps, and sunset-aware scheduling. It is not itself a supervised subsystem; there is no `detection` process and no direct MQTT involvement — the camera worker invokes these modules and publishes on their behalf.
## Key files
- `vigilar/detection/yolo.py` — YOLO object detector and category classifier (person/vehicle/pet/wildlife)
- `vigilar/detection/person.py` — MobileNet-SSD v2 detector for person/car/truck
- `vigilar/detection/pet_id.py` — MobileNetV3-Small pet identification classifier
- `vigilar/detection/trainer.py` — transfer-learning trainer for pet-ID model
- `vigilar/detection/face.py` — dlib/`face_recognition`-based face matcher against `face_profiles`/`face_embeddings`
- `vigilar/detection/wildlife.py` — wildlife species + threat-level classifier
- `vigilar/detection/package.py` — package-delivery state machine (`IDLE → PRESENT → REMINDED → COLLECTED`)
- `vigilar/detection/vehicle.py` — dominant-color and size fingerprinting for vehicles
- `vigilar/detection/zones.py` — polygon zones and intersection tests
- `vigilar/detection/heatmap.py` — accumulates detection bboxes into a grid for activity heatmaps
- `vigilar/detection/crop_manager.py` — saves detection crops into staging and training directories
- `vigilar/detection/solar.py` — stdlib NOAA sunset calculator for twilight-aware logic
## MQTT topics
**Subscribes:** none. Detection modules are library calls inside the camera worker.
**Publishes:** none directly. The camera worker publishes detection results on `vigilar/camera/{id}/motion/start` (person/vehicle, with an extra `detection` field), `vigilar/camera/{id}/pet/detected`, `vigilar/camera/{id}/wildlife/detected`, and `vigilar/pets/{pet_name}/location`.
## Database tables
No direct writes. `face.py` reads `face_profiles` and `face_embeddings` via `vigilar.storage.queries`.
## Depends on
- `storage``face.py` loads face profiles and embeddings
- `camera` — invoked as a library inside the camera worker process
## Consumed by
- `camera` — worker imports and calls these modules per frame
- `pets` — downstream consumer of pet-ID results (via MQTT topics published by camera)
- `events` — consumes the resulting detection topics off the bus
## Notes
Person and vehicle detections are piggybacked on `motion/start` rather than published on their own topics; the events processor currently collapses both back to `MOTION_START`. The package state machine in `package.py` is driven by the camera worker and uses `solar.get_sunset` to decide when a package has been "abandoned" after dusk.

View File

@@ -0,0 +1,43 @@
# events
## Purpose
Central classifier and persistence layer for everything that happens on the MQTT bus. Subscribes to the whole bus, parses topic paths into `EventType`/`Severity`/`source_id`, writes rows into `events`, evaluates the rule engine against the arm-state FSM, and executes resulting actions (send alert, broadcast system-alert, command all cameras to record).
## Key files
- `vigilar/events/processor.py``EventProcessor`: subscribes via `bus.subscribe_all`, classifies topics in `_classify_event` (a big if-chain, no routing table), writes via `insert_event`, executes rule actions
- `vigilar/events/state.py``ArmStateFSM` (HOME/AWAY/NIGHT/DISARMED), publishes `Topics.SYSTEM_ARM_STATE`, persists to `arm_state_log`
- `vigilar/events/rules.py``RuleEngine` evaluating TOML-configured rules against topic/payload/arm-state
- `vigilar/events/history.py` — thin wrapper around `get_events`/`acknowledge_event`
## MQTT topics
**Subscribes:** `vigilar/#` (wildcard via `bus.subscribe_all`)
**Publishes:**
- `vigilar/system/arm_state` (from `ArmStateFSM`)
- `vigilar/system/alert` (when a rule fires `alert_all`/`push_and_record`)
- `vigilar/camera/{id}/command/record` (per-camera record command for `push_and_record`/`record_all_cameras`)
## Database tables
- `events` — canonical event log; every classified message becomes a row
- `arm_state_log` — arm-state transitions written by the FSM
- `pet_sightings` — inserted when classification produces `PET_DETECTED`, `PET_ESCAPE`, or `UNKNOWN_ANIMAL`
- `wildlife_sightings` — inserted for `WILDLIFE_PREDATOR`/`NUISANCE`/`PASSIVE`
## Depends on
- Every publisher on the bus (camera, sensors, ups, presence, detection, health, highlights, etc.)
- `storage` — uses `vigilar.storage.queries` for all inserts
- `alerts` — calls `send_alert` from `vigilar.alerts.sender` when a rule requests it
## Consumed by
- `alerts` — triggered indirectly via rule actions
- `web` — reads `events` table through `vigilar.storage.queries` for the timeline and SSE stream
- `highlights` — consumes the `events` table for reel scoring
## Notes
Classification is a hand-written if-chain keyed on the topic path. Camera `heartbeat` and `error` topics are intentionally ignored (return `None`). Person and vehicle YOLO detections ride on `camera/{id}/motion/start` and currently collapse back to a plain `MOTION_START` event — a known artefact of the publishing quirk in the camera worker. Arm-state change messages on `vigilar/system/*` are also deliberately ignored by the classifier to avoid feedback loops.

View File

@@ -0,0 +1,39 @@
# health
## Purpose
Self-monitoring and housekeeping. Periodically checks disk usage and MQTT reachability, publishes an aggregated system-health snapshot, auto-prunes old recordings when the disk gets full, builds a daily "what happened today" digest, and drives the scheduled firing of highlight-reel and timelapse jobs.
## Key files
- `vigilar/health/monitor.py``HealthMonitor` subsystem loop: disk check every 5 min, MQTT port check every 30 s, status publish every 10 s, daily reel trigger, per-minute timelapse schedule check
- `vigilar/health/pruner.py``auto_prune` / `find_prunable_recordings`; deletes oldest unstarred recordings (and their thumbnails) until usage is back under target
- `vigilar/health/digest.py``build_digest`: counts person/vehicle events, pet and wildlife sightings, and recording totals over a time window for notification digests
- `vigilar/health/__init__.py`
## MQTT topics
**Subscribes:** none
**Publishes:**
- `vigilar/system/health` — rolled-up health status with per-check breakdown
## Database tables
- `recordings` — read in pruner (`starred = 0`, oldest first) and deleted row-by-row once the file is unlinked
- `events`, `pet_sightings`, `wildlife_sightings`, `recordings` — read in `digest.py`
- `timelapse_schedules` — read indirectly via `vigilar.highlights.timelapse.check_schedules`
## Depends on
- `storage` — all DB access
- `highlights` — calls `generate_daily_reel` and `check_schedules` on schedule
- Filesystem — `shutil.disk_usage` on `system.data_dir`
## Consumed by
- `web` — surfaces `vigilar/system/health` and the digest output in the system blueprint
- `events` — ingests `vigilar/system/health` off the bus (subscribes to everything)
## Notes
The pruner is keyed on percentages, not gigabytes: it checks `config.health.disk_warn_pct` / `disk_critical_pct` for alerting and prunes toward `config.health.auto_prune_target_pct`. It never deletes a recording whose `starred` flag is set, and it unlinks the thumbnail alongside the video. The monitor owns the scheduling clock for both `generate_daily_reel` (fires when wall-clock matches `highlights.generate_time`) and timelapse generation (polls `check_schedules` every 60 s), so neither `highlights/` nor its cron is wired directly into the supervisor.

View File

@@ -0,0 +1,35 @@
# highlights
## Purpose
Generates derived video products from recorded footage: a daily highlight reel (events scored and assembled into a short FFmpeg-concatenated clip) and scheduled per-camera timelapses. These are offline/batch jobs that read the `events` and `recordings` tables and write new rows into `recordings` tagged with a synthetic trigger.
## Key files
- `vigilar/highlights/reel.py` — event scoring map (`PET_ESCAPE=10`, `WILDLIFE_PREDATOR=9`, ...), clip selection, FFmpeg concat assembly, inserts resulting file via `insert_recording`
- `vigilar/highlights/timelapse.py` — per-camera timelapse generator; reads `timelapse_schedules`, walks recordings within the configured hour window, stitches with FFmpeg, inserts a new `recordings` row
- `vigilar/highlights/__init__.py`
## MQTT topics
**Subscribes:** No MQTT subscribers found at time of writing.
**Publishes:** No MQTT publishers found at time of writing.
## Database tables
- `events` — read-only, source for reel scoring
- `recordings` — read (source clips) and write (new reel / timelapse entries via `insert_recording`)
- `timelapse_schedules` — read by `timelapse.py` to determine which cameras / hours / generation times are active
## Depends on
- `storage` — all table access via `vigilar.storage.queries` and `vigilar.storage.schema`
- FFmpeg subprocess for clip concatenation and timelapse assembly
## Consumed by
- `web` — the `recordings` blueprint surfaces reels and timelapses alongside regular recordings
## Notes
Scoring is a hand-tuned dict in `reel.py`. The top-scored events of the day are pulled, their surrounding recordings are concatenated via FFmpeg, and the result is persisted as a new `recordings` row with a `RecordingTrigger` tag so the UI can distinguish generated from captured clips. Timelapse generation is driven by `timelapse_schedules` rows — each row defines a camera, an hour window, and a `generate_time` at which the day's frames are stitched.

View File

@@ -0,0 +1,34 @@
# pets
## Purpose
Pet rule engine. Defines composable conditions (detected-in-zone, not-seen-for-N-minutes, detected-without-person, time-of-day, etc.) and per-pet state tracking that downstream code uses to trigger pet-specific alerts and automations. The on-camera pet identification itself lives in `vigilar/detection/pet_id.py`; this package is the behavioural layer on top of those sightings.
## Key files
- `vigilar/pets/rules.py``PetState` dataclass, `evaluate_condition`, rule evaluator; reads `pet_rules` via `get_all_enabled_rules`
- `vigilar/pets/__init__.py` (empty)
## MQTT topics
**Subscribes:** No MQTT subscribers found at time of writing. The module is invoked as a library — typically from the events processor or an evaluator that already has topic payloads in hand.
**Publishes:** No MQTT publishers found at time of writing.
## Database tables
- `pet_rules` — loaded via `vigilar.storage.queries.get_all_enabled_rules`
- `pets`, `pet_sightings`, `pet_training_images` — owned schema-wise by this feature area but written from elsewhere (events processor for sightings, detection/trainer for training images, web for pets CRUD)
## Depends on
- `storage` — reads `pet_rules`
- `alerts.profiles` — reuses `is_in_time_window` for time-of-day conditions
## Consumed by
- `events` / rule evaluator — drives pet-specific alert actions
- `web` — the `pets` blueprint manages pets, training images, and rules
## Notes
`PetState` is per-pet, in-memory, and tracks last-seen camera and time, current zone with entry time, and whether a person is currently present alongside the pet. This makes rules like "cat in kitchen for more than 5 minutes while nobody is home" possible without joining SQLite on every frame. The condition schema is a plain dict with a `type` discriminator, so new condition kinds can be added without schema changes.

View File

@@ -0,0 +1,36 @@
# presence
## Purpose
Tracks whether household members are home by ICMP-pinging their phones on the LAN at a fixed interval. Derives an overall household state (EMPTY, OCCUPIED, etc.) from per-member presence and publishes both to MQTT so the rule engine and arm-state FSM can react.
## Key files
- `vigilar/presence/monitor.py``PresenceMonitor`: ping loop, per-member state machine with `departure_delay_m` grace period, publishes per-member and household-level topics
- `vigilar/presence/models.py``MemberPresence` dataclass and `derive_household_state` aggregator
- `vigilar/presence/__init__.py`
## MQTT topics
**Subscribes:** none
**Publishes:**
- `vigilar/presence/{name}` — per-member status (`HOME` or `AWAY`, plus role)
- `vigilar/presence/status` — aggregated household state and a `{name: HOME|AWAY}` map
## Database tables
none — presence state lives only in memory and on the bus.
## Depends on
- LAN reachability of configured `[presence.members]` IPs
- `ping` binary on the host
## Consumed by
- `events` — rules and the arm-state FSM react to household state changes
- `alerts` — smart alert profile matcher in `vigilar/alerts/profiles.py` branches on `HouseholdState`
## Notes
A member is marked HOME the moment a ping succeeds, but only marked AWAY after `departure_delay_m` minutes without a successful ping — this prevents a single dropped reply from flipping the household state. The monitor uses `time.monotonic()` for the grace window, so it is immune to wall-clock jumps. Per-member and household topics are republished every poll, not only on transitions, so late subscribers always get current state.

View File

@@ -0,0 +1,38 @@
# sensors
## Purpose
Ingests events from physical sensors (Zigbee door/window contacts, motion PIRs, leak sensors, GPIO-attached hardware) and republishes them onto the internal Vigilar bus in a normalized form. Maintains the current sensor state in SQLite so the web UI and rule engine always know whether a contact is open/closed, a leak is active, etc.
## Key files
- `vigilar/sensors/bridge.py``SensorBridge`: subscribes to `zigbee2mqtt/#`, normalizes payloads via `normalize_zigbee_payload`, republishes to `vigilar/sensor/{id}/{event_type}`
- `vigilar/sensors/gpio_handler.py` — direct GPIO input handling; publishes the same `vigilar/sensor/{id}/{event_type}` topics
- `vigilar/sensors/registry.py` — loads configured sensors, maps Zigbee friendly names to sensor IDs
- `vigilar/sensors/models.py``SensorEvent` dataclass and protocol enum
- `vigilar/sensors/__init__.py`
## MQTT topics
**Subscribes:** `{zigbee2mqtt.mqtt_topic_prefix}/#` (external Zigbee2MQTT broker, usually `zigbee2mqtt/#`)
**Publishes:**
- `vigilar/sensor/{id}/{event_type}` — normalized sensor events (contact_open, contact_closed, motion, leak, etc.)
## Database tables
- `sensors` — static sensor registry (read by `SensorRegistry`)
- `sensor_states` — current-state table, kept up to date via `upsert_sensor_state`
## Depends on
- External Zigbee2MQTT broker (same Mosquitto instance, `zigbee2mqtt/...` topic prefix configurable via `[zigbee2mqtt]` TOML section)
- `storage` — uses `vigilar.storage.queries.upsert_sensor_state`
## Consumed by
- `events` — classifies `vigilar/sensor/#` messages into `CONTACT_OPEN`, `CONTACT_CLOSED`, `MOTION_START`, etc. and persists them to the `events` table
- `web` — the `sensors` blueprint reads `sensors` / `sensor_states` for the dashboard
## Notes
`SensorBridge` deliberately skips `zigbee2mqtt/bridge/...` status messages and ignores unknown friendly names with a debug log. Both the Zigbee bridge and the GPIO handler produce identical `vigilar/sensor/{id}/{event_type}` topics so the rest of the system cannot tell a Zigbee door contact from a GPIO-wired one — that's intentional.

View File

@@ -0,0 +1,51 @@
# storage
## Purpose
The persistence layer. Defines every SQLite table, opens the WAL-mode database, exposes hand-written SQLAlchemy Core queries for the rest of the codebase, and provides the AES-256-CTR file encryption used to seal recordings as `.vge`. It is a library package, not a supervised subsystem process.
## Key files
- `vigilar/storage/schema.py` — SQLAlchemy Core `MetaData` and 19 `Table` definitions
- `vigilar/storage/db.py` — engine creation, WAL pragmas, `init_db`, `get_db_path`
- `vigilar/storage/queries.py` — all `insert_*`/`get_*`/`update_*`/`delete_*` helpers
- `vigilar/storage/encryption.py` — AES-256-CTR `encrypt_file` / `decrypt_stream` for `.vge` recordings
## MQTT topics
**Subscribes:** none
**Publishes:** none
## Database tables
- `cameras` — configured camera registry
- `sensors` — configured sensor registry
- `sensor_states` — current key/value state per sensor
- `events` — canonical event log written by the events processor
- `recordings` — video clip metadata (path, duration, trigger, event_id, starred)
- `system_events` — operator-facing component log (info/alert/critical)
- `arm_state_log` — arm-state FSM transitions
- `alert_log` — one row per alert delivery attempt and outcome
- `push_subscriptions` — VAPID Web Push subscription endpoints
- `pets` — known pets (name, species, breed, training count)
- `pet_sightings` — per-frame pet observations with confidence and crop path
- `wildlife_sightings` — wildlife observations with species, threat level, weather context
- `package_events` — package delivery state machine rows (detected/reminded/collected)
- `pet_training_images` — images staged for pet-ID model retraining
- `pet_rules` — per-pet rule definitions with cooldown and priority
- `face_profiles` — known faces, household flag, visit count
- `face_embeddings` — serialized face vectors linked to a profile
- `visits` — per-visit records for recognized faces
- `timelapse_schedules` — per-camera timelapse generation schedules
## Depends on
- Filesystem: SQLite database under `system.data_dir`, AES key at `/etc/vigilar/secrets/storage.key`
## Consumed by
- Every other subsystem. Camera encrypts recordings through it, events/alerts/sensors/ups/pets/highlights/presence/web all import `vigilar.storage.queries`.
## Notes
Encryption is AES-256 in **CTR mode** (no GCM), so `.vge` files are confidential but **not tamper-evident** — there is no authentication tag and a bit-flip in the ciphertext produces a corresponding bit-flip in the plaintext without detection. The 16-byte IV is prepended to each file and the plaintext is unlinked after encryption. The codebase uses SQLAlchemy Core exclusively — there are no mapped ORM classes. The database is opened in WAL mode so the web process can read while the events processor writes.

View File

@@ -0,0 +1,42 @@
# ups
## Purpose
Polls a local NUT (Network UPS Tools) daemon for battery status and publishes power state transitions onto the MQTT bus. When the UPS runs below a configured critical-runtime threshold, triggers a graceful shutdown of the host.
## Key files
- `vigilar/ups/monitor.py``UPSMonitor`: NUT client, status parser, transition detection, MQTT publisher
- `vigilar/ups/shutdown.py``ShutdownSequence`: publishes `SYSTEM_SHUTDOWN` and executes the OS shutdown
- `vigilar/ups/__init__.py`
## MQTT topics
**Subscribes:** none
**Publishes:**
- `vigilar/ups/status` (every poll; status, battery charge %, runtime seconds, input voltage, load %)
- `vigilar/ups/power_loss` (transition `OL -> OB`/`LB`)
- `vigilar/ups/restored` (transition back to `OL`)
- `vigilar/ups/low_battery` (charge below `low_battery_threshold_pct`)
- `vigilar/ups/critical` (runtime below `critical_runtime_threshold_s`, triggers shutdown)
- `vigilar/system/shutdown` (published by `ShutdownSequence`)
## Database tables
- `events` — writes `POWER_LOSS`, `POWER_RESTORED`, `LOW_BATTERY` rows via `insert_event`
- `system_events` — writes operator-facing notices (`"Power loss detected — running on battery"`, critical shutdown messages)
## Depends on
- External NUT daemon reachable at `[ups] nut_host:nut_port` via the `pynut2` client
- `storage``insert_event`, `insert_system_event`
## Consumed by
- `events` — classifies `vigilar/ups/*` into `POWER_LOSS` / `LOW_BATTERY` / `POWER_RESTORED`
- `alerts` — receives those classified events through rule actions
- `main` supervisor — reacts to `SYSTEM_SHUTDOWN` so all subsystems can drain cleanly
## Notes
The critical shutdown trigger fires when the UPS is not online AND `battery.runtime < ups.critical_runtime_threshold_s` AND no shutdown has already been triggered. The low-battery alert fires when `battery.charge < ups.low_battery_threshold_pct` while off mains. NUT reconnection uses exponential backoff capped at 120 seconds, so the monitor survives a temporary NUT restart without the supervisor having to restart it.

View File

@@ -0,0 +1,43 @@
# web
## Purpose
Flask application that exposes everything to humans: camera grid and single-camera views, event timeline, recordings browser, sensor dashboard, system/arm controls, kiosk mode, and the pets / visitors / wildlife journals. The web process runs as one `SubsystemProcess` supervised by `vigilar/main.py`, reads state directly from SQLite, and publishes a handful of config/control MQTT messages on behalf of the user.
## Key files
- `vigilar/web/app.py``create_app(cfg)` factory (note: `vigilar/web/__init__.py` is empty; the factory lives here)
- `vigilar/web/blueprints/cameras.py` — HLS streams, snapshots, per-camera view
- `vigilar/web/blueprints/events.py` — event log list, acknowledgement, and `text/event-stream` SSE endpoint for live updates
- `vigilar/web/blueprints/recordings.py` — browse, download, delete recordings; serves decrypted `.vge` content
- `vigilar/web/blueprints/sensors.py` — sensor status and history
- `vigilar/web/blueprints/system.py` — health, arm/disarm, UPS, admin settings, Web Push subscription registration, PIN management; publishes `vigilar/camera/{id}/config` for threshold updates
- `vigilar/web/blueprints/kiosk.py` — fullscreen 2x2 ambient grid
- `vigilar/web/blueprints/pets.py` — pet registration, sightings, labeling, training-image upload
- `vigilar/web/blueprints/visitors.py` — face profiles, visits, labeling, privacy controls
- `vigilar/web/blueprints/wildlife.py` — wildlife sightings journal, stats, CSV export
## MQTT topics
**Subscribes:** none. The web process does not subscribe to the bus at import time; the SSE endpoint accumulates messages in an in-process queue but nothing wires the MQTT bus into that queue today (see Notes).
**Publishes:**
- `vigilar/camera/{camera_id}/config` — runtime camera config updates (sensitivity / motion thresholds) from the system settings page
## Database tables
All access is read-mostly via `vigilar.storage.queries`. Touches essentially every table: `cameras`, `events`, `recordings`, `sensors`/`sensor_states`, `system_events`, `arm_state_log`, `alert_log`, `push_subscriptions`, `pets`/`pet_sightings`/`pet_training_images`/`pet_rules`, `face_profiles`/`face_embeddings`/`visits`, `wildlife_sightings`, `timelapse_schedules`.
## Depends on
- `storage` — engine passed via `app.config["DB_ENGINE"]` / `VIGILAR_CONFIG`
- `camera` — reads HLS segments from `cfg.system.hls_dir` and decrypts recordings via `vigilar.storage.encryption`
- `alerts` — uses `vigilar.alerts.pin.hash_pin`/`verify_pin` and manages `push_subscriptions`
- `config_writer` — persists camera/alert config changes back to TOML
## Consumed by
- Browsers and phones — HTML dashboard, PWA with Web Push (VAPID), HLS grid via `hls.js`, MJPEG fallback for single-camera low-latency view, SSE for the live event timeline
## Notes
Templates use Jinja2 with a Bootstrap 5 dark theme; the camera grid uses `hls.js` for multi-camera HLS playback with an MJPEG fallback on the single-camera page. The app is a PWA (service worker + manifest) with VAPID Web Push for mobile notifications. The event-timeline SSE endpoint lives around line 93 of `blueprints/events.py` and holds a per-client `queue.Queue` fed by a module-level `broadcast_sse_event` function. A dedicated MQTT → SSE bridge (`vigilar/web/sse_bridge.py`) runs inside the web process, subscribes to `Topics.EVENTS_PUBLISHED`, and calls `broadcast_sse_event` for every classified event emitted by the events subsystem — so the in-browser event timeline updates live without a page refresh.

337
docs/home-user-guide.md Normal file
View File

@@ -0,0 +1,337 @@
# Vigilar Home User Guide
This guide takes you from a bare mini PC to working cameras on your phone.
It assumes you are comfortable with a Linux command line and can SSH into
a machine on your network.
If you are running Vigilar on a server you already administer, see the
[Operator Guide](operator-guide.md) instead.
## What you will end up with
- A small always-on box in your house that records from your IP cameras.
- A web UI at `http://<mini-pc-ip>:49735` on your LAN.
- Push notifications on your phone when motion is detected.
- Optional nightly config + database backup to a NAS share.
```
[IP cameras] --RTSP--> [mini PC running Vigilar] --LAN--> [your phone/browser]
|
+-- optional nightly --> [NAS]
```
## What you need
- A mini PC. x86_64, 4 GB RAM minimum, 128 GB SSD minimum. Any modern
Intel NUC, Beelink, MinisForum, or similar will do.
- A USB stick (8 GB or larger) for installing Linux.
- One or more RTSP IP cameras on your LAN. See the
[Camera Hardware Guide](camera-hardware-guide.md) for picks.
- A phone for push notifications.
- Optional: a NAS on your LAN for backups.
## Step 1 — Install Linux on the mini PC
Install Debian 12 or Ubuntu Server 24.04 (or later). The official installers
walk you through it. During install:
- Choose **OpenSSH server** when asked for tasks/components.
- Set a hostname you will remember (e.g. `vigilar`).
- Create a user with `sudo`.
After install, note the LAN IP (`ip addr` on the box) and confirm you can
SSH in from your laptop.
Arch Linux works too — the installer supports both `apt` and `pacman`
but the rest of this guide assumes a Debian/Ubuntu system.
## Step 2 — Install Vigilar
SSH into the mini PC, clone the repository, and run the installer:
```bash
git clone <repo URL> vigilar
cd vigilar
sudo ./scripts/install.sh
```
`install.sh` will:
- Install system packages (`ffmpeg`, `mosquitto`, `python3`, `nut-client`),
create a locked-down `vigilar` system user, and lay out `/opt/vigilar`,
`/var/vigilar`, and `/etc/vigilar` with the right permissions.
- Build a Python virtualenv at `/opt/vigilar/venv`, install the `vigilar`
package into it, generate a 32-byte storage encryption key at
`/etc/vigilar/secrets/storage.key`, and drop a sample config at
`/etc/vigilar/vigilar.toml`.
- Install the `vigilar.service` systemd unit and a Mosquitto config that
binds the internal MQTT broker to localhost only, then restart Mosquitto.
Review the `[INFO]` and `[OK]` output. Anything marked `[FAIL]` needs
attention before you continue.
The installer finishes by printing a short list of follow-up scripts it
does **not** run for you. The ones you want now are:
```bash
sudo ./scripts/gen_vapid_keys.sh
```
That generates the VAPID keypair used for phone push notifications. If
you skip it, cameras and the web UI will still work, but phone push
will not.
The other optional follow-ups are `gen_cert.sh` (TLS certificates for
HTTPS on the LAN) and `setup_nut.sh` (UPS battery monitoring). You can
come back to those later.
## Step 3 — Edit the config and set a PIN
The installer drops a sample config at `/etc/vigilar/vigilar.toml`. Open
it in your editor:
```bash
sudoedit /etc/vigilar/vigilar.toml
```
You'll add cameras from the web UI later, so for now you only need to
set two things: the arm/disarm PIN and the web UI password.
### PIN
Vigilar stores the arm/disarm PIN as a keyed hash — there is no recovery.
Generate the hash with the CLI:
```bash
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-pin
```
Enter a PIN when prompted (twice). The command prints a line like:
```
arm_pin_hash = "…:…"
```
Copy that line into the `[system]` section of `/etc/vigilar/vigilar.toml`.
### Web UI password
Same idea for the password that protects the web UI:
```bash
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-password
```
Paste the printed `password_hash = "…"` line into the `[web]` section.
### Validate
Before starting the service, make sure the config parses:
```bash
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config validate
```
It should print `Config OK` and a short summary. If it prints
`Config INVALID`, fix the file and try again.
## Step 4 — Start it up
```bash
sudo systemctl enable --now vigilar
sudo systemctl status vigilar
```
You should see `active (running)`. Now open a browser on any LAN device:
```
http://<mini-pc-ip>:49735
```
Log in with the password you set in Step 3. The web port is configurable
under `[web] port` in `/etc/vigilar/vigilar.toml` — the default is
`49735`.
> Screenshot placeholder: `docs/images/first-run-login.png`
## Step 5 — Add your first camera
From the web UI, go to **Cameras → Add camera**. You'll need:
- A name (e.g. "Front door").
- The camera's RTSP URL — see the
[Camera Hardware Guide](camera-hardware-guide.md) for the format your
brand uses.
- Username and password for the RTSP stream (often separate from the
camera's admin login).
Click **Test**, then **Save**. The camera appears in the grid. It idles
at 2 FPS and jumps to 30 FPS when motion is detected, with a 5-second
pre-motion ring buffer so you don't lose the run-up to an event.
> Screenshot placeholder: `docs/images/add-camera.png`
Repeat for each camera. The multi-camera grid uses HLS so a handful of
streams are comfortable over LAN Wi-Fi; the single-camera view falls
back to MJPEG for lower latency.
## Step 6 — Phone push notifications
Vigilar is a Progressive Web App, so your phone installs it straight
from the browser:
1. Open `http://<mini-pc-ip>:49735` in your phone's browser on the
same Wi-Fi.
2. Log in.
3. Use your browser's "Add to Home Screen" option (Safari: share sheet;
Chrome: three-dot menu).
4. Open the installed app from your home screen.
5. When prompted, allow notifications.
As long as `gen_vapid_keys.sh` was run back in Step 2, you do not need
to configure anything else. The next motion event will push a
notification to your phone.
**Note on the event timeline:** the in-browser event list does not
update live today — you need to refresh the page to see new events.
Push notifications to your phone work independently and do arrive in
real time. Fixing live timeline updates in the browser is on the
roadmap.
## Step 7 — Optional: NAS backup
Vigilar ships a `backup.sh` script that tars the SQLite database and
the contents of `/etc/vigilar` (config + secrets) once a day. It does
**not** back up recordings — those stay on the mini PC's local disk.
Recording backup is planned as a later improvement.
### Mount the NAS share
NFS example (swap in SMB if that's what your NAS speaks):
```bash
sudo apt-get install -y nfs-common
sudo mkdir -p /mnt/nas/vigilar-backups
echo "nas.lan:/volume1/vigilar-backups /mnt/nas/vigilar-backups nfs defaults,_netdev 0 0" | sudo tee -a /etc/fstab
sudo mount -a
```
Replace `nas.lan:/volume1/vigilar-backups` with your NAS's actual host
and export path. Confirm it mounted:
```bash
mount | grep vigilar-backups
```
### Schedule `backup.sh`
Create `/etc/systemd/system/vigilar-backup.service`:
```ini
[Unit]
Description=Vigilar nightly backup
After=vigilar.service
[Service]
Type=oneshot
Environment=VIGILAR_BACKUP_DIR=/mnt/nas/vigilar-backups
Environment=VIGILAR_BACKUP_RETENTION_DAYS=30
ExecStart=/opt/vigilar/scripts/backup.sh
```
Create `/etc/systemd/system/vigilar-backup.timer`:
```ini
[Unit]
Description=Run vigilar backup nightly
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
```
Enable the timer:
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now vigilar-backup.timer
sudo systemctl list-timers | grep vigilar
```
Test it immediately without waiting for midnight:
```bash
sudo systemctl start vigilar-backup.service
ls -lh /mnt/nas/vigilar-backups/
```
You should see a `vigilar-backup-<timestamp>.tar.gz` file. Thirty days
of these will rotate automatically thanks to
`VIGILAR_BACKUP_RETENTION_DAYS`.
## Troubleshooting
### The web UI will not load
- `sudo systemctl status vigilar` — is it running?
- `journalctl -u vigilar -e` — what does the last error say?
- Port 49735 blocked or bound elsewhere?
`sudo ss -tlnp | grep 49735`.
- Wrong address? On the mini PC, `ip addr` shows its LAN IP.
### A camera will not connect
- Try the RTSP URL in VLC from your laptop (Media → Open Network
Stream). If VLC cannot play it either, it is a camera or network
problem, not Vigilar.
- Double-check credentials — many cameras have a separate RTSP login
distinct from the admin web login.
- Some cameras need a specific stream path (e.g. `/stream1`,
`/cam/realmonitor?channel=1`). See the
[Camera Hardware Guide](camera-hardware-guide.md).
### No push notifications on my phone
- Did you run `sudo ./scripts/gen_vapid_keys.sh`? `install.sh` does
not do this for you.
- Open the PWA from your home screen, not a browser tab. iOS in
particular only delivers web push to installed PWAs.
- Check the browser's notification permission for the site.
- Remember: the in-browser event timeline is not live. Refresh the
page to see events land. Push is the real-time channel.
### Service keeps restarting
- `journalctl -u vigilar -n 100` and look at the last stack trace.
- Common cause: invalid `vigilar.toml`. Validate it:
```bash
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config validate
```
- The supervisor will give up after ten consecutive restarts of the
same subsystem, so a persistent crash loop eventually stops on its
own — check the logs and fix the underlying cause.
### I forgot my PIN
There is no recovery — the PIN is only stored as a hash. Generate a new
one and replace the line in `/etc/vigilar/vigilar.toml`:
```bash
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-pin
sudoedit /etc/vigilar/vigilar.toml # replace arm_pin_hash in [system]
sudo systemctl restart vigilar
```
The same trick works for a forgotten web UI password — use
`config set-password` instead and update `password_hash` in `[web]`.
## Where to go next
- [Operator Guide](operator-guide.md) — configuration reference, backups, upgrades, security.
- [Camera Hardware Guide](camera-hardware-guide.md) — buying advice and
RTSP URL formats per brand.
- [Architecture Overview](architecture/overview.md) — how Vigilar is put
together under the hood.

667
docs/operator-guide.md Normal file
View File

@@ -0,0 +1,667 @@
# Vigilar Operator Guide
## Audience and scope
This guide is for administrators installing and operating Vigilar on a
server they already manage. It is a reference for the on-disk layout,
configuration keys, CLI, systemd integration, secrets, UPS integration,
backups, upgrades, health and pruning, remote access, and the current
set of known limitations.
If you are setting up a home system from a bare mini PC for the first
time, start with the [Home User Guide](home-user-guide.md) and return
here when you need reference-level detail.
## Layout on disk
`scripts/install.sh` lays the system out as follows. The paths are
fixed in the installer; the configuration keys that reference them are
shown in parentheses.
| Path | Owner | Mode | Purpose |
|---|---|---|---|
| `/opt/vigilar/` | `vigilar:vigilar` | 0755 | Install root and home of the service user |
| `/opt/vigilar/venv/` | `vigilar:vigilar` | 0755 | Python virtual environment with the `vigilar` entry point |
| `/etc/vigilar/` | `root:root` | 0755 | Configuration root |
| `/etc/vigilar/vigilar.toml` | `root:vigilar` | 0644 | Main config file (`VIGILAR_CONFIG`) |
| `/etc/vigilar/secrets/` | `root:root` | 0700 | Storage key, VAPID private key |
| `/etc/vigilar/secrets/storage.key` | `root:root` | 0600 | 32-byte AES-256 key for recording encryption |
| `/etc/vigilar/secrets/vapid_private.pem` | `root:root` | 0600 | VAPID signing key for Web Push |
| `/etc/vigilar/secrets/vapid_public.txt` | `root:vigilar` | 0644 | VAPID public key (base64url) |
| `/etc/vigilar/certs/` | `root:vigilar` | 0750 | TLS material |
| `/etc/vigilar/certs/cert.pem` | `root:vigilar` | 0644 | TLS certificate (`[web] tls_cert`) |
| `/etc/vigilar/certs/key.pem` | `root:vigilar` | 0640 | TLS private key (`[web] tls_key`) |
| `/var/vigilar/` | `vigilar:vigilar` | 0750 | Runtime data root |
| `/var/vigilar/data/` | `vigilar:vigilar` | 0750 | SQLite database and supporting files (`[system] data_dir`) |
| `/var/vigilar/data/vigilar.db` | `vigilar:vigilar` | 0640 | Main SQLite database (WAL mode) |
| `/var/vigilar/recordings/` | `vigilar:vigilar` | 0750 | `.vge` encrypted recordings and thumbnails (`[system] recordings_dir`) |
| `/var/vigilar/hls/` | `vigilar:vigilar` | 0750 | HLS segments and playlists (`[system] hls_dir`) |
| `/etc/systemd/system/vigilar.service` | `root:root` | 0644 | Systemd unit |
| `/etc/mosquitto/conf.d/vigilar.conf` | `root:root` | 0644 | Localhost-only MQTT config |
The `VIGILAR_CONFIG` environment variable is read by the CLI and the
web blueprints to locate `vigilar.toml`; the systemd unit sets it to
`/etc/vigilar/vigilar.toml`.
## Installation
`scripts/install.sh` is idempotent and supports Debian/Ubuntu (apt) and
Arch Linux (pacman). It performs eight phases:
1. **System dependencies.** On apt: `ffmpeg mosquitto python3
python3-venv python3-pip nut-client`. On pacman: `ffmpeg mosquitto
python python-virtualenv nut`.
2. **System user.** Creates the `vigilar` system user and group with
`/opt/vigilar` as the home directory and `/usr/sbin/nologin` as the
shell.
3. **Directories and permissions.** Creates `/var/vigilar/{data,
recordings,hls}` owned by `vigilar:vigilar` at 0750, plus
`/etc/vigilar/{secrets,certs}` with the modes shown above.
4. **Python venv.** Creates `/opt/vigilar/venv` as the `vigilar` user
and installs the project in place with `pip install "${PROJECT_DIR}"`.
5. **Storage encryption key.** Writes 32 random bytes from
`/dev/urandom` to `/etc/vigilar/secrets/storage.key` if it does not
already exist. This file is never rewritten by the installer.
6. **Sample config.** Copies the repository's `config/vigilar.toml` to
`/etc/vigilar/vigilar.toml` if a config does not already exist.
7. **Systemd unit.** Installs and enables `vigilar.service`.
8. **Mosquitto.** Installs `systemd/vigilar-mosquitto.conf` to
`/etc/mosquitto/conf.d/vigilar.conf` and restarts `mosquitto.service`.
The installer prints recommended follow-up steps: edit the TOML, then
run `gen_cert.sh`, `gen_vapid_keys.sh`, and `setup_nut.sh`, then start
the service.
### Systemd unit
`vigilar.service` runs `/opt/vigilar/venv/bin/vigilar start --config
/etc/vigilar/vigilar.toml` as `vigilar:vigilar` with
`VIGILAR_CONFIG=/etc/vigilar/vigilar.toml`. It requires
`mosquitto.service`, wants `nut-monitor.service`, and uses
`Restart=on-failure`, `RestartSec=10`, and `WatchdogSec=120`. The unit
applies `ProtectSystem=strict`, `ProtectHome`, `PrivateTmp`,
`PrivateDevices`, `NoNewPrivileges`, and a `@system-service` syscall
filter. `ReadWritePaths` is limited to `/var/vigilar/{data,recordings,
hls}`; `/etc/vigilar` is mounted read-only. Output goes to the journal
with `SyslogIdentifier=vigilar`.
### Mosquitto configuration
`vigilar-mosquitto.conf` binds a single listener on `127.0.0.1:1883`,
allows anonymous connections (localhost only), disables persistence
(all state lives in SQLite), and logs errors, warnings, notices, and
connection events to syslog. Vigilar never authenticates to the broker
and never exposes it beyond loopback.
## Configuration reference
`config/vigilar.toml` is parsed by `tomllib`, then validated by the
Pydantic models in `vigilar/config.py`. The models are the source of
truth: any unknown key is rejected, and each section has a default so
omitted sections behave sensibly.
### `[system]`
- `name` (default `"Vigilar Home Security"`): display name used in
logs and the web UI.
- `timezone` (default `"UTC"`; sample ships as
`"America/New_York"`): used for daily digests, highlight scheduling,
and timestamped file paths.
- `data_dir` (default `/var/vigilar/data`): SQLite database and
derived state.
- `recordings_dir` (default `/var/vigilar/recordings`): encrypted
`.vge` files.
- `hls_dir` (default `/var/vigilar/hls`): HLS segment output.
- `log_level` (default `"INFO"`): one of DEBUG, INFO, WARNING, ERROR.
- `arm_pin_hash` (default `""`): **deprecated.** Still parsed but
ignored at runtime. Use `[security] pin_hash` instead; run
`vigilar config set-pin` to generate the canonical hash.
### `[mqtt]`
- `host` (default `127.0.0.1`) and `port` (default `1883`): broker
address. Leave on loopback unless you deliberately run a shared
broker.
- `username`, `password` (default `""`): unused by the shipped
mosquitto config, present for operators who run their own broker.
### `[web]`
- `host` (default `0.0.0.0`) and `port` (default `49735`): Flask
listener. Change `host` to `127.0.0.1` if you front with a reverse
proxy.
- `tls_cert`, `tls_key` (default `""`): PEM paths. `gen_cert.sh`
fills these in.
- `username` (default `"admin"`): web UI login name.
- `password_hash` (default `""`): scrypt hash set via `vigilar config
set-password`.
- `session_timeout` (default `3600` seconds).
### `[zigbee2mqtt]`
- `mqtt_topic_prefix` (default `"zigbee2mqtt"`): used when subscribing
to sensor topics from an external Zigbee2MQTT bridge.
### `[ups]`
See also the UPS/NUT section below.
- `enabled` (default `true`).
- `nut_host` (default `127.0.0.1`), `nut_port` (default `3493`),
`ups_name` (default `"ups"`): matches the `[ups]` block generated by
`setup_nut.sh`.
- `poll_interval_s` (default `30`).
- `low_battery_threshold_pct` (default `20`, range 595).
- `critical_runtime_threshold_s` (default `300`).
- `shutdown_delay_s` (default `60`).
### `[storage]`
- `encrypt_recordings` (default `true`): toggles AES-256-CTR
encryption of new `.vge` files. Changing this does not re-encrypt
existing recordings.
- `key_file` (default `/etc/vigilar/secrets/storage.key`): 32-byte
raw key.
- `max_disk_usage_gb` (default `200`) and `free_space_floor_gb`
(default `10`): **legacy keys**. They are defined on the Pydantic
model and exposed in the settings UI, but no pruning or recording
code currently reads them. The real disk ceiling is the
percentage-based pruner in `[health]`. Do not rely on these two
fields to cap disk usage today.
### `[remote]`
- `enabled` (default `false`): turns on the remote-access bridge.
- `upload_bandwidth_mbps` (default `22.0`): informational ceiling.
- `remote_hls_resolution` (default `[426, 240]`), `remote_hls_fps`
(default `10`), `remote_hls_bitrate_kbps` (default `500`): quality
profile for HLS served over the tunnel.
- `max_remote_viewers` (default `4`; `0` = unlimited).
- `tunnel_ip` (default `"10.99.0.2"`): WireGuard address of the home
server, for display only.
### `[alerts.local]`
- `enabled` (default `true`).
- `syslog` (default `true`): the supervisor installs a `SysLogHandler`
on the `vigilar.alerts` logger when this is true.
- `desktop_notify` (default `false`): `notify-send` fallback for
operator-console deployments.
### `[alerts.web_push]`
- `enabled` (default `true`).
- `vapid_private_key_file` (default
`/etc/vigilar/secrets/vapid_private.pem`).
- `vapid_claim_email` (default `"mailto:admin@vigilar.local"`): used
as the VAPID `sub` claim.
### `[alerts.email]`
- `enabled` (default `false`).
- `smtp_host`, `smtp_port` (default `587`), `from_addr`, `to_addr`,
`use_tls` (default `true`).
### `[alerts.webhook]`
- `enabled` (default `false`).
- `url`, `secret`: HMAC secret signs outbound webhook bodies.
### `[[cameras]]` (array of tables)
One block per camera. Keys:
- `id`, `display_name`, `rtsp_url`: required.
- `enabled` (default `true`).
- `record_continuous` (default `false`), `record_on_motion` (default
`true`).
- `motion_sensitivity` (default `0.7`, range 0.01.0) and
`motion_min_area_px` (default `500`).
- `motion_zones`, `zones`: polygon and named-zone overrides.
- `pre_motion_buffer_s` (default `5`) and `post_motion_buffer_s`
(default `30`).
- `idle_fps` (default `2`, range 130) and `motion_fps` (default
`30`, range 160): the adaptive FPS pair.
- `retention_days` (default `30`).
- `resolution_capture` (default `[1920, 1080]`) and
`resolution_motion` (default `[640, 360]`): capture size and the
downscale used for MOG2 motion detection.
- `location` (default `INTERIOR`): `CameraLocation` enum, used for
alert profiles.
Camera IDs must be unique; the Pydantic root validator rejects
duplicates.
### `[[sensors]]` and `[sensors.gpio]`
Each `[[sensors]]` block has `id`, `display_name`, `type` (e.g.
`CONTACT`, `MOTION`, `TEMPERATURE`), `protocol` (`ZIGBEE`, `ZWAVE`,
`GPIO`), `device_address`, `location`, and `enabled` (default
`true`). `[sensors.gpio] bounce_time_ms` (default `50`) applies to all
GPIO sensors. Sensor IDs must also be unique.
### `[[rules]]`
Each rule has `id`, `description`, `conditions` (list of `{type,
value, sensor_id, event}` maps), `logic` (`AND` or `OR`, default
`AND`), `actions` (list of action names like `alert_all` or
`record_all_cameras`), and `cooldown_s` (default `60`).
### `[detection]` and `[vehicles]`
- `[detection] person_detection` (default `false`), `model_path`,
`model_config_path`, `confidence_threshold` (default `0.5`),
`cameras` (empty list means all cameras).
- `[[vehicles.known]]` entries define recognised vehicles with
`name`, `color_profile`, `size_class`, `calibration_file`.
### `[presence]`
- `enabled` (default `false`).
- `ping_interval_s` (default `30`) and `departure_delay_m` (default
`10`).
- `method`: `icmp` or `arping`.
- `[[presence.members]]` entries with `name`, `ip`, and `role`
(`adult` or `child`).
- `actions`: mapping of states (`EMPTY`, `ADULTS_HOME`, `KIDS_HOME`,
`ALL_HOME`) to arm states.
### `[health]`
This is where pruning actually lives.
- `enabled` (default `true`).
- `disk_warn_pct` (default `85`): warning threshold on the partition
hosting `data_dir`.
- `disk_critical_pct` (default `95`): critical threshold. When crossed
and `auto_prune` is true, the health monitor runs the pruner.
- `auto_prune` (default `true`).
- `auto_prune_target_pct` (default `80`): pruner deletes the oldest
non-starred recordings until disk usage drops below this percentage.
- `daily_digest` (default `true`) and `daily_digest_time` (default
`"08:00"`).
### `[pets]`, `[visitors]`, `[highlights]`, `[kiosk]`
Subsystem-specific toggles. See the subsystem references under
`docs/architecture/` for per-key behaviour. Notable defaults: `[pets]
enabled = false`, `[visitors] enabled = false`, `[highlights] enabled
= true`, `[kiosk] ambient_enabled = true`.
### `[location]` and `[security]`
- `[location] latitude`, `longitude` (default `0.0`): used for sunrise
and sunset lookups.
- `[security] pin_hash` (canonical arm/disarm PIN store): populated by
`vigilar config set-pin`, which emits a PBKDF2-SHA256 hash to paste
into the `[security]` section. The legacy `[system] arm_pin_hash`
field is deprecated; see the `[system]` section above.
- `[security] recovery_passphrase_hash`: used by the web
`/system/api/reset-pin` endpoint to authenticate PIN-reset requests.
There is no CLI helper for this field today — set it by hashing a
passphrase manually with `vigilar.alerts.pin.hash_pin` and pasting
the result into `[security]`, or leave it unset to disable recovery.
## CLI reference
The entry point is `/opt/vigilar/venv/bin/vigilar`. All commands
accept `--version`. In production, run subcommands as the service user
so file ownership and venv paths line up:
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar <subcommand>
```
The CLI exposes exactly two top-level commands: `start` and `config`.
### `vigilar start`
Starts all services under the supervisor.
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar start \
--config /etc/vigilar/vigilar.toml
```
Options: `--config/-c PATH` (defaults to `$VIGILAR_CONFIG` then
`config/vigilar.toml`); `--log-level {DEBUG,INFO,WARNING,ERROR}`
(overrides `[system] log_level`). On invocation it loads and validates
the config, configures a console log formatter, prints a startup
summary (camera count, sensor count, UPS state), then hands off to
`vigilar.main.run_supervisor`.
### `vigilar config validate`
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config validate \
-c /etc/vigilar/vigilar.toml
```
Parses and validates the TOML against the Pydantic models and prints
a summary. Exits non-zero if validation fails. Run this after every
edit before restarting the service.
### `vigilar config show`
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config show \
-c /etc/vigilar/vigilar.toml
```
Dumps the parsed config as JSON with `web.password_hash`,
`security.pin_hash`, `security.recovery_passphrase_hash`, and
`alerts.webhook.secret` redacted. Useful for confirming which
defaults Pydantic applied for keys you did not set.
### `vigilar config set-password`
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-password
```
Prompts for a web UI password (hidden, confirmed), derives a scrypt
hash (`n=16384, r=8, p=1`, random 16-byte salt, 32-byte output), and
prints a `password_hash = "salt_hex:key_hex"` line to paste into
`[web]`. It does not write the file.
### `vigilar config set-pin`
```
sudo -u vigilar /opt/vigilar/venv/bin/vigilar config set-pin
```
Prompts for an arm/disarm PIN, derives a salted PBKDF2-SHA256 hash
(600,000 iterations) via `vigilar.alerts.pin.hash_pin`, and prints a
`pin_hash = "pbkdf2_sha256$salt$dk"` line to paste into `[security]`.
Again, no file write. The same hash format is verified identically by
the web arm/disarm endpoint and by `ArmStateFSM` in the event
processor — there is one canonical PIN store.
## Secrets and security
- `/etc/vigilar/secrets/` is `root:root` mode `0700`. The `vigilar`
user cannot list it. Individual files the service needs (for
example `vapid_public.txt`) are readable by group `vigilar`.
- The storage encryption key is `/etc/vigilar/secrets/storage.key`:
32 raw bytes. **If this file is lost, every existing `.vge`
recording becomes unrecoverable.** Back it up separately (and
offline) from your tar archive whenever you take the system into
production.
- Recordings use **AES-256-CTR** (see `vigilar/storage/encryption.py`).
CTR provides confidentiality but no authentication: `.vge` files
are confidential but not tamper-evident. An attacker with write
access to the recordings directory can flip bits in a ciphertext
without detection. If tamper-evidence matters, keep the recordings
volume on integrity-verified storage (dm-integrity, ZFS with
checksums) or mirror to write-once media.
- The web UI password is a scrypt hash set by `vigilar config
set-password` and stored at `[web] password_hash`. The arm/disarm
PIN is a PBKDF2-SHA256 hash (600k iterations, salted) set by
`vigilar config set-pin` and stored at `[security] pin_hash`.
A legacy `[system] arm_pin_hash` field is still parsed but ignored
at runtime; if it's set and `[security] pin_hash` is empty, the
service logs a deprecation warning at startup and arm/disarm will
behave as if no PIN were configured until you re-run `set-pin`.
- TLS: `gen_cert.sh` uses `mkcert` if present, otherwise an `openssl`
ECDSA P-256 self-signed certificate valid for 3650 days with SANs
for `vigilar.local`, `localhost`, `127.0.0.1`, and the detected LAN
IP. It patches `[web] tls_cert`/`tls_key` into the config.
- VAPID: `gen_vapid_keys.sh` writes
`/etc/vigilar/secrets/vapid_private.pem` (mode 0600) and
`/etc/vigilar/secrets/vapid_public.txt` (the browser-side key).
- Firewall stance: the mosquitto broker and NUT daemon bind only to
`127.0.0.1`. The only port Vigilar exposes on the LAN is the web UI
port (default `49735`). Open that port only on the interface that
serves your LAN, and keep WAN exposure behind the WireGuard tunnel
described under `[remote]`.
## UPS and NUT integration
`scripts/setup_nut.sh` installs NUT, attempts to detect a USB UPS
(using `nut-scanner` first, then a short list of vendor IDs as a
fallback), and writes a standalone configuration:
- `/etc/nut/nut.conf` with `MODE=standalone`.
- `/etc/nut/ups.conf` with `[ups] driver=usbhid-ups port=auto` (the
block name `ups` matches the default `[ups] ups_name`).
- `/etc/nut/upsd.conf` with `LISTEN 127.0.0.1 3493` — loopback only.
- `/etc/nut/upsd.users` with a `vigilar` local monitoring user.
- `/etc/nut/upsmon.conf` pointing at `ups@localhost`.
It then enables `nut-driver`, `nut-server`, and `nut-monitor` (or
`upsd`/`upsmon` on distros that ship the old unit names). Test with
`upsc ups@localhost`. The Vigilar UPS subsystem polls this daemon
using the keys under `[ups]`.
## Backups
`scripts/backup.sh` produces
`${VIGILAR_BACKUP_DIR:-/var/vigilar/backups}/vigilar-backup-YYYYMMDD-HHMMSS.tar.gz`
and includes:
- A consistent SQLite snapshot produced with `sqlite3 … .backup` (or
a direct file copy if `sqlite3` is not available), plus any
`-wal`/`-shm` files.
- The entire `/etc/vigilar/` tree (config, secrets, certs).
It does **not** include `/var/vigilar/recordings` or
`/var/vigilar/hls`. Video is assumed to be either expendable or
handled by a separate storage tier.
Environment variables:
- `VIGILAR_BACKUP_DIR` — destination directory (default
`/var/vigilar/backups`).
- `VIGILAR_BACKUP_RETENTION_DAYS` — age in days after which old
archives are pruned; set to `0` to keep forever (default `30`).
The archive is `chmod 0600 root:root` because it contains secrets.
### Scheduling
You can run it from cron, as the script comment suggests
(`0 3 * * * /opt/vigilar/scripts/backup.sh`), or via a dedicated
systemd timer. A minimal pair of units, kept in your local systemd
directory (not in the repo):
```
# /etc/systemd/system/vigilar-backup.service
[Unit]
Description=Vigilar nightly backup
After=vigilar.service
[Service]
Type=oneshot
Environment=VIGILAR_BACKUP_DIR=/srv/backups/vigilar
ExecStart=/opt/vigilar/scripts/backup.sh
```
```
# /etc/systemd/system/vigilar-backup.timer
[Unit]
Description=Run Vigilar backup nightly
[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
[Install]
WantedBy=timers.target
```
Enable with `sudo systemctl enable --now vigilar-backup.timer`.
### Restore
1. `sudo systemctl stop vigilar.service`.
2. Extract the archive to a staging directory.
3. Copy `etc/vigilar/` back into `/etc/vigilar/`, preserving
permissions. Double-check `/etc/vigilar/secrets/storage.key` is
`root:root 0600`.
4. Copy the database snapshot to `/var/vigilar/data/vigilar.db` and
remove any stale `vigilar.db-wal`/`vigilar.db-shm` files.
5. `sudo chown -R vigilar:vigilar /var/vigilar/data`.
6. `sudo -u vigilar /opt/vigilar/venv/bin/vigilar config validate
-c /etc/vigilar/vigilar.toml`.
7. `sudo systemctl start vigilar.service` and watch the journal.
## Upgrades
1. `sudo systemctl stop vigilar.service`.
2. `cd /path/to/vigilar && git pull`.
3. `sudo -u vigilar /opt/vigilar/venv/bin/pip install --upgrade .`
4. Diff the shipped `config/vigilar.toml` against `/etc/vigilar/
vigilar.toml` and merge any new keys by hand; Pydantic will reject
unknown keys but is tolerant of missing keys that have defaults.
5. `sudo -u vigilar /opt/vigilar/venv/bin/vigilar config validate
-c /etc/vigilar/vigilar.toml`.
6. `sudo systemctl start vigilar.service`.
**Schema migrations:** there is no migration framework. Vigilar does
not ship Alembic; `vigilar/storage/schema.py` defines the tables
(`cameras`, `sensors`, `sensor_states`, `events`, `recordings`,
`system_events`, `arm_state_log`, `alert_log`, `push_subscriptions`,
`pets`, `pet_sightings`, `wildlife_sightings`, `package_events`,
`pet_training_images`, `pet_rules`, `face_profiles`, `face_embeddings`,
`visits`, `timelapse_schedules`) and new columns are added by code
path at startup or not at all. Take a backup before every upgrade so
you can roll back if a column assumption changes.
## Logs and health
All subsystem output goes to the journal under the `vigilar` syslog
identifier:
```
sudo journalctl -u vigilar.service -f
sudo journalctl -u vigilar.service --since "1 hour ago"
sudo journalctl -u vigilar.service -p warning
```
The alerts subsystem additionally mirrors messages to syslog via the
`vigilar.alerts` logger when `[alerts.local] syslog = true`, which is
the default; the supervisor installs the handler at startup.
Set `[system] log_level = "DEBUG"` (or pass `--log-level DEBUG` to
`vigilar start`) to trace MQTT traffic, motion scoring, and FFmpeg
invocations. Expect a significant volume increase; revert to `INFO`
once you have the evidence you need.
The only HTTP endpoint currently exposing health is
`GET /system/status` on the web UI, which returns a JSON blob with
arm state, camera counts, and sensor counts. The richer health data
(disk percentage, MQTT reachability) is published to the
`vigilar/system/health` MQTT topic by `HealthMonitor` every ten
seconds and is not yet surfaced as a REST endpoint.
## Pruning and disk management
`vigilar/health/monitor.py` runs a disk check every five minutes
against `[system] data_dir` using `shutil.disk_usage`. When usage
crosses `[health] disk_critical_pct` and `[health] auto_prune` is
true, it calls `vigilar.health.pruner.auto_prune`:
- Selects up to 20 unstarred recordings at a time, ordered oldest
first.
- Deletes the file on disk, any thumbnail, and the row from the
`recordings` table.
- Loops until disk usage drops below `[health] auto_prune_target_pct`
or no more candidates exist.
Starred recordings (`recordings.starred = 1`) are never auto-pruned.
Per-camera `retention_days` is enforced separately by the camera
subsystem. There is no hard byte ceiling; the pruner is entirely
percentage-driven. The `[storage] max_disk_usage_gb` and
`[storage] free_space_floor_gb` keys described above are not
consulted by the pruner.
## Remote access
`[remote]` controls the lower-bitrate HLS profile that Vigilar serves
through a WireGuard tunnel. The tunnel itself is not set up by this
project — you are expected to bring your own WireGuard server and
peer configuration. Once the tunnel is up:
- `enabled = true` turns on the remote bridge.
- `tunnel_ip` is the home server's address inside the tunnel (default
`10.99.0.2`), shown in the UI for reference.
- `upload_bandwidth_mbps` caps the advertised upstream.
- `remote_hls_resolution`, `remote_hls_fps`, `remote_hls_bitrate_kbps`
define the transcode profile used when a client connects through
the tunnel instead of the LAN.
- `max_remote_viewers` bounds concurrent remote sessions; set to `0`
for unlimited.
Do not expose port `49735` directly on the WAN; require the tunnel.
## Known limitations
- **Recording integrity is not authenticated.** AES-256-CTR gives you
confidentiality, not tamper-evidence. If an attacker reaches the
recordings directory they can modify ciphertext unnoticed. See the
security section.
- **Camera supervision is asymmetric.** Most subsystems run under
`SubsystemProcess` in `vigilar/main.py`, which polls every two
seconds and applies an exponential backoff up to `max_restarts=10`.
Cameras do not: `CameraManager` in `vigilar/camera/manager.py`
owns its own per-camera child processes outside that supervisor.
A repeatedly crashing camera may thrash differently from, say, a
crashing UPS poller. Watch the journal for per-camera restart
messages independently from the top-level supervisor log.
- **Legacy storage keys.** `[storage] max_disk_usage_gb` and
`[storage] free_space_floor_gb` are editable but do nothing. Use
`[health]` for real disk policy.
- **No schema migrations.** There is no Alembic (or equivalent) in
the tree. Rollbacks rely on your backup discipline.
## Troubleshooting
**Supervisor crash loops.** `journalctl -u vigilar.service` will show
a subsystem crashing and the supervisor attempting to restart it. If
the same subsystem exceeds ten restarts, the supervisor gives up on
that subsystem and logs `exceeded max restarts, giving up`. Fix the
root cause (bad config, missing secret, missing model file for
detection) and restart the unit.
**Mosquitto will not start.** Confirm that
`/etc/mosquitto/conf.d/vigilar.conf` is present and that no other
listener is bound to `127.0.0.1:1883`. Run `sudo systemctl status
mosquitto.service` and `sudo journalctl -u mosquitto.service`. The
Vigilar unit `Requires=mosquitto.service`, so Vigilar will refuse to
start until mosquitto is healthy.
**Camera thrashing.** Because cameras are not under the main
supervisor's backoff, a camera whose RTSP URL is wrong or whose
remote end is rebooting can respawn quickly. Look for repeated
`camera <id>` messages in the journal. Disable the camera in the
config (`enabled = false`) while you fix the upstream, then
re-enable.
**Disk full.** Check `[health] disk_critical_pct` and confirm
`auto_prune` is on. If the partition is already past the target
percentage and nothing is being deleted, there are no unstarred
recordings left to prune — unstar something or lower retention. The
legacy `[storage]` keys will not help here; see the pruning section.
**HLS stalls.** The HLS directory lives at `[system] hls_dir`
(default `/var/vigilar/hls`) and is mounted `ReadWritePath` in the
systemd unit. Stalls usually mean FFmpeg has died on a camera;
check the journal for FFmpeg stderr and verify the RTSP URL is still
reachable from the server with `ffprobe`.
**Config validation fails.** Run `sudo -u vigilar
/opt/vigilar/venv/bin/vigilar config validate -c
/etc/vigilar/vigilar.toml`. Pydantic error messages include the
section, key, and reason. The two common traps are duplicate camera
or sensor IDs (root validator rejects them) and a TOML table that
should be an array of tables (`[cameras]` instead of `[[cameras]]`).
**Forgotten arm PIN.** Run `vigilar config set-pin` to mint a new
hash and paste it in; restart the service. If you also forgot the
recovery passphrase set up through the UI, the web
`/system/api/reset-pin` endpoint cannot help you — fall back to the
CLI.
**Forgotten web password.** Run `vigilar config set-password` and
paste the new hash into `[web] password_hash`, then restart. No
database state needs to change.

View File

@@ -0,0 +1,998 @@
# PIN Hashing Unification Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Unify the three incompatible PIN hashing schemes in Vigilar onto `alerts/pin.py`'s PBKDF2-SHA256 implementation, and make the web arm/disarm buttons actually transition system state. Closes Gitea issue #2.
**Architecture:** One canonical hash (`alerts.pin.hash_pin` / `verify_pin`, PBKDF2-SHA256 600k iterations, salted, format `pbkdf2_sha256$salt$dk`). One canonical config key (`[security] pin_hash`). `[system] arm_pin_hash` becomes a deprecated field with a migration warning. The web arm/disarm endpoints publish a new `vigilar/system/arm_request` MQTT topic; the event processor subscribes and hands requests to `ArmStateFSM.transition()`, which verifies the PIN via `alerts.pin.verify_pin` and performs the state transition. Live state is reported back to the web on the existing `vigilar/system/arm_state` topic (already published by the FSM) and read by the web via the DB for HTTP responses.
**Tech Stack:** Python 3.11+, Pydantic v2 (config), paho-mqtt (bus), SQLAlchemy Core (storage), pytest.
---
## Background (read before starting)
Three incompatible schemes exist today:
| Component | File:line | Hash scheme | Config key |
|---|---|---|---|
| Web arm/disarm/reset | `web/blueprints/system.py:63-64, 74-75, 88` via `alerts/pin.py` | PBKDF2-SHA256, 600k iter, salted | `[security] pin_hash` |
| `ArmStateFSM` | `events/state.py:50` | Unsalted SHA-256(pin) | `[system] arm_pin_hash` |
| CLI `vigilar config set-pin` | `cli/cmd_config.py:86-95` | HMAC-SHA256(random, pin) | `[system] arm_pin_hash` |
Consequences:
1. `vigilar config set-pin` produces output no verifier understands (HMAC-format into a key the FSM reads as raw SHA-256).
2. The web arm endpoint checks the PIN but never calls the FSM — it returns `{"ok": true}` without transitioning state. The real FSM lives in the events process and is only reachable via MQTT.
3. The FSM uses unsalted SHA-256, trivially brute-forceable for 4-digit PINs.
4. `events/state.py:46` calls `hmac.compare_digest` a "HMAC comparison" — misleading; it's timing-safe equality.
The winning scheme is `alerts/pin.py`'s PBKDF2-SHA256. `[security] pin_hash` is the winning config key (security-scoped section, already used by the web path).
## File Structure
**New:**
- None. All changes modify existing files.
**Modified:**
- `vigilar/events/state.py` — rewrite `verify_pin` to delegate to `alerts/pin.verify_pin`; change `_pin_hash` source to `config.security.pin_hash`; stop re-hashing the PIN on the audit-log row.
- `vigilar/cli/cmd_config.py` — rewrite `set_pin_cmd` to use `alerts.pin.hash_pin` and emit a `[security] pin_hash` line.
- `vigilar/config.py` — add migration warning in `load_config` when `[system] arm_pin_hash` is set but `[security] pin_hash` is empty. Field stays on `SystemConfig` for backward-compat parsing but is no longer read by any runtime code.
- `vigilar/constants.py` — add `Topics.SYSTEM_ARM_REQUEST = "vigilar/system/arm_request"`.
- `vigilar/events/processor.py` — add a dedicated subscription on `Topics.SYSTEM_ARM_REQUEST` that calls `fsm.transition()` with the verified mode, pin, and triggered_by.
- `vigilar/web/blueprints/system.py``arm_system`/`disarm_system` publish on `SYSTEM_ARM_REQUEST` (without the PIN pre-check; the FSM verifies) and return 202 (accepted) with the requested mode. A thin helper wraps bus publish.
- `tests/unit/test_events.py``_make_config` moves `pin_hash` onto `security`, not `system`. Existing FSM tests continue passing against PBKDF2 hashes.
- `tests/unit/test_system_pin.py` — web tests extend to verify an arm-request message is published on MQTT.
- `docs/operator-guide.md` — update the PIN section to describe the single canonical key, the CLI output format, and the migration note.
**New tests:**
- `tests/unit/test_pin_unification.py` — end-to-end: set a PIN via the CLI helper (`hash_pin`), verify the FSM accepts it, and verify the web arm endpoint publishes the right MQTT message.
---
## Task 1: Add `Topics.SYSTEM_ARM_REQUEST` constant
**Files:**
- Modify: `vigilar/constants.py` around line 212 (the `# System` block in `class Topics`)
- Test: covered by later tasks' tests referencing the constant
- [ ] **Step 1: Add the constant**
Edit `vigilar/constants.py`. After `SYSTEM_RULES_UPDATED = "vigilar/system/rules_updated"` and before `# Classified events forwarded to the web SSE bridge ...`, add:
```python
# Web-to-FSM arm/disarm request (FSM verifies the PIN and transitions)
SYSTEM_ARM_REQUEST = "vigilar/system/arm_request"
```
- [ ] **Step 2: Verify the constant is importable**
Run: `.venv/bin/python -c "from vigilar.constants import Topics; print(Topics.SYSTEM_ARM_REQUEST)"`
Expected: `vigilar/system/arm_request`
- [ ] **Step 3: Commit**
```bash
git add vigilar/constants.py
git commit -m "feat(constants): add Topics.SYSTEM_ARM_REQUEST
Topic for web-originated arm/disarm requests that the event processor
will subscribe to and dispatch to ArmStateFSM.transition. Part of the
PIN unification work (issue #2)."
```
---
## Task 2: Rewrite `ArmStateFSM.verify_pin` to use `alerts.pin.verify_pin` (TDD)
**Files:**
- Modify: `vigilar/events/state.py` (lines 1-52 especially)
- Test: `tests/unit/test_events.py::TestArmStateFSM`
- Helper update: `tests/unit/test_events.py::_make_config` and `_pin_hash`
The FSM currently reads `config.system.arm_pin_hash` and verifies with unsalted SHA-256. We replace both: read `config.security.pin_hash`, verify via `alerts.pin.verify_pin`.
- [ ] **Step 1: Write the failing test (PBKDF2 verify path)**
Edit `tests/unit/test_events.py`. Replace the existing `_make_config` and `_pin_hash` helpers (around lines 20-30) with:
```python
def _make_config(rules=None, pin_hash=""):
return VigilarConfig(
security={"pin_hash": pin_hash},
cameras=[],
sensors=[],
rules=rules or [],
)
def _pin_hash(pin: str) -> str:
from vigilar.alerts.pin import hash_pin
return hash_pin(pin)
```
Also remove the `import hashlib` line at the top of the file if it is no longer used anywhere else in the file (grep first — `grep -n hashlib tests/unit/test_events.py` — leave it if any other test still calls `hashlib.*`).
- [ ] **Step 2: Run FSM tests to verify they now fail**
Run: `.venv/bin/pytest -q tests/unit/test_events.py::TestArmStateFSM`
Expected: Multiple failures in `test_transition_with_valid_pin`, `test_transition_with_invalid_pin`, `test_verify_pin` — failing because `ArmStateFSM` still reads `config.system.arm_pin_hash` and computes unsalted SHA-256, so it cannot validate a PBKDF2 hash.
- [ ] **Step 3: Rewrite `ArmStateFSM.verify_pin` and the constructor**
Edit `vigilar/events/state.py`. Replace the existing imports and the top of the class. Target state of lines 1-52:
```python
"""Arm state finite state machine."""
import logging
import time
from sqlalchemy.engine import Engine
from vigilar.alerts.pin import verify_pin as _verify_pin_hash
from vigilar.config import VigilarConfig
from vigilar.constants import ArmState, EventType, Severity, Topics
from vigilar.storage.queries import get_current_arm_state, insert_arm_state, insert_event
log = logging.getLogger(__name__)
class ArmStateFSM:
"""Manages DISARMED / ARMED_HOME / ARMED_AWAY state transitions."""
def __init__(self, engine: Engine, config: VigilarConfig):
self._engine = engine
self._pin_hash = config.security.pin_hash
self._state = ArmState.DISARMED
self._bus = None
self._load_initial_state()
def _load_initial_state(self) -> None:
"""Load the most recent arm state from the database."""
stored = get_current_arm_state(self._engine)
if stored and stored in ArmState.__members__:
self._state = ArmState(stored)
log.info("Loaded arm state from DB: %s", self._state)
else:
self._state = ArmState.DISARMED
log.info("No stored arm state, defaulting to DISARMED")
def set_bus(self, bus: object) -> None:
"""Attach a MessageBus for publishing state changes."""
self._bus = bus
@property
def state(self) -> ArmState:
return self._state
def verify_pin(self, pin: str) -> bool:
"""Verify a PIN against the stored PBKDF2 hash."""
if not self._pin_hash:
# No PIN configured — allow all transitions
return True
return _verify_pin_hash(pin, self._pin_hash)
```
Also update `transition()` — the audit-log call at line 72 currently re-hashes the pin with unsalted SHA-256 and passes it as `pin_hash`. Replace that whole line with `None` since a fresh PBKDF2 salt would produce a different hash each time, making the audit column useless for traceability:
```python
# Log to database (pin_hash column is no longer populated — see #2)
insert_arm_state(self._engine, new_state.value, triggered_by, None)
```
Also remove the now-unused `import hashlib` and `import hmac` at the top of the file.
- [ ] **Step 4: Run FSM tests to verify they pass**
Run: `.venv/bin/pytest -q tests/unit/test_events.py::TestArmStateFSM`
Expected: all 7 tests pass.
- [ ] **Step 5: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: 364 passing (same as baseline). If any other test fails because it constructed `VigilarConfig(system={"arm_pin_hash": ...})`, update it to `security={"pin_hash": ...}` — see Task 2a.
- [ ] **Step 6: Commit**
```bash
git add vigilar/events/state.py tests/unit/test_events.py
git commit -m "fix(events): ArmStateFSM uses PBKDF2 via alerts.pin (issue #2)
Was: unsalted SHA-256 read from [system] arm_pin_hash.
Now: PBKDF2-SHA256 600k iterations read from [security] pin_hash,
matching the web arm/disarm path and the alerts/pin module.
Also drops the redundant pin re-hash on the arm_state_log audit row
(a fresh PBKDF2 salt made the column valueless for traceability).
Part of issue #2 PIN hashing unification."
```
### Task 2a: Fix any other tests referencing the old config key
- [ ] **Step 1: Find stragglers**
Run: `grep -rn "arm_pin_hash" tests/`
- [ ] **Step 2: Update each matched test**
For every match, replace the pydantic construction pattern:
```python
VigilarConfig(system={"arm_pin_hash": ...})
```
with:
```python
VigilarConfig(security={"pin_hash": ...})
```
And if the value was a plain SHA-256 hex string, replace it with `hash_pin(pin)` from `vigilar.alerts.pin`.
- [ ] **Step 3: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: 364 passing.
- [ ] **Step 4: Commit if changes were made**
```bash
git add tests/
git commit -m "test: migrate remaining PIN hash test fixtures to security.pin_hash
Sweep after ArmStateFSM rewrite — any straggling test that still
passed arm_pin_hash via SystemConfig now uses SecurityConfig.pin_hash."
```
---
## Task 3: Rewrite `vigilar config set-pin` to output PBKDF2 under `[security] pin_hash` (TDD)
**Files:**
- Modify: `vigilar/cli/cmd_config.py` (the `set_pin_cmd` function around lines 83-95)
- Modify: `vigilar/cli/cmd_config.py` (the `show_cmd` redaction around lines 50-51, so it redacts `security.pin_hash` instead of `system.arm_pin_hash`)
- Test: `tests/unit/test_cli_set_pin.py` (new file)
The CLI currently computes an HMAC with a random secret and tells the user to paste it into `[system] arm_pin_hash`. Both the format and the destination are wrong.
- [ ] **Step 1: Create the failing test**
Create `tests/unit/test_cli_set_pin.py`:
```python
"""Tests for `vigilar config set-pin`."""
from click.testing import CliRunner
from vigilar.alerts.pin import verify_pin
from vigilar.cli.cmd_config import config_cmd
def test_set_pin_outputs_pbkdf2_hash_under_security_section():
"""The CLI must emit a hash that alerts.pin.verify_pin can validate,
and direct the user to [security] pin_hash (not [system] arm_pin_hash)."""
runner = CliRunner()
result = runner.invoke(config_cmd, ["set-pin"], input="1234\n1234\n")
assert result.exit_code == 0, result.output
# The output must direct the user to the [security] section
assert "[security]" in result.output
assert "arm_pin_hash" not in result.output
assert "pin_hash" in result.output
# Extract the emitted hash (line starts with `pin_hash = "..."`)
hash_line = next(
line for line in result.output.splitlines() if line.strip().startswith("pin_hash")
)
hash_value = hash_line.split('"')[1]
# Round-trip: the emitted hash must accept the plaintext PIN
assert verify_pin("1234", hash_value) is True
assert verify_pin("0000", hash_value) is False
# And it must be in PBKDF2 format (not the legacy HMAC "secret:mac" format)
assert hash_value.startswith("pbkdf2_sha256$")
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `.venv/bin/pytest -q tests/unit/test_cli_set_pin.py`
Expected: FAIL — either `"[security]"` not in output (the CLI still prints `[system]`), or `hash_value` starts with raw hex (HMAC format) not `pbkdf2_sha256$`.
- [ ] **Step 3: Rewrite `set_pin_cmd`**
Edit `vigilar/cli/cmd_config.py`. Replace the entire `set_pin_cmd` function (currently lines 83-95) with:
```python
@config_cmd.command("set-pin")
def set_pin_cmd() -> None:
"""Generate a PBKDF2 hash for the arm/disarm PIN."""
from vigilar.alerts.pin import hash_pin
pin = click.prompt("Enter arm/disarm PIN", hide_input=True, confirmation_prompt=True)
hash_str = hash_pin(pin)
click.echo("\nAdd this to your vigilar.toml [security] section:")
click.echo(f'pin_hash = "{hash_str}"')
```
Remove any now-unused imports at the top of the function (the old `hashlib`, `hmac`, `os` locals).
- [ ] **Step 4: Update the show_cmd redaction**
Still in `vigilar/cli/cmd_config.py`, in `show_cmd` (around lines 47-54), replace:
```python
if data.get("system", {}).get("arm_pin_hash"):
data["system"]["arm_pin_hash"] = "***"
```
with:
```python
if data.get("security", {}).get("pin_hash"):
data["security"]["pin_hash"] = "***"
```
- [ ] **Step 5: Run the CLI test to verify it passes**
Run: `.venv/bin/pytest -q tests/unit/test_cli_set_pin.py`
Expected: PASS.
- [ ] **Step 6: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: 365 passing (+1 from the new test).
- [ ] **Step 7: Commit**
```bash
git add vigilar/cli/cmd_config.py tests/unit/test_cli_set_pin.py
git commit -m "fix(cli): set-pin emits PBKDF2 under [security] pin_hash (issue #2)
Was: HMAC-SHA256(random, pin) written to [system] arm_pin_hash —
no verifier in the codebase accepted this output.
Now: PBKDF2-SHA256 via alerts.pin.hash_pin written to [security]
pin_hash, matching what the web and FSM paths verify against.
Also fixes show_cmd to redact the new location."
```
---
## Task 4: Deprecate `[system] arm_pin_hash` with a migration warning (TDD)
**Files:**
- Modify: `vigilar/config.py` — keep the field on `SystemConfig` but emit a warning in `load_config` when the old key is set and the new one isn't
- Test: `tests/unit/test_config.py` (new test)
We do NOT remove the field — existing deployments may have it set, and deleting it from the schema would cause `load_config` to raise a validation error on old configs. Instead: parse it, log a loud warning, ignore it at runtime.
- [ ] **Step 1: Write the failing test**
Edit `tests/unit/test_config.py`. Add a new test at the end:
```python
def test_deprecation_warning_for_arm_pin_hash(tmp_path, caplog):
"""Loading a config that still uses the legacy [system] arm_pin_hash
must log a clear warning pointing the user at `vigilar config set-pin`."""
import logging
cfg_path = tmp_path / "legacy.toml"
cfg_path.write_text(
'[system]\n'
'arm_pin_hash = "pbkdf2_sha256$abc$def"\n'
)
with caplog.at_level(logging.WARNING):
from vigilar.config import load_config
load_config(str(cfg_path))
messages = [r.message for r in caplog.records if r.levelno >= logging.WARNING]
assert any("arm_pin_hash" in m and "deprecated" in m.lower() for m in messages), (
f"expected deprecation warning mentioning arm_pin_hash, got: {messages}"
)
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `.venv/bin/pytest -q tests/unit/test_config.py::test_deprecation_warning_for_arm_pin_hash`
Expected: FAIL — no warning is currently emitted.
- [ ] **Step 3: Add the warning in `load_config`**
Open `vigilar/config.py`. Find `load_config` (starts near line 410). Immediately after the Pydantic parse step and before returning the config object, add:
```python
if cfg.system.arm_pin_hash and not cfg.security.pin_hash:
log.warning(
"DEPRECATED: [system] arm_pin_hash is ignored; the arm/disarm "
"PIN lives under [security] pin_hash. Run `vigilar config "
"set-pin` and paste the output into [security]."
)
```
Ensure `log = logging.getLogger(__name__)` exists at module top (add `import logging` and the log declaration if missing).
- [ ] **Step 4: Run the deprecation test to verify it passes**
Run: `.venv/bin/pytest -q tests/unit/test_config.py::test_deprecation_warning_for_arm_pin_hash`
Expected: PASS.
- [ ] **Step 5: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: 366 passing (+1).
- [ ] **Step 6: Commit**
```bash
git add vigilar/config.py tests/unit/test_config.py
git commit -m "feat(config): deprecation warning for [system] arm_pin_hash
If a config still has the legacy [system] arm_pin_hash set but no
[security] pin_hash, load_config logs a WARNING telling the operator
to re-run 'vigilar config set-pin'. The legacy field is still parsed
(so old configs don't fail validation) but ignored at runtime.
Part of issue #2 PIN hashing unification."
```
---
## Task 5: Processor subscribes to `SYSTEM_ARM_REQUEST` and dispatches to FSM (TDD)
**Files:**
- Modify: `vigilar/events/processor.py` (the `run` method around lines 56-65, and add a new private method `_handle_arm_request`)
- Test: `tests/unit/test_events.py::TestArmRequestDispatch` (new test class)
Today the FSM only transitions when someone inside the event-processor process calls `fsm.transition()` directly. There is no MQTT entry point. This task adds one: a dedicated subscription on `Topics.SYSTEM_ARM_REQUEST` that dispatches to a new `_handle_arm_request` method.
- [ ] **Step 1: Write the failing test**
Append to `tests/unit/test_events.py`:
```python
class TestArmRequestDispatch:
"""SYSTEM_ARM_REQUEST messages must reach ArmStateFSM.transition."""
def test_arm_request_calls_fsm_transition(self, test_db):
from vigilar.events.processor import EventProcessor
processor = EventProcessor.__new__(EventProcessor)
calls = []
class FakeFSM:
state = ArmState.DISARMED
def transition(self, new_state, pin="", triggered_by="system"):
calls.append((new_state, pin, triggered_by))
return True
processor._handle_arm_request(
payload={"mode": "ARMED_AWAY", "pin": "1234", "triggered_by": "web"},
fsm=FakeFSM(),
)
assert len(calls) == 1
new_state, pin, triggered_by = calls[0]
assert new_state == ArmState.ARMED_AWAY
assert pin == "1234"
assert triggered_by == "web"
def test_arm_request_ignores_bad_mode(self, test_db):
from vigilar.events.processor import EventProcessor
processor = EventProcessor.__new__(EventProcessor)
calls = []
class FakeFSM:
def transition(self, *a, **kw):
calls.append((a, kw))
return True
processor._handle_arm_request(
payload={"mode": "NONSENSE", "pin": "1234"},
fsm=FakeFSM(),
)
assert calls == []
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `.venv/bin/pytest -q tests/unit/test_events.py::TestArmRequestDispatch`
Expected: FAIL — `AttributeError: 'EventProcessor' object has no attribute '_handle_arm_request'`.
- [ ] **Step 3: Implement `_handle_arm_request`**
Edit `vigilar/events/processor.py`. After the existing `_handle_event` method (before `_classify_event`), add:
```python
def _handle_arm_request(
self,
payload: dict[str, Any],
fsm: "ArmStateFSM",
) -> None:
"""Handle an arm/disarm request received over MQTT.
Payload fields:
- mode: str — desired ArmState ("DISARMED", "ARMED_HOME", "ARMED_AWAY")
- pin: str — plaintext PIN (FSM verifies against security.pin_hash)
- triggered_by: str — origin tag for the audit log (e.g. "web")
"""
mode = payload.get("mode", "")
if mode not in ArmState.__members__:
log.warning("Ignoring arm request with invalid mode: %r", mode)
return
pin = payload.get("pin", "")
triggered_by = payload.get("triggered_by", "unknown")
fsm.transition(ArmState(mode), pin=pin, triggered_by=triggered_by)
```
Also add `ArmState` to the imports at the top of the file — change line 12 from:
```python
from vigilar.constants import EventType, Severity, Topics
```
to:
```python
from vigilar.constants import ArmState, EventType, Severity, Topics
```
- [ ] **Step 4: Run the test to verify it passes**
Run: `.venv/bin/pytest -q tests/unit/test_events.py::TestArmRequestDispatch`
Expected: both tests pass.
- [ ] **Step 5: Wire the subscription in `run`**
Still in `vigilar/events/processor.py`, inside `run()` (around the existing `bus.subscribe_all(on_message)` call on line 65), add a dedicated subscription for arm requests. Replace the block:
```python
# Subscribe to all Vigilar topics
def on_message(topic: str, payload: dict[str, Any]) -> None:
self._handle_event(topic, payload, engine, fsm, rule_engine, bus)
bus.subscribe_all(on_message)
```
with:
```python
# Subscribe to all Vigilar topics (events/motion/sensors/etc.)
def on_message(topic: str, payload: dict[str, Any]) -> None:
self._handle_event(topic, payload, engine, fsm, rule_engine, bus)
bus.subscribe_all(on_message)
# Dedicated subscription for web-originated arm/disarm requests.
# Kept separate from on_message because these are commands, not
# classifiable events.
def on_arm_request(topic: str, payload: dict[str, Any]) -> None:
self._handle_arm_request(payload, fsm)
bus.subscribe(Topics.SYSTEM_ARM_REQUEST, on_arm_request)
```
- [ ] **Step 6: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: 368 passing (+2 from the new test class).
- [ ] **Step 7: Commit**
```bash
git add vigilar/events/processor.py tests/unit/test_events.py
git commit -m "feat(events): processor handles SYSTEM_ARM_REQUEST over MQTT
Adds _handle_arm_request and a dedicated bus.subscribe on
Topics.SYSTEM_ARM_REQUEST. Payload {mode, pin, triggered_by} is
dispatched to ArmStateFSM.transition, which verifies the PIN via
alerts.pin.verify_pin and performs the state change.
This is the missing link for web /system/api/arm to actually move
the system into an armed state. Part of issue #2."
```
---
## Task 6: Web arm/disarm endpoints publish arm requests (TDD)
**Files:**
- Modify: `vigilar/web/blueprints/system.py``arm_system` and `disarm_system` publish `SYSTEM_ARM_REQUEST` via a bus client; remove the PIN pre-check (FSM verifies)
- Test: `tests/unit/test_system_pin.py` — verify the publish happens
The web endpoint currently has two bugs: (a) it verifies the PIN but never transitions state, and (b) if the PIN is wrong it returns 401 with no audit trail. We fix both by removing the local PIN check and forwarding the request to the FSM over MQTT. The FSM verifies and logs. The web response becomes 202 Accepted regardless of PIN — the client polls `/system/status` to see the actual state (if no transition occurred, the FSM logged a warning and the state is unchanged).
**Design note:** we rely on the local Mosquitto broker being trusted (127.0.0.1 only, no exposure). The PIN travels over localhost MQTT. This matches the existing convention.
- [ ] **Step 1: Extend the existing test fixtures**
Edit `tests/unit/test_system_pin.py`. At the top of the file add:
```python
from unittest.mock import MagicMock, patch
```
- [ ] **Step 2: Write the failing test**
Append to `tests/unit/test_system_pin.py`:
```python
def test_arm_publishes_arm_request_on_mqtt(app_with_pin):
"""POST /system/api/arm must publish a SYSTEM_ARM_REQUEST message
carrying the mode, pin, and a 'web' triggered_by tag."""
from vigilar.constants import Topics
fake_bus = MagicMock()
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c:
rv = c.post(
"/system/api/arm",
json={"mode": "ARMED_AWAY", "pin": "1234"},
)
assert rv.status_code == 202
assert rv.get_json()["accepted"] is True
pub.assert_called_once()
call_args = pub.call_args
# call_args.args[0] is the cfg, args[1] is the payload
payload = call_args.args[1] if len(call_args.args) > 1 else call_args.kwargs["payload"]
assert payload["mode"] == "ARMED_AWAY"
assert payload["pin"] == "1234"
assert payload["triggered_by"] == "web"
def test_disarm_publishes_arm_request(app_with_pin):
from vigilar.constants import Topics
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c:
rv = c.post("/system/api/disarm", json={"pin": "1234"})
assert rv.status_code == 202
pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["mode"] == "DISARMED"
```
Also **update** the existing tests that previously asserted the 200/401 PIN-check behavior to match the new 202 semantics. Specifically replace the bodies of `test_arm_without_pin_set`, `test_arm_correct_pin`, `test_arm_wrong_pin`, `test_disarm_correct_pin`, `test_disarm_wrong_pin` so they mock `_publish_arm_request` and assert only that the endpoint returns 202 and forwards the payload verbatim. The "wrong pin" case no longer returns 401 at the HTTP layer — the FSM logs and ignores. Example replacement for `test_arm_wrong_pin`:
```python
def test_arm_wrong_pin_still_accepted_by_web_fsm_rejects(app_with_pin):
"""HTTP layer no longer pre-checks the PIN — it forwards to the FSM
unconditionally. The FSM verifies and, on mismatch, logs a warning
and leaves the state unchanged."""
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c:
rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY", "pin": "0000"})
assert rv.status_code == 202
pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["pin"] == "0000" # forwarded verbatim — FSM will reject
```
- [ ] **Step 3: Run the new test to verify it fails**
Run: `.venv/bin/pytest -q tests/unit/test_system_pin.py::test_arm_publishes_arm_request_on_mqtt`
Expected: FAIL — `AttributeError` or `ImportError`, `_publish_arm_request` does not exist.
- [ ] **Step 4: Rewrite `arm_system`, `disarm_system`, add `_publish_arm_request` helper**
Edit `vigilar/web/blueprints/system.py`. Replace the `arm_system` and `disarm_system` handlers (the block around lines 57-77) with:
```python
def _publish_arm_request(cfg: VigilarConfig, payload: dict) -> None:
"""Publish an arm/disarm request on MQTT for the event processor to pick up."""
from vigilar.bus import MessageBus
from vigilar.constants import Topics
bus = MessageBus(cfg.mqtt, client_id="vigilar-web-arm-request")
bus.connect()
try:
bus.publish(Topics.SYSTEM_ARM_REQUEST, payload)
finally:
bus.disconnect()
@system_bp.route("/api/arm", methods=["POST"])
def arm_system():
data = request.get_json() or {}
mode = data.get("mode", "ARMED_AWAY")
pin = data.get("pin", "")
payload = {"mode": mode, "pin": pin, "triggered_by": "web"}
try:
_publish_arm_request(_get_cfg(), payload)
except Exception:
current_app.logger.exception("Failed to publish arm request")
return jsonify({"error": "bus unavailable"}), 503
return jsonify({"accepted": True, "mode": mode}), 202
@system_bp.route("/api/disarm", methods=["POST"])
def disarm_system():
data = request.get_json() or {}
pin = data.get("pin", "")
payload = {"mode": "DISARMED", "pin": pin, "triggered_by": "web"}
try:
_publish_arm_request(_get_cfg(), payload)
except Exception:
current_app.logger.exception("Failed to publish arm request")
return jsonify({"error": "bus unavailable"}), 503
return jsonify({"accepted": True, "mode": "DISARMED"}), 202
```
The `from vigilar.alerts.pin import hash_pin, verify_pin` import at the top of the file is still needed for `reset_pin`; leave it alone.
- [ ] **Step 5: Run the web PIN tests to verify they pass**
Run: `.venv/bin/pytest -q tests/unit/test_system_pin.py`
Expected: all tests in that file pass.
- [ ] **Step 6: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: all tests passing (count depends on how many existing tests you rewrote in step 2; the delta should be +2 new publish tests, same or +3 on net).
- [ ] **Step 7: Commit**
```bash
git add vigilar/web/blueprints/system.py tests/unit/test_system_pin.py
git commit -m "fix(web): arm/disarm actually transition the FSM via MQTT (issue #2)
Was: /system/api/arm verified the PIN against [security] pin_hash and
returned {ok: true} without ever calling the FSM. State never changed.
Now: the endpoint publishes a SYSTEM_ARM_REQUEST message to the local
MQTT broker. The event processor (see previous commit) picks it up,
ArmStateFSM verifies the PIN via alerts.pin.verify_pin and performs
the transition. Response is 202 Accepted; clients poll /system/status
for the new state.
Design: PIN travels over localhost-only MQTT, which matches the
existing trust boundary for the internal bus."
```
---
## Task 7: End-to-end unification test
**Files:**
- Test: `tests/unit/test_pin_unification.py` (new file)
A single test that walks the full flow using only the public API of each layer, proving that the three formerly-incompatible paths now interoperate.
- [ ] **Step 1: Write the test**
Create `tests/unit/test_pin_unification.py`:
```python
"""End-to-end test: the CLI, FSM, and web arm flow all accept the same PIN.
Regression guard for issue #2 — the three layers previously used three
incompatible hash schemes under two different config keys."""
from click.testing import CliRunner
from vigilar.alerts.pin import hash_pin, verify_pin
from vigilar.cli.cmd_config import config_cmd
from vigilar.config import SecurityConfig, VigilarConfig
from vigilar.events.state import ArmStateFSM
from vigilar.constants import ArmState
def test_cli_output_is_accepted_by_fsm(test_db):
"""Hash produced by `vigilar config set-pin` must verify against
ArmStateFSM.verify_pin, same config key, same format."""
runner = CliRunner()
result = runner.invoke(config_cmd, ["set-pin"], input="9876\n9876\n")
assert result.exit_code == 0, result.output
hash_line = next(
line for line in result.output.splitlines()
if line.strip().startswith("pin_hash")
)
hash_value = hash_line.split('"')[1]
cfg = VigilarConfig(security=SecurityConfig(pin_hash=hash_value))
fsm = ArmStateFSM(test_db, cfg)
assert fsm.verify_pin("9876") is True
assert fsm.verify_pin("0000") is False
def test_fsm_transitions_with_pin_from_alerts_module(test_db):
"""The alerts.pin module and ArmStateFSM agree on the hash format."""
stored = hash_pin("4242")
cfg = VigilarConfig(security=SecurityConfig(pin_hash=stored))
fsm = ArmStateFSM(test_db, cfg)
assert fsm.transition(ArmState.ARMED_AWAY, pin="4242", triggered_by="test") is True
assert fsm.state == ArmState.ARMED_AWAY
# Same stored hash rejects the wrong PIN
assert verify_pin("0000", stored) is False
```
- [ ] **Step 2: Run it**
Run: `.venv/bin/pytest -q tests/unit/test_pin_unification.py`
Expected: both tests pass.
- [ ] **Step 3: Run the full suite**
Run: `.venv/bin/pytest -q`
Expected: all tests passing.
- [ ] **Step 4: Commit**
```bash
git add tests/unit/test_pin_unification.py
git commit -m "test: end-to-end PIN unification regression guard (issue #2)"
```
---
## Task 8: Update the operator guide
**Files:**
- Modify: `docs/operator-guide.md` — replace stale sections about `arm_pin_hash` and the three-way mismatch note
There are four stale passages to rewrite. Each `- [ ]` step below shows the exact old text and the replacement. Line numbers are approximate; match on the text.
- [ ] **Step 1: Rewrite the `vigilar config set-pin` description**
Find this paragraph (around line 368):
```
Prompts for an arm/disarm PIN, generates a random 32-byte HMAC key,
computes `HMAC-SHA256(key, pin)`, and prints an `arm_pin_hash =
"secret_hex:mac_hex"` line to paste into `[system]`. Again, no file
write.
```
Replace with:
```
Prompts for an arm/disarm PIN, derives a salted PBKDF2-SHA256 hash
(600,000 iterations) via `vigilar.alerts.pin.hash_pin`, and prints a
`pin_hash = "pbkdf2_sha256$salt$dk"` line to paste into `[security]`.
Again, no file write. The same hash format is verified identically by
the web arm/disarm endpoint and by `ArmStateFSM` in the event
processor — there is one canonical PIN store.
```
- [ ] **Step 2: Fix the security-section summary of PIN storage**
Find this bullet (around line 390):
```
- The web UI password is a scrypt hash set by `vigilar config
set-password` and stored at `[web] password_hash`. The arm PIN is
an HMAC stored at `[system] arm_pin_hash` (and/or `[security]
pin_hash`).
```
Replace with:
```
- The web UI password is a scrypt hash set by `vigilar config
set-password` and stored at `[web] password_hash`. The arm/disarm
PIN is a PBKDF2-SHA256 hash (600k iterations, salted) set by
`vigilar config set-pin` and stored at `[security] pin_hash`.
A legacy `[system] arm_pin_hash` field is still parsed but ignored
at runtime; if it's set and `[security] pin_hash` is empty, the
service logs a deprecation warning at startup and arm/disarm will
behave as if no PIN were configured until you re-run `set-pin`.
```
- [ ] **Step 3: Remove the "Duplicate PIN fields" known limitation**
Find this bullet in the `## Known limitations` section (around line 605):
```
- **Duplicate PIN fields.** `vigilar config set-pin` writes to
`[system] arm_pin_hash`, while the web arm/disarm flow reads from
`[security] pin_hash`. Both models exist. If you set one and the
other side does not behave as expected, mirror the value manually.
```
Delete it entirely (including the blank line it shares with the surrounding bullets — re-check the section to ensure no double-blank is left behind).
- [ ] **Step 4: Grep for any remaining `arm_pin_hash` references in operator-guide.md**
Run: `grep -n "arm_pin_hash" docs/operator-guide.md`
For any remaining hits that describe the field as the primary PIN store (e.g. in the `## Configuration reference` section around line 116, 294-296, 347), rewrite them to describe `[security] pin_hash` as the primary key with a one-line note that `[system] arm_pin_hash` is deprecated and ignored. Leave any reference that's explicitly about the legacy migration path intact.
- [ ] **Step 5: Commit**
```bash
git add docs/operator-guide.md
git commit -m "docs(operator-guide): PIN hashing is unified (issue #2)
Describes the canonical [security] pin_hash key, the PBKDF2 format
emitted by 'vigilar config set-pin', and the deprecation warning for
the legacy [system] arm_pin_hash. Drops the three-way mismatch
known-limitation."
```
---
## Task 9: Open the PR
**Files:** none
- [ ] **Step 1: Push the branch**
```bash
git push -u origin fix/issue-2-pin-unification
```
- [ ] **Step 2: Open the PR via tea**
```bash
tea pulls create --login alee --repo alee/vigilar \
--head fix/issue-2-pin-unification --base main \
--title "fix: unify PIN hashing across CLI, FSM, and web (closes #2)" \
--description "$(cat <<'EOF'
Closes #2.
## Problem (recap from the issue)
Three incompatible PIN hashing schemes across two config keys:
| Component | Hash scheme | Config key |
|---|---|---|
| Web arm/disarm/reset (via alerts/pin.py) | PBKDF2-SHA256, 600k iterations, salted | [security] pin_hash |
| ArmStateFSM (events/state.py) | Unsalted SHA-256(pin) | [system] arm_pin_hash |
| CLI set-pin (cli/cmd_config.py) | HMAC-SHA256(random, pin) | [system] arm_pin_hash |
Consequences: set-pin output no verifier understood; web arm endpoint verified the PIN but never transitioned state; FSM used trivially brute-forceable unsalted SHA-256.
## Resolution
- **One canonical scheme:** PBKDF2-SHA256 via alerts/pin (hash_pin/verify_pin).
- **One canonical key:** [security] pin_hash.
- **Legacy field deprecated:** [system] arm_pin_hash is still parsed (old configs don't fail) but ignored at runtime. A WARNING logs at startup if it's set and the new key isn't.
- **Web arm/disarm now actually works:** POST /system/api/arm publishes a SYSTEM_ARM_REQUEST on localhost MQTT; the event processor subscribes, ArmStateFSM verifies the PIN and transitions. HTTP response is 202 Accepted; clients poll /system/status for the new state.
## Commits
- feat(constants): add Topics.SYSTEM_ARM_REQUEST
- fix(events): ArmStateFSM uses PBKDF2 via alerts.pin
- fix(cli): set-pin emits PBKDF2 under [security] pin_hash
- feat(config): deprecation warning for [system] arm_pin_hash
- feat(events): processor handles SYSTEM_ARM_REQUEST over MQTT
- fix(web): arm/disarm actually transition the FSM via MQTT
- test: end-to-end PIN unification regression guard
- docs(operator-guide): PIN hashing is unified
## Test plan
- [x] ArmStateFSM PBKDF2 verify path (tests/unit/test_events.py::TestArmStateFSM)
- [x] CLI set-pin emits PBKDF2 under [security] (tests/unit/test_cli_set_pin.py)
- [x] Deprecation warning fires for legacy key (tests/unit/test_config.py)
- [x] Processor dispatches SYSTEM_ARM_REQUEST to FSM (tests/unit/test_events.py::TestArmRequestDispatch)
- [x] Web arm/disarm publish SYSTEM_ARM_REQUEST, return 202 (tests/unit/test_system_pin.py)
- [x] End-to-end: CLI → FSM round-trip (tests/unit/test_pin_unification.py)
- [x] Full suite green; config/vigilar.toml unmodified after pytest (session fixture from #3)
## Migration note for operators
If you previously ran `vigilar config set-pin`, you will see a deprecation warning on the next start. Re-run the command and paste the new `pin_hash = "pbkdf2_sha256$..."` line into `[security]` (not `[system]`). The old `[system] arm_pin_hash` can then be deleted.
EOF
)"
```
- [ ] **Step 3: Verify no stray diffs**
Run: `git status && git diff origin/main -- config/vigilar.toml`
Expected: clean. `config/vigilar.toml` must not be modified.
---
## Success Criteria
- `vigilar config set-pin` output can be pasted into `[security] pin_hash` and verified by both `alerts.pin.verify_pin` and `ArmStateFSM.verify_pin`.
- A `POST /system/api/arm` with a correct PIN actually transitions the FSM (verifiable via `/system/status` or the `arm_state_log` table) — a `POST` with a wrong PIN leaves state unchanged and emits a WARNING log on the event processor side.
- Starting with a config that has only the legacy `[system] arm_pin_hash` set logs a deprecation WARNING.
- Full test suite passes; `config/vigilar.toml` stays clean (session fixture from #3 protects this).
- No file references `arm_pin_hash` outside of (a) the `SystemConfig` pydantic field definition, (b) the deprecation warning, and (c) historical spec docs under `docs/superpowers/specs/`.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,202 @@
# Vigilar Project Documentation — Design Spec
**Date:** 2026-04-05
**Status:** Approved, ready for implementation plan
**Scope:** Create user-facing and architectural documentation for Vigilar, plus a polished top-level `README.md`.
---
## 1. Goal
Vigilar currently has no top-level `README.md`, no user guide, and no architectural reference. Contributors and would-be home users have to read source code to understand what the project is or how to run it. This effort closes that gap with a single coordinated documentation pass.
The docs must:
- Give a homeowner with a mini PC a clear linear path from bare hardware to working cameras on their phone.
- Give a self-hoster a reference for config, CLI, secrets, backups, upgrades, and troubleshooting.
- Give a contributor enough architectural context to navigate the codebase without reading every file.
- Match the project's ethos: plain, no bloat, no cloud, no extra tooling.
## 2. Non-goals
- No doc site / MkDocs build (files are organized *as if* MkDocs-ready, but no tooling is added).
- No rewrites of existing code or config.
- No changes to `docs/camera-hardware-guide.md` (already exists, untouched).
- No changes to anything under `docs/superpowers/` other than this spec.
- No CI doc-linting, no link-checker automation beyond a one-time manual verification pass.
- No recording-backup-to-NAS feature work. Docs describe only what exists today; future backup improvements are noted as planned, not documented as present.
## 3. Audience & doc layout (Approach 3 — Hybrid)
Three organizing principles, one tree:
- **User-facing guides** are monolithic linear narratives (humans read top to bottom once).
- **Architecture docs** are split reference material (contributors jump to the subsystem they're touching).
- **README** is the front door that ties everything together.
Final tree:
```
README.md # NEW — top-level front door
docs/
├── home-user-guide.md # NEW — monolithic, linear
├── operator-guide.md # NEW — monolithic, reference
├── architecture/
│ ├── overview.md # NEW — process model, bus, data flow
│ ├── conventions.md # NEW — coding conventions distilled
│ └── subsystems/
│ ├── camera.md # NEW
│ ├── detection.md # NEW
│ ├── events.md # NEW
│ ├── alerts.md # NEW
│ ├── sensors.md # NEW
│ ├── ups.md # NEW
│ ├── storage.md # NEW
│ ├── highlights.md # NEW
│ ├── presence.md # NEW
│ ├── pets.md # NEW
│ ├── health.md # NEW
│ └── web.md # NEW
└── camera-hardware-guide.md # EXISTING — untouched
```
## 4. `README.md` (top-level)
Sells the project, orients newcomers, links into the doc tree.
**Required sections, in order:**
1. **Title + tagline** — "Vigilar — DIY offline-first home security"
2. **One-line pitch** (~20 words)
3. **Hero image placeholder**`![screenshot](docs/images/grid.png)` with a comment noting it's a placeholder. Do not create the image.
4. **Why Vigilar** — bullet list of differentiators: offline-first, HLS grid + MJPEG single, OpenCV MOG2 motion with 5-sec pre-motion ring buffer, event timeline + highlight reels + timelapses, visitor/pet/wildlife tracking, PWA with VAPID push (no Firebase), AES-256 encrypted recordings, NUT UPS monitoring, runs on cheap mini PC.
5. **Quick paths table** — 4 rows mapping intent → doc: home user guide, operator guide, architecture overview, camera hardware guide.
6. **60-second overview** — small ASCII diagram (cameras → mini PC → phone/browser) + 4 tech-stack bullets (Python 3.11, Flask+Bootstrap 5, SQLite WAL, MQTT, FFmpeg).
7. **Status** — "Alpha. Works on the author's hardware. Expect rough edges and breaking changes."
8. **Installation TL;DR** — 3-line fenced block (`git clone`, `sudo ./scripts/install.sh`, `sudo systemctl start vigilar`), followed by links to the full home-user and operator guides.
9. **Documentation** — nested bullet list mirroring the `docs/` tree, each entry with a one-line description.
10. **License** — GPL-3.0. If no `LICENSE` file exists, the README states the intent and notes a `LICENSE` file will follow; we do **not** create the license file as part of this effort unless asked.
11. **Contributing** — short stub: "Issues and PRs welcome. See `CLAUDE.md` for code conventions."
## 5. `docs/home-user-guide.md`
Monolithic, linear, ~15002500 words. Target reader: homeowner with a mini PC, some Linux comfort, wants cameras on phone.
**Required sections, in order:**
1. **What you'll end up with** — outcome in 2 sentences plus a small ASCII diagram.
2. **What you need** — hardware checklist: mini PC (x86_64, 4GB+ RAM, 128GB+ SSD), USB stick for OS install, one or more RTSP cameras, phone, optional NAS. Link to `camera-hardware-guide.md` for camera picks.
3. **Step 1 — Install Debian/Ubuntu Server on the mini PC** — brief, points at upstream installer docs, tells the user to enable SSH. No hand-holding on the OS install itself.
4. **Step 2 — Get Vigilar onto the box**`git clone`, `sudo ./scripts/install.sh`, plus 3 bullets summarizing what `install.sh` does (read `scripts/install.sh` at write-time to ground these bullets).
5. **Step 3 — First boot**`sudo systemctl enable --now vigilar`, then open `http://<mini-pc-ip>:49735` in a browser on the same LAN. Mention the port is configurable under `[web]` in `vigilar.toml`.
6. **Step 4 — Set your PIN** — UI walkthrough, 23 sentences, screenshot placeholder.
7. **Step 5 — Add your first camera** — UI walkthrough: RTSP URL, credentials, test stream, save. Point at `camera-hardware-guide.md` for URL formats.
8. **Step 6 — Phone push notifications (PWA)** — open web UI on phone, "Add to Home Screen", allow notifications. Under-the-hood note: VAPID keys already generated by `install.sh`.
9. **Step 7 — Optional: NAS backup of config + database** — mount NAS share at `/mnt/nas/vigilar-backups`, set `VIGILAR_BACKUP_DIR`, and set up a nightly run of `scripts/backup.sh`. The guide provides a copy-pasteable systemd service + timer snippet inline (no new units are shipped in the repo as part of this effort). **Explicitly state** that this backs up DB + `/etc/vigilar` (config + secrets) only, and that **recordings stay local** — point at a "planned" note for recording backup.
10. **Troubleshooting** — camera won't connect, no push notifications, service won't start (`journalctl -u vigilar`), motion detection too sensitive, how to reset PIN.
11. **Where to go next** — links to Operator Guide and Architecture Overview.
**Grounding rule:** every shell command must correspond to a real file in `scripts/` or a real `vigilar` CLI subcommand. Verify before writing.
## 6. `docs/operator-guide.md`
Monolithic, reference-oriented, ~25004000 words. Target reader: self-hoster tuning, upgrading, securing.
**Required sections, in order:**
1. **Audience & scope** — for admins, not first-time home users. Points at home-user-guide.md for initial setup.
2. **Layout on disk** — table of `/opt/vigilar`, `/etc/vigilar/{vigilar.toml, certs/, secrets/}`, `/var/vigilar/{data/vigilar.db, recordings/, hls/, backups/}`.
3. **Installation** — what `scripts/install.sh` does, `systemd/vigilar.service` summary, `systemd/vigilar-mosquitto.conf` summary, system dependencies (ffmpeg, mosquitto, sqlite3, Python 3.11+).
4. **Configuration reference (`vigilar.toml`)** — one subsection per `[section]` in the default TOML. Each key: default, what it controls, when to change. Sections to cover (from current `config/vigilar.toml`): `system`, `mqtt`, `web`, `zigbee2mqtt`, `ups`, `storage`, `remote`, `alerts.local`, `alerts.web_push`, `alerts.email`, plus any additional sections discovered by re-reading the TOML at write-time.
5. **CLI reference (`vigilar ...`)** — enumerated at write-time by reading `vigilar/cli/`. One subsection per top-level command. Do not guess commands.
6. **Secrets & security**`/etc/vigilar/secrets/` layout and permissions; `vigilar config set-pin`; `vigilar config set-password`; TLS via `scripts/gen_cert.sh``[web] tls_cert/tls_key`; VAPID via `scripts/gen_vapid_keys.sh`; storage encryption key (`storage.key`) — **explicit warning: do not lose it, recordings are unrecoverable without it**; recommended firewall stance (LAN-only by default).
7. **UPS / NUT integration**`scripts/setup_nut.sh`, `[ups]` options, shutdown behavior, low-battery thresholds.
8. **Backups** — what `scripts/backup.sh` captures (DB + `/etc/vigilar`) and what it does **not** (recordings); `VIGILAR_BACKUP_DIR` and `VIGILAR_BACKUP_RETENTION_DAYS`; copy-pasteable systemd service + timer snippet (inline in the doc; no new unit files added to the repo); restore procedure.
9. **Upgrades**`git pull` + `pip install -e .` + `systemctl restart vigilar`; rollback by restoring a backup tarball. If DB migrations exist, note how they're applied; if they don't, say so.
10. **Logs & health**`journalctl -u vigilar`, `log_level` in `[system]`, health endpoints (enumerated at write-time by reading `vigilar/web/blueprints/system.py` and `vigilar/health/`).
11. **Remote access**`[remote]` section, tunnel-based remote HLS, bandwidth-shaped downscaled streams, reiterated not-a-cloud.
12. **Troubleshooting** — service crash loops, MQTT broker won't start, camera worker thrashing, disk full / `free_space_floor_gb` triggered, HLS stalling.
**Grounding rule:** every TOML key, every CLI command, every file path, every endpoint must be verified against the current code before writing. Any that can't be verified must be omitted, not guessed.
## 7. `docs/architecture/overview.md`
~10001500 words. Target reader: contributor new to the codebase.
**Required sections, in order:**
1. **Design principles** — offline-first, subsystem isolation via multiprocessing, loose coupling via local MQTT bus, SQLite WAL as single durable store, SQLAlchemy Core (not ORM), adaptive FPS (2 idle / 30 on motion) with ring buffer.
2. **Process topology** — ASCII or Mermaid diagram showing parent supervisor + N subsystem processes + mosquitto + Flask web.
3. **The MQTT bus** — broker location, topic naming convention `vigilar/<subsystem>/<entity>/<event>`, retention/QoS notes (verify at write-time), rationale for MQTT over an in-process queue.
4. **Data flow: the motion → alert path** — numbered sequence from RTSP capture through motion detection, recording, event creation, highlight scoring, push notification, and UI update. Each step names the actual file/function where it happens (verify at write-time).
5. **Storage layout** — SQLite table summary (enumerate at write-time by reading `vigilar/storage/`), recordings (`.vge`, AES-256-GCM, key path), HLS segments, backups.
6. **Configuration & secrets** — TOML → Pydantic v2 validation, secrets as file paths (never inline), PIN & password hashing with constant-time compare.
7. **The web tier** — Flask + Blueprints, Jinja2 + Bootstrap 5 dark, HLS grid + MJPEG single view rationale, PWA + VAPID.
8. **What's NOT in the critical path** — remote access (optional), email alerts (optional), cloud (never).
## 8. `docs/architecture/conventions.md`
~400 words. Distilled from `CLAUDE.md` but written for human contributors, not the AI. Covers: StrEnum for string constants (`vigilar/constants.py`), SQLAlchemy Core only (no mapped ORM classes), type hints on public functions, no docstrings unless logic is non-obvious, Ruff line-length 100, multiprocessing-per-subsystem rule, MQTT topic naming, Pydantic-validated TOML config, secrets-as-file-paths.
## 9. `docs/architecture/subsystems/*.md` (12 files)
One file per subdirectory under `vigilar/`: `camera`, `detection`, `events`, `alerts`, `sensors`, `ups`, `storage`, `highlights`, `presence`, `pets`, `health`, `web`.
**Uniform template** (≈150400 words each):
```
# <Subsystem name>
## Purpose
One paragraph — what this subsystem is responsible for.
## Key files
- `vigilar/<sub>/foo.py` — role
- ...
## MQTT topics
**Subscribes:** `vigilar/...`
**Publishes:** `vigilar/...`
## Database tables
`table_name` — what it holds. Or "none."
## Depends on
- sister subsystem X (via topic Y)
## Consumed by
- sister subsystem Z (via topic W)
## Notes
Gotchas or perf notes, only if any.
```
**Grounding rule (hard):** every topic name, every table name, every file role must come from reading the actual code. If a topic cannot be found, the doc must say "no MQTT publishers found at time of writing" — not invent one. This rule is the most important verification step in the plan.
## 10. Verification checklist (before completion)
1. **Link check** — every relative link in every new file resolves to a real path.
2. **Command check** — every shell command in the user guides exists as a real script under `scripts/` or a real `vigilar` CLI subcommand.
3. **Grounding check** — every topic name, table name, file path, and endpoint is verified against code, or omitted. Nothing guessed.
4. **TOML coverage check** — every `[section]` in `config/vigilar.toml` is covered in the operator guide's configuration reference.
5. **Subsystem coverage check** — every subdirectory in `vigilar/` (matching the 12-file list) has a corresponding subsystem doc.
6. **Read-through pass** — tone and terminology consistent across all files.
7. **README link check** — all doc tree links in `README.md` resolve.
## 11. Out of scope (explicit)
- `LICENSE` file creation (the README declares GPL-3.0; creating the file is a separate request).
- Screenshot/image creation (placeholders only).
- MkDocs configuration.
- Any code changes.
- Any changes to `docs/camera-hardware-guide.md`.
- Any doc-linting CI.
- Recording-backup-to-NAS feature or docs beyond the "planned" note.
- Migration documentation beyond noting whether migrations exist.
## 12. Success criteria
- A new homeowner can go from a bare mini PC to working cameras on their phone using only `README.md` + `docs/home-user-guide.md`.
- A self-hoster can answer any "how do I configure / back up / upgrade / troubleshoot" question from `docs/operator-guide.md` alone.
- A new contributor can identify which subsystem owns a given behavior within 5 minutes using `docs/architecture/overview.md` + the subsystem files.
- Every claim in every doc is either verified against current code or explicitly flagged.

View File

@@ -11,6 +11,28 @@ from vigilar.config import VigilarConfig, load_config
from vigilar.storage.schema import metadata from vigilar.storage.schema import metadata
@pytest.fixture(autouse=True, scope="session")
def _isolate_vigilar_config(tmp_path_factory):
"""Prevent tests from writing to the real config/vigilar.toml.
Web endpoint handlers that call `_save_and_reload()` read the target
path from the VIGILAR_CONFIG env var, falling back to the relative
`"config/vigilar.toml"`. Without this fixture, any test that exercises
such an endpoint rewrites the repo's committed config file via a
Pydantic round-trip, stripping comments and non-default fields.
"""
tmp_config = tmp_path_factory.mktemp("vigilar-config") / "vigilar.toml"
prev = os.environ.get("VIGILAR_CONFIG")
os.environ["VIGILAR_CONFIG"] = str(tmp_config)
try:
yield
finally:
if prev is None:
os.environ.pop("VIGILAR_CONFIG", None)
else:
os.environ["VIGILAR_CONFIG"] = prev
def _create_test_engine(db_path: Path): def _create_test_engine(db_path: Path):
"""Create a fresh engine for testing (bypasses the global singleton).""" """Create a fresh engine for testing (bypasses the global singleton)."""
db_path.parent.mkdir(parents=True, exist_ok=True) db_path.parent.mkdir(parents=True, exist_ok=True)

View File

@@ -0,0 +1,31 @@
"""Tests for `vigilar config set-pin`."""
from click.testing import CliRunner
from vigilar.alerts.pin import verify_pin
from vigilar.cli.cmd_config import config_cmd
def test_set_pin_outputs_pbkdf2_hash_under_security_section():
"""The CLI must emit a hash that alerts.pin.verify_pin can validate,
and direct the user to [security] pin_hash (not [system] arm_pin_hash)."""
runner = CliRunner()
result = runner.invoke(config_cmd, ["set-pin"], input="1234\n1234\n")
assert result.exit_code == 0, result.output
# The output must direct the user to the [security] section
assert "[security]" in result.output
assert "arm_pin_hash" not in result.output
assert "pin_hash" in result.output
# Extract the emitted hash (line starts with `pin_hash = "..."`)
hash_line = next(
line for line in result.output.splitlines() if line.strip().startswith("pin_hash")
)
hash_value = hash_line.split('"')[1]
# Round-trip: the emitted hash must accept the plaintext PIN
assert verify_pin("1234", hash_value) is True
assert verify_pin("0000", hash_value) is False
# And it must be in PBKDF2 format (not the legacy HMAC "secret:mac" format)
assert hash_value.startswith("pbkdf2_sha256$")

View File

@@ -138,3 +138,49 @@ class TestCameraConfigLocation:
from vigilar.config import CameraConfig from vigilar.config import CameraConfig
cfg = CameraConfig(id="test", display_name="Test", rtsp_url="rtsp://x", location="EXTERIOR") cfg = CameraConfig(id="test", display_name="Test", rtsp_url="rtsp://x", location="EXTERIOR")
assert cfg.location == "EXTERIOR" assert cfg.location == "EXTERIOR"
def test_deprecation_warning_for_arm_pin_hash(tmp_path, caplog):
"""Loading a config that still uses the legacy [system] arm_pin_hash
must log a clear warning pointing the user at `vigilar config set-pin`."""
import logging
cfg_path = tmp_path / "legacy.toml"
cfg_path.write_text(
'[system]\n'
'arm_pin_hash = "pbkdf2_sha256$abc$def"\n'
)
with caplog.at_level(logging.WARNING):
from vigilar.config import load_config
load_config(str(cfg_path))
messages = [r.message for r in caplog.records if r.levelno >= logging.WARNING]
assert any("arm_pin_hash" in m and "deprecated" in m.lower() for m in messages), (
f"expected deprecation warning mentioning arm_pin_hash, got: {messages}"
)
def test_no_deprecation_warning_when_security_pin_hash_set(tmp_path, caplog):
"""No warning should fire if [security] pin_hash is populated,
regardless of whether [system] arm_pin_hash is also still present.
The warning is specifically for un-migrated configs."""
import logging
cfg_path = tmp_path / "migrated.toml"
cfg_path.write_text(
'[system]\n'
'arm_pin_hash = "pbkdf2_sha256$legacy$value"\n'
'\n'
'[security]\n'
'pin_hash = "pbkdf2_sha256$current$value"\n'
)
with caplog.at_level(logging.WARNING):
from vigilar.config import load_config
load_config(str(cfg_path))
deprecation_messages = [
r.message for r in caplog.records
if r.levelno >= logging.WARNING and "arm_pin_hash" in r.message
]
assert deprecation_messages == [], (
f"deprecation warning should not fire on migrated configs, "
f"got: {deprecation_messages}"
)

View File

@@ -1,6 +1,5 @@
"""Tests for the Phase 6 events subsystem: rules, arm state FSM, history.""" """Tests for the Phase 6 events subsystem: rules, arm state FSM, history."""
import hashlib
import time import time
import pytest import pytest
@@ -19,7 +18,7 @@ from vigilar.storage.queries import insert_event
def _make_config(rules=None, pin_hash=""): def _make_config(rules=None, pin_hash=""):
return VigilarConfig( return VigilarConfig(
system={"arm_pin_hash": pin_hash}, security={"pin_hash": pin_hash},
cameras=[], cameras=[],
sensors=[], sensors=[],
rules=rules or [], rules=rules or [],
@@ -27,7 +26,8 @@ def _make_config(rules=None, pin_hash=""):
def _pin_hash(pin: str) -> str: def _pin_hash(pin: str) -> str:
return hashlib.sha256(pin.encode()).hexdigest() from vigilar.alerts.pin import hash_pin
return hash_pin(pin)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -429,3 +429,158 @@ class TestPetEventClassification:
assert etype == EventType.WILDLIFE_PASSIVE assert etype == EventType.WILDLIFE_PASSIVE
assert sev == Severity.INFO assert sev == Severity.INFO
assert source == "front" assert source == "front"
# ---------------------------------------------------------------------------
# Classified-event broadcast (for SSE bridge)
# ---------------------------------------------------------------------------
class _RecordingBus:
"""Minimal bus fake that records publishes instead of sending them."""
def __init__(self):
self.published: list[tuple[str, dict]] = []
def publish(self, topic: str, payload: dict, qos: int = 1) -> None:
self.published.append((topic, payload))
def publish_event(self, topic: str, **kwargs) -> None:
self.publish(topic, kwargs)
class _StubFSM:
state = ArmState.DISARMED
class _StubRuleEngine:
def evaluate(self, topic, payload, state):
return []
class TestEventsPublishedBroadcast:
"""EventProcessor should publish a classified-event summary to
Topics.EVENTS_PUBLISHED after inserting, so the Flask SSE bridge
can forward it to browser clients."""
def test_handle_event_publishes_classified_payload(self, test_db):
from vigilar.events.processor import EventProcessor
from vigilar.constants import Topics
processor = EventProcessor.__new__(EventProcessor)
bus = _RecordingBus()
processor._handle_event(
topic="vigilar/camera/cam1/motion/start",
payload={"ts": 12345, "detail": "x"},
engine=test_db,
fsm=_StubFSM(),
rule_engine=_StubRuleEngine(),
bus=bus,
)
published_on_bridge_topic = [
p for t, p in bus.published if t == Topics.EVENTS_PUBLISHED
]
assert len(published_on_bridge_topic) == 1, (
f"expected exactly one publish to {Topics.EVENTS_PUBLISHED}, "
f"got publishes: {bus.published}"
)
msg = published_on_bridge_topic[0]
assert msg["type"] == EventType.MOTION_START
assert msg["severity"] == Severity.WARNING
assert msg["source_id"] == "cam1"
assert "ts" in msg
assert "id" in msg and isinstance(msg["id"], int)
def test_unclassified_topic_does_not_publish(self, test_db):
"""Heartbeats and other non-event topics must not be forwarded."""
from vigilar.events.processor import EventProcessor
from vigilar.constants import Topics
processor = EventProcessor.__new__(EventProcessor)
bus = _RecordingBus()
processor._handle_event(
topic="vigilar/camera/cam1/heartbeat",
payload={"ts": 12345},
engine=test_db,
fsm=_StubFSM(),
rule_engine=_StubRuleEngine(),
bus=bus,
)
assert not any(t == Topics.EVENTS_PUBLISHED for t, _ in bus.published)
# ---------------------------------------------------------------------------
# Arm Request Dispatch
# ---------------------------------------------------------------------------
class TestArmRequestDispatch:
"""SYSTEM_ARM_REQUEST messages must reach ArmStateFSM.transition."""
def test_arm_request_calls_fsm_transition(self, test_db):
from vigilar.events.processor import EventProcessor
processor = EventProcessor.__new__(EventProcessor)
calls = []
class FakeFSM:
state = ArmState.DISARMED
def transition(self, new_state, pin="", triggered_by="system"):
calls.append((new_state, pin, triggered_by))
return True
processor._handle_arm_request(
payload={"mode": "ARMED_AWAY", "pin": "1234", "triggered_by": "web"},
fsm=FakeFSM(),
)
assert len(calls) == 1
new_state, pin, triggered_by = calls[0]
assert new_state == ArmState.ARMED_AWAY
assert pin == "1234"
assert triggered_by == "web"
def test_arm_request_ignores_bad_mode(self, test_db):
from vigilar.events.processor import EventProcessor
processor = EventProcessor.__new__(EventProcessor)
calls = []
class FakeFSM:
def transition(self, *a, **kw):
calls.append((a, kw))
return True
processor._handle_arm_request(
payload={"mode": "NONSENSE", "pin": "1234"},
fsm=FakeFSM(),
)
assert calls == []
def test_arm_request_default_triggered_by(self, test_db):
"""Omitting triggered_by must default to 'unknown' (audit-log value)."""
from vigilar.events.processor import EventProcessor
processor = EventProcessor.__new__(EventProcessor)
calls = []
class FakeFSM:
state = ArmState.DISARMED
def transition(self, new_state, pin="", triggered_by="system"):
calls.append((new_state, pin, triggered_by))
return True
processor._handle_arm_request(
payload={"mode": "DISARMED", "pin": ""},
fsm=FakeFSM(),
)
assert len(calls) == 1
assert calls[0][2] == "unknown"

View File

@@ -37,3 +37,14 @@ def test_verify_pin_handles_unicode():
stored = hash_pin("p@ss!") stored = hash_pin("p@ss!")
assert verify_pin("p@ss!", stored) is True assert verify_pin("p@ss!", stored) is True
assert verify_pin("p@ss?", stored) is False assert verify_pin("p@ss?", stored) is False
def test_verify_pin_rejects_malformed_hash():
"""verify_pin must return False (not raise) on malformed stored hashes.
Fail-closed is load-bearing: a misconfigured or partially-migrated
[security] pin_hash must lock out transitions, not grant access."""
assert verify_pin("1234", "sha256:deadbeef") is False
assert verify_pin("1234", "garbage") is False
assert verify_pin("1234", "pbkdf2_sha256$only$two$extra") is False
# Wrong algo prefix
assert verify_pin("1234", "argon2id$salt$dk") is False

View File

@@ -0,0 +1,45 @@
"""End-to-end test: the CLI, FSM, and web arm flow all accept the same PIN.
Regression guard for issue #2 — the three layers previously used three
incompatible hash schemes under two different config keys."""
from click.testing import CliRunner
from vigilar.alerts.pin import hash_pin, verify_pin
from vigilar.cli.cmd_config import config_cmd
from vigilar.config import SecurityConfig, VigilarConfig
from vigilar.events.state import ArmStateFSM
from vigilar.constants import ArmState
def test_cli_output_is_accepted_by_fsm(test_db):
"""Hash produced by `vigilar config set-pin` must verify against
ArmStateFSM.verify_pin, same config key, same format."""
runner = CliRunner()
result = runner.invoke(config_cmd, ["set-pin"], input="9876\n9876\n")
assert result.exit_code == 0, result.output
hash_line = next(
line for line in result.output.splitlines()
if line.strip().startswith("pin_hash")
)
hash_value = hash_line.split('"')[1]
cfg = VigilarConfig(security=SecurityConfig(pin_hash=hash_value))
fsm = ArmStateFSM(test_db, cfg)
assert fsm.verify_pin("9876") is True
assert fsm.verify_pin("0000") is False
def test_fsm_transitions_with_pin_from_alerts_module(test_db):
"""The alerts.pin module and ArmStateFSM agree on the hash format."""
stored = hash_pin("4242")
cfg = VigilarConfig(security=SecurityConfig(pin_hash=stored))
fsm = ArmStateFSM(test_db, cfg)
assert fsm.transition(ArmState.ARMED_AWAY, pin="4242", triggered_by="test") is True
assert fsm.state == ArmState.ARMED_AWAY
# Same stored hash rejects the wrong PIN
assert verify_pin("0000", stored) is False

View File

@@ -0,0 +1,62 @@
"""Tests for the MQTT → Server-Sent Events bridge used by the web process."""
import json
import queue
def test_forward_event_delivers_payload_to_sse_subscribers():
"""forward_event should hand the payload to every connected SSE client."""
from vigilar.web.blueprints import events as events_bp
from vigilar.web.sse_bridge import forward_event
q: queue.Queue = queue.Queue(maxsize=10)
events_bp._sse_subscribers.append(q)
try:
payload = {
"type": "MOTION_START",
"source_id": "cam1",
"severity": "WARNING",
"ts": 1234,
}
forward_event("vigilar/events/published", payload)
raw = q.get_nowait()
assert raw.startswith("data: ")
# Strip "data: " prefix and trailing double newline
data = json.loads(raw[len("data: "):].rstrip())
assert data["type"] == "MOTION_START"
assert data["source_id"] == "cam1"
finally:
if q in events_bp._sse_subscribers:
events_bp._sse_subscribers.remove(q)
def test_start_sse_bridge_subscribes_to_events_published(monkeypatch):
"""start_sse_bridge must connect a bus and subscribe forward_event
to Topics.EVENTS_PUBLISHED — that is the single entry point for the
web process to observe classified events from the events subsystem."""
from vigilar.config import VigilarConfig
from vigilar.constants import Topics
from vigilar.web import sse_bridge
class FakeBus:
def __init__(self, config, client_id):
self.client_id = client_id
self.subscriptions: list[tuple[str, object]] = []
self.connected = False
def subscribe(self, topic, handler):
self.subscriptions.append((topic, handler))
def connect(self):
self.connected = True
monkeypatch.setattr(sse_bridge, "MessageBus", FakeBus)
bus = sse_bridge.start_sse_bridge(VigilarConfig())
assert bus.connected is True
assert len(bus.subscriptions) == 1
topic, handler = bus.subscriptions[0]
assert topic == Topics.EVENTS_PUBLISHED
assert handler is sse_bridge.forward_event

View File

@@ -1,6 +1,7 @@
"""Tests for PIN verification on arm/disarm endpoints.""" """Tests for PIN verification on arm/disarm endpoints."""
import pytest import pytest
from unittest.mock import patch
from vigilar.alerts.pin import hash_pin from vigilar.alerts.pin import hash_pin
from vigilar.config import VigilarConfig, SecurityConfig from vigilar.config import VigilarConfig, SecurityConfig
from vigilar.web.app import create_app from vigilar.web.app import create_app
@@ -29,35 +30,55 @@ def app_no_pin():
def test_arm_without_pin_set(app_no_pin): def test_arm_without_pin_set(app_no_pin):
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_no_pin.test_client() as c: with app_no_pin.test_client() as c:
rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY"}) rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY"})
assert rv.status_code == 200 assert rv.status_code == 202
assert rv.get_json()["ok"] is True pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["mode"] == "ARMED_AWAY"
assert payload["pin"] == ""
def test_arm_correct_pin(app_with_pin): def test_arm_correct_pin(app_with_pin):
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c: with app_with_pin.test_client() as c:
rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY", "pin": "1234"}) rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY", "pin": "1234"})
assert rv.status_code == 200 assert rv.status_code == 202
assert rv.get_json()["ok"] is True pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["pin"] == "1234"
def test_arm_wrong_pin(app_with_pin): def test_arm_wrong_pin_still_accepted_by_web_fsm_rejects(app_with_pin):
"""HTTP layer no longer pre-checks the PIN — it forwards to the FSM
unconditionally. The FSM verifies and, on mismatch, logs a warning
and leaves the state unchanged."""
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c: with app_with_pin.test_client() as c:
rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY", "pin": "0000"}) rv = c.post("/system/api/arm", json={"mode": "ARMED_AWAY", "pin": "0000"})
assert rv.status_code == 401 assert rv.status_code == 202
pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["pin"] == "0000" # forwarded verbatim — FSM will reject
def test_disarm_correct_pin(app_with_pin): def test_disarm_correct_pin(app_with_pin):
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c: with app_with_pin.test_client() as c:
rv = c.post("/system/api/disarm", json={"pin": "1234"}) rv = c.post("/system/api/disarm", json={"pin": "1234"})
assert rv.status_code == 200 assert rv.status_code == 202
pub.assert_called_once()
def test_disarm_wrong_pin(app_with_pin): def test_disarm_wrong_pin_still_accepted_by_web_fsm_rejects(app_with_pin):
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c: with app_with_pin.test_client() as c:
rv = c.post("/system/api/disarm", json={"pin": "9999"}) rv = c.post("/system/api/disarm", json={"pin": "9999"})
assert rv.status_code == 401 assert rv.status_code == 202
pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["pin"] == "9999" # forwarded verbatim — FSM will reject
def test_reset_pin_correct_passphrase(app_with_pin): def test_reset_pin_correct_passphrase(app_with_pin):
@@ -77,3 +98,35 @@ def test_reset_pin_wrong_passphrase(app_with_pin):
"new_pin": "5678", "new_pin": "5678",
}) })
assert rv.status_code == 401 assert rv.status_code == 401
def test_arm_publishes_arm_request_on_mqtt(app_with_pin):
"""POST /system/api/arm must publish a SYSTEM_ARM_REQUEST message
carrying the mode, pin, and a 'web' triggered_by tag."""
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c:
rv = c.post(
"/system/api/arm",
json={"mode": "ARMED_AWAY", "pin": "1234"},
)
assert rv.status_code == 202
assert rv.get_json()["ok"] is True
pub.assert_called_once()
call_args = pub.call_args
# _publish_arm_request(cfg, payload) — payload is args[1] or kwargs["payload"]
payload = call_args.args[1] if len(call_args.args) > 1 else call_args.kwargs["payload"]
assert payload["mode"] == "ARMED_AWAY"
assert payload["pin"] == "1234"
assert payload["triggered_by"] == "web"
def test_disarm_publishes_arm_request(app_with_pin):
with patch("vigilar.web.blueprints.system._publish_arm_request") as pub:
with app_with_pin.test_client() as c:
rv = c.post("/system/api/disarm", json={"pin": "1234"})
assert rv.status_code == 202
pub.assert_called_once()
payload = pub.call_args.args[1] if len(pub.call_args.args) > 1 else pub.call_args.kwargs["payload"]
assert payload["mode"] == "DISARMED"

View File

@@ -102,3 +102,36 @@ def test_recordings_page_loads():
with app.test_client() as client: with app.test_client() as client:
resp = client.get("/recordings/") resp = client.get("/recordings/")
assert resp.status_code == 200 assert resp.status_code == 200
def test_system_status_reflects_fsm_arm_state(tmp_path, monkeypatch):
"""system_status must read the current arm state from the DB,
not return a hardcoded stub. Regression guard for the web-to-FSM
async flow introduced in issue #2."""
from vigilar.config import SystemConfig, VigilarConfig
import vigilar.storage.db as db_module
from vigilar.storage.db import get_db_path
from vigilar.storage.schema import metadata
from vigilar.storage.queries import insert_arm_state
from vigilar.web.app import create_app
from sqlalchemy import create_engine
data_dir = tmp_path / "data"
data_dir.mkdir()
cfg = VigilarConfig(system=SystemConfig(data_dir=str(data_dir)))
# Build an isolated engine (bypass the module-level singleton)
db_path = get_db_path(str(data_dir))
isolated_engine = create_engine(f"sqlite:///{db_path}", echo=False)
metadata.create_all(isolated_engine)
insert_arm_state(isolated_engine, "ARMED_AWAY", "test", None)
# Patch the singleton so the blueprint's get_engine() returns our engine
monkeypatch.setattr(db_module, "_engine", isolated_engine)
app = create_app(cfg)
with app.test_client() as c:
resp = c.get("/system/status")
assert resp.status_code == 200
assert resp.get_json()["arm_state"] == "ARMED_AWAY"

View File

@@ -47,8 +47,10 @@ def show_cmd(config_path: str | None) -> None:
# Redact sensitive fields # Redact sensitive fields
if data.get("web", {}).get("password_hash"): if data.get("web", {}).get("password_hash"):
data["web"]["password_hash"] = "***" data["web"]["password_hash"] = "***"
if data.get("system", {}).get("arm_pin_hash"): if data.get("security", {}).get("pin_hash"):
data["system"]["arm_pin_hash"] = "***" data["security"]["pin_hash"] = "***"
if data.get("security", {}).get("recovery_passphrase_hash"):
data["security"]["recovery_passphrase_hash"] = "***"
if data.get("alerts", {}).get("webhook", {}).get("secret"): if data.get("alerts", {}).get("webhook", {}).get("secret"):
data["alerts"]["webhook"]["secret"] = "***" data["alerts"]["webhook"]["secret"] = "***"
click.echo(json.dumps(data, indent=2)) click.echo(json.dumps(data, indent=2))
@@ -60,12 +62,9 @@ def show_cmd(config_path: str | None) -> None:
@config_cmd.command("set-password") @config_cmd.command("set-password")
@click.option("--config", "-c", "config_path", default=None, help="Path to vigilar.toml.") @click.option("--config", "-c", "config_path", default=None, help="Path to vigilar.toml.")
def set_password_cmd(config_path: str | None) -> None: def set_password_cmd(config_path: str | None) -> None:
"""Generate a bcrypt hash for the web UI password.""" """Generate a scrypt hash for the web UI password."""
try: try:
import hashlib
password = click.prompt("Enter web UI password", hide_input=True, confirmation_prompt=True) password = click.prompt("Enter web UI password", hide_input=True, confirmation_prompt=True)
# Use SHA-256 hash (bcrypt requires external dep, but cryptography is available)
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
import os import os
@@ -82,14 +81,10 @@ def set_password_cmd(config_path: str | None) -> None:
@config_cmd.command("set-pin") @config_cmd.command("set-pin")
def set_pin_cmd() -> None: def set_pin_cmd() -> None:
"""Generate an HMAC hash for the arm/disarm PIN.""" """Generate a PBKDF2 hash for the arm/disarm PIN."""
import hashlib from vigilar.alerts.pin import hash_pin
import hmac
import os
pin = click.prompt("Enter arm/disarm PIN", hide_input=True, confirmation_prompt=True) pin = click.prompt("Enter arm/disarm PIN", hide_input=True, confirmation_prompt=True)
secret = os.urandom(32) hash_str = hash_pin(pin)
mac = hmac.new(secret, pin.encode(), hashlib.sha256).hexdigest() click.echo("\nAdd this to your vigilar.toml [security] section:")
hash_str = secret.hex() + ":" + mac click.echo(f'pin_hash = "{hash_str}"')
click.echo(f"\nAdd this to your vigilar.toml [system] section:")
click.echo(f'arm_pin_hash = "{hash_str}"')

View File

@@ -1,5 +1,6 @@
"""Configuration loading and validation via TOML + Pydantic.""" """Configuration loading and validation via TOML + Pydantic."""
import logging
import sys import sys
import tomllib import tomllib
from pathlib import Path from pathlib import Path
@@ -23,6 +24,8 @@ from vigilar.constants import (
CameraLocation, CameraLocation,
) )
log = logging.getLogger(__name__)
# --- Camera Config --- # --- Camera Config ---
class CameraConfig(BaseModel): class CameraConfig(BaseModel):
@@ -435,4 +438,13 @@ def load_config(path: str | Path | None = None) -> VigilarConfig:
raw["sensors.gpio"] = gpio_config raw["sensors.gpio"] = gpio_config
# The [[sensors]] array items remain as 'sensors' key from TOML parsing # The [[sensors]] array items remain as 'sensors' key from TOML parsing
return VigilarConfig(**raw) cfg = VigilarConfig(**raw)
if cfg.system.arm_pin_hash and not cfg.security.pin_hash:
log.warning(
"DEPRECATED: [system] arm_pin_hash is ignored; the arm/disarm "
"PIN lives under [security] pin_hash. Run `vigilar config "
"set-pin` and paste the output into [security]."
)
return cfg

View File

@@ -210,6 +210,12 @@ class Topics:
SYSTEM_ALERT = "vigilar/system/alert" SYSTEM_ALERT = "vigilar/system/alert"
SYSTEM_SHUTDOWN = "vigilar/system/shutdown" SYSTEM_SHUTDOWN = "vigilar/system/shutdown"
SYSTEM_RULES_UPDATED = "vigilar/system/rules_updated" SYSTEM_RULES_UPDATED = "vigilar/system/rules_updated"
# Web-to-FSM arm/disarm request (FSM verifies the PIN and transitions)
SYSTEM_ARM_REQUEST = "vigilar/system/arm_request"
# Classified events forwarded to the web SSE bridge (see events.processor
# and web.sse_bridge).
EVENTS_PUBLISHED = "vigilar/events/published"
# Wildcard subscriptions # Wildcard subscriptions
ALL = "vigilar/#" ALL = "vigilar/#"

View File

@@ -9,7 +9,7 @@ from sqlalchemy.engine import Engine
from vigilar.bus import MessageBus from vigilar.bus import MessageBus
from vigilar.config import VigilarConfig from vigilar.config import VigilarConfig
from vigilar.constants import EventType, Severity, Topics from vigilar.constants import ArmState, EventType, Severity, Topics
from vigilar.events.rules import RuleEngine from vigilar.events.rules import RuleEngine
from vigilar.events.state import ArmStateFSM from vigilar.events.state import ArmStateFSM
from vigilar.storage.db import get_db_path, init_db from vigilar.storage.db import get_db_path, init_db
@@ -58,12 +58,20 @@ class EventProcessor:
fsm.set_bus(bus) fsm.set_bus(bus)
bus.connect() bus.connect()
# Subscribe to all Vigilar topics # Subscribe to all Vigilar topics (events/motion/sensors/etc.)
def on_message(topic: str, payload: dict[str, Any]) -> None: def on_message(topic: str, payload: dict[str, Any]) -> None:
self._handle_event(topic, payload, engine, fsm, rule_engine, bus) self._handle_event(topic, payload, engine, fsm, rule_engine, bus)
bus.subscribe_all(on_message) bus.subscribe_all(on_message)
# Dedicated subscription for web-originated arm/disarm requests.
# Kept separate from on_message because these are commands, not
# classifiable events.
def on_arm_request(topic: str, payload: dict[str, Any]) -> None:
self._handle_arm_request(payload, fsm)
bus.subscribe(Topics.SYSTEM_ARM_REQUEST, on_arm_request)
log.info("Event processor started") log.info("Event processor started")
# Main loop # Main loop
@@ -103,6 +111,22 @@ class EventProcessor:
payload=payload, payload=payload,
) )
# Broadcast a classified summary for the web SSE bridge. This is
# independent of rule actions and DB storage — the web process
# subscribes to Topics.EVENTS_PUBLISHED and forwards each message
# to connected browser clients.
bus.publish(
Topics.EVENTS_PUBLISHED,
{
"id": event_id,
"ts": int(time.time() * 1000),
"type": event_type,
"severity": severity,
"source_id": source_id,
"payload": payload,
},
)
# Insert pet/wildlife sightings # Insert pet/wildlife sightings
if event_type in ( if event_type in (
EventType.PET_DETECTED, EventType.PET_ESCAPE, EventType.UNKNOWN_ANIMAL EventType.PET_DETECTED, EventType.PET_ESCAPE, EventType.UNKNOWN_ANIMAL
@@ -144,6 +168,26 @@ class EventProcessor:
except Exception: except Exception:
log.exception("Error processing event on %s", topic) log.exception("Error processing event on %s", topic)
def _handle_arm_request(
self,
payload: dict[str, Any],
fsm: ArmStateFSM,
) -> None:
"""Handle an arm/disarm request received over MQTT.
Payload fields:
- mode: str — desired ArmState ("DISARMED", "ARMED_HOME", "ARMED_AWAY")
- pin: str — plaintext PIN (FSM verifies against security.pin_hash)
- triggered_by: str — origin tag for the audit log (e.g. "web")
"""
mode = payload.get("mode", "")
if mode not in ArmState.__members__:
log.warning("Ignoring arm request with invalid mode: %r", mode)
return
pin = payload.get("pin", "")
triggered_by = payload.get("triggered_by", "unknown")
fsm.transition(ArmState(mode), pin=pin, triggered_by=triggered_by)
def _classify_event( def _classify_event(
self, topic: str, payload: dict[str, Any] self, topic: str, payload: dict[str, Any]
) -> tuple[str | None, str | None, str | None]: ) -> tuple[str | None, str | None, str | None]:

View File

@@ -1,12 +1,11 @@
"""Arm state finite state machine.""" """Arm state finite state machine."""
import hashlib
import hmac
import logging import logging
import time import time
from sqlalchemy.engine import Engine from sqlalchemy.engine import Engine
from vigilar.alerts.pin import verify_pin
from vigilar.config import VigilarConfig from vigilar.config import VigilarConfig
from vigilar.constants import ArmState, EventType, Severity, Topics from vigilar.constants import ArmState, EventType, Severity, Topics
from vigilar.storage.queries import get_current_arm_state, insert_arm_state, insert_event from vigilar.storage.queries import get_current_arm_state, insert_arm_state, insert_event
@@ -19,7 +18,7 @@ class ArmStateFSM:
def __init__(self, engine: Engine, config: VigilarConfig): def __init__(self, engine: Engine, config: VigilarConfig):
self._engine = engine self._engine = engine
self._pin_hash = config.system.arm_pin_hash self._pin_hash = config.security.pin_hash
self._state = ArmState.DISARMED self._state = ArmState.DISARMED
self._bus = None self._bus = None
self._load_initial_state() self._load_initial_state()
@@ -43,12 +42,11 @@ class ArmStateFSM:
return self._state return self._state
def verify_pin(self, pin: str) -> bool: def verify_pin(self, pin: str) -> bool:
"""Verify a PIN against the stored hash using HMAC comparison.""" """Verify a PIN against the stored PBKDF2 hash."""
if not self._pin_hash: if not self._pin_hash:
# No PIN configured — allow all transitions # No PIN configured — allow all transitions
return True return True
candidate = hashlib.sha256(pin.encode()).hexdigest() return verify_pin(pin, self._pin_hash)
return hmac.compare_digest(candidate, self._pin_hash)
def transition( def transition(
self, self,
@@ -68,9 +66,10 @@ class ArmStateFSM:
old_state = self._state old_state = self._state
self._state = new_state self._state = new_state
# Log to database # pin_hash is always None here: PBKDF2 uses a random salt per call, so
pin_hash = hashlib.sha256(pin.encode()).hexdigest() if pin else None # re-hashing the pin now would produce a value unrelated to the stored
insert_arm_state(self._engine, new_state.value, triggered_by, pin_hash) # hash, making the column useless for audit correlation. See issue #2.
insert_arm_state(self._engine, new_state.value, triggered_by, None)
# Log event # Log event
insert_event( insert_event(

View File

@@ -67,9 +67,27 @@ class SubsystemProcess:
def _run_web(cfg: VigilarConfig) -> None: def _run_web(cfg: VigilarConfig) -> None:
"""Run the Flask web server in a subprocess.""" """Run the Flask web server in a subprocess."""
from vigilar.web.app import create_app from vigilar.web.app import create_app
from vigilar.web.sse_bridge import start_sse_bridge
app = create_app(cfg) app = create_app(cfg)
# Forward classified events from MQTT to browser SSE clients. Failure
# here must not kill the web process — the UI still works without
# live updates, it just requires a page refresh.
sse_bus = None
try:
sse_bus = start_sse_bridge(cfg)
except Exception:
log.exception("Failed to start SSE bridge; live event timeline will not update")
try:
app.run(host=cfg.web.host, port=cfg.web.port, debug=False, use_reloader=False) app.run(host=cfg.web.host, port=cfg.web.port, debug=False, use_reloader=False)
finally:
if sse_bus is not None:
try:
sse_bus.disconnect()
except Exception:
log.exception("Error disconnecting SSE bridge bus")
def _run_event_processor(cfg: VigilarConfig) -> None: def _run_event_processor(cfg: VigilarConfig) -> None:

View File

@@ -36,8 +36,20 @@ def _save_and_reload(new_cfg: VigilarConfig) -> None:
def system_status(): def system_status():
"""JSON API: overall system health.""" """JSON API: overall system health."""
cfg = _get_cfg() cfg = _get_cfg()
arm_state = "DISARMED"
try:
from vigilar.storage.db import get_db_path, get_engine
from vigilar.storage.queries import get_current_arm_state
engine = get_engine(get_db_path(cfg.system.data_dir))
stored = get_current_arm_state(engine)
if stored:
arm_state = stored
except Exception:
current_app.logger.exception("Failed to read arm state from DB")
return jsonify({ return jsonify({
"arm_state": "DISARMED", "arm_state": arm_state,
"ups": {"status": "UNKNOWN"}, "ups": {"status": "UNKNOWN"},
"cameras_online": 0, "cameras_online": 0,
"cameras_total": len(cfg.cameras), "cameras_total": len(cfg.cameras),
@@ -54,27 +66,46 @@ def settings_page():
# --- Arm/Disarm --- # --- Arm/Disarm ---
def _publish_arm_request(cfg: VigilarConfig, payload: dict) -> None:
"""Publish an arm/disarm request on MQTT for the event processor to pick up."""
from vigilar.bus import MessageBus
from vigilar.constants import Topics
bus = MessageBus(cfg.mqtt, client_id="vigilar-web-arm-request")
bus.connect()
if not bus.connected:
raise RuntimeError("MQTT broker did not accept connection within timeout")
try:
bus.publish(Topics.SYSTEM_ARM_REQUEST, payload)
finally:
bus.disconnect()
@system_bp.route("/api/arm", methods=["POST"]) @system_bp.route("/api/arm", methods=["POST"])
def arm_system(): def arm_system():
data = request.get_json() or {} data = request.get_json() or {}
mode = data.get("mode", "ARMED_AWAY") mode = data.get("mode", "ARMED_AWAY")
pin = data.get("pin", "") pin = data.get("pin", "")
cfg = _get_cfg() payload = {"mode": mode, "pin": pin, "triggered_by": "web"}
pin_hash = cfg.security.pin_hash try:
if pin_hash and not verify_pin(pin, pin_hash): _publish_arm_request(_get_cfg(), payload)
return jsonify({"error": "Invalid PIN"}), 401 except Exception:
return jsonify({"ok": True, "state": mode}) current_app.logger.exception("Failed to publish arm request")
return jsonify({"error": "bus unavailable"}), 503
return jsonify({"ok": True, "mode": mode}), 202
@system_bp.route("/api/disarm", methods=["POST"]) @system_bp.route("/api/disarm", methods=["POST"])
def disarm_system(): def disarm_system():
data = request.get_json() or {} data = request.get_json() or {}
pin = data.get("pin", "") pin = data.get("pin", "")
cfg = _get_cfg() payload = {"mode": "DISARMED", "pin": pin, "triggered_by": "web"}
pin_hash = cfg.security.pin_hash try:
if pin_hash and not verify_pin(pin, pin_hash): _publish_arm_request(_get_cfg(), payload)
return jsonify({"error": "Invalid PIN"}), 401 except Exception:
return jsonify({"ok": True, "state": "DISARMED"}) current_app.logger.exception("Failed to publish arm request")
return jsonify({"error": "bus unavailable"}), 503
return jsonify({"ok": True, "mode": "DISARMED"}), 202
@system_bp.route("/api/reset-pin", methods=["POST"]) @system_bp.route("/api/reset-pin", methods=["POST"])

36
vigilar/web/sse_bridge.py Normal file
View File

@@ -0,0 +1,36 @@
"""Bridge classified events from MQTT to Server-Sent Events subscribers.
The events subsystem (`vigilar.events.processor.EventProcessor`) publishes
classified events to `Topics.EVENTS_PUBLISHED`. The Flask web process runs
in its own OS process, so to make the in-browser event timeline update
live it must subscribe to that topic via its own `MessageBus` client and
forward every message to `broadcast_sse_event`.
"""
import logging
from typing import Any
from vigilar.bus import MessageBus
from vigilar.config import VigilarConfig
from vigilar.constants import Topics
from vigilar.web.blueprints.events import broadcast_sse_event
log = logging.getLogger(__name__)
def forward_event(topic: str, payload: dict[str, Any]) -> None:
"""MQTT handler: forward a classified event to SSE subscribers."""
broadcast_sse_event(payload)
def start_sse_bridge(cfg: VigilarConfig) -> MessageBus:
"""Create an MQTT client that forwards classified events to SSE clients.
Returns the connected `MessageBus` so the caller can disconnect it on
shutdown.
"""
bus = MessageBus(cfg.mqtt, client_id="vigilar-web-sse-bridge")
bus.subscribe(Topics.EVENTS_PUBLISHED, forward_event)
bus.connect()
log.info("SSE bridge started: subscribed to %s", Topics.EVENTS_PUBLISHED)
return bus