vigilar/docs/superpowers/specs/2026-04-03-detection-intelligence-design.md

Detection Intelligence — Design Spec

Overview

Three features that surface richer insights from detection data already flowing through the system: per-camera activity heatmaps, a full-featured wildlife sighting journal with weather correlation, and package delivery detection with sunset-aware reminders.

Q2: Activity Heatmap

What It Does

Generates a static PNG image per camera showing where detections cluster over 7 or 30 days. Helps evaluate camera placement, identify blind spots, and visualize traffic patterns.

Implementation

New file: vigilar/detection/heatmap.py

generate_heatmap(engine, camera_id, days, frame) -> bytes:

1. Query all detection events for the camera within the time window. Events must include bounding box data — use pet_sightings, wildlife_sightings, and events table (for person/vehicle detections).
2. Accumulate bounding box centers into a 2D grid (64x36 bins, mapping to the camera's aspect ratio).
3. Apply Gaussian blur (sigma=2) for smooth heat distribution.
4. Normalize to 0-255, apply a colormap (blue → yellow → red).
5. Alpha-composite the heatmap onto the camera's most recent frame (or a representative still).
6. Return as PNG bytes.

Bounding box storage prerequisite: Detection events must include normalized bounding box coordinates in the event payload. The camera worker already sends confidence and species in payloads — add bbox (as [x, y, w, h] normalized to 0.0-1.0 of frame dimensions) to pet, wildlife, and person/vehicle detection payloads.

Caching: Generated heatmaps are stored in /var/vigilar/cache/heatmaps/{camera_id}_{days}d.png. Regenerated on request only if the cached file is older than 1 hour.

Web Route

GET /cameras/<id>/heatmap?days=7 — returns PNG image. days defaults to 7, accepts 7 or 30.

UI

Add a "Heatmap" tab on each camera's detail page. Two toggle buttons (7d / 30d) that swap the <img> src. Simple Bootstrap tab, no JavaScript framework needed.

Q3: Wildlife Journal

What It Does

A dedicated wildlife monitoring page with chronological sighting log, seasonal tracking, per-species photo galleries, weather correlation, and CSV export. Transforms wildlife detections into a naturalist's field journal.

Location Config

Add to vigilar.toml:

[location]
latitude = 45.0
longitude = -85.0

Config model:

class LocationConfig(BaseModel):
    latitude: float = 0.0
    longitude: float = 0.0

Add to VigilarConfig as location: LocationConfig. Used by both wildlife weather correlation and package detection sunset calculation.

Weather Data

New file: vigilar/detection/weather.py

WeatherFetcher class:

• Fetches current conditions from Open-Meteo API (https://api.open-meteo.com/v1/forecast). Free, no API key, no account.
• Parameters: latitude, longitude, hourly temperature + weather code.
• Caches the response for 1 hour (in-memory dict keyed by hour).
• Graceful offline degradation: if the fetch fails, returns None for temperature/conditions. Never blocks the detection pipeline.
• Called asynchronously after a wildlife sighting is inserted, not in the detection hot path.

get_conditions(lat, lon) -> dict | None: Returns {"temperature_c": 12.5, "conditions": "Partly cloudy"} or None.

Database Changes

Add columns to wildlife_sightings table:

• temperature_c (Float, nullable) — temperature at time of sighting
• conditions (String, nullable) — weather description
• bbox (String, nullable) — normalized bounding box as JSON [x, y, w, h]

Web Blueprint

New file: vigilar/web/blueprints/wildlife.py

Routes:

Route	Method	Purpose
/wildlife/	GET	Wildlife journal page
/wildlife/api/sightings	GET	Paginated sighting list (filterable by species, threat, camera, date range)
/wildlife/api/stats	GET	Aggregate stats: counts per species, firsts/lasts per year, days since last predator
/wildlife/api/frequency	GET	Species frequency by time-of-day (6 four-hour buckets: 0-4, 4-8, 8-12, 12-16, 16-20, 20-24)
/wildlife/api/gallery/<species>	GET	Detection crop thumbnails for a species
/wildlife/api/export	GET	CSV export of all sightings with weather data

Template

New file: vigilar/web/templates/wildlife/journal.html

Bootstrap 5 dark theme. Sections:

1. Summary cards — total sightings this month, species count, days since last predator, most active species
2. Seasonal firsts — "First bear: April 3", "First deer: March 12", etc. Grouped by year.
3. Sighting log — paginated table with thumbnail, species, threat badge, camera, time, temperature, conditions. Filterable.
4. Time-of-day chart — horizontal stacked bar chart per species showing when they appear (rendered with simple CSS bars or a <canvas>, no charting library dependency).
5. Species gallery — clickable species cards that expand to show a grid of detection crop thumbnails.
6. Export button — downloads CSV.

All data loaded via fetch() to the API endpoints. Auto-refresh every 60 seconds for the summary cards.

Weather Integration Flow

1. Wildlife sighting detected → inserted into wildlife_sightings (existing flow)
2. Event processor publishes WILDLIFE_* event to MQTT (existing flow)
3. A new MQTT subscriber (or inline in the event processor) calls WeatherFetcher.get_conditions() and updates the sighting row with temperature/conditions
4. If offline, temperature/conditions remain NULL — no data loss, just missing weather context

Q5: Package Detection

What It Does

Detects packages left at the front door after a delivery person leaves. Sends an alert on delivery, one reminder at the earlier of sunset or 3 hours later, and logs when the package is collected.

Detection Logic

New file: vigilar/detection/package.py

PackageTracker class, one instance per exterior camera:

State machine per tracked package:

IDLE → person detected on exterior camera
  → person leaves (no person detection for 30s)
  → check for stationary object in the detection zone
  → if stationary object found: PRESENT
  → schedule reminder at min(sunset, now + 3h)

PRESENT → REMINDED (when reminder fires)
PRESENT or REMINDED → COLLECTED (when object disappears after person returns)

Stationary object detection:

• After person departs, run YOLO on the current frame
• Look for COCO classes that could be packages: suitcase (28), handbag (26), backpack (24)
• If any are detected in approximately the same position for >30 seconds (compare bbox across 2+ frames), classify as a package
• Store the bbox and a crop image

Sunset calculation:

New file: vigilar/detection/solar.py

get_sunset(latitude, longitude, date) -> datetime.time:

• Simple solar position calculation using the NOAA solar equations
• Stdlib only (math, datetime), no dependencies
• Accuracy within ~2 minutes, sufficient for "roughly sunset" timing
• Uses LocationConfig from the config

Database

New table: package_events

Column	Type	Notes
id	Integer PK	Autoincrement
camera_id	String NOT NULL	Which camera
detected_at	Float NOT NULL	When package first seen
reminded_at	Float	When reminder was sent (NULL if not yet)
collected_at	Float	When package was picked up (NULL if still there)
status	String NOT NULL	PRESENT, REMINDED, COLLECTED
crop_path	String	Photo of the package
event_id	Integer	Linked event

Index: idx_package_camera_status on (camera_id, status)

New Event Types

Add to EventType enum:

• PACKAGE_DELIVERED — package detected after person leaves
• PACKAGE_REMINDER — sunset/3-hour reminder fired
• PACKAGE_COLLECTED — package no longer detected

Alert Integration

• PACKAGE_DELIVERED: severity INFO, push notification "Package delivered — Front Entrance"
• PACKAGE_REMINDER: severity WARNING, push notification "Package still on porch — Front Entrance (3h / sunset)"
• PACKAGE_COLLECTED: severity INFO, logged only (no push)

These plug into the existing alert profile system as new detection_type values: package_delivered, package_reminder.

Reminder Scheduling

The PackageTracker calculates the reminder time at detection:

reminder_at = min(sunset_time, detected_at + 3 * 3600)

A background check runs every 60 seconds in the event processor (or a dedicated timer thread) that scans package_events where status=PRESENT and reminded_at IS NULL and current_time >= reminder_at. When triggered, fires the PACKAGE_REMINDER event and updates the row.

Camera Worker Integration

The PackageTracker is instantiated per exterior camera in the camera worker, alongside the YOLO detector. It receives detection results from the YOLO pass and manages its own state. It publishes package events to MQTT topics:

vigilar/camera/{id}/package/delivered
vigilar/camera/{id}/package/reminder
vigilar/camera/{id}/package/collected

UI

Small "Packages" widget on the main dashboard:

• Shows active packages (status PRESENT or REMINDED) with camera name, time on porch, and crop thumbnail
• Empty state: "No packages detected" (hidden when empty to avoid clutter)
• Historical package log accessible from a "Package History" link

Dependencies

New Packages

None — all features use existing dependencies and stdlib.

New Files

File	Purpose
vigilar/detection/heatmap.py	Heatmap generation (numpy + Pillow)
vigilar/detection/weather.py	Open-Meteo weather fetcher with caching
vigilar/detection/package.py	Package detection state machine
vigilar/detection/solar.py	Sunset calculation from lat/lon
vigilar/web/blueprints/wildlife.py	Wildlife journal blueprint
vigilar/web/templates/wildlife/journal.html	Wildlife journal template

Modified Files

File	Changes
vigilar/constants.py	Add PACKAGE_DELIVERED, PACKAGE_REMINDER, PACKAGE_COLLECTED event types
vigilar/config.py	Add LocationConfig model
vigilar/storage/schema.py	Add package_events table; add columns to wildlife_sightings
vigilar/storage/queries.py	Add package and wildlife stats query functions
vigilar/camera/worker.py	Add bbox to detection payloads; instantiate PackageTracker
vigilar/events/processor.py	Handle package events; weather enrichment for wildlife
vigilar/web/blueprints/cameras.py	Add heatmap route
vigilar/web/app.py	Register wildlife blueprint

Out of Scope

• Interactive/live heatmap overlay (future — start with static PNG)
• SpeciesNet integration (separate feature, would replace YOLO wildlife classes)
• Package detection on interior cameras (only exterior)
• Multi-package tracking on same camera (track one active package per camera for simplicity)
• Charting library (use CSS bars and canvas for time-of-day chart)