vigilar/docs/superpowers/specs/2026-04-03-detection-intelligence-design.md

# Detection Intelligence — Design Spec

## Overview

Three features that surface richer insights from detection data already flowing through the system: per-camera activity heatmaps, a full-featured wildlife sighting journal with weather correlation, and package delivery detection with sunset-aware reminders.

## Q2: Activity Heatmap

### What It Does

Generates a static PNG image per camera showing where detections cluster over 7 or 30 days. Helps evaluate camera placement, identify blind spots, and visualize traffic patterns.

### Implementation

**New file: `vigilar/detection/heatmap.py`**

`generate_heatmap(engine, camera_id, days, frame) -> bytes`:
1. Query all detection events for the camera within the time window. Events must include bounding box data — use `pet_sightings`, `wildlife_sightings`, and `events` table (for person/vehicle detections).
2. Accumulate bounding box centers into a 2D grid (64x36 bins, mapping to the camera's aspect ratio).
3. Apply Gaussian blur (sigma=2) for smooth heat distribution.
4. Normalize to 0-255, apply a colormap (blue → yellow → red).
5. Alpha-composite the heatmap onto the camera's most recent frame (or a representative still).
6. Return as PNG bytes.

**Bounding box storage prerequisite:** Detection events must include normalized bounding box coordinates in the event payload. The camera worker already sends `confidence` and `species` in payloads — add `bbox` (as `[x, y, w, h]` normalized to 0.0-1.0 of frame dimensions) to pet, wildlife, and person/vehicle detection payloads.

**Caching:** Generated heatmaps are stored in `/var/vigilar/cache/heatmaps/{camera_id}_{days}d.png`. Regenerated on request only if the cached file is older than 1 hour.

### Web Route

`GET /cameras/<id>/heatmap?days=7` — returns PNG image. `days` defaults to 7, accepts 7 or 30.

### UI

Add a "Heatmap" tab on each camera's detail page. Two toggle buttons (7d / 30d) that swap the `<img>` src. Simple Bootstrap tab, no JavaScript framework needed.

## Q3: Wildlife Journal

### What It Does

A dedicated wildlife monitoring page with chronological sighting log, seasonal tracking, per-species photo galleries, weather correlation, and CSV export. Transforms wildlife detections into a naturalist's field journal.

### Location Config

Add to `vigilar.toml`:

```toml
[location]
latitude = 45.0
longitude = -85.0
```

Config model:

```python
class LocationConfig(BaseModel):
    latitude: float = 0.0
    longitude: float = 0.0
```

Add to `VigilarConfig` as `location: LocationConfig`. Used by both wildlife weather correlation and package detection sunset calculation.

### Weather Data

**New file: `vigilar/detection/weather.py`**

`WeatherFetcher` class:
- Fetches current conditions from Open-Meteo API (`https://api.open-meteo.com/v1/forecast`). Free, no API key, no account.
- Parameters: latitude, longitude, hourly temperature + weather code.
- Caches the response for 1 hour (in-memory dict keyed by hour).
- Graceful offline degradation: if the fetch fails, returns `None` for temperature/conditions. Never blocks the detection pipeline.
- Called asynchronously after a wildlife sighting is inserted, not in the detection hot path.

`get_conditions(lat, lon) -> dict | None`:
Returns `{"temperature_c": 12.5, "conditions": "Partly cloudy"}` or `None`.

### Database Changes

Add columns to `wildlife_sightings` table:
- `temperature_c` (Float, nullable) — temperature at time of sighting
- `conditions` (String, nullable) — weather description
- `bbox` (String, nullable) — normalized bounding box as JSON `[x, y, w, h]`

### Web Blueprint

**New file: `vigilar/web/blueprints/wildlife.py`**

Routes:

| Route | Method | Purpose |
|-------|--------|---------|
| `/wildlife/` | GET | Wildlife journal page |
| `/wildlife/api/sightings` | GET | Paginated sighting list (filterable by species, threat, camera, date range) |
| `/wildlife/api/stats` | GET | Aggregate stats: counts per species, firsts/lasts per year, days since last predator |
| `/wildlife/api/frequency` | GET | Species frequency by time-of-day (6 four-hour buckets: 0-4, 4-8, 8-12, 12-16, 16-20, 20-24) |
| `/wildlife/api/gallery/<species>` | GET | Detection crop thumbnails for a species |
| `/wildlife/api/export` | GET | CSV export of all sightings with weather data |

### Template

**New file: `vigilar/web/templates/wildlife/journal.html`**

Bootstrap 5 dark theme. Sections:

1. **Summary cards** — total sightings this month, species count, days since last predator, most active species
2. **Seasonal firsts** — "First bear: April 3", "First deer: March 12", etc. Grouped by year.
3. **Sighting log** — paginated table with thumbnail, species, threat badge, camera, time, temperature, conditions. Filterable.
4. **Time-of-day chart** — horizontal stacked bar chart per species showing when they appear (rendered with simple CSS bars or a `<canvas>`, no charting library dependency).
5. **Species gallery** — clickable species cards that expand to show a grid of detection crop thumbnails.
6. **Export button** — downloads CSV.

All data loaded via `fetch()` to the API endpoints. Auto-refresh every 60 seconds for the summary cards.

### Weather Integration Flow

1. Wildlife sighting detected → inserted into `wildlife_sightings` (existing flow)
2. Event processor publishes `WILDLIFE_*` event to MQTT (existing flow)
3. A new MQTT subscriber (or inline in the event processor) calls `WeatherFetcher.get_conditions()` and updates the sighting row with temperature/conditions
4. If offline, temperature/conditions remain NULL — no data loss, just missing weather context

## Q5: Package Detection

### What It Does

Detects packages left at the front door after a delivery person leaves. Sends an alert on delivery, one reminder at the earlier of sunset or 3 hours later, and logs when the package is collected.

### Detection Logic

**New file: `vigilar/detection/package.py`**

`PackageTracker` class, one instance per exterior camera:

**State machine per tracked package:**
```
IDLE → person detected on exterior camera
  → person leaves (no person detection for 30s)
  → check for stationary object in the detection zone
  → if stationary object found: PRESENT
  → schedule reminder at min(sunset, now + 3h)

PRESENT → REMINDED (when reminder fires)
PRESENT or REMINDED → COLLECTED (when object disappears after person returns)
```

**Stationary object detection:**
- After person departs, run YOLO on the current frame
- Look for COCO classes that could be packages: `suitcase` (28), `handbag` (26), `backpack` (24)
- If any are detected in approximately the same position for >30 seconds (compare bbox across 2+ frames), classify as a package
- Store the bbox and a crop image

**Sunset calculation:**

**New file: `vigilar/detection/solar.py`**

`get_sunset(latitude, longitude, date) -> datetime.time`:
- Simple solar position calculation using the NOAA solar equations
- Stdlib only (`math`, `datetime`), no dependencies
- Accuracy within ~2 minutes, sufficient for "roughly sunset" timing
- Uses `LocationConfig` from the config

### Database

**New table: `package_events`**

| Column | Type | Notes |
|--------|------|-------|
| id | Integer PK | Autoincrement |
| camera_id | String NOT NULL | Which camera |
| detected_at | Float NOT NULL | When package first seen |
| reminded_at | Float | When reminder was sent (NULL if not yet) |
| collected_at | Float | When package was picked up (NULL if still there) |
| status | String NOT NULL | PRESENT, REMINDED, COLLECTED |
| crop_path | String | Photo of the package |
| event_id | Integer | Linked event |

Index: `idx_package_camera_status` on (camera_id, status)

### New Event Types

Add to `EventType` enum:
- `PACKAGE_DELIVERED` — package detected after person leaves
- `PACKAGE_REMINDER` — sunset/3-hour reminder fired
- `PACKAGE_COLLECTED` — package no longer detected

### Alert Integration

- `PACKAGE_DELIVERED`: severity INFO, push notification "Package delivered — Front Entrance"
- `PACKAGE_REMINDER`: severity WARNING, push notification "Package still on porch — Front Entrance (3h / sunset)"
- `PACKAGE_COLLECTED`: severity INFO, logged only (no push)

These plug into the existing alert profile system as new `detection_type` values: `package_delivered`, `package_reminder`.

### Reminder Scheduling

The `PackageTracker` calculates the reminder time at detection:
```python
reminder_at = min(sunset_time, detected_at + 3 * 3600)
```

A background check runs every 60 seconds in the event processor (or a dedicated timer thread) that scans `package_events` where `status=PRESENT` and `reminded_at IS NULL` and `current_time >= reminder_at`. When triggered, fires the `PACKAGE_REMINDER` event and updates the row.

### Camera Worker Integration

The `PackageTracker` is instantiated per exterior camera in the camera worker, alongside the YOLO detector. It receives detection results from the YOLO pass and manages its own state. It publishes package events to MQTT topics:

```
vigilar/camera/{id}/package/delivered
vigilar/camera/{id}/package/reminder
vigilar/camera/{id}/package/collected
```

### UI

Small "Packages" widget on the main dashboard:
- Shows active packages (status PRESENT or REMINDED) with camera name, time on porch, and crop thumbnail
- Empty state: "No packages detected" (hidden when empty to avoid clutter)
- Historical package log accessible from a "Package History" link

## Dependencies

### New Packages
None — all features use existing dependencies and stdlib.

### New Files

| File | Purpose |
|------|---------|
| `vigilar/detection/heatmap.py` | Heatmap generation (numpy + Pillow) |
| `vigilar/detection/weather.py` | Open-Meteo weather fetcher with caching |
| `vigilar/detection/package.py` | Package detection state machine |
| `vigilar/detection/solar.py` | Sunset calculation from lat/lon |
| `vigilar/web/blueprints/wildlife.py` | Wildlife journal blueprint |
| `vigilar/web/templates/wildlife/journal.html` | Wildlife journal template |

### Modified Files

| File | Changes |
|------|---------|
| `vigilar/constants.py` | Add PACKAGE_DELIVERED, PACKAGE_REMINDER, PACKAGE_COLLECTED event types |
| `vigilar/config.py` | Add LocationConfig model |
| `vigilar/storage/schema.py` | Add package_events table; add columns to wildlife_sightings |
| `vigilar/storage/queries.py` | Add package and wildlife stats query functions |
| `vigilar/camera/worker.py` | Add bbox to detection payloads; instantiate PackageTracker |
| `vigilar/events/processor.py` | Handle package events; weather enrichment for wildlife |
| `vigilar/web/blueprints/cameras.py` | Add heatmap route |
| `vigilar/web/app.py` | Register wildlife blueprint |

## Out of Scope

- Interactive/live heatmap overlay (future — start with static PNG)
- SpeciesNet integration (separate feature, would replace YOLO wildlife classes)
- Package detection on interior cameras (only exterior)
- Multi-package tracking on same camera (track one active package per camera for simplicity)
- Charting library (use CSS bars and canvas for time-of-day chart)