vigilar/docs/superpowers/specs/2026-04-03-pet-aware-security-design.md

Pet-Aware Security Features — Design Spec

Overview

Add pet detection, identification, wildlife monitoring, and pet activity tracking to Vigilar. Transforms the system from a people/vehicle security platform into a comprehensive home awareness system that knows your pets by name, alerts when they're somewhere they shouldn't be, and tracks wildlife activity with threat-tiered notifications.

Pets

Name	Species	Breed	Description
Angel	Cat (she/her)	Black DSH	8yo, short hair, black ("pocket panther")
Taquito (Tito/Quito)	Cat (she/her)	Maine Coon mix	2yo, long hair, tan/grey/brown, small for breed, long whiskers
Milo	Dog (he/him)	Pomsky	6mo, shorter hair (not woolly), husky with squirrel tail

Camera Layout

Camera	Location Type	Pet Alert Behavior
Front Entrance (porch + driveway)	exterior	Pet = ALERT, Wildlife = tiered
Back Deck	exterior	Pet = ALERT, Wildlife = tiered
Side Entrance / Bootroom	transition	Pet = ALERT (escape risk)
Garage	transition	Pet = ALERT (escape risk)
Kitchen / Living Room	interior	Pet = LOG (location update only)

1. Detection Pipeline

Architecture: Unified Model

Replace MobileNet-SSD v2 with YOLOv8-small as the single object detection model. YOLOv8-small handles people, vehicles, AND animals in one inference pass. The pet ID classifier runs as a lightweight second stage on domestic animal crops only.

Motion Detected (MOG2, unchanged)
  → Frame Captured
  → YOLOv8-small inference (replaces MobileNet-SSD)
      → Person/Vehicle: existing event flow (unchanged behavior)
      → Cat/Dog (domestic): crop → Pet ID Classifier → named pet or "Unknown"
      → Bear/Bird/etc (wildlife): classify threat tier → event

YOLOv8-small

• Model: yolov8s.pt (~22MB)
• Input: 640x640
• COCO-80 classes including: person, car, truck, cat, dog, bear, bird, horse, cow, sheep
• Confidence threshold: 0.5 (configurable)
• Replaces both person.py and vehicle.py detection — vehicle color/size fingerprinting remains as a post-detection stage

Pet ID Classifier

• Model: MobileNetV3-Small, transfer learning from ImageNet
• Input: cropped bounding box resized to 224x224
• Output: pet name + confidence score
• High confidence (>= 0.7): auto-identified
• Low confidence (0.5 - 0.7): queued for human review in unlabeled queue
• Below 0.5: classified as "Unknown Cat" / "Unknown Dog"
• Only runs on cat/dog detections, never on wildlife
• Stored at /var/vigilar/models/pet_id.pt with backup at pet_id_backup.pt

Wildlife Threat Classification

COCO classes map directly where possible:

Threat Level	COCO Classes	Heuristic Classes
PREDATOR	bear	fox, coyote (size heuristic: medium quadruped)
NUISANCE	—	raccoon, skunk, possum (size heuristic: small quadruped)
PASSIVE	bird, horse, cow, sheep	deer (size heuristic: large quadruped)

Size heuristics for non-COCO wildlife use bounding box area as percentage of frame:

• Small (< 2%): likely raccoon, skunk, possum → NUISANCE
• Medium (2-8%): likely fox, coyote → PREDATOR
• Large (> 8%): likely deer → PASSIVE (or bear, which COCO catches directly)

Phase 2 consideration: a dedicated wildlife classifier similar to pet ID, trained on local wildlife photos. Not in scope for this spec.

2. Zone Logic & Alert Routing

Camera Location Types

Each camera gets a location field: exterior, interior, or transition. Transition zones (bootroom, garage) are treated as exterior for pet escape purposes.

Alert Matrix

Detection	Interior	Exterior/Transition
Known pet	LOG (silent location update)	ALERT (PET_ESCAPE event)
Unknown domestic animal	WARN (push notification)	WARN (push notification)
Wildlife — Predator	URGENT ALERT (shouldn't happen but covered)	URGENT ALERT
Wildlife — Nuisance	URGENT ALERT (shouldn't happen)	NOTIFY (normal push)
Wildlife — Passive	LOG	LOG (record only)

Alert Profile Integration

New detection types plug into the existing smart alert profile system. New detection_type values for profile rules:

• known_pet — identified pet (Angel, Taquito, or Milo)
• unknown_animal — domestic cat/dog not in registry
• wildlife_predator — bear, fox, coyote
• wildlife_nuisance — raccoon, skunk, possum
• wildlife_passive — deer, bird, rabbit

New location field on alert profile rules filters by the camera's location property (e.g., known_pet + exterior = push_and_record, known_pet + interior = quiet_log). When location is omitted from a rule, it matches any camera location.

New MQTT Topics

vigilar/camera/{id}/pet/detected       — pet detection with ID result
vigilar/camera/{id}/wildlife/detected  — wildlife detection with threat tier
vigilar/pets/{name}/location           — pet location updates (last-seen camera)

New Event Types

Event Type	Severity	Trigger
PET_DETECTED	INFO	Known pet identified on any camera
PET_ESCAPE	ALERT	Known pet detected in exterior/transition zone
UNKNOWN_ANIMAL	WARNING	Unregistered domestic animal detected
WILDLIFE_PREDATOR	CRITICAL	Bear, fox, coyote detected
WILDLIFE_NUISANCE	WARNING	Raccoon, skunk, possum detected
WILDLIFE_PASSIVE	INFO	Deer, bird, rabbit detected

3. Pet ID Training Interface

Two training paths feed the same classifier:

Path 1: Photo Upload

Pet Profiles page (/pets/) where users manage registered pets. Each pet shows name, species, breed, profile photo, training image count, and model status (untrained / needs more photos / trained with accuracy %).

Upload interface: drag-and-drop or file picker for JPG/PNG images. Multiple files supported. Images are auto-cropped to the animal region if a bounding box can be detected, otherwise stored as-is. Stored in /var/vigilar/pets/training/{pet_name}/.

Path 2: Label From Playback

When viewing recordings, detected animals show bounding boxes overlaid on the video. Clicking a bounding box opens a labeling popup: "Who is this?" with buttons for each registered pet of the matching species (cat detection only shows cat profiles, dog detection only shows dog profiles), plus "Unknown" and "+ New Pet" options.

Unlabeled Detection Queue

Every animal detection auto-saves a cropped image to /var/vigilar/pets/staging/. These appear in an "Unlabeled Detections" section on the Pet Profiles page as clickable thumbnails. Users can quickly label them (assigning to a pet) or dismiss them. Unlabeled crops auto-expire after 7 days.

Low-confidence identifications (0.5-0.7) are prioritized in this queue — they represent the cases where human input most improves the model.

Training Process

• Minimum 20 labeled images per pet to train
• Transfer learning from ImageNet pretrained MobileNetV3-Small
• Training triggered manually from the UI ("Train Model" button)
• UI suggests retraining when 20+ new labeled images accumulate since last training
• Training time: ~2-5 minutes on x86_64
• Previous model backed up before overwrite for rollback
• Training runs in a background process, UI shows progress

4. Pet Dashboard & Activity Features

Pet Dashboard

New section on the main dashboard or accessible via /pets/ showing:

Per-pet cards with:

• Name, species, profile photo/icon
• Current status: Safe (interior) / Check (transition zone) / Alert (exterior)
• Last known location (camera name)
• Last seen timestamp
• Active time today
• Mini location trail (last 3 hours of camera transitions)

Wildlife summary bar:

• Today's counts by threat tier (predators, nuisance, passive)
• Most recent notable sighting with timestamp and camera
• Link to full wildlife log

Activity timeline:

• Per-pet horizontal bar showing detection activity across 24 hours
• Color-coded by camera/zone (orange for transition/exterior zones)
• Integrates visually with the existing recording timeline component

Daily Pet Digest

Appended to the existing daily digest push notification. Per-pet summary including:

• Activity duration
• Rooms/zones visited
• Notable moments (zoomies, birdwatching, escape alerts)
• Wildlife summary for the day

Highlight Reel

Auto-generated short clip references based on heuristics:

• Zoomies: pet detected + high motion intensity (>0.8 threshold, configurable)
• Birdwatching: cat detected + window zone + sustained presence (>2 min)
• Wildlife encounter: any wildlife detection event
• Pet escape: PET_ESCAPE event

Highlights are bookmarks into existing recordings (start timestamp, duration, camera, description) — no additional storage. Displayed on the pet dashboard as a scrollable list with playback links.

5. Data Model

New Tables

pets

Column	Type	Notes
id	TEXT PK	UUID
name	TEXT NOT NULL	Display name
species	TEXT NOT NULL	"cat" or "dog"
breed	TEXT	Optional breed info
color_description	TEXT	For reference, not used in detection
photo_path	TEXT	Profile photo path
training_count	INTEGER	Number of labeled training images
created_at	REAL	Timestamp

pet_sightings

Column	Type	Notes
id	INTEGER PK	Autoincrement
ts	REAL NOT NULL	Detection timestamp
pet_id	TEXT FK → pets	NULL if unknown
species	TEXT NOT NULL	"cat" or "dog"
camera_id	TEXT NOT NULL	Which camera
confidence	REAL	Pet ID confidence
crop_path	TEXT	Saved bounding box crop
labeled	BOOLEAN	Human-confirmed identification
event_id	INTEGER FK → events	Linked event

wildlife_sightings

Column	Type	Notes
id	INTEGER PK	Autoincrement
ts	REAL NOT NULL	Detection timestamp
species	TEXT NOT NULL	"bear", "deer", etc.
threat_level	TEXT NOT NULL	PREDATOR, NUISANCE, or PASSIVE
camera_id	TEXT NOT NULL	Which camera
confidence	REAL	Detection confidence
crop_path	TEXT	Saved crop image
event_id	INTEGER FK → events	Linked event

pet_training_images

Column	Type	Notes
id	INTEGER PK	Autoincrement
pet_id	TEXT FK → pets	Which pet
image_path	TEXT NOT NULL	Path to crop
source	TEXT NOT NULL	"upload", "detection", or "playback"
created_at	REAL	Timestamp

Indexes

• idx_pet_sightings_ts — ts DESC
• idx_pet_sightings_pet — pet_id, ts DESC
• idx_pet_sightings_camera — camera_id, ts DESC
• idx_wildlife_ts — ts DESC
• idx_wildlife_threat — threat_level, ts DESC

6. Configuration

New Config Sections

[pets]
enabled = true
model = "yolov8s"
model_path = "/var/vigilar/models/yolov8s.pt"
confidence_threshold = 0.5

# Pet ID classifier
pet_id_enabled = true
pet_id_model_path = "/var/vigilar/models/pet_id.pt"
pet_id_threshold = 0.7
pet_id_low_confidence = 0.5

# Training data
training_dir = "/var/vigilar/pets/training"
crop_staging_dir = "/var/vigilar/pets/staging"
crop_retention_days = 7
min_training_images = 20

[pets.wildlife]
[pets.wildlife.threat_map]
predator = ["bear"]
nuisance = []
passive = ["bird", "horse", "cow", "sheep"]

[pets.wildlife.size_heuristics]
small = 0.02
medium = 0.08
large = 0.15

[pets.activity]
daily_digest = true
highlight_clips = true
zoomie_threshold = 0.8

Camera Config Addition

[[cameras]]
id = "front_entrance"
name = "Front Entrance"
location = "exterior"    # NEW FIELD
# ... existing fields

New StrEnums in constants.py

class ThreatLevel(StrEnum):
    PREDATOR = "predator"
    NUISANCE = "nuisance"
    PASSIVE = "passive"

class CameraLocation(StrEnum):
    EXTERIOR = "exterior"
    INTERIOR = "interior"
    TRANSITION = "transition"

New EventType values: PET_DETECTED, PET_ESCAPE, UNKNOWN_ANIMAL, WILDLIFE_PREDATOR, WILDLIFE_NUISANCE, WILDLIFE_PASSIVE.

7. File Storage

/var/vigilar/
  models/
    yolov8s.pt              — main detection model (~22MB)
    pet_id.pt               — trained pet classifier (~8MB)
    pet_id_backup.pt        — previous version (rollback)
  pets/
    training/               — labeled images for training
      angel/
      taquito/
      milo/
    staging/                — unlabeled detection crops (auto-expire 7d)
    profiles/               — pet profile photos

8. New Web Routes

Route	Method	Purpose
/pets/	GET	Pet dashboard and profiles page
/pets/api/status	GET	Current pet locations and status (JSON)
/pets/api/sightings	GET	Pet sighting history (JSON, filterable)
/pets/api/wildlife	GET	Wildlife sighting history (JSON, filterable)
/pets/<id>/upload	POST	Upload training photos for a pet
/pets/<id>/label	POST	Label a detection crop as a specific pet
/pets/register	POST	Register a new pet
/pets/<id>/update	POST	Update pet profile
/pets/<id>/delete	DELETE	Remove a pet
/pets/train	POST	Trigger model training
/pets/api/training-status	GET	Training progress (JSON)
/pets/api/highlights	GET	Today's highlight clips (JSON)
/pets/api/unlabeled	GET	Unlabeled detection crops queue (JSON)

9. Dependencies

New Python Packages

• ultralytics — YOLOv8 inference and model loading
• torchvision — MobileNetV3-Small for pet ID classifier (torch is a dependency of ultralytics)

Model Downloads

• YOLOv8-small: auto-downloaded by ultralytics on first run, or pre-placed at configured path
• Pet ID model: trained locally from user's labeled images, no download needed

No Removed Dependencies

• OpenCV remains for frame capture, motion detection, image manipulation
• MobileNet-SSD model files can be removed after migration is validated

10. Migration Path

The YOLOv8 model replaces MobileNet-SSD for all detection. To minimize risk:

1. Implement YOLOv8 detector with the same output interface as current MobileNet detector
2. Validate person/vehicle detection accuracy matches or exceeds MobileNet
3. Add animal detection classes and pet ID pipeline
4. Remove MobileNet code and model files once validated

Existing vehicle fingerprinting (color + size profiling) continues to work — it receives bounding box crops from the new detector instead of the old one.