From 2623eb385ecf5961f48d4b79684799da2f3ec851 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sat, 1 Nov 2025 09:03:05 -0500 Subject: [PATCH] remove --- .github/copilot-instructions.md | 207 -------------------------------- 1 file changed, 207 deletions(-) delete mode 100644 .github/copilot-instructions.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md deleted file mode 100644 index 4fc44529d..000000000 --- a/.github/copilot-instructions.md +++ /dev/null @@ -1,207 +0,0 @@ -# Frigate AI Agent Instructions - -## Project Overview - -Frigate is a **local NVR with realtime AI object detection** for IP cameras. It's a multiprocess Python service that combines: - -- Video capture and processing from IP cameras -- Realtime object detection (TensorFlow, ONNX models on CPU/GPU/AI accelerators) -- Recording with retention policies -- Event tracking and review UI -- REST API + MQTT integration - -**Key architectural philosophy**: Minimize resource use by only running expensive detection where/when motion is detected. Heavy use of multiprocessing for FPS and scalability. - -## Architecture Patterns - -### Multiprocess Communication - -Frigate uses **three primary IPC mechanisms** (NOT shared state): - -1. **ZMQ pub/sub** (for one-way broadcasts): - - - Config changes: `frigate/comms/config_updater.py` (PUB via `ipc:///tmp/cache/config`) - - Object detection signals: `frigate/comms/object_detector_signaler.py` - - Detection/event updates: detector signaler, event publishers - -2. **ZMQ req/rep** (request-reply): - - - `frigate/comms/inter_process.py` - processes request data and get responses via `ipc:///tmp/cache/comms` - - Used by `InterProcessRequestor` for sync queries - -3. **Multiprocessing Queues** (frame data): - - `detection_queue`: Camera frames → object detectors - - `tracked_objects_queue`: Detected objects → event processor - - `timeline_queue`: Events → timeline storage - -**Key pattern**: Each service publishes to ZMQ topics, others subscribe. Config changes fan out via ZMQ pub/sub to all processes without central coordination. - -### Core Services (in `frigate/app.py` FrigateApp) - -- **CameraMaintainer** (thread): Spawns camera capture/processing subprocess per camera -- **ObjectDetectProcess** (subprocess): Runs ML inference on queued frames -- **TrackedObjectProcessor** (thread): Receives detections, correlates into tracked objects, publishes events -- **EventProcessor** (thread): Manages event lifecycle, DB updates -- **RecordProcess** (subprocess): Manages video recording/retention -- **OutputProcess** (subprocess): Encodes/streams video -- **ReviewProcess** (subprocess): Processes review segments -- **EmbeddingProcess** (subprocess): Runs embeddings for semantic search/face/LPR - -**Logging pattern**: Central `log.py` uses QueueListener to collect logs from all processes into one queue to avoid multiprocess logging chaos. - -## Config System - -Configuration is **Pydantic BaseModel** hierarchy: - -- **Parsing**: YAML → Pydantic models with validators in `frigate/config/config.py` -- **Types**: `frigate/types.py` has shared enums (EventType, ObjectType, etc.) -- **Validation pattern**: Use `@field_validator` with `mode='before'` to transform/validate before assignment -- **Runtime values**: `RuntimeMotionConfig` applies frame shape transforms to masks -- **Key files**: - - `config.py` - main FrigateConfig entry point - - `camera/` - per-camera sub-configs (detect, record, snapshots, etc.) - - `classification.py` - face/LPR/audio/semantic search configs - -**When adding config**: Create Pydantic model → add to parent config → update migrations if DB schema changes. - -## Data Model & Persistence - -**Database**: SQLite with custom `SqliteVecQueueDatabase` (vector support for embeddings) - -- **Models** in `frigate/models.py`: Event, Timeline, Recordings, User, etc. (Peewee ORM) -- **Key tables**: - - `events` - detected objects (car, person, etc.) with retention policies - - `timeline` - events feed (entered_zone, audio, etc.) - - `recordings` - video segments with metadata - - `review_segments` - flagged clips for review - -**Event lifecycle**: - -1. TrackedObject detected → Event created with `false_positive=False` -2. EventProcessor updates Event (score, zones, clips, snapshots) -3. On object lost, Event gets `end_time` and is finalized - -**Migrations**: Use Peewee migrations in `migrations/` - run via `peewee_migrate.Router`. - -## Key Workflows - -### Adding a New Detector Type - -1. Create detector class in `frigate/detectors/plugins/` inheriting `DetectionApi` -2. Add config class in `frigate/detectors/detector_config.py` -3. Register in detector factory in `frigate/detectors/__init__.py` -4. Update `DEFAULT_DETECTORS` constant if it's the default - -### Object Detection Pipeline - -``` -Camera subprocess → capture frames → motion detect → - queue frame to detection_queue → - ObjectDetectProcess (inference) → - TrackedObjectProcessor (correlate detections) → - Event + tracking + DB updates -``` - -### Recording Flow - -- **24/7 recording**: Segments written every frame duration -- **Retention**: Deleted if no events + retention time elapsed -- **Cleanup**: `RecordingCleanup` task deletes old segments based on retention config - -### Frontend Translation Pattern - -- **Rule**: NEVER hardcode strings in `.ts/.tsx` files -- **Pattern**: Store strings in `web/src/locales/en.json` → import locale function → use in code -- **See**: `.cursor/rules/frontend-always-use-translation-files.mdc` - -## Common Code Patterns - -### Inter-process Config Updates - -```python -# In detector/processor: -self.config_subscriber = ConfigSubscriber(config, [ConfigUpdateEnum.cameras]) - -# In main app (FrigateApp): -publisher = ConfigPublisher() -publisher.publish("cameras", new_config) # All subscribers notified -``` - -### Event Publishing - -```python -from frigate.comms.events_updater import EventUpdatePublisher - -publisher = EventUpdatePublisher() -publisher.publish({"camera": "cam1", "label": "person", ...}) -``` - -### Shared Memory Frames - -```python -from frigate.util.image import SharedMemoryFrameManager, UntrackedSharedMemory - -frame_manager = SharedMemoryFrameManager() -shm = frame_manager.get(frame_id) # returns np.ndarray view -``` - -## Testing & Debugging - -**Test structure**: `frigate/test/test_*.py` using Python `unittest` - -- Run tests: `make run_tests` (builds Docker, runs in container) -- Key tests: config parsing, detector inference, frame processing - -**Build targets** (Makefile): - -- `make local` - builds Docker image locally with version -- `make debug` - builds with debug logging enabled -- `make run` - runs container with config volume mounted - -**Debugging multiprocess issues**: - -- Check `log_queue` output in `frigate/log.py` -- Enable `DEBUG` logging for specific modules in config -- Use `faulthandler.enable()` (already enabled in processes) for segfaults - -## Important Conventions - -- **Imports**: Run Ruff with isort (`extend-select = ["I"]`) - enforces import sorting -- **GPU/Acceleration**: Hardware detection in `frigate/util/services.py` (NVIDIA, Intel VAAPI, AMD, etc.) -- **Model paths**: Stored in `/config/model_cache/` (symlinked or volume mounted) -- **Recording paths**: `/media/frigate/recordings/` (clips in `clips/`, exports in `exports/`) -- **PID locking**: Use `setproctitle()` to name processes for debugging via `ps` - -## Files to Know - -| File | Purpose | -| ------------------------------------ | ---------------------------------------- | -| `frigate/app.py` | Main app startup, service orchestration | -| `frigate/camera/` | Camera subprocess, frame capture, motion | -| `frigate/track/object_processing.py` | Detection correlation, event publishing | -| `frigate/events/maintainer.py` | Event lifecycle management | -| `frigate/config/config.py` | Config parsing & validation | -| `frigate/comms/` | IPC (ZMQ pub/sub, req/rep) | -| `frigate/api/fastapi_app.py` | REST API setup | -| `frigate/models.py` | Database ORM models | -| `frigate/const.py` | Global constants (paths, defaults) | - -## Gotchas & Common Mistakes - -1. **Pickle compatibility**: Objects sent over multiprocess queues must be pickleable. Avoid lambdas, file handles. -2. **Config subscriptions**: Always check `mode='before'` in validators—Pydantic can be confusing. -3. **Event state confusion**: Events have transient state in `TrackedObjectProcessor` AND persistent state in DB—don't mix them. -4. **Motion masks**: Frame shape must be applied before creating `RuntimeMotionConfig`—validate in tests. -5. **ZMQ timing**: Topics must be subscribed BEFORE publisher sends; use small sleep if race condition suspected. -6. **Frontend strings**: Forgetting to use locale files breaks translations and fails linting. - -## External Integration - -- **MQTT**: Via `MqttClient` in `frigate/comms/mqtt.py` - publishes detections, accepts commands -- **Home Assistant**: Native integration via custom component (separate repo) -- **Frigate+**: Paid cloud sync service - `frigate/plus.py` handles API calls -- **Webhooks**: Event-triggered POST requests configured per-camera - ---- - -**Last updated**: Branch `review-stream-tweaks` | For architecture deep-dives, start with `frigate/app.py::FrigateApp.__init__()` to see all service wiring.