mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-04-11 17:47:37 +03:00
remove
This commit is contained in:
Josh Hawkins
parent
b5e6658bfd
commit
2623eb385e
207
.github/copilot-instructions.md
vendored
207
.github/copilot-instructions.md
vendored
@ -1,207 +0,0 @@
|
|||||||
# Frigate AI Agent Instructions
|
|
||||||
|
|
||||||
## Project Overview
|
|
||||||
|
|
||||||
Frigate is a **local NVR with realtime AI object detection** for IP cameras. It's a multiprocess Python service that combines:
|
|
||||||
|
|
||||||
- Video capture and processing from IP cameras
|
|
||||||
- Realtime object detection (TensorFlow, ONNX models on CPU/GPU/AI accelerators)
|
|
||||||
- Recording with retention policies
|
|
||||||
- Event tracking and review UI
|
|
||||||
- REST API + MQTT integration
|
|
||||||
|
|
||||||
**Key architectural philosophy**: Minimize resource use by only running expensive detection where/when motion is detected. Heavy use of multiprocessing for FPS and scalability.
|
|
||||||
|
|
||||||
## Architecture Patterns
|
|
||||||
|
|
||||||
### Multiprocess Communication
|
|
||||||
|
|
||||||
Frigate uses **three primary IPC mechanisms** (NOT shared state):
|
|
||||||
|
|
||||||
1. **ZMQ pub/sub** (for one-way broadcasts):
|
|
||||||
|
|
||||||
- Config changes: `frigate/comms/config_updater.py` (PUB via `ipc:///tmp/cache/config`)
|
|
||||||
- Object detection signals: `frigate/comms/object_detector_signaler.py`
|
|
||||||
- Detection/event updates: detector signaler, event publishers
|
|
||||||
|
|
||||||
2. **ZMQ req/rep** (request-reply):
|
|
||||||
|
|
||||||
- `frigate/comms/inter_process.py` - processes request data and get responses via `ipc:///tmp/cache/comms`
|
|
||||||
- Used by `InterProcessRequestor` for sync queries
|
|
||||||
|
|
||||||
3. **Multiprocessing Queues** (frame data):
|
|
||||||
- `detection_queue`: Camera frames → object detectors
|
|
||||||
- `tracked_objects_queue`: Detected objects → event processor
|
|
||||||
- `timeline_queue`: Events → timeline storage
|
|
||||||
|
|
||||||
**Key pattern**: Each service publishes to ZMQ topics, others subscribe. Config changes fan out via ZMQ pub/sub to all processes without central coordination.
|
|
||||||
|
|
||||||
### Core Services (in `frigate/app.py` FrigateApp)
|
|
||||||
|
|
||||||
- **CameraMaintainer** (thread): Spawns camera capture/processing subprocess per camera
|
|
||||||
- **ObjectDetectProcess** (subprocess): Runs ML inference on queued frames
|
|
||||||
- **TrackedObjectProcessor** (thread): Receives detections, correlates into tracked objects, publishes events
|
|
||||||
- **EventProcessor** (thread): Manages event lifecycle, DB updates
|
|
||||||
- **RecordProcess** (subprocess): Manages video recording/retention
|
|
||||||
- **OutputProcess** (subprocess): Encodes/streams video
|
|
||||||
- **ReviewProcess** (subprocess): Processes review segments
|
|
||||||
- **EmbeddingProcess** (subprocess): Runs embeddings for semantic search/face/LPR
|
|
||||||
|
|
||||||
**Logging pattern**: Central `log.py` uses QueueListener to collect logs from all processes into one queue to avoid multiprocess logging chaos.
|
|
||||||
|
|
||||||
## Config System
|
|
||||||
|
|
||||||
Configuration is **Pydantic BaseModel** hierarchy:
|
|
||||||
|
|
||||||
- **Parsing**: YAML → Pydantic models with validators in `frigate/config/config.py`
|
|
||||||
- **Types**: `frigate/types.py` has shared enums (EventType, ObjectType, etc.)
|
|
||||||
- **Validation pattern**: Use `@field_validator` with `mode='before'` to transform/validate before assignment
|
|
||||||
- **Runtime values**: `RuntimeMotionConfig` applies frame shape transforms to masks
|
|
||||||
- **Key files**:
|
|
||||||
- `config.py` - main FrigateConfig entry point
|
|
||||||
- `camera/` - per-camera sub-configs (detect, record, snapshots, etc.)
|
|
||||||
- `classification.py` - face/LPR/audio/semantic search configs
|
|
||||||
|
|
||||||
**When adding config**: Create Pydantic model → add to parent config → update migrations if DB schema changes.
|
|
||||||
|
|
||||||
## Data Model & Persistence
|
|
||||||
|
|
||||||
**Database**: SQLite with custom `SqliteVecQueueDatabase` (vector support for embeddings)
|
|
||||||
|
|
||||||
- **Models** in `frigate/models.py`: Event, Timeline, Recordings, User, etc. (Peewee ORM)
|
|
||||||
- **Key tables**:
|
|
||||||
- `events` - detected objects (car, person, etc.) with retention policies
|
|
||||||
- `timeline` - events feed (entered_zone, audio, etc.)
|
|
||||||
- `recordings` - video segments with metadata
|
|
||||||
- `review_segments` - flagged clips for review
|
|
||||||
|
|
||||||
**Event lifecycle**:
|
|
||||||
|
|
||||||
1. TrackedObject detected → Event created with `false_positive=False`
|
|
||||||
2. EventProcessor updates Event (score, zones, clips, snapshots)
|
|
||||||
3. On object lost, Event gets `end_time` and is finalized
|
|
||||||
|
|
||||||
**Migrations**: Use Peewee migrations in `migrations/` - run via `peewee_migrate.Router`.
|
|
||||||
|
|
||||||
## Key Workflows
|
|
||||||
|
|
||||||
### Adding a New Detector Type
|
|
||||||
|
|
||||||
1. Create detector class in `frigate/detectors/plugins/` inheriting `DetectionApi`
|
|
||||||
2. Add config class in `frigate/detectors/detector_config.py`
|
|
||||||
3. Register in detector factory in `frigate/detectors/__init__.py`
|
|
||||||
4. Update `DEFAULT_DETECTORS` constant if it's the default
|
|
||||||
|
|
||||||
### Object Detection Pipeline
|
|
||||||
|
|
||||||
```
|
|
||||||
Camera subprocess → capture frames → motion detect →
|
|
||||||
queue frame to detection_queue →
|
|
||||||
ObjectDetectProcess (inference) →
|
|
||||||
TrackedObjectProcessor (correlate detections) →
|
|
||||||
Event + tracking + DB updates
|
|
||||||
```
|
|
||||||
|
|
||||||
### Recording Flow
|
|
||||||
|
|
||||||
- **24/7 recording**: Segments written every frame duration
|
|
||||||
- **Retention**: Deleted if no events + retention time elapsed
|
|
||||||
- **Cleanup**: `RecordingCleanup` task deletes old segments based on retention config
|
|
||||||
|
|
||||||
### Frontend Translation Pattern
|
|
||||||
|
|
||||||
- **Rule**: NEVER hardcode strings in `.ts/.tsx` files
|
|
||||||
- **Pattern**: Store strings in `web/src/locales/en.json` → import locale function → use in code
|
|
||||||
- **See**: `.cursor/rules/frontend-always-use-translation-files.mdc`
|
|
||||||
|
|
||||||
## Common Code Patterns
|
|
||||||
|
|
||||||
### Inter-process Config Updates
|
|
||||||
|
|
||||||
```python
|
|
||||||
# In detector/processor:
|
|
||||||
self.config_subscriber = ConfigSubscriber(config, [ConfigUpdateEnum.cameras])
|
|
||||||
|
|
||||||
# In main app (FrigateApp):
|
|
||||||
publisher = ConfigPublisher()
|
|
||||||
publisher.publish("cameras", new_config) # All subscribers notified
|
|
||||||
```
|
|
||||||
|
|
||||||
### Event Publishing
|
|
||||||
|
|
||||||
```python
|
|
||||||
from frigate.comms.events_updater import EventUpdatePublisher
|
|
||||||
|
|
||||||
publisher = EventUpdatePublisher()
|
|
||||||
publisher.publish({"camera": "cam1", "label": "person", ...})
|
|
||||||
```
|
|
||||||
|
|
||||||
### Shared Memory Frames
|
|
||||||
|
|
||||||
```python
|
|
||||||
from frigate.util.image import SharedMemoryFrameManager, UntrackedSharedMemory
|
|
||||||
|
|
||||||
frame_manager = SharedMemoryFrameManager()
|
|
||||||
shm = frame_manager.get(frame_id) # returns np.ndarray view
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing & Debugging
|
|
||||||
|
|
||||||
**Test structure**: `frigate/test/test_*.py` using Python `unittest`
|
|
||||||
|
|
||||||
- Run tests: `make run_tests` (builds Docker, runs in container)
|
|
||||||
- Key tests: config parsing, detector inference, frame processing
|
|
||||||
|
|
||||||
**Build targets** (Makefile):
|
|
||||||
|
|
||||||
- `make local` - builds Docker image locally with version
|
|
||||||
- `make debug` - builds with debug logging enabled
|
|
||||||
- `make run` - runs container with config volume mounted
|
|
||||||
|
|
||||||
**Debugging multiprocess issues**:
|
|
||||||
|
|
||||||
- Check `log_queue` output in `frigate/log.py`
|
|
||||||
- Enable `DEBUG` logging for specific modules in config
|
|
||||||
- Use `faulthandler.enable()` (already enabled in processes) for segfaults
|
|
||||||
|
|
||||||
## Important Conventions
|
|
||||||
|
|
||||||
- **Imports**: Run Ruff with isort (`extend-select = ["I"]`) - enforces import sorting
|
|
||||||
- **GPU/Acceleration**: Hardware detection in `frigate/util/services.py` (NVIDIA, Intel VAAPI, AMD, etc.)
|
|
||||||
- **Model paths**: Stored in `/config/model_cache/` (symlinked or volume mounted)
|
|
||||||
- **Recording paths**: `/media/frigate/recordings/` (clips in `clips/`, exports in `exports/`)
|
|
||||||
- **PID locking**: Use `setproctitle()` to name processes for debugging via `ps`
|
|
||||||
|
|
||||||
## Files to Know
|
|
||||||
|
|
||||||
| File | Purpose |
|
|
||||||
| ------------------------------------ | ---------------------------------------- |
|
|
||||||
| `frigate/app.py` | Main app startup, service orchestration |
|
|
||||||
| `frigate/camera/` | Camera subprocess, frame capture, motion |
|
|
||||||
| `frigate/track/object_processing.py` | Detection correlation, event publishing |
|
|
||||||
| `frigate/events/maintainer.py` | Event lifecycle management |
|
|
||||||
| `frigate/config/config.py` | Config parsing & validation |
|
|
||||||
| `frigate/comms/` | IPC (ZMQ pub/sub, req/rep) |
|
|
||||||
| `frigate/api/fastapi_app.py` | REST API setup |
|
|
||||||
| `frigate/models.py` | Database ORM models |
|
|
||||||
| `frigate/const.py` | Global constants (paths, defaults) |
|
|
||||||
|
|
||||||
## Gotchas & Common Mistakes
|
|
||||||
|
|
||||||
1. **Pickle compatibility**: Objects sent over multiprocess queues must be pickleable. Avoid lambdas, file handles.
|
|
||||||
2. **Config subscriptions**: Always check `mode='before'` in validators—Pydantic can be confusing.
|
|
||||||
3. **Event state confusion**: Events have transient state in `TrackedObjectProcessor` AND persistent state in DB—don't mix them.
|
|
||||||
4. **Motion masks**: Frame shape must be applied before creating `RuntimeMotionConfig`—validate in tests.
|
|
||||||
5. **ZMQ timing**: Topics must be subscribed BEFORE publisher sends; use small sleep if race condition suspected.
|
|
||||||
6. **Frontend strings**: Forgetting to use locale files breaks translations and fails linting.
|
|
||||||
|
|
||||||
## External Integration
|
|
||||||
|
|
||||||
- **MQTT**: Via `MqttClient` in `frigate/comms/mqtt.py` - publishes detections, accepts commands
|
|
||||||
- **Home Assistant**: Native integration via custom component (separate repo)
|
|
||||||
- **Frigate+**: Paid cloud sync service - `frigate/plus.py` handles API calls
|
|
||||||
- **Webhooks**: Event-triggered POST requests configured per-camera
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last updated**: Branch `review-stream-tweaks` | For architecture deep-dives, start with `frigate/app.py::FrigateApp.__init__()` to see all service wiring.
|
|
||||||
Loading…
Reference in New Issue
Block a user