mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-04-11 09:37:37 +03:00
detail stream settings
This commit is contained in:
parent
7aac6b4f21
commit
33e6a8cd0e
207
.github/copilot-instructions.md
vendored
Normal file
207
.github/copilot-instructions.md
vendored
Normal file
@ -0,0 +1,207 @@
|
||||
# Frigate AI Agent Instructions
|
||||
|
||||
## Project Overview
|
||||
|
||||
Frigate is a **local NVR with realtime AI object detection** for IP cameras. It's a multiprocess Python service that combines:
|
||||
|
||||
- Video capture and processing from IP cameras
|
||||
- Realtime object detection (TensorFlow, ONNX models on CPU/GPU/AI accelerators)
|
||||
- Recording with retention policies
|
||||
- Event tracking and review UI
|
||||
- REST API + MQTT integration
|
||||
|
||||
**Key architectural philosophy**: Minimize resource use by only running expensive detection where/when motion is detected. Heavy use of multiprocessing for FPS and scalability.
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### Multiprocess Communication
|
||||
|
||||
Frigate uses **three primary IPC mechanisms** (NOT shared state):
|
||||
|
||||
1. **ZMQ pub/sub** (for one-way broadcasts):
|
||||
|
||||
- Config changes: `frigate/comms/config_updater.py` (PUB via `ipc:///tmp/cache/config`)
|
||||
- Object detection signals: `frigate/comms/object_detector_signaler.py`
|
||||
- Detection/event updates: detector signaler, event publishers
|
||||
|
||||
2. **ZMQ req/rep** (request-reply):
|
||||
|
||||
- `frigate/comms/inter_process.py` - processes request data and get responses via `ipc:///tmp/cache/comms`
|
||||
- Used by `InterProcessRequestor` for sync queries
|
||||
|
||||
3. **Multiprocessing Queues** (frame data):
|
||||
- `detection_queue`: Camera frames → object detectors
|
||||
- `tracked_objects_queue`: Detected objects → event processor
|
||||
- `timeline_queue`: Events → timeline storage
|
||||
|
||||
**Key pattern**: Each service publishes to ZMQ topics, others subscribe. Config changes fan out via ZMQ pub/sub to all processes without central coordination.
|
||||
|
||||
### Core Services (in `frigate/app.py` FrigateApp)
|
||||
|
||||
- **CameraMaintainer** (thread): Spawns camera capture/processing subprocess per camera
|
||||
- **ObjectDetectProcess** (subprocess): Runs ML inference on queued frames
|
||||
- **TrackedObjectProcessor** (thread): Receives detections, correlates into tracked objects, publishes events
|
||||
- **EventProcessor** (thread): Manages event lifecycle, DB updates
|
||||
- **RecordProcess** (subprocess): Manages video recording/retention
|
||||
- **OutputProcess** (subprocess): Encodes/streams video
|
||||
- **ReviewProcess** (subprocess): Processes review segments
|
||||
- **EmbeddingProcess** (subprocess): Runs embeddings for semantic search/face/LPR
|
||||
|
||||
**Logging pattern**: Central `log.py` uses QueueListener to collect logs from all processes into one queue to avoid multiprocess logging chaos.
|
||||
|
||||
## Config System
|
||||
|
||||
Configuration is **Pydantic BaseModel** hierarchy:
|
||||
|
||||
- **Parsing**: YAML → Pydantic models with validators in `frigate/config/config.py`
|
||||
- **Types**: `frigate/types.py` has shared enums (EventType, ObjectType, etc.)
|
||||
- **Validation pattern**: Use `@field_validator` with `mode='before'` to transform/validate before assignment
|
||||
- **Runtime values**: `RuntimeMotionConfig` applies frame shape transforms to masks
|
||||
- **Key files**:
|
||||
- `config.py` - main FrigateConfig entry point
|
||||
- `camera/` - per-camera sub-configs (detect, record, snapshots, etc.)
|
||||
- `classification.py` - face/LPR/audio/semantic search configs
|
||||
|
||||
**When adding config**: Create Pydantic model → add to parent config → update migrations if DB schema changes.
|
||||
|
||||
## Data Model & Persistence
|
||||
|
||||
**Database**: SQLite with custom `SqliteVecQueueDatabase` (vector support for embeddings)
|
||||
|
||||
- **Models** in `frigate/models.py`: Event, Timeline, Recordings, User, etc. (Peewee ORM)
|
||||
- **Key tables**:
|
||||
- `events` - detected objects (car, person, etc.) with retention policies
|
||||
- `timeline` - events feed (entered_zone, audio, etc.)
|
||||
- `recordings` - video segments with metadata
|
||||
- `review_segments` - flagged clips for review
|
||||
|
||||
**Event lifecycle**:
|
||||
|
||||
1. TrackedObject detected → Event created with `false_positive=False`
|
||||
2. EventProcessor updates Event (score, zones, clips, snapshots)
|
||||
3. On object lost, Event gets `end_time` and is finalized
|
||||
|
||||
**Migrations**: Use Peewee migrations in `migrations/` - run via `peewee_migrate.Router`.
|
||||
|
||||
## Key Workflows
|
||||
|
||||
### Adding a New Detector Type
|
||||
|
||||
1. Create detector class in `frigate/detectors/plugins/` inheriting `DetectionApi`
|
||||
2. Add config class in `frigate/detectors/detector_config.py`
|
||||
3. Register in detector factory in `frigate/detectors/__init__.py`
|
||||
4. Update `DEFAULT_DETECTORS` constant if it's the default
|
||||
|
||||
### Object Detection Pipeline
|
||||
|
||||
```
|
||||
Camera subprocess → capture frames → motion detect →
|
||||
queue frame to detection_queue →
|
||||
ObjectDetectProcess (inference) →
|
||||
TrackedObjectProcessor (correlate detections) →
|
||||
Event + tracking + DB updates
|
||||
```
|
||||
|
||||
### Recording Flow
|
||||
|
||||
- **24/7 recording**: Segments written every frame duration
|
||||
- **Retention**: Deleted if no events + retention time elapsed
|
||||
- **Cleanup**: `RecordingCleanup` task deletes old segments based on retention config
|
||||
|
||||
### Frontend Translation Pattern
|
||||
|
||||
- **Rule**: NEVER hardcode strings in `.ts/.tsx` files
|
||||
- **Pattern**: Store strings in `web/src/locales/en.json` → import locale function → use in code
|
||||
- **See**: `.cursor/rules/frontend-always-use-translation-files.mdc`
|
||||
|
||||
## Common Code Patterns
|
||||
|
||||
### Inter-process Config Updates
|
||||
|
||||
```python
|
||||
# In detector/processor:
|
||||
self.config_subscriber = ConfigSubscriber(config, [ConfigUpdateEnum.cameras])
|
||||
|
||||
# In main app (FrigateApp):
|
||||
publisher = ConfigPublisher()
|
||||
publisher.publish("cameras", new_config) # All subscribers notified
|
||||
```
|
||||
|
||||
### Event Publishing
|
||||
|
||||
```python
|
||||
from frigate.comms.events_updater import EventUpdatePublisher
|
||||
|
||||
publisher = EventUpdatePublisher()
|
||||
publisher.publish({"camera": "cam1", "label": "person", ...})
|
||||
```
|
||||
|
||||
### Shared Memory Frames
|
||||
|
||||
```python
|
||||
from frigate.util.image import SharedMemoryFrameManager, UntrackedSharedMemory
|
||||
|
||||
frame_manager = SharedMemoryFrameManager()
|
||||
shm = frame_manager.get(frame_id) # returns np.ndarray view
|
||||
```
|
||||
|
||||
## Testing & Debugging
|
||||
|
||||
**Test structure**: `frigate/test/test_*.py` using Python `unittest`
|
||||
|
||||
- Run tests: `make run_tests` (builds Docker, runs in container)
|
||||
- Key tests: config parsing, detector inference, frame processing
|
||||
|
||||
**Build targets** (Makefile):
|
||||
|
||||
- `make local` - builds Docker image locally with version
|
||||
- `make debug` - builds with debug logging enabled
|
||||
- `make run` - runs container with config volume mounted
|
||||
|
||||
**Debugging multiprocess issues**:
|
||||
|
||||
- Check `log_queue` output in `frigate/log.py`
|
||||
- Enable `DEBUG` logging for specific modules in config
|
||||
- Use `faulthandler.enable()` (already enabled in processes) for segfaults
|
||||
|
||||
## Important Conventions
|
||||
|
||||
- **Imports**: Run Ruff with isort (`extend-select = ["I"]`) - enforces import sorting
|
||||
- **GPU/Acceleration**: Hardware detection in `frigate/util/services.py` (NVIDIA, Intel VAAPI, AMD, etc.)
|
||||
- **Model paths**: Stored in `/config/model_cache/` (symlinked or volume mounted)
|
||||
- **Recording paths**: `/media/frigate/recordings/` (clips in `clips/`, exports in `exports/`)
|
||||
- **PID locking**: Use `setproctitle()` to name processes for debugging via `ps`
|
||||
|
||||
## Files to Know
|
||||
|
||||
| File | Purpose |
|
||||
| ------------------------------------ | ---------------------------------------- |
|
||||
| `frigate/app.py` | Main app startup, service orchestration |
|
||||
| `frigate/camera/` | Camera subprocess, frame capture, motion |
|
||||
| `frigate/track/object_processing.py` | Detection correlation, event publishing |
|
||||
| `frigate/events/maintainer.py` | Event lifecycle management |
|
||||
| `frigate/config/config.py` | Config parsing & validation |
|
||||
| `frigate/comms/` | IPC (ZMQ pub/sub, req/rep) |
|
||||
| `frigate/api/fastapi_app.py` | REST API setup |
|
||||
| `frigate/models.py` | Database ORM models |
|
||||
| `frigate/const.py` | Global constants (paths, defaults) |
|
||||
|
||||
## Gotchas & Common Mistakes
|
||||
|
||||
1. **Pickle compatibility**: Objects sent over multiprocess queues must be pickleable. Avoid lambdas, file handles.
|
||||
2. **Config subscriptions**: Always check `mode='before'` in validators—Pydantic can be confusing.
|
||||
3. **Event state confusion**: Events have transient state in `TrackedObjectProcessor` AND persistent state in DB—don't mix them.
|
||||
4. **Motion masks**: Frame shape must be applied before creating `RuntimeMotionConfig`—validate in tests.
|
||||
5. **ZMQ timing**: Topics must be subscribed BEFORE publisher sends; use small sleep if race condition suspected.
|
||||
6. **Frontend strings**: Forgetting to use locale files breaks translations and fails linting.
|
||||
|
||||
## External Integration
|
||||
|
||||
- **MQTT**: Via `MqttClient` in `frigate/comms/mqtt.py` - publishes detections, accepts commands
|
||||
- **Home Assistant**: Native integration via custom component (separate repo)
|
||||
- **Frigate+**: Paid cloud sync service - `frigate/plus.py` handles API calls
|
||||
- **Webhooks**: Event-triggered POST requests configured per-camera
|
||||
|
||||
---
|
||||
|
||||
**Last updated**: Branch `review-stream-tweaks` | For architecture deep-dives, start with `frigate/app.py::FrigateApp.__init__()` to see all service wiring.
|
||||
@ -91,8 +91,8 @@ export default function AnnotationOffsetSlider({ className }: Props) {
|
||||
<div className="w-full flex-1 landscape:flex">
|
||||
<Slider
|
||||
value={[annotationOffset]}
|
||||
min={-1500}
|
||||
max={1500}
|
||||
min={-2500}
|
||||
max={2500}
|
||||
step={50}
|
||||
onValueChange={handleChange}
|
||||
/>
|
||||
|
||||
1023
web/src/components/overlay/detail/TrackingDetails.tsx.bak
Normal file
1023
web/src/components/overlay/detail/TrackingDetails.tsx.bak
Normal file
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user