mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-09-30 11:56:49 +03:00
Compare commits
10
Commits
514b5ee73b
...
7448198e79
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7448198e79 | ||
|
|
775ce22204 | ||
|
|
6f24f5a595 | ||
|
|
65af0b1351 | ||
|
|
fcd05ec7bc | ||
|
|
5f6043aa92 | ||
|
|
da4037eb52 | ||
|
|
f3c09ae169 | ||
|
|
f6596ac7b0 | ||
|
|
401e19f5fe |
@@ -339,7 +339,7 @@ detect:
|
||||
# especially when using separate streams for detect and record.
|
||||
# Use this setting to make the timeline bounding boxes more closely align
|
||||
# with the recording. The value can be positive or negative.
|
||||
# TIP: Imagine there is an tracked object clip with a person walking from left to right.
|
||||
# TIP: Imagine there is a tracked object clip with a person walking from left to right.
|
||||
# If the tracked object lifecycle bounding box is consistently to the left of the person
|
||||
# then the value should be decreased. Similarly, if a person is walking from
|
||||
# left to right and the bounding box is consistently ahead of the person
|
||||
|
||||
@@ -67,15 +67,21 @@ This section can be used to set environment variables for those unable to modify
|
||||
|
||||
Variables prefixed with `FRIGATE_` can be referenced in config fields that support environment variable substitution (such as MQTT host and credentials, camera stream URLs, and ONVIF host and credentials) using the `{FRIGATE_VARIABLE_NAME}` syntax.
|
||||
|
||||
:::note
|
||||
|
||||
The `go2rtc` section is an exception. go2rtc runs as a separate process, so its stream definitions can only be substituted with variables that exist in the container's environment (set via Docker `-e`, the `environment:` section of `docker-compose.yml`, or Docker secrets). Variables defined in the `environment_vars` block above are not available to go2rtc streams. Home Assistant app users, who cannot set container environment variables, must instead put credentials directly in their go2rtc stream URLs.
|
||||
|
||||
:::
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > System > Environment variables" /> to add or edit environment variables.
|
||||
|
||||
| Field | Description |
|
||||
| --------- | --------------------------------------------------------- |
|
||||
| **Key** | The environment variable name (e.g., `FRIGATE_MQTT_USER`) |
|
||||
| **Value** | The value for the variable |
|
||||
| Field | Description |
|
||||
| ----------------- | --------------------------------------------------------- |
|
||||
| **Variable name** | The environment variable name (e.g., `FRIGATE_MQTT_USER`) |
|
||||
| **Value** | The value for the variable |
|
||||
|
||||
Variables defined here can be referenced elsewhere in your configuration using the `{FRIGATE_VARIABLE_NAME}` syntax.
|
||||
|
||||
|
||||
@@ -78,7 +78,7 @@ cameras:
|
||||
|
||||
### Configuring Minimum Volume
|
||||
|
||||
The audio detector uses volume levels in the same way that motion in a camera feed is used for object detection. This means that Frigate will not run audio detection unless the audio volume is above the configured level in order to reduce resource usage. Audio levels can vary widely between camera models so it is important to run tests to see what volume levels are. The [Debug view](/usage/live#the-single-camera-view) in the Frigate UI has an Audio tab for cameras that have the `audio` role assigned where a graph and the current levels are is displayed. The `min_volume` parameter should be set to the minimum the `RMS` level required to run audio detection.
|
||||
The audio detector uses volume levels in the same way that motion in a camera feed is used for object detection. This means that Frigate will not run audio detection unless the audio volume is above the configured level in order to reduce resource usage. Audio levels can vary widely between camera models so it is important to run tests to see what volume levels are. The [Debug view](/usage/live#the-single-camera-view) in the Frigate UI has an Audio tab for cameras that have the `audio` role assigned where a graph and the current levels are displayed. The `min_volume` parameter should be set to the minimum the `RMS` level required to run audio detection.
|
||||
|
||||
:::tip
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ title: Camera Autotracking
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
import FaqItem from "@site/src/components/FaqItem";
|
||||
|
||||
An ONVIF-capable, PTZ (pan-tilt-zoom) camera that supports relative movement within the field of view (FOV) can be configured to automatically track moving objects and keep them in the center of the frame.
|
||||
|
||||
@@ -187,30 +188,96 @@ In security and surveillance, it's common to use "spotter" cameras in combinatio
|
||||
|
||||
## Troubleshooting and FAQ
|
||||
|
||||
### The autotracker loses track of my object. Why?
|
||||
### Camera Compatibility
|
||||
|
||||
<FaqItem id="which-ptz-camera-should-i-use-for-autotracking" question="Which PTZ camera should I use for autotracking?">
|
||||
|
||||
See the community-maintained list of [ONVIF PTZ camera recommendations](cameras.md#onvif-ptz-camera-recommendations) for cameras and brands reported to work (and not work) with autotracking. This is not an exhaustive list that is frequently updated, so other cameras not listed may also work well. Frigate's autotracking was developed with a Dahua SD1A404XB-GNR (now sold as the EmpireTech PTZ1A4M-4X-S2), and Dahua / EmpireTech PTZs are the most consistently reported as working well.
|
||||
|
||||
When comparing models:
|
||||
|
||||
- Verify ONVIF support first. See [Checking ONVIF camera support](#checking-onvif-camera-support) above.
|
||||
- Favor a camera with a fast PTZ motor. Cameras with slow motors may fail [calibration](#calibration) and will struggle to keep up with objects that move across the field of view quickly.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="does-autotracking-work-with-reolink-ptz-cameras" question="Does autotracking work with Reolink PTZ cameras?">
|
||||
|
||||
No. Reolink cameras (including the TrackMix series) lack the ONVIF FOV RelativeMove firmware support that Frigate's autotracker requires, so autotracking will not work with any current Reolink PTZ. Their video streams and basic PTZ controls still work in Frigate. If you want object tracking on a Reolink PTZ, you will need to use the tracking feature built into the camera's firmware, which is proprietary and operates independently of Frigate.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="im-seeing-an-error-in-the-logs-that-my-camera-is-still-in-onvif-moving-status-what-does-this-mean" question={"I'm seeing an error in the logs that my camera \"is still in ONVIF 'MOVING' status.\" What does this mean?"}>
|
||||
|
||||
There are two possible known reasons for this (and perhaps others yet unknown): a slow PTZ motor or buggy camera firmware. Frigate uses an ONVIF parameter provided by the camera, `MoveStatus`, to determine when the PTZ's motor is moving or idle. According to some users, Hikvision PTZs (even with the latest firmware), are not updating this value after PTZ movement. Unfortunately there is no workaround to this bug in Hikvision firmware, so autotracking will not function correctly and should be disabled in your config. This may also be the case with other non-Hikvision cameras utilizing Hikvision firmware, such as some Annke models. In rare cases the vendor may provide fixed firmware on request; for example, Annke has supplied firmware that resolves this for the CZ504 (see the [camera recommendations list](cameras.md#onvif-ptz-camera-recommendations)).
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="calibration-seems-to-have-completed-but-the-camera-is-not-actually-moving-to-track-my-object-why" question="Calibration seems to have completed, but the camera is not actually moving to track my object. Why?">
|
||||
|
||||
Some cameras have firmware that reports that FOV RelativeMove, the ONVIF command that Frigate uses for autotracking, is supported. However, if the camera does not pan or tilt when an object comes into the required zone, your camera's firmware does not actually support FOV RelativeMove. One such camera is the Uniview IPC672LR-AX4DUPK. It actually moves its zoom motor instead of panning and tilting and does not follow the ONVIF standard whatsoever.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Calibration Issues
|
||||
|
||||
<FaqItem id="i-tried-calibrating-my-camera-but-the-logs-show-that-it-is-stuck-at-0-and-frigate-is-not-starting-up" question="I tried calibrating my camera, but the logs show that it is stuck at 0% and Frigate is not starting up.">
|
||||
|
||||
This is often caused by the same reason as the "MOVING" status error above - the `MoveStatus` ONVIF parameter is not changing due to a bug in your camera's firmware. Also, see the note above: Frigate's web UI and all other cameras will be unresponsive while calibration is in progress. This is expected and normal. But if you don't see log entries every few seconds for calibration progress, your camera is not compatible with autotracking.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="frigate-reports-an-error-saying-that-calibration-has-failed-why" question="Frigate reports an error saying that calibration has failed. Why?">
|
||||
|
||||
Calibration measures the amount of time it takes for Frigate to make a series of movements with your PTZ. This error message is recorded in the log if these values are too high for Frigate to support calibrated autotracking. This is often the case when your camera's motor or network connection is too slow or your camera's firmware doesn't report the motor status in a timely manner.
|
||||
|
||||
Some things to try:
|
||||
|
||||
- If your camera's firmware has a PTZ or motor speed setting, set it to the fastest available speed and calibrate again.
|
||||
- Run without calibration: remove the `movement_weights` line from your config, set `calibrate_on_startup` to `False`, and restart.
|
||||
|
||||
If calibration consistently fails, this often means your camera's motor is too slow and autotracking will behave unpredictably or won't be able to keep up with moving objects.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="autotracking-is-erratic-or-moves-the-camera-in-the-wrong-direction" question="Autotracking is erratic, moves the camera in the wrong direction, or zooms past my object. Why?">
|
||||
|
||||
Frigate uses the `movement_weights` measured during calibration to predict how far the camera needs to move to keep an object centered, so inaccurate values produce movements that don't seem to make sense: overshooting, moving the opposite direction, or zooming in on an object's last known position and losing it entirely. This is almost always a calibration issue.
|
||||
|
||||
- Remove the `movement_weights` entry from your config and restart Frigate to run without calibration. If tracking improves, try recalibrating.
|
||||
- Recalibrate several times. The `movement_weights` values should be close to each other after each run. If they vary significantly between runs, your camera may not be reporting its motor status reliably, and you may get better results without calibration.
|
||||
- If you are using zooming, a high `zoom_factor` can cause the camera to zoom in too far and lose the object. Try a lower value.
|
||||
|
||||
Remember to recalibrate whenever you change your `return_preset`, change your camera's detect `fps`, or enable zooming after calibrating with it disabled.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Tracking Behavior
|
||||
|
||||
<FaqItem id="the-autotracker-loses-track-of-my-object-why" question="The autotracker loses track of my object. Why?">
|
||||
|
||||
There are many reasons this could be the case. If you are using experimental zooming, your `zoom_factor` value might be too high, the object might be traveling too quickly, the scene might be too dark, there are not enough details in the scene (for example, a PTZ looking down on a driveway or other monotone background without a sufficient number of hard edges or corners), or the scene is otherwise less than optimal for Frigate to maintain tracking.
|
||||
|
||||
Your camera's shutter speed may also be set too low so that blurring occurs with motion. Check your camera's firmware to see if you can increase the shutter speed.
|
||||
|
||||
Watching Frigate's debug view can help to determine a possible cause. The autotracked object will have a thicker colored box around it.
|
||||
Watching Frigate's debug view can help to determine a possible cause. The autotracked object will have a thicker colored box around it. If the camera consistently zooms in on the object and then loses it, see [Autotracking is erratic, moves the camera in the wrong direction, or zooms past my object. Why?](#autotracking-is-erratic-or-moves-the-camera-in-the-wrong-direction) above.
|
||||
|
||||
### I'm seeing an error in the logs that my camera "is still in ONVIF 'MOVING' status." What does this mean?
|
||||
</FaqItem>
|
||||
|
||||
There are two possible known reasons for this (and perhaps others yet unknown): a slow PTZ motor or buggy camera firmware. Frigate uses an ONVIF parameter provided by the camera, `MoveStatus`, to determine when the PTZ's motor is moving or idle. According to some users, Hikvision PTZs (even with the latest firmware), are not updating this value after PTZ movement. Unfortunately there is no workaround to this bug in Hikvision firmware, so autotracking will not function correctly and should be disabled in your config. This may also be the case with other non-Hikvision cameras utilizing Hikvision firmware.
|
||||
|
||||
### I tried calibrating my camera, but the logs show that it is stuck at 0% and Frigate is not starting up.
|
||||
|
||||
This is often caused by the same reason as above - the `MoveStatus` ONVIF parameter is not changing due to a bug in your camera's firmware. Also, see the note above: Frigate's web UI and all other cameras will be unresponsive while calibration is in progress. This is expected and normal. But if you don't see log entries every few seconds for calibration progress, your camera is not compatible with autotracking.
|
||||
|
||||
### I'm seeing this error in the logs: "Autotracker: motion estimator couldn't get transformations". What does this mean?
|
||||
<FaqItem id="im-seeing-this-error-in-the-logs-autotracker-motion-estimator-couldnt-get-transformations-what-does-this-mean" question={"I'm seeing this error in the logs: \"Autotracker: motion estimator couldn't get transformations\". What does this mean?"}>
|
||||
|
||||
To maintain object tracking during PTZ moves, Frigate tracks the motion of your camera based on the details of the frame. If you are seeing this message, it could mean that your `zoom_factor` may be set too high, the scene around your detected object does not have enough details (like hard edges or color variations), or your camera's shutter speed is too slow and motion blur is occurring. Try reducing `zoom_factor`, finding a way to alter the scene around your object, or changing your camera's shutter speed.
|
||||
|
||||
### Calibration seems to have completed, but the camera is not actually moving to track my object. Why?
|
||||
</FaqItem>
|
||||
|
||||
Some cameras have firmware that reports that FOV RelativeMove, the ONVIF command that Frigate uses for autotracking, is supported. However, if the camera does not pan or tilt when an object comes into the required zone, your camera's firmware does not actually support FOV RelativeMove. One such camera is the Uniview IPC672LR-AX4DUPK. It actually moves its zoom motor instead of panning and tilting and does not follow the ONVIF standard whatsoever.
|
||||
<FaqItem id="why-does-object-detection-pause-briefly-when-the-camera-moves" question="Why does object detection pause briefly when the camera moves?">
|
||||
|
||||
### Frigate reports an error saying that calibration has failed. Why?
|
||||
When the PTZ moves, the entire frame changes at once. Frigate's motion detection treats sudden scene-wide changes (like a lightning flash, an infrared mode switch, or a camera move) specially and pauses detection momentarily until the scene stabilizes. This is expected and normal, and detection resumes shortly after the camera stops moving. If detection does not resume once the camera is stationary, use the [debug view](/usage/live#the-single-camera-view) to see what is happening.
|
||||
|
||||
Calibration measures the amount of time it takes for Frigate to make a series of movements with your PTZ. This error message is recorded in the log if these values are too high for Frigate to support calibrated autotracking. This is often the case when your camera's motor or network connection is too slow or your camera's firmware doesn't report the motor status in a timely manner. You can try running without calibration (just remove the `movement_weights` line from your config and restart), but if calibration fails, this often means that autotracking will behave unpredictably.
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="can-i-turn-autotracking-on-and-off-automatically" question="Can I turn autotracking on and off automatically?">
|
||||
|
||||
Yes. Autotracking can be toggled per camera at runtime over MQTT with the [`frigate/<camera_name>/ptz_autotracker/set`](../integrations/mqtt.md#frigatecamera_nameptz_autotrackerset) topic, and the [Home Assistant integration](../integrations/home-assistant.md) exposes a switch for it. This pairs well with the "spotter" camera automations described in [Usage applications](#usage-applications) above, for example only enabling autotracking at night or when nobody is home.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
@@ -156,7 +156,7 @@ Reolink has many different camera models with inconsistently supported features
|
||||
| 6MP or higher | Latest (ex: Duo3, CX-8##) | http-flv with ffmpeg 8.0, or rtsp | This uses the new http-flv-enhanced over H265 which requires ffmpeg 8.0 (Frigate's default) |
|
||||
| 6MP or higher | Older (ex: RLC-8##) | rtsp | |
|
||||
|
||||
Frigate works much better with newer reolink cameras that are setup with the below options:
|
||||
Frigate works much better with newer Reolink cameras that are setup with the below options:
|
||||
|
||||
If available, recommended settings are:
|
||||
|
||||
@@ -165,7 +165,7 @@ If available, recommended settings are:
|
||||
|
||||
#### Setup via the Add Camera Wizard
|
||||
|
||||
The Add Camera Wizard is the recommended way to add a standard Reolink camera. Before starting, make sure HTTP is enabled in the camera's advanced network settings. The wizard uses the camera's HTTP API to determine its resolution and choose the recommended stream type from the table above.
|
||||
The Add Camera Wizard is the recommended way to add a standard Reolink camera. Before starting, make sure [HTTP is enabled](https://support.reolink.com/articles/360003452893-How-to-Access-Reolink-Cameras-NVRs-Home-Hub-Locally-via-Web-Browsers/) in the camera's advanced network settings. The wizard uses the camera's HTTP API to determine its resolution and choose the recommended stream type from the table above.
|
||||
|
||||
1. Click **Add Camera** in <NavPath path="Settings > Global configuration > Camera management" />.
|
||||
2. Choose **Manual selection** as the stream detection method and select **Reolink** as the camera brand.
|
||||
@@ -192,7 +192,7 @@ Reolink's latest cameras support two way audio via go2rtc and other applications
|
||||
|
||||
NOTE: The RTSP stream can not be prefixed with `ffmpeg:`, as go2rtc needs to handle the stream to support two way audio.
|
||||
|
||||
Ensure HTTP is enabled in the camera's advanced network settings. To use two way talk with Frigate, see the [Live view documentation](/configuration/live#two-way-talk).
|
||||
Ensure [HTTP is enabled](https://support.reolink.com/articles/360003452893-How-to-Access-Reolink-Cameras-NVRs-Home-Hub-Locally-via-Web-Browsers/) in the camera's advanced network settings. To use two way talk with Frigate, see the [Live view documentation](/configuration/live#two-way-talk).
|
||||
|
||||
:::
|
||||
|
||||
@@ -204,7 +204,7 @@ go2rtc:
|
||||
- "ffmpeg:http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=username&password=password#video=copy#audio=copy#audio=opus"
|
||||
your_reolink_camera_sub:
|
||||
- "ffmpeg:http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=username&password=password"
|
||||
# example for connectin to a Reolink camera that supports two way talk
|
||||
# example for connecting to a Reolink camera that supports two way talk
|
||||
your_reolink_camera_twt:
|
||||
- "ffmpeg:http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=username&password=password#video=copy#audio=copy#audio=opus"
|
||||
- "rtsp://username:password@reolink_ip/Preview_01_sub"
|
||||
@@ -249,7 +249,7 @@ cameras:
|
||||
|
||||
:::note
|
||||
|
||||
Unifi G5s cameras and newer need a Unifi Protect server to enable rtsps stream, it's not posible to enable it in standalone mode.
|
||||
Unifi G5s cameras and newer need a Unifi Protect server to enable rtsps stream, it's not possible to enable it in standalone mode.
|
||||
|
||||
:::
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ title: Face Recognition
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
import FaqItem from "@site/src/components/FaqItem";
|
||||
|
||||
Face recognition identifies known individuals by matching detected faces with previously learned facial data. When a known `person` is recognized, their name will be added as a `sub_label`. This information is included in the UI, filters, as well as in notifications.
|
||||
|
||||
@@ -151,6 +152,14 @@ Follow these steps to begin:
|
||||
|
||||
## Creating a Robust Training Set
|
||||
|
||||
:::tip
|
||||
|
||||
**The short version:** Start with a few clear, front-facing photos of each person. As faces are detected in the Recent Recognitions tab, train clear images that scored lower, adding variety (different angles, lighting, and expressions) slowly. Diversity matters far more than volume, and low-quality images hurt recognition more than they help.
|
||||
|
||||
For a step-by-step narrative of these best practices (and the same principles applied to state and object classification), see the [Frigate Tips: Best Practices for Training](https://github.com/blakeblackshear/frigate/discussions/21374) discussion.
|
||||
|
||||
:::
|
||||
|
||||
The number of images needed for a sufficient training set for face recognition varies depending on several factors:
|
||||
|
||||
- Diversity of the dataset: A dataset with diverse images, including variations in lighting, pose, and facial expressions, will require fewer images per person than a less diverse dataset.
|
||||
@@ -181,9 +190,27 @@ When choosing images to include in the face training set it is recommended to al
|
||||
|
||||
The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.
|
||||
|
||||
Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. While each image can be used to train the system for a specific person, not all images are suitable for training.
|
||||
Each face image is labeled with a name (or `Unknown`) along with the confidence score of that recognition attempt. Images are grouped by the person they were matched against, not by who they actually are, so a group labeled with a person's name can contain a crop that is really someone else but happened to score as a partial match. The name and score shown on each individual crop describe that single attempt.
|
||||
|
||||
Refer to the guidelines below for best practices on selecting images for training.
|
||||
While each image can be used to train the system for a specific person, not all images are suitable for training. Refer to the guidelines below for best practices on selecting images for training.
|
||||
|
||||
### How Frigate Decides Who a Person Is
|
||||
|
||||
Recognition does not happen one frame at a time. While a `person` is in view, Frigate runs face recognition on many frames, not just a single frame. The final `sub_label` is decided from all of those attempts together, weighted by the area of each face (larger, closer faces count more), not from any single frame.
|
||||
|
||||
This has a few practical consequences:
|
||||
|
||||
- A handful of wrong guesses on blurry or distant frames usually do not change the result. If Frigate sees a person as "Tom, Tom, Sam, Tom, Tom," it will still conclude the person was Tom.
|
||||
- The goal is not for every individual face crop to be correct. The goal is for each person to be recognized correctly overall, across all the faces captured while they were present.
|
||||
- A single very high confidence match will not by itself assign a sub label. Recognition must be consistent. See [I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?](#i-see-scores-above-the-threshold-in-the-recent-recognitions-tab-but-a-sub-label-wasnt-assigned) below.
|
||||
|
||||
### Which Faces Are Worth Training?
|
||||
|
||||
Whether a face is worth training has little to do with what it was recognized as. A crop is a good training candidate when all of these are true:
|
||||
|
||||
- It did not already score high and correctly. Faces that are already recognized confidently add little and increase the risk of over-fitting.
|
||||
- It is clear enough to be useful: not blurry, not heavily off-axis, not infrared (gray-scale). If it is hard for you to make out the face, it will not help the model.
|
||||
- It adds something new: a different angle, lighting, expression, or distance than what you already have.
|
||||
|
||||
### Step 1 - Building a Strong Foundation
|
||||
|
||||
@@ -199,7 +226,9 @@ Once front-facing images are performing well, start choosing slightly off-angle
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I debug Face Recognition issues?
|
||||
### Getting Recognition Working
|
||||
|
||||
<FaqItem id="how-do-i-debug-face-recognition-issues" question="How do I debug Face Recognition issues?">
|
||||
|
||||
Start with the [Usage](#usage) section and re-read the [Model Requirements](#model-requirements) above.
|
||||
|
||||
@@ -217,21 +246,47 @@ Start with the [Usage](#usage) section and re-read the [Model Requirements](#mod
|
||||
- Make sure you have trained at least one face per the recommendations above.
|
||||
- Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration).
|
||||
|
||||
### Detection does not work well with blurry images?
|
||||
</FaqItem>
|
||||
|
||||
Accuracy is definitely a going to be improved with higher quality cameras / streams. It is important to look at the DORI (Detection Observation Recognition Identification) range of your camera, if that specification is posted. This specification explains the distance from the camera that a person can be detected, observed, recognized, and identified. The identification range is the most relevant here, and the distance listed by the camera is the furthest that face recognition will realistically work.
|
||||
<FaqItem id="does-face-recognition-run-on-the-recording-stream" question="Does face recognition run on the recording stream?">
|
||||
|
||||
Face recognition does not run on the recording stream, this would be suboptimal for many reasons:
|
||||
|
||||
1. The latency of accessing the recordings means the notifications would not include the names of recognized people because recognition would not complete until after.
|
||||
2. The embedding models used run on a set image size, so larger images will be scaled down to match this anyway.
|
||||
3. Motion clarity is much more important than extra pixels, over-compression and motion blur are much more detrimental to results than resolution.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Improving Accuracy and Training
|
||||
|
||||
<FaqItem id="detection-does-not-work-well-with-blurry-images" question="Detection does not work well with blurry images?">
|
||||
|
||||
Accuracy is definitely going to be improved with higher quality cameras / streams. It is important to look at the DORI (Detection Observation Recognition Identification) range of your camera, if that specification is posted. This specification explains the distance from the camera that a person can be detected, observed, recognized, and identified. The identification range is the most relevant here, and the distance listed by the camera is the furthest that face recognition will realistically work.
|
||||
|
||||
Some users have also noted that setting the stream in camera firmware to a constant bit rate (CBR) leads to better image clarity than with a variable bit rate (VBR).
|
||||
|
||||
### Why can't I bulk upload photos?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="can-i-train-faces-for-people-who-only-appear-at-night" question="Can I train faces for people who only appear at night?">
|
||||
|
||||
The embedding models are trained on color images, so gray-scale and infrared (IR) faces sit in a different feature distribution and are more easily confused with other people. Prefer color images, and avoid mixing gray-scale samples in early while you are building a foundation. If someone only ever appears at night, gray-scale training is acceptable, but keep those samples limited and as clear as possible, and add them only once color recognition is stable for your other people.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-cant-i-bulk-upload-photos" question="Why can't I bulk upload photos?">
|
||||
|
||||
It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to over-fitting in that particular scenario and hurt recognition performance.
|
||||
|
||||
### Why can't I bulk reprocess faces?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-cant-i-bulk-reprocess-faces" question="Why can't I bulk reprocess faces?">
|
||||
|
||||
Face embedding models work by breaking apart faces into different features. This means that when reprocessing an image, only images from a similar angle will have its score affected.
|
||||
|
||||
### Why do unknown people score similarly to known people?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-do-unknown-people-score-similarly-to-known-people" question="Why do unknown people score similarly to known people?">
|
||||
|
||||
This can happen for a few different reasons, but this is usually an indicator that the training set needs to be improved. This is often related to over-fitting:
|
||||
|
||||
@@ -243,31 +298,52 @@ Review your face collections and remove most of the unclear or low-quality image
|
||||
|
||||
Avoid training on images that already score highly, as this can lead to over-fitting. Instead, focus on relatively clear images that score lower (ideally with different lighting, angles, and conditions) to help the model generalize more effectively.
|
||||
|
||||
### Frigate misidentified a face. Can I tell it that a face is "not" a specific person?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="should-i-correct-a-face-that-was-recognized-as-the-wrong-person" question="Should I correct a face that was recognized as the wrong person?">
|
||||
|
||||
Only if it is a good image. Reassigning a face does add it to that person's training set, but two things are true at once:
|
||||
|
||||
- Reassigning a single misclassified frame has a small effect. The image is weighted against every other sample for that person, so correcting 1 frame out of 20 will not move recognition much. Occasional wrong guesses on poor frames are normal and do not need to be fixed.
|
||||
- Reassigning a poor image (blurry, off-angle, low-resolution, gray-scale) can hurt more than the misidentification did, because low-quality samples degrade recognition for that whole person.
|
||||
|
||||
So the decision is about image quality, not about the wrong label. If the crop is clear, well-lit, and reasonably front-facing, and it scored low or was wrong, assigning it to the correct person is useful. If you can barely make out the face yourself, ignore it; do not train it just to correct the label.
|
||||
|
||||
If a person is repeatedly misidentified, do not keep reassigning the same frame. Instead, remove low-quality or misleading images and add a few high-quality samples to the correct person. See [Why do unknown people score similarly to known people?](#why-do-unknown-people-score-similarly-to-known-people) above.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="frigate-misidentified-a-face-can-i-tell-it-that-a-face-is-not-a-specific-person" question={'Frigate misidentified a face. Can I tell it that a face is "not" a specific person?'}>
|
||||
|
||||
No, face recognition does not support negative training (i.e., explicitly telling it who someone is _not_). Instead, the best approach is to improve the training data by using a more diverse and representative set of images for each person.
|
||||
For more guidance, refer to the section above on improving recognition accuracy.
|
||||
|
||||
### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?
|
||||
This also applies to a stranger who is repeatedly matched to a known person (for example, a delivery driver recognized as you). Do not create a profile for them and do not reassign their faces to yourself, as this pollutes your training set and makes recognition worse. Leave the detection as unknown and improve the known person's training set instead. Face recognition learns who someone is, not who they are not.
|
||||
|
||||
The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
|
||||
</FaqItem>
|
||||
|
||||
### Can I use other face recognition software like DoubleTake at the same time as the built in face recognition?
|
||||
<FaqItem id="i-see-scores-above-the-threshold-in-the-recent-recognitions-tab-but-a-sub-label-wasnt-assigned" question="I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?">
|
||||
|
||||
Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Compatibility and Maintenance
|
||||
|
||||
<FaqItem id="can-i-use-other-face-recognition-software-like-doubletake-at-the-same-time-as-the-built-in-face-recognition" question="Can I use other face recognition software like DoubleTake at the same time as the built in face recognition?">
|
||||
|
||||
No, using another face recognition service will interfere with Frigate's built in face recognition. When using double-take the sub_label feature must be disabled if the built in face recognition is also desired.
|
||||
|
||||
### Does face recognition run on the recording stream?
|
||||
</FaqItem>
|
||||
|
||||
Face recognition does not run on the recording stream, this would be suboptimal for many reasons:
|
||||
|
||||
1. The latency of accessing the recordings means the notifications would not include the names of recognized people because recognition would not complete until after.
|
||||
2. The embedding models used run on a set image size, so larger images will be scaled down to match this anyway.
|
||||
3. Motion clarity is much more important than extra pixels, over-compression and motion blur are much more detrimental to results than resolution.
|
||||
|
||||
### I get an unknown error when taking a photo directly with my iPhone
|
||||
<FaqItem id="i-get-an-unknown-error-when-taking-a-photo-directly-with-my-iphone" question="I get an unknown error when taking a photo directly with my iPhone">
|
||||
|
||||
By default iOS devices will use HEIC (High Efficiency Image Container) for images, but this format is not supported for uploads. Choosing `large` as the format instead of `original` will use JPG which will work correctly.
|
||||
|
||||
### How can I delete the face database and start over?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="how-can-i-delete-the-face-database-and-start-over" question="How can I delete the face database and start over?">
|
||||
|
||||
Frigate does not store anything in its database related to face recognition. You can simply delete all of your faces through the Frigate UI or remove the contents of the `/media/frigate/clips/faces` directory.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
@@ -67,4 +67,4 @@ If your stream won't play, has no audio, uses excessive CPU, or otherwise misbeh
|
||||
|
||||
## Homekit Configuration
|
||||
|
||||
To add camera streams to Homekit Frigate must be configured in docker to use `host` networking mode. Once that is done, you can use the go2rtc WebUI (accessed via port 1984, which is disabled by default) to share export a camera to Homekit. Any changes made will automatically be saved to `/config/go2rtc_homekit.yml`.
|
||||
To add camera streams to Homekit Frigate must be configured in docker to use `host` networking mode. Once that is done, you can use the go2rtc WebUI (accessed via port 1984, which is disabled by default) to export a camera to Homekit. Any changes made will automatically be saved to `/config/go2rtc_homekit.yml`.
|
||||
|
||||
@@ -477,7 +477,7 @@ Error marking filters as finished
|
||||
Restarting ffmpeg...
|
||||
```
|
||||
|
||||
you should try to uprade to FFmpeg 7. This can be done using this config option:
|
||||
you should try to upgrade to FFmpeg 7. This can be done using this config option:
|
||||
|
||||
```yaml
|
||||
ffmpeg:
|
||||
|
||||
@@ -6,6 +6,7 @@ title: License Plate Recognition (LPR)
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
import FaqItem from "@site/src/components/FaqItem";
|
||||
|
||||
Frigate can recognize license plates on vehicles and automatically add the detected characters to the `recognized_license_plate` field or a [known](#matching) name as a `sub_label` to tracked objects of type `car` or `motorcycle`. A common use case may be to read the license plates of cars pulling into a driveway or cars passing by on a street.
|
||||
|
||||
@@ -591,7 +592,9 @@ By selecting the appropriate configuration, users can optimize their dedicated L
|
||||
|
||||
## FAQ
|
||||
|
||||
### Why isn't my license plate being detected and recognized?
|
||||
### Detection and Recognition
|
||||
|
||||
<FaqItem id="why-isnt-my-license-plate-being-detected-and-recognized" question="Why isn't my license plate being detected and recognized?">
|
||||
|
||||
Ensure that:
|
||||
|
||||
@@ -606,29 +609,43 @@ Recognized plates will show as object labels in the debug view and will appear i
|
||||
|
||||
If you are still having issues detecting plates, start with a basic configuration and see the debugging tips below.
|
||||
|
||||
### Can I run LPR without detecting `car` or `motorcycle` objects?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="can-i-run-lpr-without-detecting-car-or-motorcycle-objects" question={<>Can I run LPR without detecting <code>car</code> or <code>motorcycle</code> objects?</>}>
|
||||
|
||||
In normal LPR mode, Frigate requires a `car` or `motorcycle` to be detected first before recognizing a license plate. If you have a dedicated LPR camera, you can change the camera `type` to `"lpr"` to use the Dedicated LPR Camera algorithm. This comes with important caveats, though. See the [Dedicated LPR Cameras](#dedicated-lpr-cameras) section above.
|
||||
|
||||
### How can I improve detection accuracy?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="how-can-i-improve-detection-accuracy" question="How can I improve detection accuracy?">
|
||||
|
||||
- Use high-quality cameras with good resolution.
|
||||
- Adjust `detection_threshold` and `recognition_threshold` values.
|
||||
- Define a `format` regex to filter out invalid detections.
|
||||
|
||||
### Does LPR work at night?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="does-lpr-work-at-night" question="Does LPR work at night?">
|
||||
|
||||
Yes, but performance depends on camera quality, lighting, and infrared capabilities. Make sure your camera can capture clear images of plates at night.
|
||||
|
||||
### Can I limit LPR to specific zones?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="can-i-limit-lpr-to-specific-zones" question="Can I limit LPR to specific zones?">
|
||||
|
||||
LPR, like other Frigate enrichments, runs at the camera level rather than the zone level. While you can't restrict LPR to specific zones directly, you can control when recognition runs by setting a `min_area` value to filter out smaller detections.
|
||||
|
||||
### How can I match known plates with minor variations?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="how-can-i-match-known-plates-with-minor-variations" question="How can I match known plates with minor variations?">
|
||||
|
||||
Use `match_distance` to allow small character mismatches. Alternatively, define multiple variations in `known_plates`.
|
||||
|
||||
### How do I debug LPR issues?
|
||||
</FaqItem>
|
||||
|
||||
### Performance and Troubleshooting
|
||||
|
||||
<FaqItem id="how-do-i-debug-lpr-issues" question="How do I debug LPR issues?">
|
||||
|
||||
Start with ["Why isn't my license plate being detected and recognized?"](#why-isnt-my-license-plate-being-detected-and-recognized). If you are still having issues, work through these steps.
|
||||
|
||||
@@ -685,17 +702,23 @@ lpr:
|
||||
- Watch the debug view to see plates recognized in real-time. For non-dedicated LPR cameras, the `car` or `motorcycle` label will change to the recognized plate when LPR is enabled and working.
|
||||
- Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration).
|
||||
|
||||
### Will LPR slow down my system?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="will-lpr-slow-down-my-system" question="Will LPR slow down my system?">
|
||||
|
||||
LPR's performance impact depends on your hardware. Ensure you have at least 4GB RAM and a capable CPU or GPU for optimal results. If you are running the Dedicated LPR Camera mode, resource usage will be higher compared to users who run a model that natively detects license plates. Tune your motion detection settings for your dedicated LPR camera so that the license plate detection model runs only when necessary.
|
||||
|
||||
### I am seeing a YOLOv9 plate detection metric in Enrichment Metrics, but I have a Frigate+ or custom model that detects `license_plate`. Why is the YOLOv9 model running?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="i-am-seeing-a-yolov9-plate-detection-metric-in-enrichment-metrics-but-i-have-a-frigate-or-custom-model-that-detects-license_plate-why-is-the-yolov9-model-running" question={<>I am seeing a YOLOv9 plate detection metric in Enrichment Metrics, but I have a Frigate+ or custom model that detects <code>license_plate</code>. Why is the YOLOv9 model running?</>}>
|
||||
|
||||
The YOLOv9 license plate detector model will run (and the metric will appear) if you've enabled LPR but haven't defined `license_plate` as an object to track, either at the global or camera level.
|
||||
|
||||
If you are detecting `car` or `motorcycle` on cameras where you don't want to run LPR, make sure you disable LPR it at the camera level. And if you do want to run LPR on those cameras, make sure you define `license_plate` as an object to track.
|
||||
|
||||
### It looks like Frigate picked up my camera's timestamp or overlay text as the license plate. How can I prevent this?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="it-looks-like-frigate-picked-up-my-cameras-timestamp-or-overlay-text-as-the-license-plate-how-can-i-prevent-this" question="It looks like Frigate picked up my camera's timestamp or overlay text as the license plate. How can I prevent this?">
|
||||
|
||||
This could happen if cars or motorcycles travel close to your camera's timestamp or overlay text. You could either move the text through your camera's firmware, or apply a mask to it in Frigate.
|
||||
|
||||
@@ -703,6 +726,10 @@ If you are using a model that natively detects `license_plate`, add an _object m
|
||||
|
||||
If you are not using a model that natively detects `license_plate` or you are using dedicated LPR camera mode, only a _motion mask_ over your text is required.
|
||||
|
||||
### I see "Error running ... model" in my logs, or my inference time is very high. How can I fix this?
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="i-see-error-running--model-in-my-logs-or-my-inference-time-is-very-high-how-can-i-fix-this" question={'I see "Error running ... model" in my logs, or my inference time is very high. How can I fix this?'}>
|
||||
|
||||
This usually happens when your GPU is unable to compile or use one of the LPR models. Set your `device` to `CPU` and try again. GPU acceleration only provides a slight performance increase, and the models are lightweight enough to run without issue on most CPUs.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
+121
-65
@@ -6,6 +6,7 @@ title: Live View
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
import FaqItem from "@site/src/components/FaqItem";
|
||||
|
||||
Frigate intelligently displays your camera streams on the Live view dashboard. By default, Frigate employs "smart streaming" where camera images update once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any motion or active objects are detected, cameras seamlessly switch to a live stream.
|
||||
|
||||
@@ -341,100 +342,155 @@ When your browser runs into problems playing back your camera streams, it will l
|
||||
|
||||
## Live view FAQ
|
||||
|
||||
1. **Why don't I have audio in my Live view?**
|
||||
### Getting Live View Working
|
||||
|
||||
You must use go2rtc to hear audio in your live streams. If you have go2rtc already configured, you need to ensure your camera is sending PCMA/PCMU or AAC audio. If you can't change your camera's audio codec, you need to [transcode the audio](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-ffmpeg) using go2rtc.
|
||||
<FaqItem id="why-dont-i-have-audio-in-my-live-view" question="Why don't I have audio in my Live view?">
|
||||
|
||||
Note that the low bandwidth mode player is a video-only stream. You should not expect to hear audio when in low bandwidth mode, even if you've set up go2rtc.
|
||||
You must use go2rtc to hear audio in your live streams. If you have go2rtc already configured, you need to ensure your camera is sending PCMA/PCMU or AAC audio. If you can't change your camera's audio codec, you need to [transcode the audio](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-ffmpeg) using go2rtc.
|
||||
|
||||
2. **Frigate shows that my live stream is in "low bandwidth mode". What does this mean?**
|
||||
If the audio controls don't appear in the UI at all, verify that the Live view is actually using your go2rtc stream. If your go2rtc stream names don't match your Frigate camera name, you must map them with the `live -> streams` config (see [Setting Streams For Live UI](#setting-streams-for-live-ui) above); otherwise the UI falls back to the video-only jsmpeg player.
|
||||
|
||||
Frigate intelligently selects the live streaming technology based on a number of factors (user-selected modes like two-way talk, camera settings, browser capabilities, available bandwidth) and prioritizes showing an actual up-to-date live view of your camera's stream as quickly as possible.
|
||||
Note that the low bandwidth mode player is a video-only stream. You should not expect to hear audio when in low bandwidth mode, even if you've set up go2rtc.
|
||||
|
||||
When you have go2rtc configured, Live view initially attempts to load and play back your stream with a clearer, fluent stream technology (MSE). An initial timeout, a low bandwidth condition that would cause buffering of the stream, or decoding errors in the stream will cause Frigate to switch to the stream defined by the `detect` role, using the jsmpeg format. This is what the UI labels as "low bandwidth mode". On Live dashboards, the mode will automatically reset when smart streaming is configured and activity stops. Continuous streaming mode does not have an automatic reset mechanism, but you can use the _Reset_ option to force a reload of your stream.
|
||||
</FaqItem>
|
||||
|
||||
If you are using continuous streaming or you are loading more than a few high resolution streams at once on the dashboard, your browser may struggle to begin playback of your streams before the timeout. Frigate always prioritizes showing a live stream as quickly as possible, even if it is a lower quality jsmpeg stream. You can use the "Reset" link/button to try loading your high resolution stream again.
|
||||
<FaqItem id="i-have-unmuted-some-cameras-on-my-dashboard-but-i-do-not-hear-sound-why" question="I have unmuted some cameras on my dashboard, but I do not hear sound. Why?">
|
||||
|
||||
Errors in stream playback (e.g., connection failures, codec issues, or buffering timeouts) that cause the fallback to low bandwidth mode (jsmpeg) are logged to the browser console for easier debugging. These errors may include:
|
||||
- Network issues (e.g., MSE or WebRTC network connection problems).
|
||||
- Unsupported codecs or stream formats (e.g., H.265 in WebRTC, which is not supported in some browsers).
|
||||
- Buffering timeouts or low bandwidth conditions causing fallback to jsmpeg.
|
||||
- Browser compatibility problems (e.g., iOS Safari limitations with MSE).
|
||||
If your camera is streaming (as indicated by a red dot in the upper right, or if it has been set to continuous streaming mode), your browser may be blocking audio until you interact with the page. This is an intentional browser limitation. See [this article](https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide#autoplay_availability). Many browsers have a whitelist feature to change this behavior.
|
||||
|
||||
To view browser console logs:
|
||||
1. Open the Frigate Live View in your browser.
|
||||
2. Open the browser's Developer Tools (F12 or right-click > Inspect > Console tab).
|
||||
3. Reproduce the error (e.g., load a problematic stream or simulate network issues).
|
||||
4. Look for messages prefixed with the camera name.
|
||||
</FaqItem>
|
||||
|
||||
These logs help identify if the issue is player-specific (MSE vs. WebRTC) or related to camera configuration (e.g., go2rtc streams, codecs). If you see frequent errors:
|
||||
- Verify your camera's H.264/AAC settings (see [Frigate's camera settings recommendations](#camera-settings-recommendations)).
|
||||
- Check go2rtc configuration for transcoding (e.g., audio to AAC/OPUS).
|
||||
- Test with a different stream via the UI dropdown (if `live -> streams` is configured).
|
||||
- For WebRTC-specific issues, ensure port 8555 is forwarded and candidates are set (see [WebRTC Extra Configuration](#webrtc-extra-configuration)).
|
||||
- If your cameras are streaming at a high resolution, your browser may be struggling to load all of the streams before the buffering timeout occurs. Frigate prioritizes showing a true live view as quickly as possible. If the fallback occurs often, change your live view settings to use a lower bandwidth substream.
|
||||
<FaqItem id="my-live-view-shows-a-black-screen-or-doesnt-load-but-the-debug-view-works-why" question="My live view shows a black screen or doesn't load, but the debug view works. Why?">
|
||||
|
||||
3. **It doesn't seem like my cameras are streaming on the Live dashboard. Why?**
|
||||
The debug view plays the `detect` stream processed by Frigate itself, while the Live view plays your go2rtc stream directly in the browser. If the debug view works but the Live view doesn't, your browser usually can't decode what the camera is sending, most often H.265 video or an incompatible audio track.
|
||||
|
||||
On the default Live dashboard ("All Cameras"), your camera images will update once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any activity is detected, cameras seamlessly switch to a full-resolution live stream. If you want to customize this behavior, use a camera group.
|
||||
Work through the [go2rtc troubleshooting guide](/troubleshooting/go2rtc#live-view-is-black-buffering-or-stuck-in-low-bandwidth-mode) to isolate the problem. Two fixes resolve the majority of cases:
|
||||
|
||||
4. **I see a strange diagonal line on my live view, but my recordings look fine. How can I fix it?**
|
||||
1. Restream through go2rtc's FFmpeg module by prefixing your source with `ffmpeg:`, for example `- ffmpeg:rtsp://user:password@192.168.1.5:554/stream`.
|
||||
2. If that doesn't help, transcode to compatible codecs: `- ffmpeg:rtsp://user:password@192.168.1.5:554/stream#video=h264#audio=aac#hardware`.
|
||||
|
||||
This is caused by incorrect dimensions set in your detect width or height (or incorrectly auto-detected), causing the jsmpeg player's rendering engine to display a slightly distorted image. You should enlarge the width and height of your `detect` resolution up to a standard aspect ratio (example: 640x352 becomes 640x360, and 800x443 becomes 800x450, 2688x1520 becomes 2688x1512, etc). If changing the resolution to match a standard (4:3, 16:9, or 32:9, etc) aspect ratio does not solve the issue, you can enable "compatibility mode" in your camera group dashboard's stream settings. Depending on your browser and device, more than a few cameras in compatibility mode may not be supported, so only use this option if changing your `detect` width and height fails to resolve the color artifacts and diagonal line.
|
||||
</FaqItem>
|
||||
|
||||
5. **How does "smart streaming" work?**
|
||||
<FaqItem id="how-do-i-get-the-best-live-view-experience-in-home-assistant" question="How do I get the best live view experience in Home Assistant?">
|
||||
|
||||
Because a static image of a scene looks exactly the same as a live stream with no motion or activity, smart streaming updates your camera images once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any activity (motion or object/audio detection) occurs, cameras seamlessly switch to a live stream.
|
||||
For a full-resolution, low-latency live view in Home Assistant dashboards, use the [Advanced Camera Card](https://card.camera) with the [go2rtc live provider](https://card.camera/#/configuration/cameras/live-provider?id=go2rtc), which streams directly from Frigate's bundled go2rtc. This also supports audio and [two-way talk](#two-way-talk) on capable cameras. See the [Home Assistant integration docs](/integrations/home-assistant) for setup.
|
||||
|
||||
This static image is pulled from the stream defined in your config with the `detect` role. When activity is detected, images from the `detect` stream immediately begin updating at ~5 frames per second so you can see the activity until the live player is loaded and begins playing. This usually only takes a second or two. If the live player times out, buffers, or has streaming errors, the jsmpeg player is loaded and plays a video-only stream from the `detect` role. When activity ends, the players are destroyed and a static image is displayed until activity is detected again, and the process repeats.
|
||||
</FaqItem>
|
||||
|
||||
Smart streaming depends on having your camera's motion `threshold` and `contour_area` config values dialed in. Use the Motion Tuner in Settings in the UI to tune these values in real-time.
|
||||
### Streaming Behavior
|
||||
|
||||
This is Frigate's default and recommended setting because it results in a significant bandwidth savings, especially for high resolution cameras.
|
||||
<FaqItem id="how-does-smart-streaming-work" question={'How does "smart streaming" work?'}>
|
||||
|
||||
6. **I have unmuted some cameras on my dashboard, but I do not hear sound. Why?**
|
||||
Because a static image of a scene looks exactly the same as a live stream with no motion or activity, smart streaming updates your camera images once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any activity (motion or object/audio detection) occurs, cameras seamlessly switch to a live stream.
|
||||
|
||||
If your camera is streaming (as indicated by a red dot in the upper right, or if it has been set to continuous streaming mode), your browser may be blocking audio until you interact with the page. This is an intentional browser limitation. See [this article](https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide#autoplay_availability). Many browsers have a whitelist feature to change this behavior.
|
||||
This static image is pulled from the stream defined in your config with the `detect` role. When activity is detected, images from the `detect` stream immediately begin updating at ~5 frames per second so you can see the activity until the live player is loaded and begins playing. This usually only takes a second or two. If the live player times out, buffers, or has streaming errors, the jsmpeg player is loaded and plays a video-only stream from the `detect` role. When activity ends, the players are destroyed and a static image is displayed until activity is detected again, and the process repeats.
|
||||
|
||||
7. **My camera streams have lots of visual artifacts / distortion.**
|
||||
Smart streaming depends on having your camera's motion `threshold` and `contour_area` config values dialed in. Use the Motion Tuner in Settings in the UI to tune these values in real-time.
|
||||
|
||||
Some cameras don't include the hardware to support multiple connections to the high resolution stream, and this can cause unexpected behavior. In this case it is recommended to [restream](./restream.md) the high resolution stream so that it can be used for live view and recordings.
|
||||
This is Frigate's default and recommended setting because it results in a significant bandwidth savings, especially for high resolution cameras.
|
||||
|
||||
8. **Why does my camera stream switch aspect ratios on the Live dashboard?**
|
||||
</FaqItem>
|
||||
|
||||
Your camera may change aspect ratios on the dashboard because Frigate uses different streams for different purposes. With go2rtc and Smart Streaming, Frigate shows a static image from the `detect` stream when no activity is present, and switches to the live stream when motion is detected. The camera image will change size if your streams use different aspect ratios.
|
||||
<FaqItem id="it-doesnt-seem-like-my-cameras-are-streaming-on-the-live-dashboard-why" question="It doesn't seem like my cameras are streaming on the Live dashboard. Why?">
|
||||
|
||||
To prevent this, make the `detect` stream match the go2rtc live stream's aspect ratio (resolution does not need to match, just the aspect ratio). You can either adjust the camera's output resolution or set the `width` and `height` values in your config's `detect` section to a resolution with an aspect ratio that matches.
|
||||
On the default Live dashboard ("All Cameras"), your camera images will update once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any activity is detected, cameras seamlessly switch to a full-resolution live stream. If you want to customize this behavior, use a camera group.
|
||||
|
||||
Example: Resolutions from two streams
|
||||
- Mismatched (may cause aspect ratio switching on the dashboard):
|
||||
- Live/go2rtc stream: 1920x1080 (16:9)
|
||||
- Detect stream: 640x352 (~1.82:1, not 16:9)
|
||||
</FaqItem>
|
||||
|
||||
- Matched (prevents switching):
|
||||
- Live/go2rtc stream: 1920x1080 (16:9)
|
||||
- Detect stream: 640x360 (16:9)
|
||||
<FaqItem id="frigate-shows-that-my-live-stream-is-in-low-bandwidth-mode-what-does-this-mean" question={'Frigate shows that my live stream is in "low bandwidth mode". What does this mean?'}>
|
||||
|
||||
You can update the detect settings in your camera config to match the aspect ratio of your go2rtc live stream. For example:
|
||||
Frigate intelligently selects the live streaming technology based on a number of factors (user-selected modes like two-way talk, camera settings, browser capabilities, available bandwidth) and prioritizes showing an actual up-to-date live view of your camera's stream as quickly as possible.
|
||||
|
||||
```yaml
|
||||
cameras:
|
||||
front_door:
|
||||
detect:
|
||||
width: 640
|
||||
height: 360 # set this to 360 instead of 352
|
||||
ffmpeg:
|
||||
inputs:
|
||||
- path: rtsp://127.0.0.1:8554/front_door # main stream 1920x1080
|
||||
roles:
|
||||
- record
|
||||
- path: rtsp://127.0.0.1:8554/front_door_sub # sub stream 640x352
|
||||
roles:
|
||||
- detect
|
||||
```
|
||||
When you have go2rtc configured, Live view initially attempts to load and play back your stream with a clearer, fluent stream technology (MSE). An initial timeout, a low bandwidth condition that would cause buffering of the stream, or decoding errors in the stream will cause Frigate to switch to the stream defined by the `detect` role, using the jsmpeg format. This is what the UI labels as "low bandwidth mode". On Live dashboards, the mode will automatically reset when smart streaming is configured and activity stops. Continuous streaming mode does not have an automatic reset mechanism, but you can use the _Reset_ option to force a reload of your stream.
|
||||
|
||||
The same applies to your `record` stream: if its aspect ratio differs from your `detect` stream, your recordings will appear in a different shape than the live view. For consistent framing across live view and recordings, use the same aspect ratio for all of a camera's streams (the resolution can still differ).
|
||||
If you are using continuous streaming or you are loading more than a few high resolution streams at once on the dashboard, your browser may struggle to begin playback of your streams before the timeout. Frigate always prioritizes showing a live stream as quickly as possible, even if it is a lower quality jsmpeg stream. You can use the "Reset" link/button to try loading your high resolution stream again.
|
||||
|
||||
9. **Why does Frigate prefer MSE over WebRTC for live view?**
|
||||
Errors in stream playback (e.g., connection failures, codec issues, or buffering timeouts) that cause the fallback to low bandwidth mode (jsmpeg) are logged to the browser console for easier debugging. These errors may include:
|
||||
|
||||
Frigate prefers MSE because it delivers a better out-of-the-box experience than WebRTC on nearly every axis that matters for a security camera system. MSE is an open standard optimized and supported by all modern browsers, works without any extra configuration (WebRTC requires port forwarding and candidate setup, and lacks H.265 support in some browsers), and requires no internet access for NAT traversal. More importantly, MSE runs over TCP, so every frame arrives and is decoded in order, so nothing is ever silently skipped. WebRTC optimizes for latency over UDP by discarding late or incomplete frames, which works against you on cellular or spotty Wi-Fi: you can end up with frozen video, visual corruption, or gaps in the feed without ever knowing you missed something. Frigate's enhanced MSE player has adaptive speed playback and has been tuned for latency and connection robustness that meets or exceeds WebRTC, so you get near-real-time playback with a guarantee that when the video plays, every frame is actually there - which, for an NVR whose whole purpose is letting you see what happened, matters more than shaving fractions of a second off a latency number. That's why Frigate defaults to MSE and reserves WebRTC for cases that require it, like two-way talk.
|
||||
- Network issues (e.g., MSE or WebRTC network connection problems).
|
||||
- Unsupported codecs or stream formats (e.g., H.265 in WebRTC, which is not supported in some browsers).
|
||||
- Buffering timeouts or low bandwidth conditions causing fallback to jsmpeg.
|
||||
- Browser compatibility problems (e.g., iOS Safari limitations with MSE).
|
||||
|
||||
To view browser console logs:
|
||||
|
||||
1. Open the Frigate Live View in your browser.
|
||||
2. Open the browser's Developer Tools (F12 or right-click > Inspect > Console tab).
|
||||
3. Reproduce the error (e.g., load a problematic stream or simulate network issues).
|
||||
4. Look for messages prefixed with the camera name.
|
||||
|
||||
These logs help identify if the issue is player-specific (MSE vs. WebRTC) or related to camera configuration (e.g., go2rtc streams, codecs). If you see frequent errors:
|
||||
|
||||
- Verify your camera's H.264/AAC settings (see [Frigate's camera settings recommendations](#camera-settings-recommendations)).
|
||||
- Check go2rtc configuration for transcoding (e.g., audio to AAC/OPUS).
|
||||
- Test with a different stream via the UI dropdown (if `live -> streams` is configured).
|
||||
- For WebRTC-specific issues, ensure port 8555 is forwarded and candidates are set (see [WebRTC Extra Configuration](#webrtc-extra-configuration)).
|
||||
- If your cameras are streaming at a high resolution, your browser may be struggling to load all of the streams before the buffering timeout occurs. Frigate prioritizes showing a true live view as quickly as possible. If the fallback occurs often, change your live view settings to use a lower bandwidth substream.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-is-my-live-view-delayed-or-lagging-behind-real-time" question="Why is my live view delayed or lagging behind real time?">
|
||||
|
||||
A delay when a stream first starts is usually caused by your camera's I-frame (keyframe) interval. Playback cannot begin until a keyframe arrives, so an interval set higher than your camera's frame rate makes the stream take longer to start. Set the I-frame interval to match the frame rate (or "1x" on Reolink) per the [camera settings recommendations](#camera-settings-recommendations).
|
||||
|
||||
A stream that starts on time but falls further behind live is buffering, which is usually the browser struggling to decode too many high-resolution streams at once. Select a lower-bandwidth substream for your dashboards (see [Setting Streams For Live UI](#setting-streams-for-live-ui)), reduce the number of streams open at once, or improve the network connection between your browser and Frigate. Frigate's player automatically speeds up playback to catch up to live after buffering, and falls back to low bandwidth mode if it stalls for too long. The _Reset_ option forces a fresh connection at the live edge.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-does-frigate-prefer-mse-over-webrtc-for-live-view" question="Why does Frigate prefer MSE over WebRTC for live view?">
|
||||
|
||||
Frigate prefers MSE because it delivers a better out-of-the-box experience than WebRTC on nearly every axis that matters for a security camera system. MSE is an open standard optimized and supported by all modern browsers, works without any extra configuration (WebRTC requires port forwarding and candidate setup, and lacks H.265 support in some browsers), and requires no internet access for NAT traversal. More importantly, MSE runs over TCP, so every frame arrives and is decoded in order, so nothing is ever silently skipped. WebRTC optimizes for latency over UDP by discarding late or incomplete frames, which works against you on cellular or spotty Wi-Fi: you can end up with frozen video, visual corruption, or gaps in the feed without ever knowing you missed something. Frigate's enhanced MSE player has adaptive speed playback and has been tuned for latency and connection robustness that meets or exceeds WebRTC, so you get near-real-time playback with a guarantee that when the video plays, every frame is actually there - which, for an NVR whose whole purpose is letting you see what happened, matters more than shaving fractions of a second off a latency number. That's why Frigate defaults to MSE and reserves WebRTC for cases that require it, like two-way talk.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Video Quality Issues
|
||||
|
||||
<FaqItem id="i-see-a-strange-diagonal-line-on-my-live-view-but-my-recordings-look-fine-how-can-i-fix-it" question="I see a strange diagonal line on my live view, but my recordings look fine. How can I fix it?">
|
||||
|
||||
This is caused by incorrect dimensions set in your detect width or height (or incorrectly auto-detected), causing the jsmpeg player's rendering engine to display a slightly distorted image. You should enlarge the width and height of your `detect` resolution up to a standard aspect ratio (example: 640x352 becomes 640x360, and 800x443 becomes 800x450, 2688x1520 becomes 2688x1512, etc). If changing the resolution to match a standard (4:3, 16:9, or 32:9, etc) aspect ratio does not solve the issue, you can enable "compatibility mode" in your camera group dashboard's stream settings. Depending on your browser and device, more than a few cameras in compatibility mode may not be supported, so only use this option if changing your `detect` width and height fails to resolve the color artifacts and diagonal line.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="my-camera-streams-have-lots-of-visual-artifacts-or-distortion" question="My camera streams have lots of visual artifacts / distortion.">
|
||||
|
||||
Some cameras don't include the hardware to support multiple connections to the high resolution stream, and this can cause unexpected behavior. In this case it is recommended to [restream](./restream.md) the high resolution stream so that it can be used for live view and recordings.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="why-does-my-camera-stream-switch-aspect-ratios-on-the-live-dashboard" question="Why does my camera stream switch aspect ratios on the Live dashboard?">
|
||||
|
||||
Your camera may change aspect ratios on the dashboard because Frigate uses different streams for different purposes. With go2rtc and Smart Streaming, Frigate shows a static image from the `detect` stream when no activity is present, and switches to the live stream when motion is detected. The camera image will change size if your streams use different aspect ratios.
|
||||
|
||||
To prevent this, make the `detect` stream match the go2rtc live stream's aspect ratio (resolution does not need to match, just the aspect ratio). You can either adjust the camera's output resolution or set the `width` and `height` values in your config's `detect` section to a resolution with an aspect ratio that matches.
|
||||
|
||||
Example: Resolutions from two streams
|
||||
|
||||
- Mismatched (may cause aspect ratio switching on the dashboard):
|
||||
- Live/go2rtc stream: 1920x1080 (16:9)
|
||||
- Detect stream: 640x352 (~1.82:1, not 16:9)
|
||||
|
||||
- Matched (prevents switching):
|
||||
- Live/go2rtc stream: 1920x1080 (16:9)
|
||||
- Detect stream: 640x360 (16:9)
|
||||
|
||||
You can update the detect settings in your camera config to match the aspect ratio of your go2rtc live stream. For example:
|
||||
|
||||
```yaml
|
||||
cameras:
|
||||
front_door:
|
||||
detect:
|
||||
width: 640
|
||||
height: 360 # set this to 360 instead of 352
|
||||
ffmpeg:
|
||||
inputs:
|
||||
- path: rtsp://127.0.0.1:8554/front_door # main stream 1920x1080
|
||||
roles:
|
||||
- record
|
||||
- path: rtsp://127.0.0.1:8554/front_door_sub # sub stream 640x352
|
||||
roles:
|
||||
- detect
|
||||
```
|
||||
|
||||
The same applies to your `record` stream: if its aspect ratio differs from your `detect` stream, your recordings will appear in a different shape than the live view. For consistent framing across live view and recordings, use the same aspect ratio for all of a camera's streams (the resolution can still differ).
|
||||
|
||||
</FaqItem>
|
||||
|
||||
@@ -66,7 +66,7 @@ motion:
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
Lower values mean motion detection is more sensitive to changes in color, making it more likely for example to detect motion when a brown dogs blends in with a brown fence or a person wearing a red shirt blends in with a red car. If the threshold is too low however, it may detect things like grass blowing in the wind, shadows, etc. to be detected as motion.
|
||||
Lower values mean motion detection is more sensitive to changes in color, making it more likely for example to detect motion when a brown dog blends in with a brown fence or a person wearing a red shirt blends in with a red car. If the threshold is too low however, it may detect things like grass blowing in the wind, shadows, etc. to be detected as motion.
|
||||
|
||||
Watching the motion boxes in the debug view, increase the threshold until you only see motion that is visible to the eye. Once this is done, it is important to test and ensure that desired motion is still detected.
|
||||
|
||||
|
||||
@@ -725,7 +725,7 @@ The inference time was determined on a rk3588 with 3 NPU cores.
|
||||
|
||||
To convert a onnx model to the rknn format using the [rknn-toolkit2](https://github.com/airockchip/rknn-toolkit2/) you have to:
|
||||
|
||||
- Place one ore more models in onnx format in the directory `config/model_cache/rknn_cache/onnx` on your docker host (this might require `sudo` privileges).
|
||||
- Place one or more models in onnx format in the directory `config/model_cache/rknn_cache/onnx` on your docker host (this might require `sudo` privileges).
|
||||
- Save the configuration file under `config/conv2rknn.yaml` (see below for details).
|
||||
- Run `docker exec <frigate_container_id> python3 /opt/conv2rknn.py`. If the conversion was successful, the rknn models will be placed in `config/model_cache/rknn_cache`.
|
||||
|
||||
@@ -743,13 +743,13 @@ config:
|
||||
quant_img_RGB2BGR: true
|
||||
```
|
||||
|
||||
Explanation of the paramters:
|
||||
Explanation of the parameters:
|
||||
|
||||
- `soc`: A list of all SoCs you want to build the rknn model for. If you don't specify this parameter, the script tries to find out your SoC and builds the rknn model for this one.
|
||||
- `quantization`: true: 8 bit integer (i8) quantization, false: 16 bit float (fp16). Default: false.
|
||||
- `output_name`: The output name of the model. The following variables are available:
|
||||
- `quant`: "i8" or "fp16" depending on the config
|
||||
- `input_basename`: the basename of the input model (e.g. "my_model" if the input model is calles "my_model.onnx")
|
||||
- `input_basename`: the basename of the input model (e.g. "my_model" if the input model is called "my_model.onnx")
|
||||
- `soc`: the SoC this model was build for (e.g. "rk3588")
|
||||
- `tk_version`: Version of `rknn-toolkit2` (e.g. "2.3.0")
|
||||
- **example**: Specifying `output_name = "frigate-{quant}-{input_basename}-{soc}-v{tk_version}"` could result in a model called `frigate-i8-my_model-rk3588-v2.3.0.rknn`.
|
||||
|
||||
@@ -36,7 +36,7 @@ Any detection below `min_score` will be immediately thrown out and never tracked
|
||||
|
||||
### Threshold
|
||||
|
||||
`threshold` is used to determine that the object is a true positive. Once an object is detected with a score >= `threshold` object is considered a true positive. If `threshold` is too low then some higher scoring false positives may create an tracked object. If `threshold` is too high then true positive tracked objects may be missed due to the object never scoring high enough.
|
||||
`threshold` is used to determine that the object is a true positive. Once an object is detected with a score >= `threshold` object is considered a true positive. If `threshold` is too low then some higher scoring false positives may create a tracked object. If `threshold` is too high then true positive tracked objects may be missed due to the object never scoring high enough.
|
||||
|
||||
## Configuring Object Scores
|
||||
|
||||
|
||||
@@ -226,7 +226,7 @@ For tips on getting the best results from Semantic Search (choosing between thum
|
||||
|
||||
## Triggers
|
||||
|
||||
Triggers utilize Semantic Search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes a specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
|
||||
Triggers utilize Semantic Search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
|
||||
|
||||
:::note
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ Let's look at an example use case: I want to record any cars that enter my drive
|
||||
|
||||
One might simply think "Why not just run object detection any time there is motion around the driveway area and notify if the bounding box is in that zone?"
|
||||
|
||||
With that approach, what video is related to the car that entered the driveway? Did it come from the left or right? Was it parked across the street for an hour before turning into the driveway? One approach is to just record 24/7 or for motion (on any changed changed pixels) and not attempt to do that at all. This is what most other NVRs do. Just don't even try to identify a start and end for that object since it's hard and you will be wrong some portion of the time.
|
||||
With that approach, what video is related to the car that entered the driveway? Did it come from the left or right? Was it parked across the street for an hour before turning into the driveway? One approach is to just record 24/7 or for motion (on any changed pixels) and not attempt to do that at all. This is what most other NVRs do. Just don't even try to identify a start and end for that object since it's hard and you will be wrong some portion of the time.
|
||||
|
||||
Couldn't you just look at when motion stopped and started? Motion for a video feed is nothing more than looking for pixels that are different than they were in previous frames. If the car entered the driveway while someone was mowing the grass, how would you know which motion was for the car and which was for the person when they mow along the driveway or street? What if another car was driving the other direction on the street? Or what if its a windy day and the bush by your mailbox is blowing around?
|
||||
|
||||
@@ -61,4 +61,4 @@ Now you have to determine which of the bounding boxes in this frame should be ma
|
||||
|
||||
Now let's assume that those other 3 cars were already being tracked as stationary objects, so the car driving down the street is a new 4th car. The object tracker knows we have had 3 cars and we now have 4. As the new car approaches the parked cars, the bounding boxes for all 4 cars is predicted based on the previous frames. The predicted boxes for the parked cars is pretty much a 100% overlap with the bounding boxes in the new frame. The parked cars are slam dunk matches to the tracking ids they had before and the only one left is the remaining bounding box which gets assigned to the new car. This results in a much lower error rate. Not perfect, but better.
|
||||
|
||||
The most difficult scenario that causes IDs to be assigned incorrectly is when an object completely occludes another object. When a car drives in front of another car and its no longer visible, a bounding box disappeared and it's a bit of a toss up when assigning the id since it's difficult to know which one is in front of the other. This happens for cars passing in front of other cars fairly often. It's something that we want to improve in the future.
|
||||
The most difficult scenario that causes IDs to be assigned incorrectly is when an object completely occludes another object. When a car drives in front of another car and it's no longer visible, a bounding box disappeared and it's a bit of a toss up when assigning the id since it's difficult to know which one is in front of the other. This happens for cars passing in front of other cars fairly often. It's something that we want to improve in the future.
|
||||
|
||||
@@ -94,7 +94,7 @@ The following sections contain additional setup steps that are only required if
|
||||
|
||||
By default, the Raspberry Pi limits the amount of memory available to the GPU. In order to use ffmpeg hardware acceleration, you must increase the available memory by setting `gpu_mem` to the maximum recommended value in `config.txt` as described in the [official docs](https://www.raspberrypi.org/documentation/computers/config_txt.html#memory-options).
|
||||
|
||||
Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with it's own power supply. Some have reported success with <a href="https://amzn.to/3a2mH0P" target="_blank" rel="nofollow noopener sponsored">this</a> (affiliate link).
|
||||
Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with its own power supply. Some have reported success with <a href="https://amzn.to/3a2mH0P" target="_blank" rel="nofollow noopener sponsored">this</a> (affiliate link).
|
||||
|
||||
### Hailo-8
|
||||
|
||||
|
||||
@@ -9,6 +9,8 @@ The best way to integrate with Home Assistant is to use the [official integratio
|
||||
|
||||
### Preparation
|
||||
|
||||
Frigate itself must be installed and running before setting up the integration. See the [installation documentation](../frigate/installation.md) for details.
|
||||
|
||||
The Frigate integration requires the `mqtt` integration to be installed and
|
||||
manually configured first.
|
||||
|
||||
@@ -122,7 +124,7 @@ Use `http://<frigate_device_ip>:8971` as the URL for the integration so that aut
|
||||
|
||||
The above URL assumes you have [disabled TLS](../configuration/tls).
|
||||
By default, TLS is enabled and Frigate will be using a self-signed certificate. HomeAssistant will fail to connect HTTPS to port 8971 since it fails to verify the self-signed certificate.
|
||||
Either disable TLS and use HTTP from HomeAssistant, or configure Frigate to be acessible with a valid certificate.
|
||||
Either disable TLS and use HTTP from HomeAssistant, or configure Frigate to be accessible with a valid certificate.
|
||||
|
||||
:::
|
||||
|
||||
|
||||
@@ -280,7 +280,7 @@ Same data available at `/api/stats` published at a configurable interval.
|
||||
|
||||
### `frigate/camera_activity`
|
||||
|
||||
Returns data about each camera, its current features, and if it is detecting motion, objects, etc. Can be triggered by publising to `frigate/onConnect`
|
||||
Returns data about each camera, its current features, and if it is detecting motion, objects, etc. Can be triggered by publishing to `frigate/onConnect`
|
||||
|
||||
### `frigate/profile/set`
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ The [Advanced Camera Card](https://card.camera/#/README) is a Home Assistant das
|
||||
|
||||
## [Double Take](https://github.com/skrashevich/double-take)
|
||||
|
||||
[Double Take](https://github.com/skrashevich/double-take) provides an unified UI and API for processing and training images for facial recognition.
|
||||
[Double Take](https://github.com/skrashevich/double-take) provides a unified UI and API for processing and training images for facial recognition.
|
||||
It supports automatically setting the sub labels in Frigate for person objects that are detected and recognized.
|
||||
This is a fork (with fixed errors and new features) of [original Double Take](https://github.com/jakowenko/double-take) project which, unfortunately, isn't being maintained by author.
|
||||
|
||||
@@ -53,7 +53,7 @@ This is a fork (with fixed errors and new features) of [original Double Take](ht
|
||||
|
||||
## [Scrypted - Frigate bridge plugin](https://github.com/apocaliss92/scrypted-frigate-bridge)
|
||||
|
||||
[Scrypted - Frigate bridge](https://github.com/apocaliss92/scrypted-frigate-bridge) is an plugin that allows to ingest Frigate detections, motion, videoclips on Scrypted as well as provide templates to export rebroadcast configurations on Frigate.
|
||||
[Scrypted - Frigate bridge](https://github.com/apocaliss92/scrypted-frigate-bridge) is a plugin that allows you to ingest Frigate detections, motion, videoclips on Scrypted as well as provide templates to export rebroadcast configurations on Frigate.
|
||||
|
||||
## [Strix](https://github.com/eduard256/Strix)
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ For the best results, follow these guidelines. You may also want to review the d
|
||||
|
||||
## AI suggested labels
|
||||
|
||||
If you have an active Frigate+ subscription, new uploads will be scanned for the objects configured for you camera and you will see suggested labels as light blue boxes when annotating in Frigate+. These suggestions are processed via a queue and typically complete within a minute after uploading, but processing times can be longer.
|
||||
If you have an active Frigate+ subscription, new uploads will be scanned for the objects configured for your camera and you will see suggested labels as light blue boxes when annotating in Frigate+. These suggestions are processed via a queue and typically complete within a minute after uploading, but processing times can be longer.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -3,6 +3,10 @@ id: first_model
|
||||
title: Requesting your first model
|
||||
---
|
||||
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
|
||||
## Step 1: Upload and annotate your images
|
||||
|
||||
Before requesting your first model, you will need to upload and verify at least 10 images to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate.
|
||||
@@ -16,13 +20,21 @@ For more detailed recommendations, you can refer to the docs on [annotating](./a
|
||||
Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [this part of the documentation](./index.md#available-model-types). If you are unsure which type to request, you can test the base model for each version from the "Base Models" tab. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours.
|
||||

|
||||
|
||||
## Step 3: Set your model id in the config
|
||||
## Step 3: Set your model
|
||||
|
||||
You will receive an email notification when your Frigate+ model is ready.
|
||||

|
||||
|
||||
Models available in Frigate+ can be used with a special model path. No other information needs to be configured because it fetches the remaining config from Frigate+ automatically.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > System > Detectors and model" />. In the **Detection Model** section, choose the **Frigate+** tab. Select your new Frigate+ model from the **Available Frigate+ models** dropdown, then click **Save**. Restart Frigate to apply the change.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
detectors: ...
|
||||
|
||||
@@ -30,22 +42,46 @@ model:
|
||||
path: plus://<your_model_id>
|
||||
```
|
||||
|
||||
:::note
|
||||
|
||||
Model IDs are not secret values and can be shared freely. Access to your model is protected by your API key.
|
||||
|
||||
:::
|
||||
|
||||
:::tip
|
||||
|
||||
When setting the plus model id, all other fields should be removed as these are configured automatically with the Frigate+ model config
|
||||
|
||||
:::
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
:::note
|
||||
|
||||
Model IDs are not secret values and can be shared freely. Access to your model is protected by your API key.
|
||||
|
||||
:::
|
||||
|
||||
## Step 4: Adjust your object filters for higher scores
|
||||
|
||||
Frigate+ models generally have much higher scores than the default model provided in Frigate. You will likely need to increase your `threshold` and `min_score` values. Here is an example of how these values can be refined, but you should expect these to evolve as your model improves. For more information about how `threshold` and `min_score` are related, see the docs on [object filters](../configuration/object_filters.md#object-scores).
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > Global configuration > Objects" />. Under **Object filters**, set **Min Score** and **Threshold** for each object type, then click **Save**.
|
||||
|
||||
| Object | Min Score | Threshold |
|
||||
| ----------------- | --------- | --------- |
|
||||
| **dog** | .7 | .9 |
|
||||
| **cat** | .65 | .8 |
|
||||
| **face** | .7 | |
|
||||
| **package** | .65 | .9 |
|
||||
| **license_plate** | .6 | |
|
||||
| **amazon** | .75 | |
|
||||
| **ups** | .75 | |
|
||||
| **fedex** | .75 | |
|
||||
| **person** | .65 | .85 |
|
||||
| **car** | .65 | .85 |
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
objects:
|
||||
filters:
|
||||
@@ -75,3 +111,6 @@ objects:
|
||||
min_score: .65
|
||||
threshold: .85
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
@@ -39,7 +39,9 @@ The per-clip variation is typically quite low and is mostly an artifact of keyfr
|
||||
|
||||
Debug Replay lets you re-run Frigate's detection pipeline against a section of recorded video without manually configuring a dummy camera. It automatically extracts the recording, creates a temporary camera with the same detection settings as the original, and loops the clip through the pipeline so you can observe detections in real time.
|
||||
|
||||
Debug Replay isn't intended to be a one-stop pane for all Frigate diagnostics or a comprehensive debugging environment for every Frigate feature. It merely makes it easier to spin up a "dummy camera" and perform some common adjustments in real-time. You'll still need to use the normal tools (logs, an MQTT client, etc) to debug your feature.
|
||||
The replay camera behaves like a live camera feed rather than History's video player: it loops the clip continuously as Frigate analyzes it and has no playback controls, so you cannot pause, scrub, or step through it frame by frame.
|
||||
|
||||
Debug Replay isn't intended to be a one-stop pane for all Frigate diagnostics or a comprehensive debugging environment for every Frigate feature. It merely makes it easier to spin up a "dummy camera" and perform some common adjustments in real time. You'll still need to use the normal tools (logs, an MQTT client, etc) to debug your feature.
|
||||
|
||||
### When to use
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ There are many possible causes for a USB coral not being detected and some are O
|
||||
|
||||
:::tip
|
||||
|
||||
Using `lsusb` or checking the hardware page in HA OS will show as `1a6e:089a Global Unichip Corp.` until Frigate runs an inferance using the coral. So don't worry about the identification until after Frigate has attempted to detect the coral.
|
||||
Using `lsusb` or checking the hardware page in HA OS will show as `1a6e:089a Global Unichip Corp.` until Frigate runs an inference using the coral. So don't worry about the identification until after Frigate has attempted to detect the coral.
|
||||
|
||||
:::
|
||||
|
||||
@@ -43,13 +43,13 @@ Some users have reported that this older device runs an older kernel causing iss
|
||||
3. Start the docker container with Coral TPU enabled in the config
|
||||
4. The TPU would be detected but a few moments later it would disconnect.
|
||||
5. While leaving the TPU device plugged in, restart the NAS using the reboot command in the UI. Do NOT unplug the NAS/power it off etc.
|
||||
6. Open the control panel - info scree. The coral TPU will now be recognised as a USB Device - google inc
|
||||
6. Open the control panel - info screen. The coral TPU will now be recognized as a USB Device - google inc
|
||||
7. Start the frigate container. Everything should work now!
|
||||
|
||||
### QNAP NAS
|
||||
|
||||
QNAP NAS devices, such as the TS-253A, may use connected Coral TPU devices if [QuMagie](https://www.qnap.com/en/software/qumagie) is installed along with its QNAP AI Core extension. If any of the features (`facial recognition`, `object recognition`, or `similar photo recognition`) are enabled, Container Station applications such as `Frigate` or `CodeProject.AI Server` will be unable to initialize the TPU device in use.
|
||||
To allow the Coral TPU device to be discovered, the you must either:
|
||||
To allow the Coral TPU device to be discovered, you must either:
|
||||
|
||||
1. [Disable the AI recognition features in QuMagie](https://docs.qnap.com/application/qumagie/2.x/en-us/configuring-qnap-ai-core-settings-FB13CE03.html),
|
||||
2. Remove the QNAP AI Core extension or
|
||||
@@ -76,7 +76,7 @@ This is an issue due to outdated gasket driver when being used with new linux ke
|
||||
|
||||
### Not detected on Raspberry Pi5
|
||||
|
||||
A kernel update to the RPi5 means an upate to config.txt is required, see [the raspberry pi forum for more info](https://forums.raspberrypi.com/viewtopic.php?t=363682&sid=cb59b026a412f0dc041595951273a9ca&start=25)
|
||||
A kernel update to the RPi5 means an update to config.txt is required, see [the raspberry pi forum for more info](https://forums.raspberrypi.com/viewtopic.php?t=363682&sid=cb59b026a412f0dc041595951273a9ca&start=25)
|
||||
|
||||
Specifically, add the following to config.txt
|
||||
|
||||
@@ -87,7 +87,7 @@ dtoverlay=pcie-32bit-dma-pi5
|
||||
|
||||
## Only One PCIe Coral Is Detected With Coral Dual EdgeTPU
|
||||
|
||||
Coral Dual EdgeTPU is one card with two identical TPU cores. Each core has it's own PCIe interface and motherboard needs to have two PCIe busses on the m.2 slot to make them both work.
|
||||
Coral Dual EdgeTPU is one card with two identical TPU cores. Each core has its own PCIe interface and motherboard needs to have two PCIe busses on the m.2 slot to make them both work.
|
||||
|
||||
E-key slot implemented to full m.2 electromechanical specification has two PCIe busses. Most motherboard manufacturers implement only one PCIe bus in m.2 E-key connector (this is why only one TPU is working). Some SBCs can have only USB bus on m.2 connector, ie none of TPUs will work.
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ You can open `chrome://media-internals/` in another tab and then try to playback
|
||||
|
||||
### What do I do if my cameras sub stream is not good enough?
|
||||
|
||||
Frigate generally [recommends cameras with configurable sub streams](/frigate/hardware.md). However, if your camera does not have a sub stream that a suitable resolution, the main stream can be resized.
|
||||
Frigate generally [recommends cameras with configurable sub streams](/frigate/hardware.md). However, if your camera does not have a sub stream that is a suitable resolution, the main stream can be resized.
|
||||
|
||||
To do this efficiently the following setup is required:
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ id: recordings
|
||||
title: Recordings Errors
|
||||
---
|
||||
|
||||
import FaqItem from "@site/src/components/FaqItem";
|
||||
|
||||
## Why are my recordings not working? (empty Recordings, "No recordings found for this time")
|
||||
|
||||
If Frigate shows live video but the History view is empty, or you see "No recordings found for this time", the cause is almost always in one of the three categories below. Segments are first written to the RAM cache and are only moved to disk if they match a retention policy _and_ the camera's `record` stream is producing valid, storable video. Work through the categories in order: retention configuration is by far the most common cause.
|
||||
@@ -19,7 +21,7 @@ A healthy camera logs lines like `Copied /media/frigate/recordings/{segment_path
|
||||
|
||||
### Retention configuration issues
|
||||
|
||||
#### Recording is enabled, but nothing is saved
|
||||
<FaqItem id="recording-is-enabled-but-nothing-is-saved" question="Recording is enabled, but nothing is saved">
|
||||
|
||||
This is the single most common cause. Setting `record.enabled: True` on its own does **not** keep any footage: **continuous recording is disabled by default**, and segments in the cache are only moved to disk if they match a configured retention policy. You must configure at least one of `continuous`, `motion`, `alerts`, or `detections` retention.
|
||||
|
||||
@@ -34,7 +36,9 @@ record:
|
||||
|
||||
See [Recording](/configuration/record) for the full set of common configurations, including reduced-storage and alerts-only setups.
|
||||
|
||||
#### Motion or event-only recording keeps less than you expect
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="motion-or-event-only-recording-keeps-less-than-you-expect" question="Motion or event-only recording keeps less than you expect">
|
||||
|
||||
If you only configured `motion`, `alerts`, or `detections` retention (with no `continuous`), Frigate keeps footage selectively based on the retention `mode`:
|
||||
|
||||
@@ -44,20 +48,26 @@ If you only configured `motion`, `alerts`, or `detections` retention (with no `c
|
||||
|
||||
If you expected continuous footage but only configured motion/event retention, add a `continuous` retention period as shown above. To verify motion is actually being detected, watch the motion boxes in the debug view or the Motion Tuner in the UI.
|
||||
|
||||
#### Alert and detection recordings require working object detection
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="alert-and-detection-recordings-require-working-object-detection" question="Alert and detection recordings require working object detection">
|
||||
|
||||
`alerts` and `detections` retention only keep footage that overlaps a tracked object, so they depend on object detection running:
|
||||
|
||||
- **Detection must be enabled.** If `detect: enabled: False`, no alerts or detections are ever created, so alert/detection retention keeps nothing. (Continuous and motion retention still work with detection disabled.)
|
||||
- **The object must be supported by your model.** If you track an object your model doesn't support (for example `deer` or `license_plate` on the default model), Frigate never detects it and never records for it. Check your logs for warnings such as `... is configured to track ['deer'] objects, which are not supported by the current model` and remove unsupported objects or switch to a model (e.g. [Frigate+](/plus/)) that includes them.
|
||||
|
||||
#### You're following an outdated guide
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="youre-following-an-outdated-guide" question="You're following an outdated guide">
|
||||
|
||||
Configuration keys change between major versions. The old `clips` config, for example, has not existed for a long time. If you copied a config from an old blog post or video, verify every key against the current [reference config](/configuration/advanced/reference).
|
||||
|
||||
</FaqItem>
|
||||
|
||||
### Camera and stream issues
|
||||
|
||||
#### Incompatible audio codec (recordings silently fail to save)
|
||||
<FaqItem id="incompatible-audio-codec-recordings-silently-fail-to-save" question="Incompatible audio codec (recordings silently fail to save)">
|
||||
|
||||
Frigate stores recordings in an MP4 container, and some camera audio codecs (most commonly `pcm_alaw`, `pcm_mulaw`, or other G.711 variants) **cannot be placed in an MP4 container**. When this happens, ffmpeg fails to write the segment and no recording is saved, even though the live view works fine. This is a frequent cause on Tapo, TP-Link VIGI, and some Reolink cameras.
|
||||
|
||||
@@ -72,7 +82,9 @@ cameras:
|
||||
# or preset-record-generic to record with no audio
|
||||
```
|
||||
|
||||
#### The record stream isn't connecting
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="the-record-stream-isnt-connecting" question="The record stream isn't connecting">
|
||||
|
||||
A message like `No new recording segments were created for <camera> in the last 120s` means ffmpeg cannot read the `record` stream. To diagnose:
|
||||
|
||||
@@ -81,17 +93,11 @@ A message like `No new recording segments were created for <camera> in the last
|
||||
- Test the exact RTSP URL (with the correct path, port, and credentials) in VLC or `ffplay`.
|
||||
- If you restream through go2rtc, make sure the `record` input path points at the correct go2rtc stream name. Copying a config between cameras without updating the stream name is a common mistake.
|
||||
|
||||
#### Recordings play back with no video (or won't play at all)
|
||||
|
||||
Frigate copies the `record` stream directly without re-encoding, so playback depends on your browser supporting the camera's codec. H265/HEVC recordings may not be playable in some browsers. If recordings appear as audio-only or a black screen, your camera is likely sending a codec your browser can't decode. Configure the camera to output **H264** for maximum compatibility.
|
||||
|
||||
#### Segments are only ~1 second long
|
||||
|
||||
If the record stream uses a "Smart Codec"/H.264+ mode or changes encoding parameters mid-stream, corrupt timestamps cause segments to be split far too frequently and fill the cache. This produces the "Too many unprocessed recording segments" warning. See [that section below](#i-see-the-message-warning--too-many-unprocessed-recording-segments-in-cache-for-camera-this-likely-indicates-an-issue-with-the-detect-stream) for the full diagnosis.
|
||||
</FaqItem>
|
||||
|
||||
### Storage and mounting issues
|
||||
|
||||
#### The storage volume isn't mounted correctly
|
||||
<FaqItem id="the-storage-volume-isnt-mounted-correctly" question="The storage volume isn't mounted correctly">
|
||||
|
||||
If the recordings volume (`/media/frigate`) points at the wrong location, isn't writable, or a network/encrypted mount failed to mount at boot, Frigate cannot save recordings, or it silently writes to the boot drive and then purges aggressively because the drive appears far smaller than expected.
|
||||
|
||||
@@ -100,21 +106,114 @@ If the recordings volume (`/media/frigate`) points at the wrong location, isn't
|
||||
- For a mount that may fail intermittently, protecting the mount point with `chattr +i` on an empty directory forces Frigate to error out (rather than silently writing to the boot drive) when the mount is missing.
|
||||
- Check `dmesg` and system logs for filesystem or I/O errors around the time recordings disappeared.
|
||||
|
||||
If recordings _are_ being written but the copy is too slow to keep up, see the ["Unable to keep up with recording segments"](#i-see-the-message-warning--unable-to-keep-up-with-recording-segments-in-cache-for-camera-keeping-the-5-most-recent-segments-out-of-6-and-discarding-the-rest) section below.
|
||||
If recordings _are_ being written but the copy is too slow to keep up, see the ["Unable to keep up with recording segments"](#i-see-the-message-warning--unable-to-keep-up-with-recording-segments-in-cache-for-camera-keeping-the-5-most-recent-segments-out-of-6-and-discarding-the-rest) question below.
|
||||
|
||||
## I have Frigate configured for motion recording only, but it still seems to be recording even with no motion. Why?
|
||||
</FaqItem>
|
||||
|
||||
You'll want to:
|
||||
## Recordings won't play back
|
||||
|
||||
- Make sure your camera's timestamp is masked out with a motion mask. Even if there is no motion occurring in your scene, your motion settings may be sensitive enough to count your timestamp as motion.
|
||||
- If you have audio detection enabled, keep in mind that audio that is heard above `min_volume` is considered motion.
|
||||
- [Tune your motion detection settings](/configuration/motion_detection) either by editing your config file or by using the UI's Motion Tuner.
|
||||
<FaqItem id="pipeline-error-decode" question={"Recordings won't play back: \"PIPELINE_ERROR_DECODE\" (or \"Media failed to decode\")"}>
|
||||
|
||||
## I see the message: WARNING : Unable to keep up with recording segments in cache for camera. Keeping the 5 most recent segments out of 6 and discarding the rest...
|
||||
When a recording refuses to play in the Frigate UI and you see an error like `Failed to play recordings (error 3): PIPELINE_ERROR_DECODE`, the message is coming from **your browser**, not from Frigate. `PIPELINE_ERROR_DECODE` is emitted exclusively by the media pipeline in **Chromium-based browsers** (Chrome, Edge, Brave, Vivaldi, Opera, Arc, and the Android WebView used by many in-app browsers) when the browser cannot decode a video or audio packet in the recording. WebKit browsers (Safari) report the same underlying problem with a different message, usually `Media failed to decode` or `DECODER_ERROR_NOT_SUPPORTED`.
|
||||
|
||||
Frigate copies the `record` stream to disk **without re-encoding it**, so the browser must decode exactly what your camera produced, and Chromium's decoder is far stricter about malformed or nonstandard media than VLC or ffmpeg.
|
||||
|
||||
:::warning
|
||||
|
||||
The same recording playing perfectly in VLC, decoding cleanly with `ffprobe`/`ffmpeg`, or having a valid MP4 container does **not** mean the browser can decode it. VLC and ffmpeg are much more tolerant of codec quirks and damaged packets than a browser's media pipeline, so a "valid" file can still trigger `PIPELINE_ERROR_DECODE`. This is outside of Frigate's control, because Frigate never modifies the recording stream.
|
||||
|
||||
:::
|
||||
|
||||
#### Step 1: Confirm it is a browser issue
|
||||
|
||||
Open the same recording in **Firefox** or **Safari**. Firefox and Safari both use a different media engine and cannot produce `PIPELINE_ERROR_DECODE`, so if playback works there you have confirmed a client-side codec or decoder problem rather than a bad recording. Switching browsers is a workaround, not a fix; the remaining steps address the root cause so that Chromium browsers work too.
|
||||
|
||||
#### Step 2: Rule out H.265 / HEVC
|
||||
|
||||
Browser support for H.265 (HEVC) is limited and depends on the operating system, GPU, hardware acceleration, and browser version, which makes it the most common cause of this error. Options, in order of reliability:
|
||||
|
||||
- **Record H.264 instead.** Configure the camera's `record`/main stream to output H.264, the most compatible codec across all browsers. See [camera settings recommendations](/configuration/live#camera-settings-recommendations).
|
||||
- **Transcode to H.264 with go2rtc.** If you must keep HEVC on the camera, have go2rtc re-encode the recording stream. This increases CPU usage; add `#hardware` to use the GPU where available:
|
||||
|
||||
```yaml
|
||||
go2rtc:
|
||||
streams:
|
||||
your_camera:
|
||||
# transcode video to h264 and audio to aac; #hardware uses the GPU if available
|
||||
- "ffmpeg:rtsp://user:password@CAMERA_IP:554/stream#video=h264#audio=aac#hardware"
|
||||
cameras:
|
||||
your_camera:
|
||||
ffmpeg:
|
||||
inputs:
|
||||
- path: rtsp://127.0.0.1:8554/your_camera
|
||||
input_args: preset-rtsp-restream
|
||||
roles:
|
||||
- record
|
||||
```
|
||||
|
||||
The `#video=h264` parameter only takes effect with the `ffmpeg:` source module; adding it to a plain `rtsp://` go2rtc source does nothing.
|
||||
|
||||
- **Keep HEVC but improve compatibility.** If your browser and OS do support HEVC, set [`apple_compatibility`](/configuration/camera_specific#h265-cameras-via-safari) on the camera. Some players (Safari and other clients) require a specific HEVC stream format that this option corrects:
|
||||
|
||||
```yaml
|
||||
cameras:
|
||||
your_camera:
|
||||
ffmpeg:
|
||||
apple_compatibility: true
|
||||
```
|
||||
|
||||
You may also need to enable HEVC and hardware decoding in the browser itself (for example, Chrome's Settings → System → "Use hardware acceleration when available"). HEVC hardware support varies widely by GPU, OS, and browser version.
|
||||
|
||||
#### Step 3: Clean up damaged packets from the camera
|
||||
|
||||
If the error is **intermittent** (the same recording plays after a page refresh, or fails only after playing for a while), the camera is most likely emitting occasional corrupt or malformed packets. Some camera models are more prone to this than others. Routing the stream through go2rtc's `ffmpeg` module often "cleans up" the stream enough for the browser to decode it, even without changing the codec:
|
||||
|
||||
```yaml
|
||||
go2rtc:
|
||||
streams:
|
||||
your_camera:
|
||||
- "ffmpeg:rtsp://user:password@CAMERA_IP:554/stream#video=h264#audio=aac"
|
||||
```
|
||||
|
||||
#### Step 4: Fix incompatible or corrupt audio
|
||||
|
||||
Audio is one of the most common culprits, and a decode failure on the audio track fails the whole recording. Make sure the camera outputs **AAC** audio, transcode the audio to AAC with go2rtc (`#audio=aac`), or drop audio entirely. See [Incompatible audio codec](#incompatible-audio-codec-recordings-silently-fail-to-save) for a preset-based way to do this.
|
||||
|
||||
#### Step 5: Avoid "smart" / "+" codecs and check the keyframe interval
|
||||
|
||||
- Disable any **"Smart Codec"**, **"H.264+"**, or **"H.265+"** feature in the camera. These nonstandard modes drop keyframes and change encoding parameters mid-stream, producing exactly the kind of packets a browser refuses to decode. (They also cause [short recording segments](#segments-are-only-1-second-long).)
|
||||
- Set the camera's **I-frame (keyframe) interval equal to the frame rate** (for example `20` for a 20 fps stream). Long keyframe intervals slow the start of playback and make decode errors more likely.
|
||||
|
||||
#### Step 6: Consider bitrate and the client hardware
|
||||
|
||||
The browser decodes the video locally, so a stream that is too demanding can fail on one device while playing on another:
|
||||
|
||||
- A **very high bitrate or resolution** (for example a 4K/8MP HEVC main stream) can overwhelm a low-power tablet, phone, or SBC and stall the decoder. Test the same recording on a desktop; if it plays there, lower the camera's bitrate or record a lower-resolution profile.
|
||||
- Errors that name the client's GPU decoder, such as `VaapiVideoDecoder: failed Initialize()ing the frame pool`, indicate a browser hardware-decode problem. Toggling the browser's "Use hardware acceleration" setting (on or off) often resolves these.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="recordings-play-back-with-no-video-or-wont-play-at-all" question="Recordings play back with no video (or won't play at all)">
|
||||
|
||||
Frigate copies the `record` stream directly without re-encoding, so playback depends on your browser supporting the camera's codec. H265/HEVC recordings may not be playable in some browsers. If recordings appear as audio-only or a black screen, your camera is likely sending a codec your browser can't decode. Configure the camera to output **H264** for maximum compatibility.
|
||||
|
||||
If playback instead fails with an explicit `PIPELINE_ERROR_DECODE` or `Media failed to decode` error, see [Recordings won't play back with "PIPELINE_ERROR_DECODE"](#pipeline-error-decode) above.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
## Recording cache warnings and errors
|
||||
|
||||
<FaqItem id="segments-are-only-1-second-long" question="Segments are only ~1 second long">
|
||||
|
||||
If the record stream uses a "Smart Codec"/H.264+ mode or changes encoding parameters mid-stream, corrupt timestamps cause segments to be split far too frequently and fill the cache. This produces the "Too many unprocessed recording segments" warning. See [that question below](#i-see-the-message-warning--too-many-unprocessed-recording-segments-in-cache-for-camera-this-likely-indicates-an-issue-with-the-detect-stream) for the full diagnosis.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="i-see-the-message-warning--unable-to-keep-up-with-recording-segments-in-cache-for-camera-keeping-the-5-most-recent-segments-out-of-6-and-discarding-the-rest" question="I see the message: WARNING : Unable to keep up with recording segments in cache for camera. Keeping the 5 most recent segments out of 6 and discarding the rest...">
|
||||
|
||||
This warning means the recording maintainer cannot move recording segments from the RAM cache to disk fast enough. When the cache fills up, Frigate discards the oldest segments to avoid running out of memory and crashing, so you lose recorded footage. This is almost always a storage throughput or system resource problem. Work through the steps below to identify which.
|
||||
|
||||
### Step 1: Enable recording debug logging
|
||||
#### Step 1: Enable recording debug logging
|
||||
|
||||
The first step is to measure how long each segment takes to move from the RAM cache to disk. Enable debug logging for the recording maintainer:
|
||||
|
||||
@@ -132,14 +231,14 @@ DEBUG : Copied /media/frigate/recordings/{segment_path} in 0.2 seconds.
|
||||
|
||||
Let this run until the warnings begin to appear, so you can confirm whether the disk is actually slowing down at the moment the error occurs.
|
||||
|
||||
### Step 2: Interpret the copy times
|
||||
#### Step 2: Interpret the copy times
|
||||
|
||||
The copy duration tells you which direction to investigate:
|
||||
|
||||
- **Consistently longer than ~1 second**: your storage cannot keep up with the incoming recordings. Continue with Steps 3–5 to diagnose the slow storage.
|
||||
- **Consistently well under 1 second**: storage is fast enough, and the problem is more likely CPU or resource contention. Skip to Step 6.
|
||||
|
||||
### Step 3: Check RAM, swap, cache, and disk utilization
|
||||
#### Step 3: Check RAM, swap, cache, and disk utilization
|
||||
|
||||
If CPU, RAM, disk throughput, or bus I/O is insufficient, nothing inside Frigate will help. Review each aspect of available system resources while the warnings are occurring.
|
||||
|
||||
@@ -175,19 +274,21 @@ services:
|
||||
|
||||
NOTE: These are hard limits for the container, so be sure there is enough headroom above what `docker stats` shows for your container. It will immediately halt if it hits `<MAXRAM>`. In general, keeping all cache and tmp filespace in RAM is preferable to disk I/O where possible.
|
||||
|
||||
### Step 4: Check your storage type
|
||||
#### Step 4: Check your storage type
|
||||
|
||||
Mounting a network share is a popular option for storing recordings, but it can lead to reduced copy times and cause problems. Some users have found that using `NFS` instead of `SMB` considerably decreased copy times and fixed the issue. It is also important to ensure that the network connection between the device running Frigate and the network share is stable and fast. A saturated or unreliable link will stall copies.
|
||||
|
||||
### Step 5: Check your mount options
|
||||
#### Step 5: Check your mount options
|
||||
|
||||
Some users found that mounting a drive via `fstab` with the `sync` option dramatically reduced performance and led to this issue. Using `async` instead greatly reduced copy times.
|
||||
|
||||
### Step 6: Rule out CPU load
|
||||
#### Step 6: Rule out CPU load
|
||||
|
||||
If the copy times are consistently under 1 second but you still see the warning, the machine's CPU load is likely too high for Frigate to have the resources to keep up. Try temporarily shutting down other services, and any resource-intensive Frigate features, to see if the issue improves.
|
||||
|
||||
## I see the message: WARNING : Too many unprocessed recording segments in cache for camera. This likely indicates an issue with the detect stream...
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="i-see-the-message-warning--too-many-unprocessed-recording-segments-in-cache-for-camera-this-likely-indicates-an-issue-with-the-detect-stream" question="I see the message: WARNING : Too many unprocessed recording segments in cache for camera. This likely indicates an issue with the detect stream...">
|
||||
|
||||
This warning means that the detect stream for the affected camera has fallen behind or stopped processing frames. Frigate's recording cache holds segments waiting to be analyzed by the detector. When more than 6 segments pile up without being processed, Frigate discards the oldest ones to prevent the cache from filling up.
|
||||
|
||||
@@ -197,11 +298,11 @@ This error is a **symptom**, not the root cause. The actual cause is always logg
|
||||
|
||||
:::
|
||||
|
||||
### Step 1: Get the full logs
|
||||
#### Step 1: Get the full logs
|
||||
|
||||
Collect complete Frigate logs from startup through the first occurrence of the error. Look for errors or warnings that appear **before** the "Too many unprocessed" messages begin. That is where the root cause will be found.
|
||||
|
||||
### Step 2: Check the cache directory
|
||||
#### Step 2: Check the cache directory
|
||||
|
||||
Exec into the Frigate container and inspect the recording cache:
|
||||
|
||||
@@ -211,7 +312,7 @@ docker exec -it frigate ls -la /tmp/cache
|
||||
|
||||
Each camera should have a small number of `.mp4` segment files. If one camera has significantly more files than others, that camera is the source of the problem. A problem with a single camera can cascade and cause all cameras to show this error.
|
||||
|
||||
### Step 3: Verify segment duration
|
||||
#### Step 3: Verify segment duration
|
||||
|
||||
Recording segments should be approximately 10 seconds long. Run `ffprobe` on segments in the cache to check:
|
||||
|
||||
@@ -233,7 +334,7 @@ You don't have to run `ffprobe` by hand to catch this. Open a camera's **Camera
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Check for a stuck detector
|
||||
#### Step 4: Check for a stuck detector
|
||||
|
||||
If the detect stream is not processing frames, segments will accumulate. Common causes:
|
||||
|
||||
@@ -242,7 +343,7 @@ If the detect stream is not processing frames, segments will accumulate. Common
|
||||
- **Model too large**: Use smaller model variants (e.g., YOLO `s` or `t` size, not `e` or `x`). Use 320x320 input size rather than 640x640 unless you have a powerful dedicated detector.
|
||||
- **Virtualization**: Running Frigate in a VM (especially Proxmox) can cause the detector to hang or stall. This is a known issue with GPU/TPU passthrough in virtualized environments and is not something Frigate can fix. Running Frigate in Docker on bare metal is recommended.
|
||||
|
||||
### Step 5: Check for GPU hangs
|
||||
#### Step 5: Check for GPU hangs
|
||||
|
||||
On the host machine, check `dmesg` for GPU-related errors:
|
||||
|
||||
@@ -252,7 +353,7 @@ dmesg | grep -i -E "gpu|drm|reset|hang"
|
||||
|
||||
Messages like `trying reset from guc_exec_queue_timedout_job` or similar GPU reset/hang messages indicate a driver or hardware issue. Ensure your kernel and GPU drivers (especially Intel) are up to date.
|
||||
|
||||
### Step 6: Verify hardware acceleration configuration
|
||||
#### Step 6: Verify hardware acceleration configuration
|
||||
|
||||
An incorrect `hwaccel_args` preset can cause ffmpeg to fail silently or consume excessive CPU, starving the detector of resources.
|
||||
|
||||
@@ -260,11 +361,11 @@ An incorrect `hwaccel_args` preset can cause ffmpeg to fail silently or consume
|
||||
- For h265 cameras, use the corresponding h265 preset (e.g., `preset-intel-qsv-h265`).
|
||||
- Note that `hwaccel_args` are only relevant for the detect stream. Frigate does not decode the record stream.
|
||||
|
||||
### Step 7: Verify go2rtc stream configuration
|
||||
#### Step 7: Verify go2rtc stream configuration
|
||||
|
||||
Ensure that the ffmpeg source names in your go2rtc configuration match the correct camera stream. A misconfigured stream name (e.g., copying a config from one camera to another without updating the stream reference) will cause the wrong stream to be used or the stream to fail entirely.
|
||||
|
||||
### Step 8: Check system resources
|
||||
#### Step 8: Check system resources
|
||||
|
||||
If none of the above apply, the issue may be a general resource constraint. Monitor the following on your host:
|
||||
|
||||
@@ -275,7 +376,9 @@ If none of the above apply, the issue may be a general resource constraint. Moni
|
||||
|
||||
Try temporarily disabling resource-intensive features like `genai` and `face_recognition` to see if the issue resolves. This can help isolate whether the detector is being starved of resources.
|
||||
|
||||
## I see the message: ERROR : Error occurred when attempting to maintain recording cache
|
||||
</FaqItem>
|
||||
|
||||
<FaqItem id="i-see-the-message-error--error-occurred-when-attempting-to-maintain-recording-cache" question="I see the message: ERROR : Error occurred when attempting to maintain recording cache">
|
||||
|
||||
This message means the recording maintainer hit an error while moving segments from the cache to disk. It is a **generic wrapper**: the actual cause is always logged on the **very next line**. Frigate usually recovers and keeps running, but any affected segments are lost, so it is worth resolving.
|
||||
|
||||
@@ -287,27 +390,41 @@ Always read the line immediately following this message. `Error occurred when at
|
||||
|
||||
Because these are operating-system-level errors, they must be resolved on the **host**, not within Frigate's configuration. The most common underlying errors are below.
|
||||
|
||||
### [Errno 28] No space left on device
|
||||
#### [Errno 28] No space left on device
|
||||
|
||||
The filesystem Frigate is writing to is full. Things to check:
|
||||
|
||||
- **The recordings volume is genuinely full.** Check free space on the host with `df -h` for the path mapped to `/media/frigate`, and review the **Storage** page in the Frigate UI.
|
||||
- **The disk shows free space but is still "full".** This usually means the filesystem has run out of **inodes** (check with `df -i`), or recordings are landing on a different, smaller filesystem than you expect because of an incorrect bind mount. See [The storage volume isn't mounted correctly](#the-storage-volume-isnt-mounted-correctly) above.
|
||||
- **`/tmp/cache` is full.** If you mounted `/tmp/cache` as a small `tmpfs`, a backlog of segments can fill it. Increase the tmpfs size, or address whatever is causing segments to pile up (see the [Too many unprocessed recording segments](#i-see-the-message-warning--too-many-unprocessed-recording-segments-in-cache-for-camera-this-likely-indicates-an-issue-with-the-detect-stream) section above).
|
||||
- **`/tmp/cache` is full.** If you mounted `/tmp/cache` as a small `tmpfs`, a backlog of segments can fill it. Increase the tmpfs size, or address whatever is causing segments to pile up (see the [Too many unprocessed recording segments](#i-see-the-message-warning--too-many-unprocessed-recording-segments-in-cache-for-camera-this-likely-indicates-an-issue-with-the-detect-stream) question above).
|
||||
- **The host blocks writes before Frigate can purge.** On some systems (for example Unraid with a fill-up threshold), the host stops writes before Frigate's emergency cleanup can run. Leave more headroom on the volume, or lower your retention so Frigate purges sooner.
|
||||
|
||||
### [Errno 17] File exists (with ffmpeg "Error writing trailer" or "unable to re-open output file")
|
||||
#### [Errno 17] File exists (with ffmpeg "Error writing trailer" or "unable to re-open output file")
|
||||
|
||||
Errors like `[Errno 17] File exists: '/media/frigate/recordings/.../<camera>'`, often alongside ffmpeg errors such as `Unable to re-open ... output file for shifting data` or `Error writing trailer: No such file or directory`, are a hallmark of an **unreliable network share** (NFS or SMB). The mount is dropping, serving stale directory entries, or mishandling file locking.
|
||||
|
||||
- Confirm the network connection to the NAS is stable and fast. An intermittent link produces these errors sporadically.
|
||||
- Prefer **NFS over SMB** for the recordings mount; several users have found NFS more reliable and faster.
|
||||
- Review your `fstab`/mount options for settings that hurt consistency or performance (see the `sync` vs `async` note in the [Unable to keep up with recording segments](#i-see-the-message-warning--unable-to-keep-up-with-recording-segments-in-cache-for-camera-keeping-the-5-most-recent-segments-out-of-6-and-discarding-the-rest) section above).
|
||||
- Review your `fstab`/mount options for settings that hurt consistency or performance (see the `sync` vs `async` note in the [Unable to keep up with recording segments](#i-see-the-message-warning--unable-to-keep-up-with-recording-segments-in-cache-for-camera-keeping-the-5-most-recent-segments-out-of-6-and-discarding-the-rest) question above).
|
||||
- Enable `frigate.record.maintainer` debug logging to confirm whether the errors line up with the share becoming unavailable.
|
||||
|
||||
### Errors referencing a camera you manually renamed or removed
|
||||
#### Errors referencing a camera you manually renamed or removed
|
||||
|
||||
If the next-line error references a camera name that no longer exists in your config, orphaned data is left over from a rename or removal in a persistent `/tmp/cache` volume.
|
||||
|
||||
- Using a `tmpfs` mount for `/tmp/cache` as recommended in the [installation docs](/frigate/installation#storage) prevents stale cache files under the old camera name from surviving a restart, which avoids this issue entirely.
|
||||
- If errors persist, stop Frigate and remove any leftover segments for the old camera name from `/tmp/cache`.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
## Other recording questions
|
||||
|
||||
<FaqItem id="i-have-frigate-configured-for-motion-recording-only-but-it-still-seems-to-be-recording-even-with-no-motion-why" question="I have Frigate configured for motion recording only, but it still seems to be recording even with no motion. Why?">
|
||||
|
||||
You'll want to:
|
||||
|
||||
- Make sure your camera's timestamp is masked out with a motion mask. Even if there is no motion occurring in your scene, your motion settings may be sensitive enough to count your timestamp as motion.
|
||||
- If you have audio detection enabled, keep in mind that audio that is heard above `min_volume` is considered motion.
|
||||
- [Tune your motion detection settings](/configuration/motion_detection) either by editing your config file or by using the UI's Motion Tuner.
|
||||
|
||||
</FaqItem>
|
||||
|
||||
@@ -91,7 +91,7 @@ To act on many objects at once, Ctrl/Cmd-click or right-click to start a selecti
|
||||
|
||||
1. Semantic Search is used in conjunction with the other filters available on the Explore page. Use a combination of traditional filtering and Semantic Search for the best results.
|
||||
2. Use the thumbnail search type when searching for particular objects in the scene. Use the description search type when attempting to discern the intent of your object.
|
||||
3. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that with multi-modal (`thumbnail` and `description`) searches, results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the the ones without them are more relevant.
|
||||
3. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that with multi-modal (`thumbnail` and `description`) searches, results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the ones without them are more relevant.
|
||||
4. Make your search language and tone closely match exactly what you're looking for. If you are using thumbnail search, **phrase your query as an image caption**. Searching for "red car" may not work as well as "red sedan driving down a residential street on a sunny day".
|
||||
5. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well.
|
||||
6. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you.
|
||||
|
||||
@@ -60,7 +60,7 @@ You can optionally overlay live streaming statistics (stream type, bandwidth, la
|
||||
|
||||
Right-clicking (or long-pressing) a camera tile opens a context menu with quick controls: an **audio volume** control for streams that support audio, **Mute / Unmute all cameras**, **show or hide streaming statistics**, the **debug view**, **notification** options, and, for admins, turning the camera on or off. If the audio control doesn't appear, see [Audio Support](/configuration/live#audio-support). Audio requires go2rtc configured with a compatible codec.
|
||||
|
||||
A **Low-bandwidth mode** notice may also appear in the context menu with a **Reset** option appears when Frigate has fallen back to the lower-quality jsmpeg stream. See the [Live view FAQ](/configuration/live#live-view-faq) for why this happens.
|
||||
A **Low-bandwidth mode** notice may also appear in the context menu with a **Reset** option when Frigate has fallen back to the lower-quality jsmpeg stream. See the [Live view FAQ](/configuration/live#live-view-faq) for why this happens.
|
||||
|
||||
For non-default groups, the context menu also exposes **Streaming Settings** for that camera, which let you choose:
|
||||
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import React, { useState, useEffect } from "react";
|
||||
import Heading from "@theme/Heading";
|
||||
import styles from "./styles.module.css";
|
||||
|
||||
// A single FAQ entry.
|
||||
//
|
||||
// The question is a real anchored heading (via @theme/Heading), so on desktop
|
||||
// it gets the standard hover "#" hash link and the answer is always shown. On
|
||||
// mobile the heading text is a button that toggles its answer, keeping long
|
||||
// FAQ pages short. The desktop/mobile split is pure CSS (Docusaurus breakpoint:
|
||||
// 996px), so there is no hydration flash. The answer is always rendered into
|
||||
// the DOM, so search engines and the docs AI bot can read it regardless of
|
||||
// layout or collapsed state. The heading id resolves deep links on both layouts
|
||||
// and auto-expands the entry on mobile when it is the link target.
|
||||
export default function FaqItem({ id, question, children }) {
|
||||
const [open, setOpen] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
const openIfTargeted = () => {
|
||||
if (window.location.hash === `#${id}`) {
|
||||
setOpen(true);
|
||||
}
|
||||
};
|
||||
openIfTargeted();
|
||||
window.addEventListener("hashchange", openIfTargeted);
|
||||
return () => window.removeEventListener("hashchange", openIfTargeted);
|
||||
}, [id]);
|
||||
|
||||
const toggle = () => {
|
||||
const next = !open;
|
||||
setOpen(next);
|
||||
// Reflect the entry in the URL like clicking the heading anchor, so an
|
||||
// opened answer is shareable. Use replaceState to avoid history spam and
|
||||
// an abrupt scroll. Clear it on close if it currently points here.
|
||||
if (next) {
|
||||
if (window.location.hash !== `#${id}`) {
|
||||
window.history.replaceState(null, "", `#${id}`);
|
||||
}
|
||||
} else if (window.location.hash === `#${id}`) {
|
||||
window.history.replaceState(
|
||||
null,
|
||||
"",
|
||||
window.location.pathname + window.location.search,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className={styles.item} data-open={open || undefined}>
|
||||
<Heading as="h4" id={id} className={styles.heading}>
|
||||
<button
|
||||
type="button"
|
||||
className={styles.toggle}
|
||||
aria-expanded={open}
|
||||
aria-controls={`${id}-content`}
|
||||
onClick={toggle}
|
||||
>
|
||||
<span className={styles.question}>{question}</span>
|
||||
</button>
|
||||
</Heading>
|
||||
<div id={`${id}-content`} className={styles.content}>
|
||||
{children}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
/*
|
||||
* FAQ entry: collapsible on mobile, static heading + expanded answer on
|
||||
* desktop. The split is pure CSS (Docusaurus breakpoint: 996px) so there is
|
||||
* no hydration flash. The answer is always rendered into the DOM, so search
|
||||
* engines and the docs AI bot can read it regardless of layout or state.
|
||||
*/
|
||||
|
||||
.item {
|
||||
scroll-margin-top: calc(var(--ifm-navbar-height) + 1rem);
|
||||
}
|
||||
|
||||
.heading {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* Mobile: the heading text is a full-width clickable toggle row. */
|
||||
.toggle {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
width: 100%;
|
||||
padding: 0.85rem 0;
|
||||
border: none;
|
||||
border-bottom: 1px solid var(--ifm-color-emphasis-200);
|
||||
background: none;
|
||||
color: inherit;
|
||||
font: inherit;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.toggle::before {
|
||||
content: "";
|
||||
flex: 0 0 auto;
|
||||
width: 0.5rem;
|
||||
height: 0.5rem;
|
||||
border-right: 2px solid currentColor;
|
||||
border-bottom: 2px solid currentColor;
|
||||
transform: rotate(-45deg);
|
||||
transition: transform var(--ifm-transition-fast, 200ms) ease;
|
||||
}
|
||||
|
||||
.item[data-open] .toggle::before {
|
||||
transform: rotate(45deg);
|
||||
}
|
||||
|
||||
.question {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.content {
|
||||
display: none;
|
||||
padding: 0 0 0.85rem;
|
||||
}
|
||||
|
||||
.item[data-open] .content {
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* Hide the hover hash link on mobile (no hover; avoids a stray empty line). */
|
||||
.heading :global(.hash-link) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Desktop: render as a normal expanded heading + answer. */
|
||||
@media (min-width: 997px) {
|
||||
.heading {
|
||||
margin: 1.75rem 0 0.85rem;
|
||||
}
|
||||
|
||||
.toggle {
|
||||
display: inline;
|
||||
width: auto;
|
||||
padding: 0;
|
||||
border: none;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.toggle::before {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.content {
|
||||
display: block;
|
||||
padding: 0 0 0.5rem 1rem;
|
||||
border-left: 2px solid var(--ifm-color-emphasis-200);
|
||||
}
|
||||
|
||||
.heading :global(.hash-link) {
|
||||
display: inline;
|
||||
}
|
||||
}
|
||||
@@ -400,6 +400,9 @@ def verify_objects_track(
|
||||
)
|
||||
camera_config.objects.track = valid_objects
|
||||
|
||||
for label in invalid_objects:
|
||||
camera_config.objects.filters.pop(label, None)
|
||||
|
||||
|
||||
def verify_lpr_and_face(
|
||||
frigate_config: FrigateConfig, camera_config: CameraConfig
|
||||
|
||||
@@ -200,6 +200,9 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
)
|
||||
|
||||
for model_config in self.config.classification.custom.values():
|
||||
if not model_config.enabled:
|
||||
continue
|
||||
|
||||
self.realtime_processors.append(
|
||||
CustomStateClassificationProcessor(
|
||||
self.config, model_config, self.requestor, self.metrics
|
||||
@@ -332,6 +335,25 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
for processor in self.post_processors:
|
||||
processor.update_config(topic, payload)
|
||||
|
||||
def _remove_custom_classification_processor(self, model_name: str) -> None:
|
||||
"""Shut down and drop any running processor for a custom model."""
|
||||
remaining = []
|
||||
for processor in self.realtime_processors:
|
||||
if (
|
||||
isinstance(
|
||||
processor,
|
||||
(
|
||||
CustomStateClassificationProcessor,
|
||||
CustomObjectClassificationProcessor,
|
||||
),
|
||||
)
|
||||
and processor.model_config.name == model_name
|
||||
):
|
||||
processor.shutdown()
|
||||
else:
|
||||
remaining.append(processor)
|
||||
self.realtime_processors = remaining
|
||||
|
||||
def _handle_custom_classification_update(
|
||||
self, topic: str, model_config: Any
|
||||
) -> None:
|
||||
@@ -339,23 +361,7 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
model_name = topic.split("/")[-1]
|
||||
|
||||
if model_config is None:
|
||||
remaining = []
|
||||
for processor in self.realtime_processors:
|
||||
if (
|
||||
isinstance(
|
||||
processor,
|
||||
(
|
||||
CustomStateClassificationProcessor,
|
||||
CustomObjectClassificationProcessor,
|
||||
),
|
||||
)
|
||||
and processor.model_config.name == model_name
|
||||
):
|
||||
processor.shutdown()
|
||||
else:
|
||||
remaining.append(processor)
|
||||
self.realtime_processors = remaining
|
||||
|
||||
self._remove_custom_classification_processor(model_name)
|
||||
logger.info(
|
||||
f"Successfully removed classification processor for model: {model_name}"
|
||||
)
|
||||
@@ -363,20 +369,29 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
|
||||
self.config.classification.custom[model_name] = model_config
|
||||
|
||||
# Check if processor already exists
|
||||
# A disabled model must not run; tear down any existing processor and
|
||||
# do not register a new one.
|
||||
if not model_config.enabled:
|
||||
self._remove_custom_classification_processor(model_name)
|
||||
logger.info(f"Disabled classification processor for model: {model_name}")
|
||||
return
|
||||
|
||||
for processor in self.realtime_processors:
|
||||
if isinstance(
|
||||
processor,
|
||||
(
|
||||
CustomStateClassificationProcessor,
|
||||
CustomObjectClassificationProcessor,
|
||||
),
|
||||
if (
|
||||
isinstance(
|
||||
processor,
|
||||
(
|
||||
CustomStateClassificationProcessor,
|
||||
CustomObjectClassificationProcessor,
|
||||
),
|
||||
)
|
||||
and processor.model_config.name == model_name
|
||||
):
|
||||
if processor.model_config.name == model_name:
|
||||
logger.debug(
|
||||
f"Classification processor for model {model_name} already exists, skipping"
|
||||
)
|
||||
return
|
||||
processor.model_config = model_config
|
||||
logger.debug(
|
||||
f"Updated config for classification processor: {model_name}"
|
||||
)
|
||||
return
|
||||
|
||||
if model_config.state_config is not None:
|
||||
processor = CustomStateClassificationProcessor(
|
||||
@@ -702,7 +717,11 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
and "license_plate" not in camera_config.objects.track
|
||||
)
|
||||
|
||||
if not dedicated_lpr_enabled and len(self.config.classification.custom) == 0:
|
||||
has_enabled_custom = any(
|
||||
c.enabled for c in self.config.classification.custom.values()
|
||||
)
|
||||
|
||||
if not dedicated_lpr_enabled and not has_enabled_custom:
|
||||
# no active features that use this data
|
||||
return
|
||||
|
||||
|
||||
@@ -57,6 +57,12 @@ class BaseEmbedding(ABC):
|
||||
def _preprocess_inputs(self, raw_inputs: Any) -> Any:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _bgr_to_rgb(frame: Any) -> Any:
|
||||
if isinstance(frame, np.ndarray) and frame.ndim == 3:
|
||||
return np.ascontiguousarray(frame[:, :, ::-1])
|
||||
return frame
|
||||
|
||||
def _process_image(self, image, output: str = "RGB") -> Image.Image:
|
||||
if isinstance(image, str):
|
||||
if image.startswith("http"):
|
||||
|
||||
@@ -73,7 +73,7 @@ class FaceNetEmbedding(BaseEmbedding):
|
||||
self.tensor_output_details = self.runner.get_output_details()
|
||||
|
||||
def _preprocess_inputs(self, raw_inputs):
|
||||
pil = self._process_image(raw_inputs[0])
|
||||
pil = self._process_image(self._bgr_to_rgb(raw_inputs[0]))
|
||||
|
||||
# handle images larger than input size
|
||||
width, height = pil.size
|
||||
@@ -159,7 +159,7 @@ class ArcfaceEmbedding(BaseEmbedding):
|
||||
)
|
||||
|
||||
def _preprocess_inputs(self, raw_inputs):
|
||||
pil = self._process_image(raw_inputs[0])
|
||||
pil = self._process_image(self._bgr_to_rgb(raw_inputs[0]))
|
||||
|
||||
# handle images larger than input size
|
||||
width, height = pil.size
|
||||
|
||||
@@ -281,6 +281,11 @@ class GenAIClient:
|
||||
"""Whether the configured model exposes a per-request thinking toggle."""
|
||||
return False
|
||||
|
||||
@property
|
||||
def supports_embeddings(self) -> bool:
|
||||
"""Whether the configured model can generate embeddings via embed()."""
|
||||
return False
|
||||
|
||||
def list_models(self) -> list[str]:
|
||||
"""Return the list of model names available from this provider.
|
||||
|
||||
|
||||
@@ -121,5 +121,6 @@ class GenAIClientManager:
|
||||
"models": client.list_models(),
|
||||
"roles": [r.value for r in genai_cfg.roles],
|
||||
"supports_toggleable_thinking": client.supports_toggleable_thinking,
|
||||
"supports_embeddings": client.supports_embeddings,
|
||||
}
|
||||
return result
|
||||
|
||||
@@ -38,6 +38,37 @@ def _encode_thought_signature(signature: bytes | None) -> str | None:
|
||||
return base64.b64encode(signature).decode("ascii")
|
||||
|
||||
|
||||
def _decode_data_uri(url: str) -> tuple[str, bytes] | None:
|
||||
"""Decode a ``data:`` URI into ``(mime_type, bytes)``; None if not a data URI."""
|
||||
if not isinstance(url, str) or not url.startswith("data:"):
|
||||
return None
|
||||
try:
|
||||
header, b64 = url.split(",", 1)
|
||||
mime = header[len("data:") :].split(";")[0] or "image/jpeg"
|
||||
return mime, base64.b64decode(b64)
|
||||
except (ValueError, binascii.Error):
|
||||
return None
|
||||
|
||||
|
||||
def _parts_from_content(content: Any) -> list[types.Part]:
|
||||
"""Convert OpenAI-style message content (str or multimodal list) to Gemini parts."""
|
||||
if isinstance(content, list):
|
||||
parts: list[types.Part] = []
|
||||
for item in content:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
if item.get("type") == "text":
|
||||
parts.append(types.Part.from_text(text=item.get("text") or ""))
|
||||
elif item.get("type") == "image_url":
|
||||
decoded = _decode_data_uri((item.get("image_url") or {}).get("url", ""))
|
||||
if decoded is not None:
|
||||
mime, data = decoded
|
||||
parts.append(types.Part.from_bytes(data=data, mime_type=mime))
|
||||
# Gemini rejects empty parts; fall back to a single space.
|
||||
return parts or [types.Part.from_text(text=" ")]
|
||||
return [types.Part.from_text(text=content or "")]
|
||||
|
||||
|
||||
def _stats_from_gemini_usage(usage: Any) -> dict[str, Any] | None:
|
||||
"""Build a stats dict from a Gemini usage_metadata object."""
|
||||
prompt_tokens = getattr(usage, "prompt_token_count", None)
|
||||
@@ -227,9 +258,7 @@ class GeminiClient(GenAIClient):
|
||||
)
|
||||
else: # user
|
||||
gemini_messages.append(
|
||||
types.Content(
|
||||
role="user", parts=[types.Part.from_text(text=content)]
|
||||
)
|
||||
types.Content(role="user", parts=_parts_from_content(content))
|
||||
)
|
||||
|
||||
# Convert tools to Gemini format
|
||||
@@ -485,9 +514,7 @@ class GeminiClient(GenAIClient):
|
||||
)
|
||||
else: # user
|
||||
gemini_messages.append(
|
||||
types.Content(
|
||||
role="user", parts=[types.Part.from_text(text=content)]
|
||||
)
|
||||
types.Content(role="user", parts=_parts_from_content(content))
|
||||
)
|
||||
|
||||
# Convert tools to Gemini format
|
||||
@@ -553,7 +580,7 @@ class GeminiClient(GenAIClient):
|
||||
# Use streaming API
|
||||
content_parts: list[str] = []
|
||||
reasoning_parts: list[str] = []
|
||||
tool_calls_by_index: dict[int, dict[str, Any]] = {}
|
||||
tool_calls_accum: list[dict[str, Any]] = []
|
||||
finish_reason = "stop"
|
||||
usage_stats: dict[str, Any] | None = None
|
||||
|
||||
@@ -600,7 +627,11 @@ class GeminiClient(GenAIClient):
|
||||
content_parts.append(part.text)
|
||||
yield ("content_delta", part.text)
|
||||
elif part.function_call:
|
||||
# Handle function call
|
||||
# Gemini streams complete function calls (not partial
|
||||
# argument deltas), so each part is a distinct tool
|
||||
# call. Append rather than accumulate by name — the
|
||||
# latter concatenated parallel/repeated calls into one
|
||||
# invalid arguments string (e.g. `{...}{...}`).
|
||||
try:
|
||||
arguments = (
|
||||
dict(part.function_call.args)
|
||||
@@ -610,40 +641,16 @@ class GeminiClient(GenAIClient):
|
||||
except Exception:
|
||||
arguments = {}
|
||||
|
||||
# Store tool call
|
||||
tool_call_id = part.function_call.name or ""
|
||||
tool_call_name = part.function_call.name or ""
|
||||
|
||||
# Check if we already have this tool call
|
||||
found_index = None
|
||||
for idx, tc in tool_calls_by_index.items():
|
||||
if tc["name"] == tool_call_name:
|
||||
found_index = idx
|
||||
break
|
||||
|
||||
if found_index is None:
|
||||
found_index = len(tool_calls_by_index)
|
||||
tool_calls_by_index[found_index] = {
|
||||
"id": tool_call_id,
|
||||
"name": tool_call_name,
|
||||
"arguments": "",
|
||||
"thought_signature": None,
|
||||
tool_calls_accum.append(
|
||||
{
|
||||
"id": part.function_call.name or "",
|
||||
"name": part.function_call.name or "",
|
||||
"arguments": arguments,
|
||||
"thought_signature": getattr(
|
||||
part, "thought_signature", None
|
||||
),
|
||||
}
|
||||
|
||||
# Accumulate arguments
|
||||
if arguments:
|
||||
tool_calls_by_index[found_index]["arguments"] += (
|
||||
json.dumps(arguments)
|
||||
if isinstance(arguments, dict)
|
||||
else str(arguments)
|
||||
)
|
||||
|
||||
# Capture latest thought_signature for this call
|
||||
chunk_sig = getattr(part, "thought_signature", None)
|
||||
if chunk_sig:
|
||||
tool_calls_by_index[found_index][
|
||||
"thought_signature"
|
||||
] = chunk_sig
|
||||
)
|
||||
|
||||
# Build final message
|
||||
full_content = "".join(content_parts).strip() or None
|
||||
@@ -651,25 +658,20 @@ class GeminiClient(GenAIClient):
|
||||
|
||||
# Convert tool calls to list format
|
||||
tool_calls_list = None
|
||||
if tool_calls_by_index:
|
||||
tool_calls_list = []
|
||||
for tc in tool_calls_by_index.values():
|
||||
try:
|
||||
# Try to parse accumulated arguments as JSON
|
||||
parsed_args = json.loads(tc["arguments"])
|
||||
except (json.JSONDecodeError, Exception):
|
||||
parsed_args = tc["arguments"]
|
||||
|
||||
tool_calls_list.append(
|
||||
{
|
||||
"id": tc["id"],
|
||||
"name": tc["name"],
|
||||
"arguments": parsed_args,
|
||||
"thought_signature": _encode_thought_signature(
|
||||
tc.get("thought_signature")
|
||||
),
|
||||
}
|
||||
)
|
||||
if tool_calls_accum:
|
||||
tool_calls_list = [
|
||||
{
|
||||
"id": tc["id"],
|
||||
"name": tc["name"],
|
||||
"arguments": tc["arguments"]
|
||||
if isinstance(tc["arguments"], dict)
|
||||
else {},
|
||||
"thought_signature": _encode_thought_signature(
|
||||
tc.get("thought_signature")
|
||||
),
|
||||
}
|
||||
for tc in tool_calls_accum
|
||||
]
|
||||
finish_reason = "tool_calls"
|
||||
|
||||
if usage_stats is not None:
|
||||
|
||||
@@ -128,6 +128,11 @@ class LlamaCppClient(GenAIClient):
|
||||
_text_baseline_tokens: int | None
|
||||
_media_marker: str
|
||||
|
||||
@property
|
||||
def supports_embeddings(self) -> bool:
|
||||
"""llama.cpp exposes an /embeddings endpoint for any loaded model."""
|
||||
return True
|
||||
|
||||
def _init_provider(self) -> str | None:
|
||||
"""Initialize the client and query model metadata from the server."""
|
||||
self.provider_options = {
|
||||
|
||||
@@ -423,9 +423,18 @@ class OpenAIClient(GenAIClient):
|
||||
for tc in tool_calls_by_index.values():
|
||||
try:
|
||||
# Parse accumulated arguments as JSON
|
||||
parsed_args = json.loads(tc["arguments"])
|
||||
except (json.JSONDecodeError, Exception):
|
||||
parsed_args = tc["arguments"]
|
||||
parsed_args = json.loads(tc["arguments"] or "{}")
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
logger.warning(
|
||||
"Failed to parse streamed tool call arguments for %s",
|
||||
tc["name"],
|
||||
)
|
||||
parsed_args = {}
|
||||
|
||||
# Downstream (ToolCall model) requires a dict; never leak a
|
||||
# partial/invalid arguments string.
|
||||
if not isinstance(parsed_args, dict):
|
||||
parsed_args = {}
|
||||
|
||||
tool_calls_list.append(
|
||||
{
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Tests that disabled custom classification models are not registered or run."""
|
||||
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
# Mock TFLite before importing the maintainer / classification modules
|
||||
_MOCK_MODULES = [
|
||||
"tflite_runtime",
|
||||
"tflite_runtime.interpreter",
|
||||
"ai_edge_litert",
|
||||
"ai_edge_litert.interpreter",
|
||||
]
|
||||
for mod in _MOCK_MODULES:
|
||||
if mod not in sys.modules:
|
||||
sys.modules[mod] = MagicMock()
|
||||
|
||||
from frigate.data_processing.real_time.custom_classification import ( # noqa: E402
|
||||
CustomObjectClassificationProcessor,
|
||||
)
|
||||
from frigate.embeddings.maintainer import EmbeddingMaintainer # noqa: E402
|
||||
|
||||
|
||||
class TestCustomClassificationEnabledGating(unittest.TestCase):
|
||||
"""A model with enabled: false must not keep a processor registered."""
|
||||
|
||||
def _make_maintainer(self) -> EmbeddingMaintainer:
|
||||
# Bypass the heavy __init__; only the attributes touched by the
|
||||
# config update path are needed for these tests.
|
||||
maintainer = EmbeddingMaintainer.__new__(EmbeddingMaintainer)
|
||||
maintainer.realtime_processors = []
|
||||
maintainer.config = MagicMock()
|
||||
maintainer.config.classification.custom = {}
|
||||
maintainer.requestor = MagicMock()
|
||||
maintainer.metrics = MagicMock()
|
||||
maintainer.event_metadata_publisher = MagicMock()
|
||||
return maintainer
|
||||
|
||||
def _make_model_config(self, name: str, enabled: bool) -> MagicMock:
|
||||
model_config = MagicMock()
|
||||
model_config.name = name
|
||||
model_config.enabled = enabled
|
||||
model_config.state_config = None
|
||||
return model_config
|
||||
|
||||
def _make_processor(self, name: str) -> MagicMock:
|
||||
processor = MagicMock(spec=CustomObjectClassificationProcessor)
|
||||
processor.model_config = MagicMock()
|
||||
processor.model_config.name = name
|
||||
return processor
|
||||
|
||||
def test_disabled_update_tears_down_existing_processor(self):
|
||||
"""Toggling a running model to disabled shuts down and drops its processor."""
|
||||
maintainer = self._make_maintainer()
|
||||
processor = self._make_processor("atli")
|
||||
maintainer.realtime_processors = [processor]
|
||||
|
||||
maintainer._handle_custom_classification_update(
|
||||
"config/classification/custom/atli",
|
||||
self._make_model_config("atli", enabled=False),
|
||||
)
|
||||
|
||||
processor.shutdown.assert_called_once()
|
||||
self.assertEqual(maintainer.realtime_processors, [])
|
||||
|
||||
def test_disabled_update_does_not_register_processor(self):
|
||||
"""A disabled model that has no processor is never registered."""
|
||||
maintainer = self._make_maintainer()
|
||||
|
||||
maintainer._handle_custom_classification_update(
|
||||
"config/classification/custom/atli",
|
||||
self._make_model_config("atli", enabled=False),
|
||||
)
|
||||
|
||||
self.assertEqual(maintainer.realtime_processors, [])
|
||||
|
||||
def test_disabled_update_leaves_other_processors_untouched(self):
|
||||
"""Disabling one model must not affect other running processors."""
|
||||
maintainer = self._make_maintainer()
|
||||
other = self._make_processor("simbi")
|
||||
maintainer.realtime_processors = [other]
|
||||
|
||||
maintainer._handle_custom_classification_update(
|
||||
"config/classification/custom/atli",
|
||||
self._make_model_config("atli", enabled=False),
|
||||
)
|
||||
|
||||
other.shutdown.assert_not_called()
|
||||
self.assertEqual(maintainer.realtime_processors, [other])
|
||||
|
||||
def test_removed_model_tears_down_processor(self):
|
||||
"""A None payload (model deleted) still shuts down its processor."""
|
||||
maintainer = self._make_maintainer()
|
||||
processor = self._make_processor("atli")
|
||||
maintainer.realtime_processors = [processor]
|
||||
|
||||
maintainer._handle_custom_classification_update(
|
||||
"config/classification/custom/atli", None
|
||||
)
|
||||
|
||||
processor.shutdown.assert_called_once()
|
||||
self.assertEqual(maintainer.realtime_processors, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -397,6 +397,43 @@ class TestConfig(unittest.TestCase):
|
||||
assert "dog" in frigate_config.cameras["back"].objects.filters
|
||||
assert frigate_config.cameras["back"].objects.filters["dog"].threshold == 0.7
|
||||
|
||||
def test_unsupported_tracked_object_pruned_from_track_and_filters(self):
|
||||
# "unicorn" is not in the model labelmap, so it must be removed from the
|
||||
# tracked objects AND from the object filters, otherwise a stale filter
|
||||
# entry lingers in the parsed config.
|
||||
config = {
|
||||
"mqtt": {"host": "mqtt"},
|
||||
"cameras": {
|
||||
"back": {
|
||||
"ffmpeg": {
|
||||
"inputs": [
|
||||
{"path": "rtsp://10.0.0.1:554/video", "roles": ["detect"]}
|
||||
]
|
||||
},
|
||||
"detect": {
|
||||
"height": 1080,
|
||||
"width": 1920,
|
||||
"fps": 5,
|
||||
},
|
||||
"objects": {
|
||||
"track": ["person", "unicorn"],
|
||||
"filters": {
|
||||
"person": {"threshold": 0.7},
|
||||
"unicorn": {"threshold": 0.7},
|
||||
},
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
frigate_config = FrigateConfig(**config)
|
||||
objects = frigate_config.cameras["back"].objects
|
||||
assert "unicorn" not in objects.track
|
||||
assert "unicorn" not in objects.filters
|
||||
# supported entries are left untouched
|
||||
assert "person" in objects.track
|
||||
assert "person" in objects.filters
|
||||
|
||||
def test_global_object_mask(self):
|
||||
config = {
|
||||
"mqtt": {"host": "mqtt"},
|
||||
|
||||
@@ -0,0 +1,496 @@
|
||||
"""Smoke tests for GenAI chat providers.
|
||||
|
||||
Each provider's ``chat_with_tools_stream`` is driven with a canned "test
|
||||
response" so the two conversion layers are exercised without any network:
|
||||
|
||||
1. Frigate (OpenAI-style) messages -> provider-native request format
|
||||
2. provider-native response -> Frigate ``("kind", value)`` stream events
|
||||
|
||||
These guard against regressions such as tool-call arguments arriving as raw
|
||||
strings instead of dicts (which crash the ``ToolCall`` model), and multimodal
|
||||
user content (a list of text/image parts, as injected by ``get_live_context``)
|
||||
crashing message conversion.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
from frigate.config import GenAIConfig, GenAIProviderEnum
|
||||
from frigate.genai import PROVIDERS, load_providers
|
||||
|
||||
load_providers()
|
||||
|
||||
# A minimal but valid JPEG data URI, mirroring what get_live_context injects.
|
||||
_TINY_JPEG = base64.b64encode(b"\xff\xd8\xff\xd9").decode("ascii")
|
||||
_IMAGE_DATA_URI = f"data:image/jpeg;base64,{_TINY_JPEG}"
|
||||
|
||||
# Conversation ending in a multimodal user message (text + live image), the
|
||||
# exact shape the chat endpoint builds after a get_live_context tool result.
|
||||
MULTIMODAL_MESSAGES = [
|
||||
{"role": "system", "content": "You are a test assistant."},
|
||||
{"role": "user", "content": "what do you see on the front camera?"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_live_context",
|
||||
"arguments": json.dumps({"camera": "front"}),
|
||||
},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_1",
|
||||
"name": "get_live_context",
|
||||
"content": json.dumps({"camera": "front"}),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Here is the current live image from camera 'front'.",
|
||||
},
|
||||
{"type": "image_url", "image_url": {"url": _IMAGE_DATA_URI}},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
SIMPLE_MESSAGES = [
|
||||
{"role": "system", "content": "You are a test assistant."},
|
||||
{"role": "user", "content": "hello"},
|
||||
]
|
||||
|
||||
TOOLS = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "search_objects",
|
||||
"description": "Search tracked objects",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {"label": {"type": "string"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def _make_client(provider: str, **cfg_overrides):
|
||||
"""Build a provider client offline (no model validation, no network)."""
|
||||
cfg = GenAIConfig(provider=provider, **cfg_overrides)
|
||||
cls = PROVIDERS[GenAIProviderEnum(provider)]
|
||||
return cls(cfg, timeout=5, validate_model=False)
|
||||
|
||||
|
||||
def _collect(client, messages, tools=TOOLS):
|
||||
"""Drain chat_with_tools_stream into a list of (kind, value) events."""
|
||||
|
||||
async def _run():
|
||||
events = []
|
||||
async for event in client.chat_with_tools_stream(
|
||||
messages=messages, tools=tools, tool_choice="auto"
|
||||
):
|
||||
events.append(event)
|
||||
return events
|
||||
|
||||
return asyncio.run(_run())
|
||||
|
||||
|
||||
def _final_message(events) -> dict:
|
||||
messages = [value for (kind, value) in events if kind == "message"]
|
||||
assert messages, f"stream produced no final message: {events}"
|
||||
return messages[-1]
|
||||
|
||||
|
||||
def _assert_tool_args_are_dicts(final: dict) -> None:
|
||||
"""Every returned tool call must expose arguments as a dict, never a string."""
|
||||
for tool_call in final.get("tool_calls") or []:
|
||||
assert isinstance(tool_call["arguments"], dict), (
|
||||
f"tool call arguments must be a dict, got "
|
||||
f"{type(tool_call['arguments']).__name__}: {tool_call['arguments']!r}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# OpenAI
|
||||
# ---------------------------------------------------------------------------
|
||||
def _openai_tc(index, id=None, name=None, arguments=None):
|
||||
return SimpleNamespace(
|
||||
index=index,
|
||||
id=id,
|
||||
function=SimpleNamespace(name=name, arguments=arguments),
|
||||
)
|
||||
|
||||
|
||||
def _openai_chunk(content=None, tool_calls=None, finish_reason=None, usage=None):
|
||||
delta = SimpleNamespace(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
reasoning_content=None,
|
||||
reasoning=None,
|
||||
)
|
||||
choice = SimpleNamespace(delta=delta, finish_reason=finish_reason)
|
||||
return SimpleNamespace(choices=[choice], usage=usage)
|
||||
|
||||
|
||||
class TestOpenAIProvider(unittest.TestCase):
|
||||
def _client(self):
|
||||
return _make_client(
|
||||
"openai", model="gpt-4o", api_key="k", base_url="http://localhost:9999/v1"
|
||||
)
|
||||
|
||||
def test_stream_tool_call_arguments_are_dict(self):
|
||||
# Arguments arrive split across chunks, as the real API streams them.
|
||||
chunks = [
|
||||
_openai_chunk(
|
||||
tool_calls=[
|
||||
_openai_tc(0, id="c1", name="search_objects", arguments='{"label":')
|
||||
]
|
||||
),
|
||||
_openai_chunk(tool_calls=[_openai_tc(0, arguments=' "person"}')]),
|
||||
_openai_chunk(finish_reason="tool_calls"),
|
||||
]
|
||||
client = self._client()
|
||||
client.provider.chat.completions.create = MagicMock(return_value=iter(chunks))
|
||||
|
||||
final = _final_message(_collect(client, SIMPLE_MESSAGES))
|
||||
self.assertEqual(final["finish_reason"], "tool_calls")
|
||||
self.assertEqual(len(final["tool_calls"]), 1)
|
||||
_assert_tool_args_are_dicts(final)
|
||||
self.assertEqual(final["tool_calls"][0]["arguments"], {"label": "person"})
|
||||
|
||||
def test_stream_content_response(self):
|
||||
chunks = [
|
||||
_openai_chunk(content="hel"),
|
||||
_openai_chunk(content="lo"),
|
||||
_openai_chunk(finish_reason="stop"),
|
||||
]
|
||||
client = self._client()
|
||||
client.provider.chat.completions.create = MagicMock(return_value=iter(chunks))
|
||||
|
||||
events = _collect(client, SIMPLE_MESSAGES)
|
||||
deltas = [v for (k, v) in events if k == "content_delta"]
|
||||
self.assertEqual("".join(deltas), "hello")
|
||||
self.assertEqual(_final_message(events)["content"], "hello")
|
||||
|
||||
def test_multimodal_message_does_not_crash(self):
|
||||
client = self._client()
|
||||
client.provider.chat.completions.create = MagicMock(
|
||||
return_value=iter([_openai_chunk(content="ok", finish_reason="stop")])
|
||||
)
|
||||
# Passing the OpenAI-native multimodal list through must not raise.
|
||||
final = _final_message(_collect(client, MULTIMODAL_MESSAGES))
|
||||
self.assertEqual(final["content"], "ok")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Gemini
|
||||
# ---------------------------------------------------------------------------
|
||||
def _gemini_part(text=None, thought=False, function_call=None, thought_signature=None):
|
||||
return SimpleNamespace(
|
||||
text=text,
|
||||
thought=thought,
|
||||
function_call=function_call,
|
||||
thought_signature=thought_signature,
|
||||
)
|
||||
|
||||
|
||||
def _gemini_chunk(parts, finish_reason=None, usage_metadata=None):
|
||||
candidate = SimpleNamespace(
|
||||
content=SimpleNamespace(parts=parts), finish_reason=finish_reason
|
||||
)
|
||||
return SimpleNamespace(candidates=[candidate], usage_metadata=usage_metadata)
|
||||
|
||||
|
||||
def _gemini_stream(chunks):
|
||||
async def _agen(*args, **kwargs):
|
||||
for chunk in chunks:
|
||||
yield chunk
|
||||
|
||||
return _agen
|
||||
|
||||
|
||||
class TestGeminiProvider(unittest.TestCase):
|
||||
def _client(self):
|
||||
return _make_client("gemini", model="gemini-2.5-flash", api_key="k")
|
||||
|
||||
def _patch_stream(self, client, chunks):
|
||||
client.provider = MagicMock()
|
||||
client.provider.aio.models.generate_content_stream = AsyncMock(
|
||||
side_effect=_gemini_stream(chunks)
|
||||
)
|
||||
|
||||
def test_stream_parallel_tool_calls_stay_separate_dicts(self):
|
||||
# Regression: Gemini streams complete function calls. Two calls to the
|
||||
# same tool must NOT be merged into one concatenated arguments string.
|
||||
from google.genai.types import FinishReason
|
||||
|
||||
chunks = [
|
||||
_gemini_chunk(
|
||||
parts=[
|
||||
_gemini_part(
|
||||
function_call=SimpleNamespace(
|
||||
name="search_objects", args={"label": "person"}
|
||||
)
|
||||
),
|
||||
_gemini_part(
|
||||
function_call=SimpleNamespace(
|
||||
name="search_objects", args={"limit": 1}
|
||||
)
|
||||
),
|
||||
],
|
||||
finish_reason=FinishReason.STOP,
|
||||
),
|
||||
]
|
||||
client = self._client()
|
||||
self._patch_stream(client, chunks)
|
||||
|
||||
final = _final_message(_collect(client, SIMPLE_MESSAGES))
|
||||
self.assertEqual(final["finish_reason"], "tool_calls")
|
||||
self.assertEqual(len(final["tool_calls"]), 2)
|
||||
_assert_tool_args_are_dicts(final)
|
||||
self.assertEqual(final["tool_calls"][0]["arguments"], {"label": "person"})
|
||||
self.assertEqual(final["tool_calls"][1]["arguments"], {"limit": 1})
|
||||
|
||||
def test_stream_content_response(self):
|
||||
from google.genai.types import FinishReason
|
||||
|
||||
chunks = [
|
||||
_gemini_chunk(parts=[_gemini_part(text="hel")]),
|
||||
_gemini_chunk(
|
||||
parts=[_gemini_part(text="lo")], finish_reason=FinishReason.STOP
|
||||
),
|
||||
]
|
||||
client = self._client()
|
||||
self._patch_stream(client, chunks)
|
||||
|
||||
events = _collect(client, SIMPLE_MESSAGES)
|
||||
deltas = [v for (k, v) in events if k == "content_delta"]
|
||||
self.assertEqual("".join(deltas), "hello")
|
||||
self.assertEqual(_final_message(events)["content"], "hello")
|
||||
|
||||
def test_multimodal_message_converts_without_crash(self):
|
||||
# Regression: a user message with list content (text + image_url) used
|
||||
# to be handed to Part.from_text(text=<list>) and raise ValidationError.
|
||||
from google.genai.types import FinishReason
|
||||
|
||||
client = self._client()
|
||||
self._patch_stream(
|
||||
client,
|
||||
[
|
||||
_gemini_chunk(
|
||||
parts=[_gemini_part(text="ok")], finish_reason=FinishReason.STOP
|
||||
)
|
||||
],
|
||||
)
|
||||
final = _final_message(_collect(client, MULTIMODAL_MESSAGES))
|
||||
self.assertEqual(final["content"], "ok")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ollama
|
||||
# ---------------------------------------------------------------------------
|
||||
class TestOllamaProvider(unittest.TestCase):
|
||||
def _client(self):
|
||||
return _make_client("ollama", model="llama3", base_url="http://localhost:9999")
|
||||
|
||||
def _run_with_response(self, client, response, messages):
|
||||
# Ollama uses a non-streaming call when tools are present, via an
|
||||
# internally-constructed async client.
|
||||
fake_async = MagicMock()
|
||||
fake_async.chat = AsyncMock(return_value=response)
|
||||
with patch(
|
||||
"frigate.genai.plugins.ollama.OllamaAsyncClient",
|
||||
return_value=fake_async,
|
||||
):
|
||||
return _collect(client, messages)
|
||||
|
||||
def test_tool_call_arguments_are_dict(self):
|
||||
response = {
|
||||
"message": {
|
||||
"content": "",
|
||||
"tool_calls": [
|
||||
{
|
||||
"function": {
|
||||
"name": "search_objects",
|
||||
"arguments": {"label": "person"},
|
||||
}
|
||||
}
|
||||
],
|
||||
},
|
||||
"done": True,
|
||||
"done_reason": "stop",
|
||||
"eval_count": 5,
|
||||
"prompt_eval_count": 3,
|
||||
"eval_duration": 1_000_000,
|
||||
}
|
||||
client = self._client()
|
||||
final = _final_message(
|
||||
self._run_with_response(client, response, SIMPLE_MESSAGES)
|
||||
)
|
||||
self.assertEqual(final["finish_reason"], "tool_calls")
|
||||
_assert_tool_args_are_dicts(final)
|
||||
self.assertEqual(final["tool_calls"][0]["arguments"], {"label": "person"})
|
||||
|
||||
def test_multimodal_message_normalizes_image(self):
|
||||
# Ollama needs content as a string with images pulled into a separate
|
||||
# field; the normalizer must extract both without crashing.
|
||||
response = {
|
||||
"message": {"content": "ok"},
|
||||
"done": True,
|
||||
"done_reason": "stop",
|
||||
}
|
||||
client = self._client()
|
||||
final = _final_message(
|
||||
self._run_with_response(client, response, MULTIMODAL_MESSAGES)
|
||||
)
|
||||
self.assertEqual(final["content"], "ok")
|
||||
|
||||
def test_normalize_multimodal_content(self):
|
||||
from frigate.genai.plugins.ollama import _normalize_multimodal_content
|
||||
|
||||
text, images = _normalize_multimodal_content(MULTIMODAL_MESSAGES[-1]["content"])
|
||||
self.assertIn("live image", text)
|
||||
self.assertEqual(len(images), 1)
|
||||
self.assertEqual(images[0], b"\xff\xd8\xff\xd9")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# llama.cpp
|
||||
# ---------------------------------------------------------------------------
|
||||
class _FakeStreamResponse:
|
||||
def __init__(self, lines):
|
||||
self._lines = lines
|
||||
|
||||
def raise_for_status(self):
|
||||
return None
|
||||
|
||||
async def aiter_lines(self):
|
||||
for line in self._lines:
|
||||
yield line
|
||||
|
||||
|
||||
class _FakeStreamCtx:
|
||||
def __init__(self, lines):
|
||||
self._resp = _FakeStreamResponse(lines)
|
||||
|
||||
async def __aenter__(self):
|
||||
return self._resp
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
return False
|
||||
|
||||
|
||||
class _FakeAsyncClient:
|
||||
def __init__(self, lines):
|
||||
self._lines = lines
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
return False
|
||||
|
||||
def stream(self, method, url, json=None):
|
||||
return _FakeStreamCtx(self._lines)
|
||||
|
||||
|
||||
class TestLlamaCppProvider(unittest.TestCase):
|
||||
def _client(self):
|
||||
return _make_client("llamacpp", model="m", base_url="http://localhost:9999")
|
||||
|
||||
def _run_with_lines(self, client, lines, messages):
|
||||
with patch(
|
||||
"frigate.genai.plugins.llama_cpp.httpx.AsyncClient",
|
||||
return_value=_FakeAsyncClient(lines),
|
||||
):
|
||||
return _collect(client, messages)
|
||||
|
||||
def test_stream_tool_call_arguments_are_dict(self):
|
||||
lines = [
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 0,
|
||||
"id": "c1",
|
||||
"function": {
|
||||
"name": "search_objects",
|
||||
"arguments": '{"label":',
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
),
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 0,
|
||||
"function": {"arguments": ' "person"}'},
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
),
|
||||
"data: "
|
||||
+ json.dumps({"choices": [{"delta": {}, "finish_reason": "tool_calls"}]}),
|
||||
"data: [DONE]",
|
||||
]
|
||||
client = self._client()
|
||||
final = _final_message(self._run_with_lines(client, lines, SIMPLE_MESSAGES))
|
||||
self.assertEqual(final["finish_reason"], "tool_calls")
|
||||
_assert_tool_args_are_dicts(final)
|
||||
self.assertEqual(final["tool_calls"][0]["arguments"], {"label": "person"})
|
||||
|
||||
def test_stream_content_response(self):
|
||||
lines = [
|
||||
"data: " + json.dumps({"choices": [{"delta": {"content": "hel"}}]}),
|
||||
"data: " + json.dumps({"choices": [{"delta": {"content": "lo"}}]}),
|
||||
"data: "
|
||||
+ json.dumps({"choices": [{"delta": {}, "finish_reason": "stop"}]}),
|
||||
"data: [DONE]",
|
||||
]
|
||||
client = self._client()
|
||||
events = self._run_with_lines(client, lines, SIMPLE_MESSAGES)
|
||||
deltas = [v for (k, v) in events if k == "content_delta"]
|
||||
self.assertEqual("".join(deltas), "hello")
|
||||
self.assertEqual(_final_message(events)["content"], "hello")
|
||||
|
||||
def test_multimodal_message_does_not_crash(self):
|
||||
lines = [
|
||||
"data: " + json.dumps({"choices": [{"delta": {"content": "ok"}}]}),
|
||||
"data: "
|
||||
+ json.dumps({"choices": [{"delta": {}, "finish_reason": "stop"}]}),
|
||||
"data: [DONE]",
|
||||
]
|
||||
client = self._client()
|
||||
final = _final_message(self._run_with_lines(client, lines, MULTIMODAL_MESSAGES))
|
||||
self.assertEqual(final["content"], "ok")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -40,18 +40,18 @@ class TestGpuStats(unittest.TestCase):
|
||||
"driver": "i915",
|
||||
"pid": "100",
|
||||
"engines": {
|
||||
"render": (1_000_000_000, 0),
|
||||
"video": (5_000_000_000, 0),
|
||||
"video-enhance": (200_000_000, 0),
|
||||
"compute": (0, 0),
|
||||
"render": (1_000_000_000, 0, 1),
|
||||
"video": (5_000_000_000, 0, 1),
|
||||
"video-enhance": (200_000_000, 0, 1),
|
||||
"compute": (0, 0, 1),
|
||||
},
|
||||
},
|
||||
("0000:00:02.0", "2", "200"): {
|
||||
"driver": "i915",
|
||||
"pid": "200",
|
||||
"engines": {
|
||||
"render": (0, 0),
|
||||
"compute": (2_000_000_000, 0),
|
||||
"render": (0, 0, 1),
|
||||
"compute": (2_000_000_000, 0, 1),
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -60,18 +60,18 @@ class TestGpuStats(unittest.TestCase):
|
||||
"driver": "i915",
|
||||
"pid": "100",
|
||||
"engines": {
|
||||
"render": (1_200_000_000, 0),
|
||||
"video": (5_500_000_000, 0),
|
||||
"video-enhance": (300_000_000, 0),
|
||||
"compute": (0, 0),
|
||||
"render": (1_200_000_000, 0, 1),
|
||||
"video": (5_500_000_000, 0, 1),
|
||||
"video-enhance": (300_000_000, 0, 1),
|
||||
"compute": (0, 0, 1),
|
||||
},
|
||||
},
|
||||
("0000:00:02.0", "2", "200"): {
|
||||
"driver": "i915",
|
||||
"pid": "200",
|
||||
"engines": {
|
||||
"render": (0, 0),
|
||||
"compute": (2_100_000_000, 0),
|
||||
"render": (0, 0, 1),
|
||||
"compute": (2_100_000_000, 0, 1),
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -92,6 +92,64 @@ class TestGpuStats(unittest.TestCase):
|
||||
},
|
||||
}
|
||||
|
||||
@patch("frigate.stats.intel_gpu_info.intel_gpu_name_resolver.get_names")
|
||||
@patch("frigate.util.services.time.sleep")
|
||||
@patch("frigate.util.services.time.monotonic")
|
||||
@patch("frigate.util.services._read_intel_drm_fdinfo")
|
||||
def test_intel_gpu_stats_xe_capacity(
|
||||
self, read_fdinfo, monotonic, sleep, get_names
|
||||
):
|
||||
# Xe engines report cumulative cycles paired with total cycles, plus a
|
||||
# per-class capacity. drm-cycles-* is summed across every instance of a
|
||||
# class, so on Battlemage (capacity 2 for vcs/vecs) busy/total must be
|
||||
# divided by capacity to land in 0-100%.
|
||||
monotonic.side_effect = [0.0, 1.0]
|
||||
get_names.return_value = {"0000:03:00.0": "Intel Arc"}
|
||||
|
||||
# Deltas over the window (busy, total): render 200/1000 cap 1 = 20%,
|
||||
# video 800/1000 cap 2 = 40%, video-enhance 400/1000 cap 2 = 20%,
|
||||
# compute 100/1000 cap 1 = 10%. Without the capacity divisor video/
|
||||
# video-enhance would read 80%/40% and dec would clamp at 100%.
|
||||
snapshot_a = {
|
||||
("0000:03:00.0", "1", "300"): {
|
||||
"driver": "xe",
|
||||
"pid": "300",
|
||||
"engines": {
|
||||
"render": (0, 0, 1),
|
||||
"video": (0, 0, 2),
|
||||
"video-enhance": (0, 0, 2),
|
||||
"compute": (0, 0, 1),
|
||||
},
|
||||
},
|
||||
}
|
||||
snapshot_b = {
|
||||
("0000:03:00.0", "1", "300"): {
|
||||
"driver": "xe",
|
||||
"pid": "300",
|
||||
"engines": {
|
||||
"render": (200, 1000, 1),
|
||||
"video": (800, 1000, 2),
|
||||
"video-enhance": (400, 1000, 2),
|
||||
"compute": (100, 1000, 1),
|
||||
},
|
||||
},
|
||||
}
|
||||
read_fdinfo.side_effect = [snapshot_a, snapshot_b]
|
||||
|
||||
intel_stats = get_intel_gpu_stats(None)
|
||||
|
||||
assert intel_stats == {
|
||||
"0000:03:00.0": {
|
||||
"name": "Intel Arc",
|
||||
"vendor": "intel",
|
||||
"gpu": "90.0%",
|
||||
"mem": "-%",
|
||||
"compute": "30.0%",
|
||||
"dec": "60.0%",
|
||||
"clients": {"300": "90.0%"},
|
||||
},
|
||||
}
|
||||
|
||||
@patch("frigate.util.services._read_intel_drm_fdinfo")
|
||||
def test_intel_gpu_stats_no_clients(self, read_fdinfo):
|
||||
read_fdinfo.return_value = {}
|
||||
|
||||
@@ -360,7 +360,7 @@ def _read_intel_drm_fdinfo(target_pdev: str | None) -> dict:
|
||||
if key in snapshot:
|
||||
continue
|
||||
|
||||
engines: dict[str, tuple[int, int]] = {}
|
||||
engines: dict[str, tuple[int, int, int]] = {}
|
||||
|
||||
if driver == "i915":
|
||||
for fkey, engine in _I915_ENGINE_KEYS.items():
|
||||
@@ -368,19 +368,34 @@ def _read_intel_drm_fdinfo(target_pdev: str | None) -> dict:
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
engines[engine] = (int(raw.split()[0]), 0)
|
||||
engines[engine] = (int(raw.split()[0]), 0, 1)
|
||||
except (ValueError, IndexError):
|
||||
continue
|
||||
else:
|
||||
for suffix, engine in _XE_ENGINE_KEYS.items():
|
||||
busy_raw = fields.get(f"drm-cycles-{suffix}")
|
||||
total_raw = fields.get(f"drm-total-cycles-{suffix}")
|
||||
|
||||
if not (busy_raw and total_raw):
|
||||
continue
|
||||
|
||||
# drm-cycles-* is summed across every instance of the engine
|
||||
# class while drm-total-cycles-* tracks a single instance, so
|
||||
# busy/total scales up to the capacity (e.g. Battlemage
|
||||
# reports 2 for vcs/vecs). Capture it to divide back out;
|
||||
# absent means a single engine, so default to 1.
|
||||
capacity_raw = fields.get(f"drm-engine-capacity-{suffix}")
|
||||
|
||||
try:
|
||||
capacity = int(capacity_raw.split()[0]) if capacity_raw else 1
|
||||
except (ValueError, IndexError):
|
||||
capacity = 1
|
||||
|
||||
try:
|
||||
engines[engine] = (
|
||||
int(busy_raw.split()[0]),
|
||||
int(total_raw.split()[0]),
|
||||
max(1, capacity),
|
||||
)
|
||||
except (ValueError, IndexError):
|
||||
continue
|
||||
@@ -450,11 +465,13 @@ def get_intel_gpu_stats(
|
||||
pid_pct = per_pdev_pid_pct.setdefault(pdev, {})
|
||||
|
||||
client_total = 0.0
|
||||
for engine, (busy_b, total_b) in data_b["engines"].items():
|
||||
for engine, (busy_b, total_b, capacity) in data_b["engines"].items():
|
||||
if engine not in engine_pct:
|
||||
continue
|
||||
|
||||
busy_a, total_a = data_a["engines"].get(engine, (busy_b, total_b))
|
||||
busy_a, total_a, _ = data_a["engines"].get(
|
||||
engine, (busy_b, total_b, capacity)
|
||||
)
|
||||
|
||||
if data_b["driver"] == "i915":
|
||||
delta = max(0, busy_b - busy_a)
|
||||
@@ -464,7 +481,9 @@ def get_intel_gpu_stats(
|
||||
delta_total = total_b - total_a
|
||||
if delta_total <= 0:
|
||||
continue
|
||||
pct = min(100.0, delta_busy / delta_total * 100.0)
|
||||
# Normalize by capacity so a class with N engine instances
|
||||
# (busy summed across all N) reports 0-100%, not 0-N*100%.
|
||||
pct = min(100.0, delta_busy / (delta_total * capacity) * 100.0)
|
||||
|
||||
engine_pct[engine] += pct
|
||||
client_total += pct
|
||||
|
||||
Generated
+3
-3
@@ -8099,9 +8099,9 @@
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/follow-redirects": {
|
||||
"version": "1.15.11",
|
||||
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz",
|
||||
"integrity": "sha512-deG2P0JfjrTxl50XGCDyfI97ZGVCxIpfKYmfyrQ54n5FO/0gfIES8C/Psl6kWVDolizcaaxZJnTS0QSMxvnsBQ==",
|
||||
"version": "1.16.0",
|
||||
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.0.tgz",
|
||||
"integrity": "sha512-y5rN/uOsadFT/JfYwhxRS5R7Qce+g3zG97+JrtFZlC9klX/W5hD7iiLzScI4nZqUS7DNUdhPgw4xI8W2LuXlUw==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "individual",
|
||||
|
||||
@@ -123,7 +123,7 @@
|
||||
"error": {
|
||||
"failed": "Failed to queue export: {{error}}",
|
||||
"endTimeMustAfterStartTime": "End time must be after start time",
|
||||
"noVaildTimeSelected": "No valid time range selected"
|
||||
"noValidTimeSelected": "No valid time range selected"
|
||||
}
|
||||
},
|
||||
"fromTimeline": {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"documentTitle": "Classification Models - Frigate",
|
||||
"disabled": "Disabled",
|
||||
"details": {
|
||||
"scoreInfo": "Score represents the average classification confidence across all detections of this object.",
|
||||
"none": "None",
|
||||
@@ -64,7 +65,20 @@
|
||||
"title": "Edit Classification Model",
|
||||
"descriptionState": "Edit the classes for this state classification model. Changes will require retraining the model.",
|
||||
"descriptionObject": "Edit the object type and classification type for this object classification model.",
|
||||
"stateClassesInfo": "Note: Changing state classes requires retraining the model with the updated classes."
|
||||
"enabled": "Enabled",
|
||||
"enabledDesc": "Run this model. When disabled, it stops running and no longer classifies.",
|
||||
"saveAttempts": "Save Attempts",
|
||||
"saveAttemptsDesc": "Number of classification attempt images to keep for the recent classifications UI.",
|
||||
"motion": "Run on Motion",
|
||||
"motionDesc": "Run classification when motion is detected within the configured crop.",
|
||||
"interval": "Interval",
|
||||
"intervalDesc": "Seconds between periodic classification runs. Leave empty to run only on motion.",
|
||||
"intervalPlaceholder": "No interval",
|
||||
"stateClassesInfo": "Model updated. Retrain the model for the class changes to take effect.",
|
||||
"errors": {
|
||||
"saveAttemptsInvalid": "Save attempts must be a whole number of 0 or greater",
|
||||
"intervalInvalid": "Interval must be a whole number greater than 0"
|
||||
}
|
||||
},
|
||||
"deleteDatasetImages": {
|
||||
"title": "Delete Dataset Images",
|
||||
|
||||
@@ -76,7 +76,7 @@
|
||||
},
|
||||
"offset": {
|
||||
"label": "Annotation Offset",
|
||||
"desc": "This data comes from your camera's detect feed but is overlayed on images from the the record feed. It is unlikely that the two streams are perfectly in sync. As a result, the bounding box and the footage will not line up perfectly. You can use this setting to offset the annotations forward or backward in time to better align them with the recorded footage.",
|
||||
"desc": "This data comes from your camera's detect feed but is overlaid on images from the record feed. It is unlikely that the two streams are perfectly in sync. As a result, the bounding box and the footage will not line up perfectly. You can use this setting to offset the annotations forward or backward in time to better align them with the recorded footage.",
|
||||
"millisecondsToOffset": "Milliseconds to offset detect annotations by. <em>Default: 0</em>",
|
||||
"tips": "Lower the value if the video playback is ahead of the boxes and path points, and increase the value if the video playback is behind them. This value can be negative.",
|
||||
"toast": {
|
||||
|
||||
@@ -1053,7 +1053,7 @@
|
||||
},
|
||||
"createUser": {
|
||||
"title": "Create New User",
|
||||
"desc": "Add a new user account and specify an role for access to areas of the Frigate UI.",
|
||||
"desc": "Add a new user account and specify a role for access to areas of the Frigate UI.",
|
||||
"usernameOnlyInclude": "Username may only include letters, numbers, . or _",
|
||||
"confirmPassword": "Please confirm your password"
|
||||
},
|
||||
@@ -1490,7 +1490,14 @@
|
||||
"keyLabel": "Key",
|
||||
"valueLabel": "Value",
|
||||
"keyPlaceholder": "New key",
|
||||
"remove": "Remove"
|
||||
"remove": "Remove",
|
||||
"providerNameLabel": "Provider name",
|
||||
"providerNamePlaceholder": "e.g., openai",
|
||||
"variableNameLabel": "Variable name",
|
||||
"variableNamePlaceholder": "e.g., MY_VARIABLE",
|
||||
"loggerNameLabel": "Logger name",
|
||||
"loggerNamePlaceholder": "e.g., frigate.record",
|
||||
"keyPatternError": "Use only letters, numbers, hyphens, and underscores (no spaces)"
|
||||
},
|
||||
"knownPlates": {
|
||||
"namePlaceholder": "e.g., Wife's Car",
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
import {
|
||||
Form,
|
||||
FormControl,
|
||||
FormDescription,
|
||||
FormField,
|
||||
FormItem,
|
||||
FormLabel,
|
||||
@@ -17,6 +18,7 @@ import {
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Label } from "@/components/ui/label";
|
||||
import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
@@ -50,14 +52,25 @@ type ClassificationModelEditDialogProps = {
|
||||
type ObjectClassificationType = "sub_label" | "attribute";
|
||||
|
||||
type ObjectFormData = {
|
||||
enabled: boolean;
|
||||
saveAttempts: number;
|
||||
objectLabel: string;
|
||||
objectType: ObjectClassificationType;
|
||||
};
|
||||
|
||||
type StateFormData = {
|
||||
enabled: boolean;
|
||||
saveAttempts: number;
|
||||
motion: boolean;
|
||||
interval?: number;
|
||||
classes: string[];
|
||||
};
|
||||
|
||||
const DEFAULT_SAVE_ATTEMPTS = {
|
||||
object: 200,
|
||||
state: 100,
|
||||
} as const;
|
||||
|
||||
export default function ClassificationModelEditDialog({
|
||||
open,
|
||||
model,
|
||||
@@ -71,6 +84,10 @@ export default function ClassificationModelEditDialog({
|
||||
const isStateModel = model.state_config !== undefined;
|
||||
const isObjectModel = model.object_config !== undefined;
|
||||
|
||||
const defaultSaveAttempts = isObjectModel
|
||||
? DEFAULT_SAVE_ATTEMPTS.object
|
||||
: DEFAULT_SAVE_ATTEMPTS.state;
|
||||
|
||||
const objectLabels = useMemo(() => {
|
||||
if (!config) return [];
|
||||
|
||||
@@ -93,8 +110,17 @@ export default function ClassificationModelEditDialog({
|
||||
|
||||
// Define form schema based on model type
|
||||
const formSchema = useMemo(() => {
|
||||
const sharedFields = {
|
||||
enabled: z.boolean(),
|
||||
saveAttempts: z.coerce
|
||||
.number({ message: t("edit.errors.saveAttemptsInvalid") })
|
||||
.int(t("edit.errors.saveAttemptsInvalid"))
|
||||
.min(0, t("edit.errors.saveAttemptsInvalid")),
|
||||
};
|
||||
|
||||
if (isObjectModel) {
|
||||
return z.object({
|
||||
...sharedFields,
|
||||
objectLabel: z
|
||||
.string()
|
||||
.min(1, t("wizard.step1.errors.objectLabelRequired")),
|
||||
@@ -103,6 +129,17 @@ export default function ClassificationModelEditDialog({
|
||||
} else {
|
||||
// State model
|
||||
return z.object({
|
||||
...sharedFields,
|
||||
motion: z.boolean(),
|
||||
interval: z.preprocess(
|
||||
(val) =>
|
||||
val === "" || val === null || val === undefined ? undefined : val,
|
||||
z.coerce
|
||||
.number({ message: t("edit.errors.intervalInvalid") })
|
||||
.int(t("edit.errors.intervalInvalid"))
|
||||
.positive(t("edit.errors.intervalInvalid"))
|
||||
.optional(),
|
||||
),
|
||||
classes: z
|
||||
.array(z.string())
|
||||
.min(1, t("wizard.step1.errors.classRequired"))
|
||||
@@ -129,12 +166,18 @@ export default function ClassificationModelEditDialog({
|
||||
resolver: zodResolver(formSchema),
|
||||
defaultValues: isObjectModel
|
||||
? ({
|
||||
enabled: model.enabled,
|
||||
saveAttempts: model.save_attempts ?? defaultSaveAttempts,
|
||||
objectLabel: model.object_config?.objects?.[0] || "",
|
||||
objectType:
|
||||
(model.object_config
|
||||
?.classification_type as ObjectClassificationType) || "sub_label",
|
||||
} as ObjectFormData)
|
||||
: ({
|
||||
enabled: model.enabled,
|
||||
saveAttempts: model.save_attempts ?? defaultSaveAttempts,
|
||||
motion: model.state_config?.motion ?? false,
|
||||
interval: model.state_config?.interval,
|
||||
classes: [""], // Will be populated from dataset
|
||||
} as StateFormData),
|
||||
mode: "onChange",
|
||||
@@ -151,6 +194,8 @@ export default function ClassificationModelEditDialog({
|
||||
if (open) {
|
||||
if (isObjectModel) {
|
||||
form.reset({
|
||||
enabled: model.enabled,
|
||||
saveAttempts: model.save_attempts ?? defaultSaveAttempts,
|
||||
objectLabel: model.object_config?.objects?.[0] || "",
|
||||
objectType:
|
||||
(model.object_config
|
||||
@@ -158,6 +203,10 @@ export default function ClassificationModelEditDialog({
|
||||
} as ObjectFormData);
|
||||
} else {
|
||||
form.reset({
|
||||
enabled: model.enabled,
|
||||
saveAttempts: model.save_attempts ?? defaultSaveAttempts,
|
||||
motion: model.state_config?.motion ?? false,
|
||||
interval: model.state_config?.interval,
|
||||
classes: [""],
|
||||
} as StateFormData);
|
||||
}
|
||||
@@ -166,7 +215,15 @@ export default function ClassificationModelEditDialog({
|
||||
mutateDataset();
|
||||
}
|
||||
}
|
||||
}, [open, isObjectModel, isStateModel, model, form, mutateDataset]);
|
||||
}, [
|
||||
open,
|
||||
isObjectModel,
|
||||
isStateModel,
|
||||
model,
|
||||
form,
|
||||
mutateDataset,
|
||||
defaultSaveAttempts,
|
||||
]);
|
||||
|
||||
// Update form with classes from dataset when loaded
|
||||
useEffect(() => {
|
||||
@@ -233,6 +290,7 @@ export default function ClassificationModelEditDialog({
|
||||
setIsSaving(true);
|
||||
try {
|
||||
if (isObjectModel) {
|
||||
// object model save
|
||||
const objectData = data as ObjectFormData;
|
||||
|
||||
// Update the config
|
||||
@@ -243,9 +301,10 @@ export default function ClassificationModelEditDialog({
|
||||
classification: {
|
||||
custom: {
|
||||
[model.name]: {
|
||||
enabled: model.enabled,
|
||||
enabled: objectData.enabled,
|
||||
name: model.name,
|
||||
threshold: model.threshold,
|
||||
save_attempts: objectData.saveAttempts,
|
||||
object_config: {
|
||||
objects: [objectData.objectLabel],
|
||||
classification_type: objectData.objectType,
|
||||
@@ -260,7 +319,34 @@ export default function ClassificationModelEditDialog({
|
||||
position: "top-center",
|
||||
});
|
||||
} else {
|
||||
// state model save
|
||||
const stateData = data as StateFormData;
|
||||
|
||||
const stateConfig: { motion: boolean; interval?: number | null } = {
|
||||
motion: stateData.motion,
|
||||
};
|
||||
if (stateData.interval != null) {
|
||||
stateConfig.interval = stateData.interval;
|
||||
} else if (model.state_config?.interval != null) {
|
||||
stateConfig.interval = null;
|
||||
}
|
||||
|
||||
await axios.put("/config/set", {
|
||||
requires_restart: 0,
|
||||
update_topic: `config/classification/custom/${model.name}`,
|
||||
config_data: {
|
||||
classification: {
|
||||
custom: {
|
||||
[model.name]: {
|
||||
enabled: stateData.enabled,
|
||||
save_attempts: stateData.saveAttempts,
|
||||
state_config: stateConfig,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const newClasses = stateData.classes.filter(
|
||||
(c) => c.trim().length > 0,
|
||||
);
|
||||
@@ -307,11 +393,11 @@ export default function ClassificationModelEditDialog({
|
||||
if (renamePromises.length > 0) {
|
||||
await Promise.all(renamePromises);
|
||||
await mutate(`classification/${model.name}/dataset`);
|
||||
toast.success(t("toast.success.updatedModel"), {
|
||||
toast.success(t("edit.stateClassesInfo"), {
|
||||
position: "top-center",
|
||||
});
|
||||
} else {
|
||||
toast.info(t("edit.stateClassesInfo"), {
|
||||
toast.success(t("toast.success.updatedModel"), {
|
||||
position: "top-center",
|
||||
});
|
||||
}
|
||||
@@ -359,6 +445,29 @@ export default function ClassificationModelEditDialog({
|
||||
<div className="space-y-6">
|
||||
<Form {...form}>
|
||||
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">
|
||||
<FormField
|
||||
control={form.control}
|
||||
name="enabled"
|
||||
render={({ field }) => (
|
||||
<FormItem className="flex flex-row items-center justify-between gap-4">
|
||||
<div className="space-y-0.5">
|
||||
<FormLabel className="text-primary-variant">
|
||||
{t("edit.enabled")}
|
||||
</FormLabel>
|
||||
<FormDescription className="text-xs">
|
||||
{t("edit.enabledDesc")}
|
||||
</FormDescription>
|
||||
</div>
|
||||
<FormControl>
|
||||
<Switch
|
||||
checked={field.value}
|
||||
onCheckedChange={field.onChange}
|
||||
/>
|
||||
</FormControl>
|
||||
</FormItem>
|
||||
)}
|
||||
/>
|
||||
|
||||
{isObjectModel && (
|
||||
<>
|
||||
<FormField
|
||||
@@ -520,6 +629,77 @@ export default function ClassificationModelEditDialog({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{isStateModel && (
|
||||
<>
|
||||
<FormField
|
||||
control={form.control}
|
||||
name="motion"
|
||||
render={({ field }) => (
|
||||
<FormItem className="flex flex-row items-center justify-between gap-4">
|
||||
<div className="space-y-0.5">
|
||||
<FormLabel className="text-primary-variant">
|
||||
{t("edit.motion")}
|
||||
</FormLabel>
|
||||
<FormDescription className="text-xs">
|
||||
{t("edit.motionDesc")}
|
||||
</FormDescription>
|
||||
</div>
|
||||
<FormControl>
|
||||
<Switch
|
||||
checked={field.value}
|
||||
onCheckedChange={field.onChange}
|
||||
/>
|
||||
</FormControl>
|
||||
</FormItem>
|
||||
)}
|
||||
/>
|
||||
|
||||
<FormField
|
||||
control={form.control}
|
||||
name="interval"
|
||||
render={({ field }) => (
|
||||
<FormItem>
|
||||
<FormLabel className="text-primary-variant">
|
||||
{t("edit.interval")}
|
||||
</FormLabel>
|
||||
<FormControl>
|
||||
<Input
|
||||
className="h-8"
|
||||
inputMode="numeric"
|
||||
placeholder={t("edit.intervalPlaceholder")}
|
||||
{...field}
|
||||
value={field.value ?? ""}
|
||||
/>
|
||||
</FormControl>
|
||||
<FormDescription className="text-xs">
|
||||
{t("edit.intervalDesc")}
|
||||
</FormDescription>
|
||||
<FormMessage />
|
||||
</FormItem>
|
||||
)}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
|
||||
<FormField
|
||||
control={form.control}
|
||||
name="saveAttempts"
|
||||
render={({ field }) => (
|
||||
<FormItem>
|
||||
<FormLabel className="text-primary-variant">
|
||||
{t("edit.saveAttempts")}
|
||||
</FormLabel>
|
||||
<FormControl>
|
||||
<Input className="h-8" inputMode="numeric" {...field} />
|
||||
</FormControl>
|
||||
<FormDescription className="text-xs">
|
||||
{t("edit.saveAttemptsDesc")}
|
||||
</FormDescription>
|
||||
<FormMessage />
|
||||
</FormItem>
|
||||
)}
|
||||
/>
|
||||
|
||||
<div className="flex flex-col gap-3 pt-3 sm:flex-row sm:justify-end sm:gap-4">
|
||||
<Button
|
||||
type="button"
|
||||
|
||||
@@ -7,7 +7,13 @@ const environmentVars: SectionConfigOverrides = {
|
||||
advancedFields: [],
|
||||
uiSchema: {
|
||||
additionalProperties: {
|
||||
"ui:options": { size: "lg" },
|
||||
"ui:options": {
|
||||
size: "lg",
|
||||
additionalPropertyKeyLabel:
|
||||
"configForm.additionalProperties.variableNameLabel",
|
||||
additionalPropertyKeyPlaceholder:
|
||||
"configForm.additionalProperties.variableNamePlaceholder",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@@ -9,7 +9,15 @@ const genai: SectionConfigOverrides = {
|
||||
uiSchema: {
|
||||
"ui:options": { disableNestedCard: true },
|
||||
"*": {
|
||||
"ui:options": { disableNestedCard: true },
|
||||
"ui:options": {
|
||||
disableNestedCard: true,
|
||||
additionalPropertyKeyLabel:
|
||||
"configForm.additionalProperties.providerNameLabel",
|
||||
additionalPropertyKeyPlaceholder:
|
||||
"configForm.additionalProperties.providerNamePlaceholder",
|
||||
additionalPropertyKeyPattern: "^[a-zA-Z0-9_-]+$",
|
||||
preventKeyRename: true,
|
||||
},
|
||||
"ui:order": [
|
||||
"provider",
|
||||
"api_key",
|
||||
|
||||
@@ -12,7 +12,13 @@ const logger: SectionConfigOverrides = {
|
||||
},
|
||||
logs: {
|
||||
additionalProperties: {
|
||||
"ui:options": { enumI18nPrefix: "logger.logLevel" },
|
||||
"ui:options": {
|
||||
enumI18nPrefix: "logger.logLevel",
|
||||
additionalPropertyKeyLabel:
|
||||
"configForm.additionalProperties.loggerNameLabel",
|
||||
additionalPropertyKeyPlaceholder:
|
||||
"configForm.additionalProperties.loggerNamePlaceholder",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@@ -6,12 +6,14 @@ import {
|
||||
StrictRJSFSchema,
|
||||
WrapIfAdditionalTemplateProps,
|
||||
} from "@rjsf/utils";
|
||||
import { useEffect, useMemo, useState, type FocusEvent } from "react";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Label } from "@/components/ui/label";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { LuTrash2 } from "react-icons/lu";
|
||||
import type { ConfigFormContext } from "@/types/configForm";
|
||||
|
||||
export function WrapIfAdditionalTemplate<
|
||||
T = unknown,
|
||||
@@ -30,6 +32,7 @@ export function WrapIfAdditionalTemplate<
|
||||
onKeyRenameBlur,
|
||||
readonly,
|
||||
required,
|
||||
registry,
|
||||
schema,
|
||||
uiSchema,
|
||||
} = props;
|
||||
@@ -38,6 +41,55 @@ export function WrapIfAdditionalTemplate<
|
||||
|
||||
const additional = ADDITIONAL_PROPERTY_FLAG in schema;
|
||||
|
||||
const uiOptions = getUiOptions(uiSchema);
|
||||
const keyIsReadonly = uiOptions.additionalPropertyKeyReadonly === true;
|
||||
|
||||
const keyLabelKey =
|
||||
typeof uiOptions.additionalPropertyKeyLabel === "string"
|
||||
? uiOptions.additionalPropertyKeyLabel
|
||||
: undefined;
|
||||
const keyPlaceholderKey =
|
||||
typeof uiOptions.additionalPropertyKeyPlaceholder === "string"
|
||||
? uiOptions.additionalPropertyKeyPlaceholder
|
||||
: undefined;
|
||||
const keyPattern =
|
||||
typeof uiOptions.additionalPropertyKeyPattern === "string"
|
||||
? uiOptions.additionalPropertyKeyPattern
|
||||
: undefined;
|
||||
const preventKeyRename = uiOptions.preventKeyRename === true;
|
||||
|
||||
const formContext = registry?.formContext as ConfigFormContext | undefined;
|
||||
|
||||
// optionally, lock the key once it's been saved
|
||||
const baseline = formContext?.baselineFormData;
|
||||
const keyLocked =
|
||||
preventKeyRename &&
|
||||
typeof label === "string" &&
|
||||
!!baseline &&
|
||||
Object.prototype.hasOwnProperty.call(baseline, label);
|
||||
|
||||
// controlled key value so we can validate live and block invalid renames.
|
||||
const [keyValue, setKeyValue] = useState<string>(label ?? "");
|
||||
useEffect(() => {
|
||||
setKeyValue(label ?? "");
|
||||
}, [label]);
|
||||
|
||||
const keyRegex = useMemo(
|
||||
() => (keyPattern ? new RegExp(keyPattern) : undefined),
|
||||
[keyPattern],
|
||||
);
|
||||
const keyError = useMemo(() => {
|
||||
if (!keyRegex || keyLocked) return null;
|
||||
if (!keyRegex.test(keyValue)) {
|
||||
return t("configForm.additionalProperties.keyPatternError", {
|
||||
ns: "views/settings",
|
||||
defaultValue:
|
||||
"Use only letters, numbers, hyphens, and underscores (no spaces)",
|
||||
});
|
||||
}
|
||||
return null;
|
||||
}, [keyRegex, keyLocked, keyValue, t]);
|
||||
|
||||
if (!additional) {
|
||||
return (
|
||||
<div className={classNames} style={style}>
|
||||
@@ -47,20 +99,26 @@ export function WrapIfAdditionalTemplate<
|
||||
}
|
||||
|
||||
const keyId = `${id}-key`;
|
||||
const keyLabel = t("configForm.additionalProperties.keyLabel", {
|
||||
ns: "views/settings",
|
||||
});
|
||||
const keyLabel = keyLabelKey
|
||||
? t(keyLabelKey, { ns: "views/settings" })
|
||||
: t("configForm.additionalProperties.keyLabel", { ns: "views/settings" });
|
||||
const valueLabel = t("configForm.additionalProperties.valueLabel", {
|
||||
ns: "views/settings",
|
||||
});
|
||||
const keyPlaceholder = t("configForm.additionalProperties.keyPlaceholder", {
|
||||
ns: "views/settings",
|
||||
});
|
||||
const keyPlaceholder = keyPlaceholderKey
|
||||
? t(keyPlaceholderKey, { ns: "views/settings" })
|
||||
: t("configForm.additionalProperties.keyPlaceholder", {
|
||||
ns: "views/settings",
|
||||
});
|
||||
const removeLabel = t("configForm.additionalProperties.remove", {
|
||||
ns: "views/settings",
|
||||
});
|
||||
const uiOptions = getUiOptions(uiSchema);
|
||||
const keyIsReadonly = uiOptions.additionalPropertyKeyReadonly === true;
|
||||
|
||||
const commitKeyRename = (e: FocusEvent<HTMLInputElement>) => {
|
||||
if (readonly) return;
|
||||
if (keyError) return;
|
||||
onKeyRenameBlur?.(e);
|
||||
};
|
||||
|
||||
return (
|
||||
<div
|
||||
@@ -70,23 +128,30 @@ export function WrapIfAdditionalTemplate<
|
||||
{!keyIsReadonly && (
|
||||
<div className="col-span-12 space-y-2 md:col-span-2">
|
||||
{displayLabel && <Label htmlFor={keyId}>{keyLabel}</Label>}
|
||||
{keyIsReadonly ? (
|
||||
{keyLocked ? (
|
||||
<div
|
||||
id={keyId}
|
||||
className="flex items-center text-sm text-muted-foreground"
|
||||
className="flex items-center break-all text-sm text-primary-variant"
|
||||
>
|
||||
{label}
|
||||
</div>
|
||||
) : (
|
||||
<Input
|
||||
id={keyId}
|
||||
name={keyId}
|
||||
required={required}
|
||||
defaultValue={label}
|
||||
placeholder={keyPlaceholder}
|
||||
disabled={disabled || readonly}
|
||||
onBlur={!readonly ? onKeyRenameBlur : undefined}
|
||||
/>
|
||||
<>
|
||||
<Input
|
||||
id={keyId}
|
||||
name={keyId}
|
||||
required={required}
|
||||
value={keyValue}
|
||||
placeholder={keyPlaceholder}
|
||||
disabled={disabled || readonly}
|
||||
onChange={(e) => setKeyValue(e.target.value)}
|
||||
onBlur={!readonly ? commitKeyRename : undefined}
|
||||
aria-invalid={keyError ? true : undefined}
|
||||
/>
|
||||
{keyError && (
|
||||
<p className="text-xs text-destructive">{keyError}</p>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import type { WidgetProps } from "@rjsf/utils";
|
||||
import { useMemo } from "react";
|
||||
import { useEffect, useMemo } from "react";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import useSWR from "swr";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import type { ConfigFormContext } from "@/types/configForm";
|
||||
import type { GenAIModelsResponse } from "@/types/chat";
|
||||
|
||||
const GENAI_ROLES = ["embeddings", "descriptions", "chat"] as const;
|
||||
|
||||
@@ -37,10 +39,24 @@ export function GenAIRolesWidget(props: WidgetProps) {
|
||||
const selectedRoles = useMemo(() => normalizeValue(value), [value]);
|
||||
const providerKey = useMemo(() => getProviderKey(id), [id]);
|
||||
|
||||
// Compute occupied roles directly from formData. The computation is
|
||||
// trivially cheap (iterate providers × 3 roles max) so we skip an
|
||||
// intermediate memoization layer whose formData dependency would
|
||||
// never produce a cache hit (new object reference on every change).
|
||||
const { data: genaiInfo } = useSWR<GenAIModelsResponse>("genai/models", {
|
||||
revalidateOnFocus: false,
|
||||
});
|
||||
|
||||
const embeddingsSupported = useMemo(() => {
|
||||
if (!providerKey) return true;
|
||||
const info = genaiInfo?.[providerKey];
|
||||
return info ? info.supports_embeddings : true;
|
||||
}, [genaiInfo, providerKey]);
|
||||
|
||||
const availableRoles = useMemo(
|
||||
() =>
|
||||
embeddingsSupported
|
||||
? GENAI_ROLES
|
||||
: GENAI_ROLES.filter((role) => role !== "embeddings"),
|
||||
[embeddingsSupported],
|
||||
);
|
||||
|
||||
const occupiedRoles = useMemo(() => {
|
||||
const occupied = new Set<string>();
|
||||
const fd = formContext?.formData;
|
||||
@@ -64,6 +80,12 @@ export function GenAIRolesWidget(props: WidgetProps) {
|
||||
return occupied;
|
||||
}, [formContext?.formData, providerKey]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!embeddingsSupported && selectedRoles.includes("embeddings")) {
|
||||
onChange(selectedRoles.filter((role) => role !== "embeddings"));
|
||||
}
|
||||
}, [embeddingsSupported, selectedRoles, onChange]);
|
||||
|
||||
const toggleRole = (role: string, enabled: boolean) => {
|
||||
if (enabled) {
|
||||
if (!selectedRoles.includes(role)) {
|
||||
@@ -78,7 +100,7 @@ export function GenAIRolesWidget(props: WidgetProps) {
|
||||
return (
|
||||
<div className="rounded-lg border border-secondary-highlight bg-background_alt p-2 pr-0 md:max-w-md">
|
||||
<div className="grid gap-2">
|
||||
{GENAI_ROLES.map((role) => {
|
||||
{availableRoles.map((role) => {
|
||||
const checked = selectedRoles.includes(role);
|
||||
const roleDisabled = !checked && occupiedRoles.has(role);
|
||||
const label = t(`configForm.genaiRoles.options.${role}`, {
|
||||
|
||||
@@ -134,7 +134,7 @@ export default function ExportDialog({
|
||||
}
|
||||
|
||||
if (!range) {
|
||||
toast.error(t("export.toast.error.noVaildTimeSelected"), {
|
||||
toast.error(t("export.toast.error.noValidTimeSelected"), {
|
||||
position: "top-center",
|
||||
});
|
||||
return false;
|
||||
@@ -665,7 +665,7 @@ export function ExportContent({
|
||||
}
|
||||
|
||||
if (!range) {
|
||||
toast.error(t("export.toast.error.noVaildTimeSelected"), {
|
||||
toast.error(t("export.toast.error.noValidTimeSelected"), {
|
||||
position: "top-center",
|
||||
});
|
||||
return;
|
||||
|
||||
@@ -149,7 +149,7 @@ export default function MobileReviewSettingsDrawer({
|
||||
|
||||
if (!range) {
|
||||
toast.error(
|
||||
t("export.toast.error.noVaildTimeSelected", {
|
||||
t("export.toast.error.noValidTimeSelected", {
|
||||
ns: "components/dialog",
|
||||
}),
|
||||
{
|
||||
|
||||
@@ -27,7 +27,7 @@ import axios from "axios";
|
||||
import { toast } from "sonner";
|
||||
import useSWR from "swr";
|
||||
import { FrigateConfig } from "@/types/frigateConfig";
|
||||
import { reviewQueries } from "@/utils/zoneEdutUtil";
|
||||
import { removeRequiredZoneQuery, reviewQueries } from "@/utils/zoneEdutUtil";
|
||||
import IconWrapper from "../ui/icon-wrapper";
|
||||
import { buttonVariants } from "@/components/ui/button";
|
||||
import { Trans, useTranslation } from "react-i18next";
|
||||
@@ -153,6 +153,30 @@ export default function PolygonItem({
|
||||
cameraConfig?.review.alerts.required_zones || [],
|
||||
cameraConfig?.review.detections.required_zones || [],
|
||||
);
|
||||
const genaiQueries = removeRequiredZoneQuery(
|
||||
polygon.name,
|
||||
polygon.camera,
|
||||
"objects.genai",
|
||||
cameraConfig?.objects.genai.required_zones || [],
|
||||
);
|
||||
const snapshotQueries = removeRequiredZoneQuery(
|
||||
polygon.name,
|
||||
polygon.camera,
|
||||
"snapshots",
|
||||
cameraConfig?.snapshots.required_zones || [],
|
||||
);
|
||||
const mqttQueries = removeRequiredZoneQuery(
|
||||
polygon.name,
|
||||
polygon.camera,
|
||||
"mqtt",
|
||||
cameraConfig?.mqtt.required_zones || [],
|
||||
);
|
||||
const autotrackQueries = removeRequiredZoneQuery(
|
||||
polygon.name,
|
||||
polygon.camera,
|
||||
"onvif.autotracking",
|
||||
cameraConfig?.onvif.autotracking.required_zones || [],
|
||||
);
|
||||
// Also delete from profiles that have overrides for this zone
|
||||
let profileQueries = "";
|
||||
if (allProfileNames && cameraConfig) {
|
||||
@@ -165,7 +189,7 @@ export default function PolygonItem({
|
||||
}
|
||||
}
|
||||
}
|
||||
url = `cameras.${polygon.camera}.zones.${polygon.name}${alertQueries}${detectionQueries}${profileQueries}`;
|
||||
url = `cameras.${polygon.camera}.zones.${polygon.name}${alertQueries}${detectionQueries}${genaiQueries}${snapshotQueries}${mqttQueries}${autotrackQueries}${profileQueries}`;
|
||||
}
|
||||
|
||||
await axios
|
||||
|
||||
@@ -108,7 +108,11 @@ export default function ZoneEditPane({
|
||||
}
|
||||
const inRequiredZones =
|
||||
cam.review.alerts.required_zones.includes(polygon.name) ||
|
||||
cam.review.detections.required_zones.includes(polygon.name);
|
||||
cam.review.detections.required_zones.includes(polygon.name) ||
|
||||
cam.objects.genai.required_zones.includes(polygon.name) ||
|
||||
cam.snapshots.required_zones.includes(polygon.name) ||
|
||||
cam.mqtt.required_zones.includes(polygon.name) ||
|
||||
cam.onvif.autotracking.required_zones.includes(polygon.name);
|
||||
const hasProfileOverride = Object.values(cam.profiles ?? {}).some(
|
||||
(profile) => profile?.zones && polygon.name in profile.zones,
|
||||
);
|
||||
|
||||
@@ -43,6 +43,7 @@ export type GenAIProviderInfo = {
|
||||
models: string[];
|
||||
roles: string[];
|
||||
supports_toggleable_thinking: boolean;
|
||||
supports_embeddings: boolean;
|
||||
};
|
||||
|
||||
export type GenAIModelsResponse = Record<string, GenAIProviderInfo>;
|
||||
|
||||
@@ -370,6 +370,7 @@ export type CustomClassificationModelConfig = {
|
||||
};
|
||||
};
|
||||
motion: boolean;
|
||||
interval?: number;
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -1,3 +1,33 @@
|
||||
// Build a config/set query fragment that removes `name` from a
|
||||
// required_zones list on the given camera section (e.g. "snapshots",
|
||||
// "mqtt", "objects.genai", "onvif.autotracking"), rebuilding the
|
||||
// remaining entries. When removing the name empties the list, the
|
||||
// required_zones key itself is deleted so the field reverts to its
|
||||
// default instead of retaining the now-stale zone name. Returns an empty
|
||||
// string when `name` is not present so unrelated sections are untouched.
|
||||
export const removeRequiredZoneQuery = (
|
||||
name: string,
|
||||
camera: string,
|
||||
section: string,
|
||||
zones: string[],
|
||||
) => {
|
||||
const remaining = new Set<string>(zones || []);
|
||||
|
||||
if (!remaining.has(name)) {
|
||||
return "";
|
||||
}
|
||||
|
||||
remaining.delete(name);
|
||||
|
||||
const key = `cameras.${camera}.${section}.required_zones`;
|
||||
|
||||
if (remaining.size === 0) {
|
||||
return `&${key}`;
|
||||
}
|
||||
|
||||
return [...remaining].map((zone) => `&${key}=${zone}`).join("");
|
||||
};
|
||||
|
||||
export const reviewQueries = (
|
||||
name: string,
|
||||
review_alerts: boolean,
|
||||
|
||||
@@ -3,6 +3,7 @@ import ClassificationModelWizardDialog from "@/components/classification/Classif
|
||||
import ClassificationModelEditDialog from "@/components/classification/ClassificationModelEditDialog";
|
||||
import ActivityIndicator from "@/components/indicators/activity-indicator";
|
||||
import { ImageShadowOverlay } from "@/components/overlay/ImageShadowOverlay";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button, buttonVariants } from "@/components/ui/button";
|
||||
import { ToggleGroup, ToggleGroupItem } from "@/components/ui/toggle-group";
|
||||
import useOptimisticState from "@/hooks/use-optimistic-state";
|
||||
@@ -330,7 +331,10 @@ function ModelCard({ config, onClick, onUpdate, onDelete }: ModelCardProps) {
|
||||
{coverImage ? (
|
||||
<>
|
||||
<img
|
||||
className="size-full"
|
||||
className={cn(
|
||||
"size-full",
|
||||
!config.enabled && "opacity-50 grayscale",
|
||||
)}
|
||||
src={`${baseUrl}clips/${config.name}/dataset/${coverImage.name}/${coverImage.img}`}
|
||||
/>
|
||||
<ImageShadowOverlay lowerClassName="h-[30%] z-0" />
|
||||
@@ -338,6 +342,14 @@ function ModelCard({ config, onClick, onUpdate, onDelete }: ModelCardProps) {
|
||||
) : (
|
||||
<Skeleton className="flex size-full items-center justify-center" />
|
||||
)}
|
||||
{!config.enabled && (
|
||||
<Badge
|
||||
variant="secondary"
|
||||
className="absolute right-2 top-2 z-40 text-primary-variant"
|
||||
>
|
||||
{t("disabled")}
|
||||
</Badge>
|
||||
)}
|
||||
<div className="absolute bottom-2 left-3 text-lg text-white smart-capitalize">
|
||||
{config.name}
|
||||
</div>
|
||||
|
||||
@@ -348,6 +348,19 @@ function MotionPreviewClip({
|
||||
}
|
||||
}, [clipStart, clipEnd, playbackRate, preview]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!videoRef.current || !preview || !videoPlaying) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (isSafari || (isFirefox && isMobile)) {
|
||||
// These browsers step frames manually; rebuild the interval at the new rate
|
||||
resetPlayback();
|
||||
} else {
|
||||
videoRef.current.playbackRate = playbackRate;
|
||||
}
|
||||
}, [playbackRate, preview, videoPlaying, resetPlayback]);
|
||||
|
||||
const drawDimOverlay = useCallback(() => {
|
||||
if (!dimOverlayCanvasRef.current) {
|
||||
return;
|
||||
|
||||
@@ -305,7 +305,7 @@ export default function MotionSearchView({
|
||||
const handleExportPreview = useCallback(() => {
|
||||
if (!exportRange) {
|
||||
toast.error(
|
||||
t("export.toast.error.noVaildTimeSelected", {
|
||||
t("export.toast.error.noValidTimeSelected", {
|
||||
ns: "components/dialog",
|
||||
}),
|
||||
{
|
||||
@@ -351,7 +351,7 @@ export default function MotionSearchView({
|
||||
const handleExportSave = useCallback(() => {
|
||||
if (!exportRange || !selectedCamera) {
|
||||
toast.error(
|
||||
t("export.toast.error.noVaildTimeSelected", {
|
||||
t("export.toast.error.noValidTimeSelected", {
|
||||
ns: "components/dialog",
|
||||
}),
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user