Compare commits

...

8 Commits

Author SHA1 Message Date
Kai Curry
3ce5c93489
Merge e888dc347a into 352d271fe4 2026-02-23 17:13:08 -01:00
Josh Hawkins
352d271fe4
Update HA docs with MQTT example (#22098)
* update HA docs with MQTT example

* format block as yaml
2026-02-23 10:25:03 -06:00
Kai Curry
a6e11a59d6
docs: Add detail to face recognition MQTT update docs (#21942)
* Add detail to face recognition MQTT update docs

Clarify that the weighted average favors larger faces and
higher-confidence detections, that unknown attempts are excluded,
and document when name/score will be null/0.0.

* Fix score decimal in MQTT face recognition documentation

`0.0` in JSON is just `0`.

* Clarify score is a running weighted average

* Simplify MQTT tracked_object_update docs with inline comments

Move scoring logic details to face recognition docs and keep
MQTT reference concise with inline field comments and links.

* fix (expand) lpr doc link

* rm obvious lpr comments

---------

Co-authored-by: Kai Curry <kai@wjerk.com>
2026-02-23 06:46:55 -07:00
Kai Curry
a7d8d13d9a
docs: Add frame selection and clean copy details to snapshots docs (#21946)
* docs: Add frame selection and clean copy details to snapshots docs

Document how Frigate selects the best frame for snapshots, explain the
difference between regular snapshots and clean copies, fix internal
links to use absolute paths, and highlight Frigate+ as the primary
reason to keep clean_copy enabled if regular snapshot is configured clean.

* revert - do not use the word event

* rm clean copy is only saved when `clean_copy` is enabled

* Simplified the Frame Selection section down to a single paragraph.

* rm note about snapshot file ext change from png to webp

---------

Co-authored-by: Kai Curry <kai@wjerk.com>
2026-02-23 06:45:29 -07:00
Kai Curry
e888dc347a docs: Simplify face recognition alerts section, move notification details
Condense the alerts/notifications/automations section to focus on
what users need to know. Move notification-specific details to the
notifications doc. Remove HA automation examples (covered by HA
integration docs).
2026-02-09 17:48:48 -01:00
Kai Curry
aeab5705ea MQTT is "running" (incremental updates), UI shows the "final" weighted average 2026-02-09 16:37:21 -01:00
Kai Curry
dc865d51dc Expand FAQ on face recognition scoring with weighted average details 2026-02-09 16:30:21 -01:00
Kai Curry
a7925e9b9f docs: Improve face recognition documentation with alerts, automations, and UI details
Add new section explaining how face recognition relates to alerts and
notifications, including Home Assistant automation examples. Expand
configuration parameter descriptions and Recent Recognitions tab
documentation with score color coding, event grouping, and debugging tips.
2026-02-09 16:19:09 -01:00
5 changed files with 83 additions and 18 deletions

View File

@ -5,6 +5,18 @@ title: Face Recognition
Face recognition identifies known individuals by matching detected faces with previously learned facial data. When a known `person` is recognized, their name will be added as a `sub_label`. This information is included in the UI, filters, as well as in notifications.
## Alerts and Notifications
Face recognition does not affect whether an alert is created — alerts are based on tracked objects like `person` in your `review.alerts.labels` and your [zone requirements](./review). The `face` label is an [attribute label](/plus/#available-label-types), not a tracked object, so it cannot trigger alerts on its own.
When a face is recognized, the person's name is added as a `sub_label` on the tracked object. This name appears in the Frigate UI, in [built-in notifications](/configuration/notifications), and is published via [MQTT](/integrations/mqtt).
:::note
There is no built-in way to only create alerts for specific recognized faces. Neither `face`, `person-verified`, nor specific person names can be used in `review.alerts.labels`. To trigger automations based on face recognition results, use the [official Frigate integration's sensors](/integrations/home-assistant) and/or the [MQTT data](/integrations/mqtt) Frigate publishes.
:::
## Model Requirements
### Face Detection
@ -69,9 +81,9 @@ Fine-tune face recognition with these optional parameters at the global level of
- Default: `0.9`.
- `min_faces`: Min face recognitions for the sub label to be applied to the person object.
- Default: `1`
- `save_attempts`: Number of images of recognized faces to save for training.
- `save_attempts`: Maximum number of face attempt images to keep in the training folder. Frigate saves a face image after each recognition attempt; when the limit is reached, the oldest image is deleted. These images are displayed in the Face Library's Recent Recognitions tab.
- Default: `200`.
- `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this.
- `blur_confidence_filter`: Enables a filter that measures face image blurriness (using Laplacian variance) and reduces the recognition confidence score accordingly. Blurrier images receive a larger penalty (up to -0.06 for very blurry, down to 0 for clear images), making it harder for blurry faces to meet the `recognition_threshold`.
- Default: `True`.
- `device`: Target a specific device to run the face recognition model on (multi-GPU installation).
- Default: `None`.
@ -118,9 +130,19 @@ When choosing images to include in the face training set it is recommended to al
The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.
Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. While each image can be used to train the system for a specific person, not all images are suitable for training.
Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. The score is color-coded based on your configured thresholds:
Refer to the guidelines below for best practices on selecting images for training.
- **Green**: score >= `recognition_threshold` (default `0.9`) — a confident match
- **Orange**: score >= `unknown_score` (default `0.8`) — a potential match
- **Red**: score < `unknown_score` — unknown or no match
When an event has multiple recognition attempts, the face cards are displayed within a group. The group shows the recognized person's name if one was identified, or "Unknown" if not. Within the group, each individual face card shows its own recognition score. Frigate uses a weighted average across all attempts for a person object to determine whether to assign a name (`sub_label`) — so a single high-scoring card does not guarantee the person will be identified (see the [FAQ](#i-see-scores-above-the-threshold-in-the-recent-recognitions-tab-but-a-sub-label-wasnt-assigned) for more details).
If the weighted average did not meet the `recognition_threshold`, there is no place in the UI to see it. The weighted average is published in the `score` field of the [`frigate/tracked_object_update`](/integrations/mqtt.md#face-recognition-update) MQTT topic after each recognition attempt, regardless of whether it meets the threshold. This is the most useful tool for debugging why a sub label was or wasn't assigned.
Clicking a face card navigates to the Tracked Object Details for the associated event. To select face cards for deletion, right-click (or Ctrl/Cmd+click) individual cards, or use Ctrl+A to select all. A delete button will appear in the toolbar once cards are selected. Removing cards from the Recent Recognitions tab only removes the saved attempt images — it does not affect recognition accuracy or training data.
While each image can be used to train the system for a specific person, not all images are suitable for training. Refer to the guidelines below for best practices on selecting images for training.
### Step 1 - Building a Strong Foundation
@ -157,6 +179,8 @@ Start with the [Usage](#usage) section and re-read the [Model Requirements](#mod
- Make sure you have trained at least one face per the recommendations above.
- Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration).
3. To see recognition scores for an event, check the **Face Library** > **Recent Recognitions** tab. Face cards from the same event are grouped together, with the group header showing the combined result. Each card within the group shows its individual recognition score with [color coding](#understanding-the-recent-recognitions-tab). The **Tracked Object Details** view only shows the final weighted average score (in parentheses next to the top score) if a `sub_label` was assigned.
### Detection does not work well with blurry images?
Accuracy is definitely a going to be improved with higher quality cameras / streams. It is important to look at the DORI (Detection Observation Recognition Identification) range of your camera, if that specification is posted. This specification explains the distance from the camera that a person can be detected, observed, recognized, and identified. The identification range is the most relevant here, and the distance listed by the camera is the furthest that face recognition will realistically work.
@ -190,7 +214,15 @@ For more guidance, refer to the section above on improving recognition accuracy.
### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?
The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
Frigate considers recognition scores across all attempts for each person object. The score shown in the UI is the final weighted average across all attempts, while MQTT publishes a running weighted average that updates after each attempt. The weighting favors larger faces (by pixel area, capped at 4000px) and higher-confidence detections. Attempts scored at or below `unknown_score` are excluded from the average.
A sub label will only be assigned if:
- At least `min_faces` recognition attempts have been recorded.
- A single person name has the most detections (no ties).
- The weighted average score meets the `recognition_threshold`.
This avoids cases where a single high-confidence recognition would throw off the results.
### Can I use other face recognition software like DoubleTake at the same time as the built in face recognition?

View File

@ -49,7 +49,7 @@ Once notifications are enabled, press the `Register for Notifications` button on
## Supported Notifications
Currently notifications are only supported for review alerts. More notifications will be supported in the future.
Currently notifications are only supported for review alerts. More notifications will be supported in the future. If [face recognition](/configuration/face_recognition) identifies a person during the alert, their name will be included in the notification. However, built-in notifications cannot be configured to only fire for specific recognized faces.
:::note

View File

@ -9,4 +9,25 @@ Snapshots are accessible in the UI in the Explore pane. This allows for quick su
To only save snapshots for objects that enter a specific zone, [see the zone docs](./zones.md#restricting-snapshots-to-specific-zones)
Snapshots sent via MQTT are configured in the [config file](https://docs.frigate.video/configuration/) under `cameras -> your_camera -> mqtt`
Snapshots sent via MQTT are configured in the [config file](/configuration) under `cameras -> your_camera -> mqtt`
## Frame Selection
Frigate does not save every frame — it picks a single "best" frame for each tracked object and uses it for both the snapshot and clean copy. As the object is tracked across frames, Frigate continuously evaluates whether the current frame is better than the previous best based on detection confidence, object size, and the presence of key attributes like faces or license plates. Frames where the object touches the edge of the frame are deprioritized. The snapshot is written to disk once tracking ends using whichever frame was determined to be the best.
MQTT snapshots are published more frequently — each time a better thumbnail frame is found during tracking, or when the current best image is older than `best_image_timeout` (default: 60s). These use their own annotation settings configured under `cameras -> your_camera -> mqtt`.
## Clean Copy
Frigate can produce up to two snapshot files per event, each used in different places:
| Version | File | Annotations | Used by |
| --- | --- | --- | --- |
| **Regular snapshot** | `<camera>-<id>.jpg` | Respects your `timestamp`, `bounding_box`, `crop`, and `height` settings | API (`/api/events/<id>/snapshot.jpg`), MQTT (`<camera>/<label>/snapshot`), Explore pane in the UI |
| **Clean copy** | `<camera>-<id>-clean.webp` | Always unannotated — no bounding box, no timestamp, no crop, full resolution | API (`/api/events/<id>/snapshot-clean.webp`), [Frigate+](/plus/first_model) submissions, "Download Clean Snapshot" in the UI |
MQTT snapshots are configured separately under `cameras -> your_camera -> mqtt` and are unrelated to the clean copy.
The clean copy is required for submitting events to [Frigate+](/plus/first_model) — if you plan to use Frigate+, keep `clean_copy` enabled regardless of your other snapshot settings.
If you are not using Frigate+ and `timestamp`, `bounding_box`, and `crop` are all disabled, the regular snapshot is already effectively clean, so `clean_copy` provides no benefit and only uses additional disk space. You can safely set `clean_copy: False` in this case.

View File

@ -16,7 +16,15 @@ See the [MQTT integration
documentation](https://www.home-assistant.io/integrations/mqtt/) for more
details.
In addition, MQTT must be enabled in your Frigate configuration file and Frigate must be connected to the same MQTT server as Home Assistant for many of the entities created by the integration to function.
In addition, MQTT must be enabled in your Frigate configuration file and Frigate must be connected to the same MQTT server as Home Assistant for many of the entities created by the integration to function, e.g.:
```yaml
mqtt:
enabled: True
host: mqtt.server.com # the address of your HA server that's running the MQTT integration
user: your_mqtt_broker_username
password: your_mqtt_broker_password
```
### Integration installation
@ -95,12 +103,12 @@ services:
If you are using Home Assistant Add-on, the URL should be one of the following depending on which Add-on variant you are using. Note that if you are using the Proxy Add-on, you should NOT point the integration at the proxy URL. Just enter the same URL used to access Frigate directly from your network.
| Add-on Variant | URL |
| -------------------------- | ----------------------------------------- |
| Frigate | `http://ccab4aaf-frigate:5000` |
| Frigate (Full Access) | `http://ccab4aaf-frigate-fa:5000` |
| Frigate Beta | `http://ccab4aaf-frigate-beta:5000` |
| Frigate Beta (Full Access) | `http://ccab4aaf-frigate-fa-beta:5000` |
| Add-on Variant | URL |
| -------------------------- | -------------------------------------- |
| Frigate | `http://ccab4aaf-frigate:5000` |
| Frigate (Full Access) | `http://ccab4aaf-frigate-fa:5000` |
| Frigate Beta | `http://ccab4aaf-frigate-beta:5000` |
| Frigate Beta (Full Access) | `http://ccab4aaf-frigate-fa-beta:5000` |
### Frigate running on a separate machine

View File

@ -120,7 +120,7 @@ Message published for each changed tracked object. The first message is publishe
### `frigate/tracked_object_update`
Message published for updates to tracked object metadata, for example:
Message published for updates to tracked object metadata. All messages include an `id` field which is the tracked object's event ID, and can be used to look up the event via the API or match it to items in the UI.
#### Generative AI Description Update
@ -134,12 +134,14 @@ Message published for updates to tracked object metadata, for example:
#### Face Recognition Update
Published after each recognition attempt, regardless of whether the score meets `recognition_threshold`. See the [Face Recognition](/configuration/face_recognition) documentation for details on how scoring works.
```json
{
"type": "face",
"id": "1607123955.475377-mxklsc",
"name": "John",
"score": 0.95,
"name": "John", // best matching person, or null if no match
"score": 0.95, // running weighted average across all recognition attempts
"camera": "front_door_cam",
"timestamp": 1607123958.748393
}
@ -147,11 +149,13 @@ Message published for updates to tracked object metadata, for example:
#### License Plate Recognition Update
Published when a license plate is recognized on a car object. See the [License Plate Recognition](/configuration/license_plate_recognition) documentation for details.
```json
{
"type": "lpr",
"id": "1607123955.475377-mxklsc",
"name": "John's Car",
"name": "John's Car", // known name for the plate, or null
"plate": "123ABC",
"score": 0.95,
"camera": "driveway_cam",