mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-07-09 21:41:15 +03:00
update face recognition docs
This commit is contained in:
parent
e6cac50250
commit
9289253ba8
@ -151,6 +151,14 @@ Follow these steps to begin:
|
|||||||
|
|
||||||
## Creating a Robust Training Set
|
## Creating a Robust Training Set
|
||||||
|
|
||||||
|
:::tip
|
||||||
|
|
||||||
|
**The short version:** Start with a few clear, front-facing photos of each person. As faces are detected in the Recent Recognitions tab, train clear images that scored lower, adding variety (different angles, lighting, and expressions) slowly. Diversity matters far more than volume, and low-quality images hurt recognition more than they help.
|
||||||
|
|
||||||
|
For a step-by-step narrative of these best practices (and the same principles applied to state and object classification), see the [Frigate Tips: Best Practices for Training](https://github.com/blakeblackshear/frigate/discussions/21374) discussion.
|
||||||
|
|
||||||
|
:::
|
||||||
|
|
||||||
The number of images needed for a sufficient training set for face recognition varies depending on several factors:
|
The number of images needed for a sufficient training set for face recognition varies depending on several factors:
|
||||||
|
|
||||||
- Diversity of the dataset: A dataset with diverse images, including variations in lighting, pose, and facial expressions, will require fewer images per person than a less diverse dataset.
|
- Diversity of the dataset: A dataset with diverse images, including variations in lighting, pose, and facial expressions, will require fewer images per person than a less diverse dataset.
|
||||||
@ -181,9 +189,27 @@ When choosing images to include in the face training set it is recommended to al
|
|||||||
|
|
||||||
The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.
|
The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.
|
||||||
|
|
||||||
Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. While each image can be used to train the system for a specific person, not all images are suitable for training.
|
Each face image is labeled with a name (or `Unknown`) along with the confidence score of that recognition attempt. Images are grouped by the person they were matched against, not by who they actually are, so a group labeled with a person's name can contain a crop that is really someone else but happened to score as a partial match. The name and score shown on each individual crop describe that single attempt.
|
||||||
|
|
||||||
Refer to the guidelines below for best practices on selecting images for training.
|
While each image can be used to train the system for a specific person, not all images are suitable for training. Refer to the guidelines below for best practices on selecting images for training.
|
||||||
|
|
||||||
|
### How Frigate Decides Who a Person Is
|
||||||
|
|
||||||
|
Recognition does not happen one frame at a time. While a `person` is in view, Frigate runs face recognition on many frames and collects every attempt. The final `sub_label` is decided from all of those attempts together, weighted by the area of each face (larger, closer faces count more), not from any single frame.
|
||||||
|
|
||||||
|
This has a few practical consequences:
|
||||||
|
|
||||||
|
- A handful of wrong guesses on blurry or distant frames usually do not change the result. If Frigate sees a person as "Tom, Tom, Sam, Tom, Tom," it will still conclude the person was Tom.
|
||||||
|
- The goal is not for every individual face crop to be correct. The goal is for each person to be recognized correctly overall, across all the faces captured while they were present.
|
||||||
|
- A single very high confidence match will not by itself assign a sub label. Recognition must be consistent. See [I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?](#i-see-scores-above-the-threshold-in-the-recent-recognitions-tab-but-a-sub-label-wasnt-assigned) below.
|
||||||
|
|
||||||
|
### Which Faces Are Worth Training?
|
||||||
|
|
||||||
|
Whether a face is worth training has little to do with what it was recognized as. A crop is a good training candidate when all of these are true:
|
||||||
|
|
||||||
|
- It did not already score high and correctly. Faces that are already recognized confidently add little and increase the risk of over-fitting.
|
||||||
|
- It is clear enough to be useful: not blurry, not heavily off-axis, not infrared (gray-scale). If it is hard for you to make out the face, it will not help the model.
|
||||||
|
- It adds something new: a different angle, lighting, expression, or distance than what you already have.
|
||||||
|
|
||||||
### Step 1 - Building a Strong Foundation
|
### Step 1 - Building a Strong Foundation
|
||||||
|
|
||||||
@ -223,6 +249,10 @@ Accuracy is definitely a going to be improved with higher quality cameras / stre
|
|||||||
|
|
||||||
Some users have also noted that setting the stream in camera firmware to a constant bit rate (CBR) leads to better image clarity than with a variable bit rate (VBR).
|
Some users have also noted that setting the stream in camera firmware to a constant bit rate (CBR) leads to better image clarity than with a variable bit rate (VBR).
|
||||||
|
|
||||||
|
### Can I train faces for people who only appear at night?
|
||||||
|
|
||||||
|
The embedding models are trained on color images, so gray-scale and infrared (IR) faces sit in a different feature distribution and are more easily confused with other people. Prefer color images, and avoid mixing gray-scale samples in early while you are building a foundation. If someone only ever appears at night, gray-scale training is acceptable, but keep those samples limited and as clear as possible, and add them only once color recognition is stable for your other people.
|
||||||
|
|
||||||
### Why can't I bulk upload photos?
|
### Why can't I bulk upload photos?
|
||||||
|
|
||||||
It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to over-fitting in that particular scenario and hurt recognition performance.
|
It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to over-fitting in that particular scenario and hurt recognition performance.
|
||||||
@ -243,11 +273,24 @@ Review your face collections and remove most of the unclear or low-quality image
|
|||||||
|
|
||||||
Avoid training on images that already score highly, as this can lead to over-fitting. Instead, focus on relatively clear images that score lower (ideally with different lighting, angles, and conditions) to help the model generalize more effectively.
|
Avoid training on images that already score highly, as this can lead to over-fitting. Instead, focus on relatively clear images that score lower (ideally with different lighting, angles, and conditions) to help the model generalize more effectively.
|
||||||
|
|
||||||
|
### Should I correct a face that was recognized as the wrong person?
|
||||||
|
|
||||||
|
Only if it is a good image. Reassigning a face does add it to that person's training set, but two things are true at once:
|
||||||
|
|
||||||
|
- Reassigning a single misclassified frame has a small effect. The image is weighted against every other sample for that person, so correcting 1 frame out of 20 will not move recognition much. Occasional wrong guesses on poor frames are normal and do not need to be fixed.
|
||||||
|
- Reassigning a poor image (blurry, off-angle, low-resolution, gray-scale) can hurt more than the misidentification did, because low-quality samples degrade recognition for that whole person.
|
||||||
|
|
||||||
|
So the decision is about image quality, not about the wrong label. If the crop is clear, well-lit, and reasonably front-facing, and it scored low or was wrong, assigning it to the correct person is useful. If you can barely make out the face yourself, ignore it; do not train it just to correct the label.
|
||||||
|
|
||||||
|
If a person is repeatedly misidentified, do not keep reassigning the same frame. Instead, remove low-quality or misleading images and add a few high-quality samples to the correct person. See [Why do unknown people score similarly to known people?](#why-do-unknown-people-score-similarly-to-known-people) above.
|
||||||
|
|
||||||
### Frigate misidentified a face. Can I tell it that a face is "not" a specific person?
|
### Frigate misidentified a face. Can I tell it that a face is "not" a specific person?
|
||||||
|
|
||||||
No, face recognition does not support negative training (i.e., explicitly telling it who someone is _not_). Instead, the best approach is to improve the training data by using a more diverse and representative set of images for each person.
|
No, face recognition does not support negative training (i.e., explicitly telling it who someone is _not_). Instead, the best approach is to improve the training data by using a more diverse and representative set of images for each person.
|
||||||
For more guidance, refer to the section above on improving recognition accuracy.
|
For more guidance, refer to the section above on improving recognition accuracy.
|
||||||
|
|
||||||
|
This also applies to a stranger who is repeatedly matched to a known person (for example, a delivery driver recognized as you). Do not create a profile for them and do not reassign their faces to yourself, as this pollutes your training set and makes recognition worse. Leave the detection as unknown and improve the known person's training set instead. Face recognition learns who someone is, not who they are not.
|
||||||
|
|
||||||
### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?
|
### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?
|
||||||
|
|
||||||
The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
|
The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user