diff --git a/docs/docs/configuration/camera_specific.md b/docs/docs/configuration/camera_specific.md index 1cff19ecd3..4a034354da 100644 --- a/docs/docs/configuration/camera_specific.md +++ b/docs/docs/configuration/camera_specific.md @@ -156,7 +156,7 @@ Reolink has many different camera models with inconsistently supported features | 6MP or higher | Latest (ex: Duo3, CX-8##) | http-flv with ffmpeg 8.0, or rtsp | This uses the new http-flv-enhanced over H265 which requires ffmpeg 8.0 (Frigate's default) | | 6MP or higher | Older (ex: RLC-8##) | rtsp | | -Frigate works much better with newer reolink cameras that are setup with the below options: +Frigate works much better with newer Reolink cameras that are setup with the below options: If available, recommended settings are: @@ -165,7 +165,7 @@ If available, recommended settings are: #### Setup via the Add Camera Wizard -The Add Camera Wizard is the recommended way to add a standard Reolink camera. Before starting, make sure HTTP is enabled in the camera's advanced network settings. The wizard uses the camera's HTTP API to determine its resolution and choose the recommended stream type from the table above. +The Add Camera Wizard is the recommended way to add a standard Reolink camera. Before starting, make sure [HTTP is enabled](https://support.reolink.com/articles/360003452893-How-to-Access-Reolink-Cameras-NVRs-Home-Hub-Locally-via-Web-Browsers/) in the camera's advanced network settings. The wizard uses the camera's HTTP API to determine its resolution and choose the recommended stream type from the table above. 1. Click **Add Camera** in . 2. Choose **Manual selection** as the stream detection method and select **Reolink** as the camera brand. @@ -192,7 +192,7 @@ Reolink's latest cameras support two way audio via go2rtc and other applications NOTE: The RTSP stream can not be prefixed with `ffmpeg:`, as go2rtc needs to handle the stream to support two way audio. -Ensure HTTP is enabled in the camera's advanced network settings. To use two way talk with Frigate, see the [Live view documentation](/configuration/live#two-way-talk). +Ensure [HTTP is enabled](https://support.reolink.com/articles/360003452893-How-to-Access-Reolink-Cameras-NVRs-Home-Hub-Locally-via-Web-Browsers/) in the camera's advanced network settings. To use two way talk with Frigate, see the [Live view documentation](/configuration/live#two-way-talk). ::: diff --git a/docs/docs/configuration/face_recognition.md b/docs/docs/configuration/face_recognition.md index 2039374bff..b9af946ee9 100644 --- a/docs/docs/configuration/face_recognition.md +++ b/docs/docs/configuration/face_recognition.md @@ -6,6 +6,7 @@ title: Face Recognition import ConfigTabs from "@site/src/components/ConfigTabs"; import TabItem from "@theme/TabItem"; import NavPath from "@site/src/components/NavPath"; +import FaqItem from "@site/src/components/FaqItem"; Face recognition identifies known individuals by matching detected faces with previously learned facial data. When a known `person` is recognized, their name will be added as a `sub_label`. This information is included in the UI, filters, as well as in notifications. @@ -151,6 +152,14 @@ Follow these steps to begin: ## Creating a Robust Training Set +:::tip + +**The short version:** Start with a few clear, front-facing photos of each person. As faces are detected in the Recent Recognitions tab, train clear images that scored lower, adding variety (different angles, lighting, and expressions) slowly. Diversity matters far more than volume, and low-quality images hurt recognition more than they help. + +For a step-by-step narrative of these best practices (and the same principles applied to state and object classification), see the [Frigate Tips: Best Practices for Training](https://github.com/blakeblackshear/frigate/discussions/21374) discussion. + +::: + The number of images needed for a sufficient training set for face recognition varies depending on several factors: - Diversity of the dataset: A dataset with diverse images, including variations in lighting, pose, and facial expressions, will require fewer images per person than a less diverse dataset. @@ -181,9 +190,27 @@ When choosing images to include in the face training set it is recommended to al The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching. -Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. While each image can be used to train the system for a specific person, not all images are suitable for training. +Each face image is labeled with a name (or `Unknown`) along with the confidence score of that recognition attempt. Images are grouped by the person they were matched against, not by who they actually are, so a group labeled with a person's name can contain a crop that is really someone else but happened to score as a partial match. The name and score shown on each individual crop describe that single attempt. -Refer to the guidelines below for best practices on selecting images for training. +While each image can be used to train the system for a specific person, not all images are suitable for training. Refer to the guidelines below for best practices on selecting images for training. + +### How Frigate Decides Who a Person Is + +Recognition does not happen one frame at a time. While a `person` is in view, Frigate runs face recognition on many frames, not just a single frame. The final `sub_label` is decided from all of those attempts together, weighted by the area of each face (larger, closer faces count more), not from any single frame. + +This has a few practical consequences: + +- A handful of wrong guesses on blurry or distant frames usually do not change the result. If Frigate sees a person as "Tom, Tom, Sam, Tom, Tom," it will still conclude the person was Tom. +- The goal is not for every individual face crop to be correct. The goal is for each person to be recognized correctly overall, across all the faces captured while they were present. +- A single very high confidence match will not by itself assign a sub label. Recognition must be consistent. See [I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?](#i-see-scores-above-the-threshold-in-the-recent-recognitions-tab-but-a-sub-label-wasnt-assigned) below. + +### Which Faces Are Worth Training? + +Whether a face is worth training has little to do with what it was recognized as. A crop is a good training candidate when all of these are true: + +- It did not already score high and correctly. Faces that are already recognized confidently add little and increase the risk of over-fitting. +- It is clear enough to be useful: not blurry, not heavily off-axis, not infrared (gray-scale). If it is hard for you to make out the face, it will not help the model. +- It adds something new: a different angle, lighting, expression, or distance than what you already have. ### Step 1 - Building a Strong Foundation @@ -199,7 +226,9 @@ Once front-facing images are performing well, start choosing slightly off-angle ## FAQ -### How do I debug Face Recognition issues? +### Getting Recognition Working + + Start with the [Usage](#usage) section and re-read the [Model Requirements](#model-requirements) above. @@ -217,21 +246,47 @@ Start with the [Usage](#usage) section and re-read the [Model Requirements](#mod - Make sure you have trained at least one face per the recommendations above. - Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration). -### Detection does not work well with blurry images? + + + + +Face recognition does not run on the recording stream, this would be suboptimal for many reasons: + +1. The latency of accessing the recordings means the notifications would not include the names of recognized people because recognition would not complete until after. +2. The embedding models used run on a set image size, so larger images will be scaled down to match this anyway. +3. Motion clarity is much more important than extra pixels, over-compression and motion blur are much more detrimental to results than resolution. + + + +### Improving Accuracy and Training + + Accuracy is definitely a going to be improved with higher quality cameras / streams. It is important to look at the DORI (Detection Observation Recognition Identification) range of your camera, if that specification is posted. This specification explains the distance from the camera that a person can be detected, observed, recognized, and identified. The identification range is the most relevant here, and the distance listed by the camera is the furthest that face recognition will realistically work. Some users have also noted that setting the stream in camera firmware to a constant bit rate (CBR) leads to better image clarity than with a variable bit rate (VBR). -### Why can't I bulk upload photos? + + + + +The embedding models are trained on color images, so gray-scale and infrared (IR) faces sit in a different feature distribution and are more easily confused with other people. Prefer color images, and avoid mixing gray-scale samples in early while you are building a foundation. If someone only ever appears at night, gray-scale training is acceptable, but keep those samples limited and as clear as possible, and add them only once color recognition is stable for your other people. + + + + It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to over-fitting in that particular scenario and hurt recognition performance. -### Why can't I bulk reprocess faces? + + + Face embedding models work by breaking apart faces into different features. This means that when reprocessing an image, only images from a similar angle will have its score affected. -### Why do unknown people score similarly to known people? + + + This can happen for a few different reasons, but this is usually an indicator that the training set needs to be improved. This is often related to over-fitting: @@ -243,31 +298,52 @@ Review your face collections and remove most of the unclear or low-quality image Avoid training on images that already score highly, as this can lead to over-fitting. Instead, focus on relatively clear images that score lower (ideally with different lighting, angles, and conditions) to help the model generalize more effectively. -### Frigate misidentified a face. Can I tell it that a face is "not" a specific person? + + + + +Only if it is a good image. Reassigning a face does add it to that person's training set, but two things are true at once: + +- Reassigning a single misclassified frame has a small effect. The image is weighted against every other sample for that person, so correcting 1 frame out of 20 will not move recognition much. Occasional wrong guesses on poor frames are normal and do not need to be fixed. +- Reassigning a poor image (blurry, off-angle, low-resolution, gray-scale) can hurt more than the misidentification did, because low-quality samples degrade recognition for that whole person. + +So the decision is about image quality, not about the wrong label. If the crop is clear, well-lit, and reasonably front-facing, and it scored low or was wrong, assigning it to the correct person is useful. If you can barely make out the face yourself, ignore it; do not train it just to correct the label. + +If a person is repeatedly misidentified, do not keep reassigning the same frame. Instead, remove low-quality or misleading images and add a few high-quality samples to the correct person. See [Why do unknown people score similarly to known people?](#why-do-unknown-people-score-similarly-to-known-people) above. + + + + No, face recognition does not support negative training (i.e., explicitly telling it who someone is _not_). Instead, the best approach is to improve the training data by using a more diverse and representative set of images for each person. For more guidance, refer to the section above on improving recognition accuracy. -### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned? +This also applies to a stranger who is repeatedly matched to a known person (for example, a delivery driver recognized as you). Do not create a profile for them and do not reassign their faces to yourself, as this pollutes your training set and makes recognition worse. Leave the detection as unknown and improve the known person's training set instead. Face recognition learns who someone is, not who they are not. + + + + The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results. -### Can I use other face recognition software like DoubleTake at the same time as the built in face recognition? + + +### Compatibility and Maintenance + + No, using another face recognition service will interfere with Frigate's built in face recognition. When using double-take the sub_label feature must be disabled if the built in face recognition is also desired. -### Does face recognition run on the recording stream? + -Face recognition does not run on the recording stream, this would be suboptimal for many reasons: - -1. The latency of accessing the recordings means the notifications would not include the names of recognized people because recognition would not complete until after. -2. The embedding models used run on a set image size, so larger images will be scaled down to match this anyway. -3. Motion clarity is much more important than extra pixels, over-compression and motion blur are much more detrimental to results than resolution. - -### I get an unknown error when taking a photo directly with my iPhone + By default iOS devices will use HEIC (High Efficiency Image Container) for images, but this format is not supported for uploads. Choosing `large` as the format instead of `original` will use JPG which will work correctly. -### How can I delete the face database and start over? + + + Frigate does not store anything in its database related to face recognition. You can simply delete all of your faces through the Frigate UI or remove the contents of the `/media/frigate/clips/faces` directory. + + diff --git a/docs/docs/configuration/license_plate_recognition.md b/docs/docs/configuration/license_plate_recognition.md index acd81c633c..eae92a9815 100644 --- a/docs/docs/configuration/license_plate_recognition.md +++ b/docs/docs/configuration/license_plate_recognition.md @@ -6,6 +6,7 @@ title: License Plate Recognition (LPR) import ConfigTabs from "@site/src/components/ConfigTabs"; import TabItem from "@theme/TabItem"; import NavPath from "@site/src/components/NavPath"; +import FaqItem from "@site/src/components/FaqItem"; Frigate can recognize license plates on vehicles and automatically add the detected characters to the `recognized_license_plate` field or a [known](#matching) name as a `sub_label` to tracked objects of type `car` or `motorcycle`. A common use case may be to read the license plates of cars pulling into a driveway or cars passing by on a street. @@ -591,7 +592,9 @@ By selecting the appropriate configuration, users can optimize their dedicated L ## FAQ -### Why isn't my license plate being detected and recognized? +### Detection and Recognition + + Ensure that: @@ -606,29 +609,43 @@ Recognized plates will show as object labels in the debug view and will appear i If you are still having issues detecting plates, start with a basic configuration and see the debugging tips below. -### Can I run LPR without detecting `car` or `motorcycle` objects? + + +Can I run LPR without detecting car or motorcycle objects?}> In normal LPR mode, Frigate requires a `car` or `motorcycle` to be detected first before recognizing a license plate. If you have a dedicated LPR camera, you can change the camera `type` to `"lpr"` to use the Dedicated LPR Camera algorithm. This comes with important caveats, though. See the [Dedicated LPR Cameras](#dedicated-lpr-cameras) section above. -### How can I improve detection accuracy? + + + - Use high-quality cameras with good resolution. - Adjust `detection_threshold` and `recognition_threshold` values. - Define a `format` regex to filter out invalid detections. -### Does LPR work at night? + + + Yes, but performance depends on camera quality, lighting, and infrared capabilities. Make sure your camera can capture clear images of plates at night. -### Can I limit LPR to specific zones? + + + LPR, like other Frigate enrichments, runs at the camera level rather than the zone level. While you can't restrict LPR to specific zones directly, you can control when recognition runs by setting a `min_area` value to filter out smaller detections. -### How can I match known plates with minor variations? + + + Use `match_distance` to allow small character mismatches. Alternatively, define multiple variations in `known_plates`. -### How do I debug LPR issues? + + +### Performance and Troubleshooting + + Start with ["Why isn't my license plate being detected and recognized?"](#why-isnt-my-license-plate-being-detected-and-recognized). If you are still having issues, work through these steps. @@ -685,17 +702,23 @@ lpr: - Watch the debug view to see plates recognized in real-time. For non-dedicated LPR cameras, the `car` or `motorcycle` label will change to the recognized plate when LPR is enabled and working. - Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration). -### Will LPR slow down my system? + + + LPR's performance impact depends on your hardware. Ensure you have at least 4GB RAM and a capable CPU or GPU for optimal results. If you are running the Dedicated LPR Camera mode, resource usage will be higher compared to users who run a model that natively detects license plates. Tune your motion detection settings for your dedicated LPR camera so that the license plate detection model runs only when necessary. -### I am seeing a YOLOv9 plate detection metric in Enrichment Metrics, but I have a Frigate+ or custom model that detects `license_plate`. Why is the YOLOv9 model running? + + +I am seeing a YOLOv9 plate detection metric in Enrichment Metrics, but I have a Frigate+ or custom model that detects license_plate. Why is the YOLOv9 model running?}> The YOLOv9 license plate detector model will run (and the metric will appear) if you've enabled LPR but haven't defined `license_plate` as an object to track, either at the global or camera level. If you are detecting `car` or `motorcycle` on cameras where you don't want to run LPR, make sure you disable LPR it at the camera level. And if you do want to run LPR on those cameras, make sure you define `license_plate` as an object to track. -### It looks like Frigate picked up my camera's timestamp or overlay text as the license plate. How can I prevent this? + + + This could happen if cars or motorcycles travel close to your camera's timestamp or overlay text. You could either move the text through your camera's firmware, or apply a mask to it in Frigate. @@ -703,6 +726,10 @@ If you are using a model that natively detects `license_plate`, add an _object m If you are not using a model that natively detects `license_plate` or you are using dedicated LPR camera mode, only a _motion mask_ over your text is required. -### I see "Error running ... model" in my logs, or my inference time is very high. How can I fix this? + + + This usually happens when your GPU is unable to compile or use one of the LPR models. Set your `device` to `CPU` and try again. GPU acceleration only provides a slight performance increase, and the models are lightweight enough to run without issue on most CPUs. + + diff --git a/docs/docs/plus/first_model.md b/docs/docs/plus/first_model.md index e9523f6b98..98095554b5 100644 --- a/docs/docs/plus/first_model.md +++ b/docs/docs/plus/first_model.md @@ -3,6 +3,10 @@ id: first_model title: Requesting your first model --- +import ConfigTabs from "@site/src/components/ConfigTabs"; +import TabItem from "@theme/TabItem"; +import NavPath from "@site/src/components/NavPath"; + ## Step 1: Upload and annotate your images Before requesting your first model, you will need to upload and verify at least 10 images to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. @@ -16,13 +20,21 @@ For more detailed recommendations, you can refer to the docs on [annotating](./a Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [this part of the documentation](./index.md#available-model-types). If you are unsure which type to request, you can test the base model for each version from the "Base Models" tab. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. ![Plus Models Page](/img/plus/plus-models.jpg) -## Step 3: Set your model id in the config +## Step 3: Set your model You will receive an email notification when your Frigate+ model is ready. ![Model Ready Email](/img/plus/model-ready-email.jpg) Models available in Frigate+ can be used with a special model path. No other information needs to be configured because it fetches the remaining config from Frigate+ automatically. + + + +Navigate to . In the **Detection Model** section, choose the **Frigate+** tab. Select your new Frigate+ model from the **Available Frigate+ models** dropdown, then click **Save**. Restart Frigate to apply the change. + + + + ```yaml detectors: ... @@ -30,22 +42,46 @@ model: path: plus:// ``` -:::note - -Model IDs are not secret values and can be shared freely. Access to your model is protected by your API key. - -::: - :::tip When setting the plus model id, all other fields should be removed as these are configured automatically with the Frigate+ model config ::: + + + +:::note + +Model IDs are not secret values and can be shared freely. Access to your model is protected by your API key. + +::: + ## Step 4: Adjust your object filters for higher scores Frigate+ models generally have much higher scores than the default model provided in Frigate. You will likely need to increase your `threshold` and `min_score` values. Here is an example of how these values can be refined, but you should expect these to evolve as your model improves. For more information about how `threshold` and `min_score` are related, see the docs on [object filters](../configuration/object_filters.md#object-scores). + + + +Navigate to . Under **Object filters**, set **Min Score** and **Threshold** for each object type, then click **Save**. + +| Object | Min Score | Threshold | +| ----------------- | --------- | --------- | +| **dog** | .7 | .9 | +| **cat** | .65 | .8 | +| **face** | .7 | | +| **package** | .65 | .9 | +| **license_plate** | .6 | | +| **amazon** | .75 | | +| **ups** | .75 | | +| **fedex** | .75 | | +| **person** | .65 | .85 | +| **car** | .65 | .85 | + + + + ```yaml objects: filters: @@ -75,3 +111,6 @@ objects: min_score: .65 threshold: .85 ``` + + + diff --git a/docs/src/components/FaqItem/index.jsx b/docs/src/components/FaqItem/index.jsx new file mode 100644 index 0000000000..51d4c072e0 --- /dev/null +++ b/docs/src/components/FaqItem/index.jsx @@ -0,0 +1,66 @@ +import React, { useState, useEffect } from "react"; +import Heading from "@theme/Heading"; +import styles from "./styles.module.css"; + +// A single FAQ entry. +// +// The question is a real anchored heading (via @theme/Heading), so on desktop +// it gets the standard hover "#" hash link and the answer is always shown. On +// mobile the heading text is a button that toggles its answer, keeping long +// FAQ pages short. The desktop/mobile split is pure CSS (Docusaurus breakpoint: +// 996px), so there is no hydration flash. The answer is always rendered into +// the DOM, so search engines and the docs AI bot can read it regardless of +// layout or collapsed state. The heading id resolves deep links on both layouts +// and auto-expands the entry on mobile when it is the link target. +export default function FaqItem({ id, question, children }) { + const [open, setOpen] = useState(false); + + useEffect(() => { + const openIfTargeted = () => { + if (window.location.hash === `#${id}`) { + setOpen(true); + } + }; + openIfTargeted(); + window.addEventListener("hashchange", openIfTargeted); + return () => window.removeEventListener("hashchange", openIfTargeted); + }, [id]); + + const toggle = () => { + const next = !open; + setOpen(next); + // Reflect the entry in the URL like clicking the heading anchor, so an + // opened answer is shareable. Use replaceState to avoid history spam and + // an abrupt scroll. Clear it on close if it currently points here. + if (next) { + if (window.location.hash !== `#${id}`) { + window.history.replaceState(null, "", `#${id}`); + } + } else if (window.location.hash === `#${id}`) { + window.history.replaceState( + null, + "", + window.location.pathname + window.location.search, + ); + } + }; + + return ( +
+ + + +
+ {children} +
+
+ ); +} diff --git a/docs/src/components/FaqItem/styles.module.css b/docs/src/components/FaqItem/styles.module.css new file mode 100644 index 0000000000..7c367212b7 --- /dev/null +++ b/docs/src/components/FaqItem/styles.module.css @@ -0,0 +1,87 @@ +/* + * FAQ entry: collapsible on mobile, static heading + expanded answer on + * desktop. The split is pure CSS (Docusaurus breakpoint: 996px) so there is + * no hydration flash. The answer is always rendered into the DOM, so search + * engines and the docs AI bot can read it regardless of layout or state. + */ + +.item { + scroll-margin-top: calc(var(--ifm-navbar-height) + 1rem); +} + +.heading { + margin: 0; +} + +/* Mobile: the heading text is a full-width clickable toggle row. */ +.toggle { + display: flex; + align-items: center; + gap: 0.6rem; + width: 100%; + padding: 0.85rem 0; + border: none; + border-bottom: 1px solid var(--ifm-color-emphasis-200); + background: none; + color: inherit; + font: inherit; + text-align: left; + cursor: pointer; +} + +.toggle::before { + content: ""; + flex: 0 0 auto; + width: 0.5rem; + height: 0.5rem; + border-right: 2px solid currentColor; + border-bottom: 2px solid currentColor; + transform: rotate(-45deg); + transition: transform var(--ifm-transition-fast, 200ms) ease; +} + +.item[data-open] .toggle::before { + transform: rotate(45deg); +} + +.content { + display: none; + padding: 0 0 0.85rem; +} + +.item[data-open] .content { + display: block; +} + +/* Hide the hover hash link on mobile (no hover; avoids a stray empty line). */ +.heading :global(.hash-link) { + display: none; +} + +/* Desktop: render as a normal expanded heading + answer. */ +@media (min-width: 997px) { + .heading { + margin: 1.75rem 0 0.5rem; + } + + .toggle { + display: inline; + width: auto; + padding: 0; + border: none; + cursor: default; + } + + .toggle::before { + display: none; + } + + .content { + display: block; + padding: 0; + } + + .heading :global(.hash-link) { + display: inline; + } +}