administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-11-15 18:13:01 +03:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Add Frigate+ model selection guide (#24981)

Browse Source
This commit is contained in:
Blake Blackshear
2026-11-15 06:57:07 -07:00
committed by GitHub
parent cc94cbc2da
commit d3b7d4ea44
4 changed files with 138 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/plus/choosing_a_model.md
+121
View File
@@ -0,0 +1,121 @@
---
id: choosing_a_model
title: Choosing a model
---
Frigate+ offers several model types, sizes, and input resolutions. This page compares their accuracy and speed to help you decide which one to request. Make sure the model type you choose is supported by your detector, as shown in the [supported detector types](./index.md#supported-detector-types) table.
## Recommendations
- **YOLOv9-s at 320** is the balanced default if your hardware runs it well. It detects about twice as many objects as MobileDet, and 15 more per 100 than YOLO-NAS-s at 320, at the same false alarm rate.
- **Moving up a size at 320** helps more than moving to 640 for most cameras. YOLOv9-m at 320 detects more objects than YOLOv9-s at 640 (60% vs 59%) with lower GPU latency.
- **640 input** is rarely needed. Frigate doesn't run detection on the full frame. It crops a region around motion and resizes it to the model's input size, so with a 320 model it zooms in on a smaller area of the frame and small objects are no longer small to the model. 640 models detect small objects better within a large region, but not compared to a 320 model looking at a smaller, zoomed-in region. See [object size and input resolution](#object-size-and-input-resolution) for details.
- **YOLOv9-c** gains little over YOLOv9-m: about 2 points of detection rate at 320 and none at 640, with more latency. On CPU it takes 1.4 to 1.7 times as long as YOLOv9-m.
- **YOLOv9-t at 640** is no better than YOLOv9-s at 320 (55% vs 54%) and is slower on both GPU and CPU.
- **YOLO-NAS-s** and **MobileDet** are only worth considering on hardware that can't run YOLOv9. Everywhere else, a YOLOv9 model of similar speed is more accurate.
Results on your own hardware and cameras will differ, so test with the base models before requesting a model.
### Thresholds
Each model's threshold for 95% precision is listed in the [accuracy](#accuracy) table. YOLOv9 models reach it at about 0.70 to 0.77. MobileDet (0.82) and YOLO-NAS-s (0.86 to 0.89) need higher thresholds, so with Frigate's default threshold they produce more false positives than the YOLOv9 models. See [object filters](../configuration/object_filters.md#object-scores) for how to adjust thresholds.
## How these were measured
The numbers on this page are for the 2026.3 base models.
- **Images:** 10,947 images from the Frigate+ validation set, from real Frigate cameras. Images that share a source with training images are excluded.
- **Processing:** every model runs through Frigate's own detector code and post-processing (Frigate 0.18.2), exactly as deployed.
- **Labels:** results cover the 12 most common labels: `car`, `person`, `waste_bin`, `license_plate`, `dog`, `cat`, `face`, `bicycle`, `bird`, `deer`, `bbq_grill`, and `package`.
- **Single frames:** each image is scored on its own. In Frigate, an object has to score well across several frames before it becomes a tracked object, so brief one-frame mistakes rarely reach you. Use these numbers to compare models with each other, not as event rates.
**Detection rate at 95% precision** is the main accuracy number. It's the share of real objects a model detects when its threshold is set so that 95% of its detections are correct (about 1 false detection in 20). Every model is held to the same false alarm rate, so the detection rates compare directly. The threshold that achieves this is shown too, because it differs between models.
## Accuracy
| Model | Input | Detection rate at 95% precision | Missed objects per 100 | Threshold for 95% precision | AP50 |
| ---------- | ----- | ------------------------------- | ---------------------- | --------------------------- | ---- |
| MobileDet | 320 | 24% | 76 | 0.82 | 0.47 |
| YOLO-NAS-s | 320 | 39% | 61 | 0.86 | 0.44 |
| YOLO-NAS-s | 640 | 48% | 52 | 0.89 | 0.55 |
| YOLOv9-t | 320 | 45% | 55 | 0.69 | 0.44 |
| YOLOv9-t | 640 | 55% | 45 | 0.70 | 0.54 |
| YOLOv9-s | 320 | 54% | 46 | 0.74 | 0.54 |
| YOLOv9-s | 640 | 59% | 41 | 0.75 | 0.61 |
| YOLOv9-m | 320 | 60% | 40 | 0.75 | 0.58 |
| YOLOv9-m | 640 | 64% | 36 | 0.77 | 0.64 |
| YOLOv9-c | 320 | 62% | 38 | 0.75 | 0.60 |
| YOLOv9-c | 640 | 63% | 37 | 0.77 | 0.64 |
AP50 is the standard COCO metric (IoU 0.5) averaged over the 12 labels. It summarizes accuracy across every threshold, while the detection rate describes one realistic setting.
:::note
These AP50 numbers can't be compared to the COCO results published for YOLOv9, YOLO-NAS, and other models. Those results are measured on the COCO dataset, while these are measured on images from security cameras, with a different set of labels (several, such as `license_plate`, `face`, and `package`, aren't in COCO) and through Frigate's own post-processing. Published COCO results also usually report mAP averaged over IoU 0.5 to 0.95 rather than AP50. Use these numbers only to compare Frigate+ models with each other.
:::
### By object size
AP50 by object size:
| Model | Input | Small | Medium | Large |
| ---------- | ----- | ----- | ------ | ----- |
| MobileDet | 320 | 0.23 | 0.62 | 0.71 |
| YOLO-NAS-s | 320 | 0.23 | 0.57 | 0.60 |
| YOLO-NAS-s | 640 | 0.42 | 0.63 | 0.58 |
| YOLOv9-t | 320 | 0.22 | 0.60 | 0.67 |
| YOLOv9-t | 640 | 0.42 | 0.63 | 0.57 |
| YOLOv9-s | 320 | 0.35 | 0.70 | 0.73 |
| YOLOv9-s | 640 | 0.51 | 0.69 | 0.69 |
| YOLOv9-m | 320 | 0.40 | 0.73 | 0.74 |
| YOLOv9-m | 640 | 0.54 | 0.71 | 0.72 |
| YOLOv9-c | 320 | 0.44 | 0.73 | 0.77 |
| YOLOv9-c | 640 | 0.54 | 0.71 | 0.74 |
In these single-image results, 640 input mostly helps small objects, such as distant people, license plates, and faces, and doesn't help medium and large objects. For example, license plate AP50 rises from 0.32 to 0.55 for YOLOv9-s. These numbers overstate the benefit of 640 in Frigate, as explained below.
#### Object size and input resolution
Frigate doesn't send the full camera frame to the model. It crops a region around motion or a tracked object and resizes that region to the model's input size. With a 320 model the regions are smaller, so Frigate effectively zooms in on that part of the frame, and an object that is small in the full frame is no longer small to the model. A 640 model detects small objects better than a 320 model when both look at the same large region, but not when compared to a 320 model looking at a smaller, zoomed-in region.
640 input can still help in scenes with large objects in the foreground and small objects in the background at the same time, such as a doorbell camera that sees a person on the porch and another person across the street. The region has to be large enough to fit the nearby person, so the distant person is small within it. Frigate already handles much of this case: when an object is small relative to the region it was detected in, Frigate runs a separate detection on a smaller region around it.
### By label
AP50 by label:
| Label | MobileDet 320 | YOLO-NAS-s 320 | YOLOv9-t 320 | YOLOv9-s 320 | YOLOv9-m 320 | YOLOv9-c 320 | YOLO-NAS-s 640 | YOLOv9-t 640 | YOLOv9-s 640 | YOLOv9-m 640 | YOLOv9-c 640 |
| --------------- | ------------- | -------------- | ------------ | ------------ | ------------ | ------------ | -------------- | ------------ | ------------ | ------------ | ------------ |
| `car` | 0.73 | 0.72 | 0.70 | 0.77 | 0.81 | 0.82 | 0.82 | 0.79 | 0.83 | 0.85 | 0.84 |
| `person` | 0.65 | 0.62 | 0.62 | 0.72 | 0.75 | 0.76 | 0.71 | 0.69 | 0.76 | 0.78 | 0.77 |
| `waste_bin` | 0.52 | 0.49 | 0.49 | 0.55 | 0.57 | 0.58 | 0.60 | 0.60 | 0.62 | 0.62 | 0.63 |
| `license_plate` | 0.31 | 0.31 | 0.23 | 0.32 | 0.36 | 0.39 | 0.55 | 0.46 | 0.55 | 0.60 | 0.61 |
| `dog` | 0.38 | 0.35 | 0.39 | 0.53 | 0.53 | 0.57 | 0.40 | 0.48 | 0.53 | 0.55 | 0.58 |
| `cat` | 0.41 | 0.37 | 0.39 | 0.51 | 0.57 | 0.59 | 0.43 | 0.46 | 0.58 | 0.61 | 0.62 |
| `face` | 0.39 | 0.41 | 0.23 | 0.33 | 0.42 | 0.45 | 0.60 | 0.43 | 0.57 | 0.63 | 0.63 |
| `bicycle` | 0.46 | 0.43 | 0.40 | 0.51 | 0.56 | 0.55 | 0.54 | 0.49 | 0.58 | 0.62 | 0.60 |
| `bird` | 0.30 | 0.18 | 0.33 | 0.42 | 0.47 | 0.50 | 0.23 | 0.30 | 0.39 | 0.43 | 0.38 |
| `deer` | 0.51 | 0.50 | 0.51 | 0.62 | 0.69 | 0.71 | 0.59 | 0.56 | 0.60 | 0.70 | 0.67 |
| `bbq_grill` | 0.68 | 0.60 | 0.69 | 0.77 | 0.78 | 0.81 | 0.74 | 0.74 | 0.81 | 0.83 | 0.84 |
| `package` | 0.35 | 0.28 | 0.34 | 0.46 | 0.47 | 0.49 | 0.39 | 0.43 | 0.50 | 0.48 | 0.50 |
## Speed
Average time per detection in milliseconds, measured through Frigate's detector code and including post-processing. Lower is better. A blank cell means that combination was not measured.
| Model | Input | NVIDIA RTX 5060 Ti (ONNX, CUDA) | Intel i9-13900K (OpenVINO, CPU) | Intel i9-13900K (ONNX, CPU) | Intel i9-13900K (TFLite, CPU) |
| ---------- | ----- | ------------------------------- | ------------------------------- | --------------------------- | ----------------------------- |
| MobileDet | 320 | | | | 8.4 |
| YOLO-NAS-s | 320 | 4.9 | 8.4 | 8.9 | |
| YOLO-NAS-s | 640 | 7.1 | 29.6 | 28.6 | |
| YOLOv9-t | 320 | 2.9 | 4.3 | 6.5 | |
| YOLOv9-t | 640 | 5.0 | 12.9 | 17.0 | |
| YOLOv9-s | 320 | 3.9 | 9.3 | 11.5 | |
| YOLOv9-s | 640 | 6.3 | 33.1 | 32.6 | |
| YOLOv9-m | 320 | 5.2 | 23.2 | 19.8 | |
| YOLOv9-m | 640 | 10.1 | 88.1 | 69.5 | |
| YOLOv9-c | 320 | 5.4 | 38.8 | 27.5 | |
| YOLOv9-c | 640 | 11.1 | 147.8 | 99.5 | |
CPU times vary by about ±15% between runs. Speeds on other hardware, such as Google Coral, Hailo, Intel iGPUs and NPUs, AMD GPUs, and Rockchip NPUs, are not included here. Test the base models on your own hardware to compare speeds.
docs/docs/plus/first_model.md
+1 -1
View File
@@ -17,7 +17,7 @@ For more detailed recommendations, you can refer to the docs on [annotating](./a
## Step 2: Submit a model request ## Step 2: Submit a model request
Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [this part of the documentation](./index.md#available-model-types). If you are unsure which type to request, you can test the base model for each version from the "Base Models" tab. For most hardware, `yolov9` is recommended; `mobiledet` is a legacy model type and will be removed in 2027. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [available model types](./index.md#available-model-types) and [choosing a model](./choosing_a_model.md). If you are unsure which type to request, you can test the base model for each version from the "Base Models" tab. For most hardware, `yolov9` is recommended; `mobiledet` is a legacy model type and will be removed in 2027. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours.
![Plus Models Page](/img/plus/plus-models.jpg) ![Plus Models Page](/img/plus/plus-models.jpg)
## Step 3: Set your model ## Step 3: Set your model
docs/docs/plus/index.md
+15 -15
View File
@@ -13,13 +13,13 @@ Information on how to integrate Frigate+ with Frigate can be found in the [integ
There are three model types offered in Frigate+, `yolov9`, `yolonas`, and `mobiledet` (legacy). All of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types). There are three model types offered in Frigate+, `yolov9`, `yolonas`, and `mobiledet` (legacy). All of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types).
Not all model types are supported by all detectors, so it's important to choose a model type to match your detector as shown in the table under [supported detector types](#supported-detector-types). You can test model types for compatibility and speed on your hardware by using the base models. Not all model types are supported by all detectors, so it's important to choose a model type to match your detector as shown in the table under [supported detector types](#supported-detector-types). You can test model types for compatibility and speed on your hardware by using the base models. For accuracy and speed comparisons between model types, sizes, and resolutions, see [choosing a model](./choosing_a_model.md).
| Model Type | Description | | Model Type | Description |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `yolov9` | A leading SOTA (state of the art) object detection model with similar performance to yolonas, but on a wider range of hardware options. Runs on most hardware. | | `yolov9` | A leading SOTA (state of the art) object detection model. More accurate than yolonas at similar speeds, and runs on a wider range of hardware. Runs on most hardware. |
| `yolonas` | A newer architecture that offers slightly higher accuracy and improved detection of small objects. Runs on Intel, NVidia GPUs, and AMD GPUs. | | `yolonas` | An alternative architecture for hardware that can't run yolov9. Runs on Intel, NVidia GPUs, and AMD GPUs. |
| `mobiledet` | Legacy. Based on the same architecture as the default model included with Frigate. Runs on Google Coral devices and CPUs. | | `mobiledet` | Legacy. Based on the same architecture as the default model included with Frigate. Runs on Google Coral devices and CPUs. |
:::warning :::warning
@@ -29,15 +29,15 @@ Not all model types are supported by all detectors, so it's important to choose
### YOLOv9 Details ### YOLOv9 Details
YOLOv9 models are available in `t`, `s`, `c`, and `edgetpu` variants, and an `m` size is coming soon. When requesting a `yolov9` model, you will be prompted to choose a variant. If you want the model to be compatible with a Google Coral, you will need to choose the `edgetpu` variant, which requires Frigate 0.17 or later. If you are unsure what variant to choose, you should perform some tests with the base models to find the performance level that suits you. The `s` size is most similar to the current `yolonas` models in terms of inference times and accuracy, and a good place to start is the `320x320` resolution model for `yolov9s`. The `c` size is larger and offers higher accuracy at the cost of slower inference, so it is best suited to more powerful hardware such as GPUs. YOLOv9 models are available in `t`, `s`, `m`, `c`, and `edgetpu` variants. When requesting a `yolov9` model, you will be prompted to choose a variant. If you want the model to be compatible with a Google Coral, you will need to choose the `edgetpu` variant, which requires Frigate 0.17 or later. If you are unsure what variant to choose, you should perform some tests with the base models to find the performance level that suits you. The `s` size has similar inference times to the current `yolonas` models with noticeably higher accuracy, and a good place to start is the `320x320` resolution model for `yolov9s`. The `m` size offers higher accuracy at the cost of slower inference, so it is best suited to more powerful hardware such as GPUs. The `c` size is slower still and offers little accuracy gain over `m`. See [choosing a model](./choosing_a_model.md) for detailed accuracy and speed comparisons.
| Variant | Size | Parameters | Speed and accuracy | Best suited for | | Variant | Size | Parameters | Speed and accuracy | Best suited for |
| --------- | -------------------- | ---------- | ----------------------------------------------------------- | -------------------------------------------------- | | --------- | ------- | ---------- | ------------------------------------------------------------- | -------------------------------------------------- |
| `t` | Tiny | ~2.0M | Fastest inference, lowest accuracy | Low power hardware or a large number of cameras | | `t` | Tiny | ~2.0M | Fastest inference, lowest accuracy | Low power hardware or a large number of cameras |
| `s` | Small | ~7.1M | Balanced. Similar inference times and accuracy to `yolonas` | Most users. Start with `320x320` | | `s` | Small | ~7.1M | Balanced. Similar inference times to `yolonas`, more accurate | Most users. Start with `320x320` |
| `m` | Medium (coming soon) | ~20.0M | Slower than `s` with higher accuracy | Mid-range GPUs and NPUs | | `m` | Medium | ~20.0M | Slower than `s` with higher accuracy | Mid-range GPUs and NPUs |
| `c` | Compact | ~25.3M | Highest accuracy, slowest inference | More powerful hardware such as GPUs | | `c` | Compact | ~25.3M | Similar accuracy to `m`, slowest inference | More powerful hardware such as GPUs |
| `edgetpu` | N/A | N/A | Optimized to run on the Coral's Edge TPU | Google Coral devices only (requires Frigate 0.17+) | | `edgetpu` | N/A | N/A | Optimized to run on the Coral's Edge TPU | Google Coral devices only (requires Frigate 0.17+) |
Parameter counts are for the standard YOLOv9 architectures. Actual inference times depend on your hardware and the input resolution you choose, so test with the base models before requesting a model. Parameter counts are for the standard YOLOv9 architectures. Actual inference times depend on your hardware and the input resolution you choose, so test with the base models before requesting a model.
@@ -49,7 +49,7 @@ When switching to YOLOv9, you may need to adjust your thresholds for some object
#### Hailo Support #### Hailo Support
If you have a Hailo device, you will need to specify the hardware you have when submitting a model request because they are not cross compatible. Hailo models are available for the `t`, `s`, and `c` variants on Hailo8, Hailo8L, and Hailo8R devices. Please test using the available base models before submitting your model request. If you have a Hailo device, you will need to specify the hardware you have when submitting a model request because they are not cross compatible. Hailo models are available for the `t`, `s`, `m`, and `c` variants on Hailo8, Hailo8L, and Hailo8R devices. Please test using the available base models before submitting your model request.
#### Rockchip (RKNN) Support #### Rockchip (RKNN) Support
docs/sidebars.ts
+1
View File
@@ -160,6 +160,7 @@ const sidebars: SidebarsConfig = {
], ],
"Frigate+": [ "Frigate+": [
"plus/index", "plus/index",
"plus/choosing_a_model",
"plus/annotating", "plus/annotating",
"plus/first_model", "plus/first_model",
"plus/faq", "plus/faq",