administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-09-26 17:48:59 +03:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Implement UI for managing multiple models (#24023)

Browse Source 
* Implement hardware detection and UI management

* Cleanup Frigate+ detection

* Don't count model as changed

* Fixes for audio map error

* Add descriptions

* Enforce that all model must exist

* Fix hardware picking

* Docs fixes

* WebUI cleanup

* Cleanup handling of scenes

* UI refinement

* Cleanup recommended UI

* test fixews
This commit is contained in:
Nicolas Mowen
2026-08-22 11:40:42 -05:00
committed by Josh Hawkins
parent 79ea68caa2
commit 07ba2357e6
59 changed files with 2554 additions and 2587 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/data/object_detectors_models.yaml
+34 -130
View File
@@ -4,8 +4,8 @@ edgeTPU:
- key: mobiledet
label: Mobiledet
recommended: true
download: A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`.
ui: Navigate to **Settings > System > Detectors and model** and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`.
download: A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with a model's `path`.
ui: Navigate to **Settings > System > Detection models** and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown.
yaml: |-
models:
- devices:
@@ -13,9 +13,9 @@ edgeTPU:
- key: yolov9
label: YOLOv9
recommended: false
download: "[Download the model](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite), bind mount the file into the container, and provide the path with `model.path`. Note that the linked model requires a 17-label [labelmap file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) that includes only 17 COCO classes."
download: "[Download the model](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite), bind mount the file into the container, and provide the path with a model's `path`. Note that the linked model requires a 17-label [labelmap file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) that includes only 17 COCO classes."
ui: |-
Navigate to **Settings > System > Detectors and model** and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to **Settings > System > Detection models** and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure the model settings:
| Field | Value |
| ---------------------------------------- | ----------------------------------------------------------------- |
@@ -44,7 +44,7 @@ hailo8l:
recommended: true
download: If no custom model path or URL is provided, the Hailo detector automatically downloads the default model (YOLOv6n) from the Hailo Model Zoo on first startup based on the detected hardware. Once cached under `/config/model_cache/hailo`, the model works fully offline.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to **Settings > System > Detection models** and select **Hailo** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure the model settings:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -86,7 +86,7 @@ hailo8l:
recommended: false
download: For SSD-based models, provide either a model path or URL to your compiled SSD model. The integration will first check the local path before downloading if necessary. The model file is cached under `/config/model_cache/hailo`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to **Settings > System > Detection models** and select **Hailo** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure the model settings:
| Field | Value |
| --------------------------------------- | ------ |
@@ -143,7 +143,7 @@ openvino:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** (or **Intel NPU**) from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -171,7 +171,7 @@ openvino:
recommended: false
download: An OpenVINO model is provided in the container at `/openvino-model/ssdlite_mobilenet_v2.xml` and is used by this detector type by default. The model comes from Intel's Open Model Zoo [SSDLite MobileNet V2](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssdlite_mobilenet_v2) and is converted to an FP16 precision IR model.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** (or **Intel NPU**) from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------ |
@@ -206,7 +206,7 @@ openvino:
python3 yolo_to_onnx.py -m yolov7-320
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** (or **Intel NPU**) from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -243,7 +243,7 @@ openvino:
The input image size in this notebook is set to 320x320. This results in lower CPU usage and faster inference times without impacting performance in most cases due to the way Frigate crops video frames to areas of interest before running detection. The notebook and config can be updated to 640x640 if desired.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -271,7 +271,7 @@ openvino:
recommended: false
download: YOLOx models can be downloaded [from the YOLOx repo](https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/ONNXRuntime).
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ------------------------------------- | -------------------------------- |
@@ -308,7 +308,7 @@ openvino:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Intel GPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| --------------------------------------- | --------------------------------- |
@@ -402,7 +402,7 @@ openvino:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `CPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **CPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------- |
@@ -455,7 +455,7 @@ appleSilicon:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and add a model. The ZMQ endpoint is not reported by the hardware probe, so set `devices` to `zmq:tcp://host.docker.internal:5555` in YAML. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -491,7 +491,7 @@ appleSilicon:
python3 yolo_to_onnx.py -m yolov7-320
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and add a model. The ZMQ endpoint is not reported by the hardware probe, so set `devices` to `zmq:tcp://host.docker.internal:5555` in YAML. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -544,7 +544,7 @@ onnx:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -588,7 +588,7 @@ onnx:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| --------------------------------------- | --------------------------------- |
@@ -623,7 +623,7 @@ onnx:
The input image size in this notebook is set to 320x320. This results in lower CPU usage and faster inference times without impacting performance in most cases due to the way Frigate crops video frames to areas of interest before running detection. The notebook and config can be updated to 640x640 if desired.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -651,7 +651,7 @@ onnx:
recommended: false
download: YOLOx models can be downloaded [from the YOLOx repo](https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/ONNXRuntime).
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -747,7 +747,7 @@ onnx:
EOF
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------- |
@@ -783,7 +783,7 @@ onnx:
python3 yolo_to_onnx.py -m yolov7-320
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select your GPU from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -812,9 +812,9 @@ cpu:
- key: ssd
label: MobileNet v2
recommended: true
download: A TensorFlow Lite model is provided in the container at `/cpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`.
download: A TensorFlow Lite model is provided in the container at `/cpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with a model's `path`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **CPU** from the detector type dropdown and click **Add**. Configure the number of threads and click **Add** again to add additional CPU detectors as needed (one per camera is recommended).
Navigate to **Settings > System > Detection models** and select **CPU** from the **Hardware** dropdown and set **Detectors** to the number of detection processes to run (one per camera is recommended).
| Field | Value |
| ----------------- | ----- |
@@ -832,7 +832,7 @@ deepstack:
recommended: true
download: This detector runs object detection over the network against a CodeProject.AI or DeepStack server, so no model is downloaded into Frigate itself. Visit the [CodeProject.AI official website](https://www.codeproject.com/Articles/5322557/CodeProject-AI-Server-AI-the-easy-way) to download and install the AI server on your preferred device (e.g. Raspberry Pi, Nvidia Jetson, or other compatible hardware) before configuring the detector.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **DeepStack** from the detector type dropdown and click **Add**. Set the API URL to point to your CodeProject.AI server (e.g., `http://<your_codeproject_ai_server_ip>:<port>/v1/vision/detection`).
Navigate to **Settings > System > Detection models** and add a model. The CodeProject.AI server is not reported by the hardware probe, so set `devices` to `deepstack:http://<your_codeproject_ai_server_ip>:<port>/v1/vision/detection` in YAML.
| Field | Value |
| ------------- | ---------------------------------------------------------------------- |
@@ -859,7 +859,7 @@ memryx:
MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the container at `/memryx_models/model_folder/`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **MemryX MX3** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -893,7 +893,7 @@ memryx:
MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the container at `/memryx_models/model_folder/`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **MemryX MX3** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -926,7 +926,7 @@ memryx:
MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the container at `/memryx_models/model_folder/`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **MemryX MX3** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -959,7 +959,7 @@ memryx:
MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the container at `/memryx_models/model_folder/`.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **MemryX MX3** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -1005,7 +1005,7 @@ tensorrt:
- USE_FP16=false
```
ui: |-
Navigate to **Settings > System > Detectors and model** and select **TensorRT** from the detector type dropdown and click **Add**, then set the device to `0` (the default GPU index). Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **NVIDIA Jetson** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------------------ |
@@ -1035,7 +1035,7 @@ synaptics:
recommended: true
download: A synap model is provided in the container at `/mobilenet.synap` and is used by this detector type by default. The model comes from the [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
ui: |-
Navigate to **Settings > System > Detectors and model** and select **Synaptics** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **Synaptics NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------- |
@@ -1068,7 +1068,7 @@ rknn:
You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models.
ui: |-
Navigate to **Settings > System > Detectors and model** and, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** select **Rockchip NPU** from the **Hardware** dropdown, then open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------- |
@@ -1108,7 +1108,7 @@ rknn:
**Note:** The pre-trained YOLO-NAS weights from DeciAI are subject to their license and can't be used commercially. For more information, see: https://docs.deci.ai/super-gradients/latest/LICENSE.YOLONAS.html
ui: |-
Navigate to **Settings > System > Detectors and model** and, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** select **Rockchip NPU** from the **Hardware** dropdown, then open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ----------------------------------------------------------------------- |
@@ -1145,7 +1145,7 @@ rknn:
You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models.
ui: |-
Navigate to **Settings > System > Detectors and model** and, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** select **Rockchip NPU** from the **Hardware** dropdown, then open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------------------- |
@@ -1182,7 +1182,7 @@ axengine:
recommended: true
download: A yolov9 axmodel is provided in the container at `/axmodels` and is used by this detector type by default. The AXEngine detector downloads its default model from HuggingFace on first startup; once cached, the model works fully offline.
ui: |-
Navigate to **Settings > System > Detectors and model** and select **AXEngine NPU** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to **Settings > System > Detection models** and select **AXERA NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -1195,20 +1195,6 @@ axengine:
| **Model Input D Type** | `int` |
| **Object Detection Model Type** | `yolo-generic` |
yaml: |-
<<<<<<< HEAD
detectors:
axengine:
type: axengine
model:
path: frigate-yolov9-tiny
model_type: yolo-generic
width: 320
height: 320
input_dtype: int
input_pixel_format: bgr
labelmap_path: /labelmap/coco-80.txt
=======
models:
- devices:
- axengine
@@ -1219,85 +1205,3 @@ axengine:
input_dtype: int
input_pixel_format: bgr
labelmap_path: /labelmap/coco-80.txt
degirumAiServer:
title: DeGirum AI Server
models:
- key: ai-server-inference
label: AI Server Inference
recommended: true
download: |-
Launch a DeGirum AI server as a Docker container, then point the detector at it. Add this to your `docker-compose.yml`:
```yaml
degirum_detector:
container_name: degirum
image: degirum/aiserver:latest
privileged: true
ports:
- "8778:8778"
```
Set `location` to the server's service name, container name, or `host:port`.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `degirum` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
models:
- devices:
- degirum:degirum
path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300
height: 300
input_pixel_format: rgb
degirumLocal:
title: DeGirum Local
models:
- key: local-inference
label: Local Inference
recommended: true
download: Run hardware directly inside the Frigate container with `@local`, removing the AI server hop. The matching device runtime (e.g. the Hailo runtime) must be installed in the container; confirm it with `degirum sys-info`.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `@local` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
models:
- devices:
- degirum:@local
path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300
height: 300
input_pixel_format: rgb
degirumCloud:
title: DeGirum AI Hub Cloud
models:
- key: ai-hub-cloud-inference
label: AI Hub Cloud Inference
recommended: true
download: Run inferences on DeGirum's [AI Hub](https://hub.degirum.com) cloud with `@cloud`. Sign up, create an access token, and set it as `token`. Network latency may require lowering your detection fps.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `@cloud` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
models:
- devices:
- degirum:@cloud
path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300
height: 300
input_pixel_format: rgb
>>>>>>> 34363affa (Refactor detector and model management)
docs/docs/configuration/advanced/system.md
+1 -1
View File
@@ -177,7 +177,7 @@ Custom models may also require different input tensor formats. The colorspace co
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and open the **Custom Model** tab to configure the model path, dimensions, and input format.
Navigate to <NavPath path="Settings > System > Detection models" /> and, on the model you want to change, open the **Custom Model** tab to configure the model path, dimensions, and input format.
| Field | Description |
| --------------------------------------------- | ------------------------------------ |
docs/docs/configuration/config.md
+4 -4
View File
@@ -154,7 +154,7 @@ Here are some common starter configuration examples. These can be configured thr
1. Navigate to <NavPath path="Settings > System > MQTT" /> and configure the MQTT connection to your Home Assistant Mosquitto broker
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `Raspberry Pi (H.264)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
3. Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown
4. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
5. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
6. Navigate to <NavPath path="Settings > Global configuration > Camera management" /> and add your camera with the appropriate RTSP stream URL
@@ -232,7 +232,7 @@ cameras:
1. Navigate to <NavPath path="Settings > System > MQTT" /> and set **Enable MQTT** to off
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `VAAPI (Intel/AMD GPU)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
3. Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown
4. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
5. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
6. Navigate to <NavPath path="Settings > Global configuration > Camera management" /> and add your camera with the appropriate RTSP stream URL
@@ -308,8 +308,8 @@ cameras:
1. Navigate to <NavPath path="Settings > System > MQTT" /> and configure the connection to your MQTT broker
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `VAAPI (Intel/AMD GPU)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `openvino` and **Device** `AUTO`
4. On the same page, in the **Custom Model** tab, configure the OpenVINO model path and settings
3. Navigate to <NavPath path="Settings > System > Detection models" /> and select **Intel GPU** from the **Hardware** dropdown
4. On the same model, open the **Custom Model** tab and configure the OpenVINO model path and settings
5. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
6. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
7. Navigate to <NavPath path="Settings > Global configuration > Camera management" /> and add your camera with the appropriate RTSP stream URL
docs/docs/configuration/object_detectors.md
+8 -103
View File
@@ -165,7 +165,7 @@ See [common Edge TPU troubleshooting steps](/troubleshooting/edgetpu) if the Edg
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`.
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown.
</TabItem>
<TabItem value="yaml">
@@ -184,7 +184,7 @@ models:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `usb:0` and `usb:1` as the device for each.
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown and check each Coral the model should run on.
</TabItem>
<TabItem value="yaml">
@@ -206,7 +206,7 @@ _warning: may have [compatibility issues](https://github.com/blakeblackshear/fri
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then leave the device field empty.
Navigate to <NavPath path="Settings > System > Detection models" /> and select the **Coral EdgeTPU** entry from the **Hardware** dropdown.
</TabItem>
<TabItem value="yaml">
@@ -225,7 +225,7 @@ models:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `pci`.
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (PCIe)** from the **Hardware** dropdown.
</TabItem>
<TabItem value="yaml">
@@ -244,7 +244,7 @@ models:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `pci:0` and `pci:1` as the device for each.
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (PCIe)** from the **Hardware** dropdown and check each Coral the model should run on.
</TabItem>
<TabItem value="yaml">
@@ -264,7 +264,7 @@ models:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors with different device types (e.g., `usb` and `pci`).
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown. USB and PCIe Corals are listed as separate hardware, so mixing the two on one model has to be done in YAML.
</TabItem>
<TabItem value="yaml">
@@ -354,7 +354,7 @@ Intel NPUs cannot be used under Home Assistant OS, which does not include the NP
:::warning
The network-based detectors (Deepstack, DeGirum, and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, DeGirum ignores `zoo` and `token`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
The network-based detectors (Deepstack and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
:::
@@ -536,7 +536,7 @@ When using CPU detectors, you can add one CPU detector per camera. Adding more d
:::warning
The network-based detectors (Deepstack, DeGirum, and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, DeGirum ignores `zoo` and `token`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
The network-based detectors (Deepstack and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
:::
@@ -809,101 +809,6 @@ Explanation of the parameters:
- **example**: Specifying `output_name = "frigate-{quant}-{input_basename}-{soc}-v{tk_version}"` could result in a model called `frigate-i8-my_model-rk3588-v2.3.0.rknn`.
- `config`: Configuration passed to `rknn-toolkit2` for model conversion. For an explanation of all available parameters have a look at section "2.2. Model configuration" of [this manual](https://github.com/MarcA711/rknn-toolkit2/releases/download/v2.3.2/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.3.2_EN.pdf).
<<<<<<< HEAD
=======
## DeGirum
:::warning
The network-based detectors (Deepstack, DeGirum, and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, DeGirum ignores `zoo` and `token`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
:::
DeGirum is a detector that can use any type of hardware listed on [their website](https://hub.degirum.com). DeGirum can be used with local hardware through a DeGirum AI Server, or through the use of `@local`. You can also connect directly to DeGirum's AI Hub to run inferences. **Please Note:** This detector _cannot_ be used for commercial purposes.
### Configuration {#configuration-degirum}
#### AI Server Inference
Before starting with the config file for this section, you must first launch an AI server. DeGirum has an AI server ready to use as a docker container. Add this to your `docker-compose.yml` to get started:
```yaml
degirum_detector:
container_name: degirum
image: degirum/aiserver:latest
privileged: true
ports:
- "8778:8778"
```
All supported hardware will automatically be found on your AI server host as long as relevant runtimes and drivers are properly installed on your machine. Refer to [DeGirum's docs site](https://docs.degirum.com/pysdk/runtimes-and-drivers) if you have any trouble.
Once completed, configure the detector as follows:
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumAiServer.models} />
The model is set on the same `models` entry as the DeGirum device. You can set it to:
- A model listed on the [AI Hub](https://hub.degirum.com)
- If this is what you choose to do, the correct model will be downloaded onto your machine before running.
- A local directory acting as a zoo. See DeGirum's docs site [for more information](https://docs.degirum.com/pysdk/user-guide-pysdk/organizing-models#model-zoo-directory-structure).
- A path to some model.json.
```yaml
models:
- devices:
- degirum:<location>
path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1 # directory to model .json and file
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
#### Local Inference
It is also possible to eliminate the need for an AI server and run the hardware directly. The benefit of this approach is that you eliminate any bottlenecks that occur when transferring prediction results from the AI server docker container to the frigate one. However, the method of implementing local inference is different for every device and hardware combination, so it's usually more trouble than it's worth. A general guideline to achieve this would be:
1. Ensuring that the frigate docker container has the runtime you want to use. So for instance, running `@local` for Hailo means making sure the container you're using has the Hailo runtime installed.
2. To double check the runtime is detected by the DeGirum detector, make sure the `degirum sys-info` command properly shows whatever runtimes you mean to install.
3. Create a DeGirum detector in your configuration.
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumLocal.models} />
Once the DeGirum device is set up, you can choose a model on the same `models` entry in the `config.yml` file.
```yaml
models:
- devices:
- degirum:<location>
path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
#### AI Hub Cloud Inference
If you do not possess whatever hardware you want to run, there's also the option to run cloud inferences. Do note that your detection fps might need to be lowered as network latency does significantly slow down this method of detection. For use with Frigate, we highly recommend using a local AI server as described above. To set up cloud inferences,
1. Sign up at [DeGirum's AI Hub](https://hub.degirum.com).
2. Get an access token.
3. Create a DeGirum detector in your configuration.
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumCloud.models} />
Once the DeGirum device is set up, you can choose a model on the same `models` entry in the `config.yml` file.
```yaml
models:
- devices:
- degirum:<location>
path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
>>>>>>> 34363affa (Refactor detector and model management)
## AXERA
Hardware accelerated object detection is supported on the following SoCs:
docs/docs/guides/getting_started.md
+3 -3
View File
@@ -204,8 +204,8 @@ You need to refer to **Configure hardware acceleration** above to enable the con
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `OpenVINO` and **Device** `GPU`
2. On the same page, in the **Custom Model** tab, configure the model settings for OpenVINO:
1. Navigate to <NavPath path="Settings > System > Detection models" /> and select **Intel GPU** from the **Hardware** dropdown
2. On the same model, open the **Custom Model** tab and configure the model settings for OpenVINO:
| Field | Value |
| ---------------------------------------- | ------------------------------------------ |
@@ -270,7 +270,7 @@ services:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`.
Navigate to <NavPath path="Settings > System > Detection models" /> and select **Coral EdgeTPU (USB)** from the **Hardware** dropdown.
</TabItem>
<TabItem value="yaml">
docs/docs/integrations/plus.md
+1 -1
View File
@@ -59,7 +59,7 @@ You can view all of your submitted images at [https://plus.frigate.video](https:
Once you have [requested your first model](../plus/first_model.md) and gotten your own model ID, it can be used with a special model path. No other information needs to be configured for Frigate+ models because it fetches the remaining config from Frigate+ automatically.
You can either choose the new model from the <NavPath path="Settings > System > Detectors and model" /> pane in the Frigate UI (the **Frigate+ Model** tab), or manually set the model at the root level in your config:
You can either choose the new model from the <NavPath path="Settings > System > Detection models" /> pane in the Frigate UI (on the **Frigate+** tab of the model you want to change), or set it on that model in your config:
```yaml
models:
docs/docs/plus/first_model.md
+1 -1
View File
@@ -30,7 +30,7 @@ Models available in Frigate+ can be used with a special model path. No other inf
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" />. In the **Detection Model** section, choose the **Frigate+** tab. Select your new Frigate+ model from the **Available Frigate+ models** dropdown, then click **Save**. Restart Frigate to apply the change.
Navigate to <NavPath path="Settings > System > Detection models" />. On the model you want to change, choose the **Frigate+** tab and select your new Frigate+ model from the **Available Frigate+ models** dropdown, then click **Save**. Restart Frigate to apply the change.
</TabItem>
<TabItem value="yaml">
docs/docs/troubleshooting/common_errors.md
+1 -1
View File
@@ -131,7 +131,7 @@ The process was killed by the CPU for executing an unsupported instruction. Ther
<FaqItem id="onnx-invalidprotobuf" question="ONNX Runtime InvalidProtobuf / failed to load model">
ONNX Runtime could not parse the model file. The file exists but its contents are not a valid ONNX model, usually a corrupted or interrupted download in `model_cache`, or the wrong file pointed at by `model.path`. Delete the cached model file so Frigate re-downloads it, and confirm `model.path` points at an actual `.onnx` model. See [ONNX detector configuration](/configuration/object_detectors#onnx).
ONNX Runtime could not parse the model file. The file exists but its contents are not a valid ONNX model, usually a corrupted or interrupted download in `model_cache`, or the wrong file pointed at by a model's `path`. Delete the cached model file so Frigate re-downloads it, and confirm the model's `path` points at an actual `.onnx` model. See [ONNX detector configuration](/configuration/object_detectors#onnx).
</FaqItem>
docs/scripts/lib/nav_map.py
+1 -2
View File
@@ -63,8 +63,7 @@ SYSTEM_NAV: dict[str, tuple[str, str]] = {
"environment_vars": ("System", "Environment variables"),
"telemetry": ("System", "Telemetry"),
"birdseye": ("System", "Birdseye"),
"detectors": ("System", "Detectors and model"),
"model": ("System", "Detectors and model"),
"models": ("System", "Detection models"),
}
# All known top-level config section keys
docs/static/frigate-api.yaml
Vendored
+101
View File
@@ -4010,6 +4010,49 @@ paths:
security:
- frigateAdminAuth: []
x-required-role: admin
/hardware/probe:
get:
tags:
- Hardware
summary: Probe Hardware
description: |-
**Access:** Admin role required.
Get the object detection hardware attached to this system.
Args:
refresh: Probe again instead of returning the cached result
Returns:
Every kind of detection hardware that was found
operationId: probe_hardware_hardware_probe_get
parameters:
- name: refresh
in: query
required: false
schema:
type: boolean
default: false
title: Refresh
responses:
'200':
description: Successful Response
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/DetectionHardware'
title: Response Probe Hardware Hardware Probe Get
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- frigateAdminAuth: []
x-required-role: admin
/events:
get:
tags:
@@ -7842,6 +7885,46 @@ components:
required:
- ids
title: DeleteFaceImagesBody
DetectionHardware:
properties:
key:
type: string
title: Hardware key
description: Stable identifier for this kind of hardware.
detector:
type: string
title: Detector type
description: The detector that drives this hardware.
name:
type: string
title: Hardware name
description: Human readable name for this kind of hardware.
units:
items:
$ref: '#/components/schemas/HardwareUnit'
type: array
title: Units
description: Each physical piece of this hardware that was found.
count:
type: integer
title: Unit count
description: How many units were found.
unlimited:
type: boolean
title: Unlimited detectors
description: Whether this hardware can run more inference processes
than there are units.
type: object
required:
- key
- detector
- name
- units
- count
- unlimited
title: DetectionHardware
description: A kind of detection hardware, and every unit of it that was
found.
EventCreateResponse:
properties:
success:
@@ -8569,6 +8652,24 @@ components:
title: Detail
type: object
title: HTTPValidationError
HardwareUnit:
properties:
device:
type: string
title: Device string
description: The value to put in a model's devices list, for example
'edgetpu:pci:1'.
label:
type: string
title: Unit label
description: How to identify this unit among others of the same kind,
for example 'PCIe 1'.
type: object
required:
- device
- label
title: HardwareUnit
description: One physical piece of hardware.
Last24HoursReview:
properties:
reviewed_alert: