administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-10-22 22:52:48 +03:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Merge 8a40ea5c87 into 4ee12e6237

Browse Source
This commit is contained in:
Rami Mouro
2026-07-06 11:28:07 -06:00
committed by GitHub
parent 4ee12e6237 8a40ea5c87
commit 651414d063
14 changed files with 3764 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/data/object_detectors_models.json
+13
View File
@@ -265,6 +265,19 @@
}
]
},
"qualcomm": {
"title": "Qualcomm",
"models": [
{
"key": "ssd_mobiledet",
"label": "SSD MobileDet",
"recommended": true,
"download": "The default SSD MobileDet TFLite model (`/cpu_model.tflite`) bundled with the Frigate container is used automatically. This model is INT8 quantized and compatible with the QNN HTP backend.\n\nOnly `.tflite` models are supported with this detector. A custom model must be INT8 quantized for the Hexagon NPU.",
"ui": "Navigate to **Settings > System > Detectors and model** and select **Qualcomm** from the detector type dropdown and click **Add**. The bundled SSD MobileDet model is used automatically.\n\nTo use a custom model, configure in the **Custom Model** tab:\n\n| Field | Value |\n| ---------------------------------------- | ---------------------------------- |\n| **Custom object detector model path** | `/config/your_custom_model.tflite` |\n| **Object detection model input width** | `320` |\n| **Object detection model input height** | `320` |\n| **Label map for custom object detector** | `/labelmap/coco-80.txt` |",
"yaml": "detectors:\n qualcomm_npu:\n type: qualcomm_tfl"
}
]
},
"rknn": {
"title": "RKNN",
"models": [
docs/docs/configuration/object_detectors.md
+19
View File
@@ -53,6 +53,10 @@ Frigate supports multiple different detectors that work on different types of ha
- [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs.
**Qualcomm** <CommunityBadge />
- [Qualcomm](#qualcomm): TFLite models can run on Qualcomm SoCs with a Hexagon NPU (e.g. IQ9100, QCS6490) via the QNN TFLite delegate.
**Synaptics** <CommunityBadge />
- [Synaptics](#synaptics): synap models can run on Synaptics devices(e.g astra machina) with included NPUs.
@@ -652,6 +656,21 @@ When configuring the Synap detector, you have to specify the model: a local **pa
<ModelConfigDropdown detectorTitle="Synaptics" models={objectDetectorsModels.synaptics.models} />
## Qualcomm
Hardware accelerated object detection is supported on the following Qualcomm SoCs with a Hexagon NPU:
- IQ9100 / IQ-9075 EVK
- QCS6490 / RB3 Gen 2 Vision Kit / Rubik Pi 3
This implementation uses the QNN TFLite delegate (`libQnnTFLiteDelegate.so`) to accelerate TFLite model inference on the Qualcomm Hexagon Tensor Processor (HTP / NPU). The delegate library and device drivers are provided by the host operating system and made available to the container via [CDI (Container Device Interface)](https://docs.docker.com/build/building/cdi/).
See the [installation docs](../frigate/installation.md#qualcomm) for information on setting up CDI and configuring the hardware.
### Configuration
<ModelConfigDropdown detectorTitle="Qualcomm" models={objectDetectorsModels.qualcomm.models} />
## Rockchip platform
Hardware accelerated object detection is supported on the following SoCs:
docs/docs/frigate/hardware.md
+15
View File
@@ -107,6 +107,10 @@ Frigate supports multiple different detectors that work on different types of ha
- [AXEngine](#axera): axera models can run on AXERA NPUs via AXEngine, delivering highly efficient object detection.
**Qualcomm** <CommunityBadge />
- [Qualcomm](#qualcomm): TFLite models can run on Qualcomm SoCs with a Hexagon NPU (e.g. IQ9100, QCS6490) via the QNN TFLite delegate.
:::
### Hailo-8
@@ -295,6 +299,17 @@ The inference time of a rk3588 with all 3 cores enabled is typically 25-30 ms fo
| Name | AXERA AX650N/AX8850N Inference Time |
| ---------------- | ----------------------------------- |
| yolov9-tiny | ~ 4 ms |
### Qualcomm
Frigate supports hardware object detection on Qualcomm SoCs with a Hexagon NPU, including the IQ9100 and QCS6490 (RB3 Gen 2). The QNN TFLite delegate is used to accelerate inference on the HTP (Hexagon Tensor Processor).
A single detector is typically sufficient for multiple camera streams. The default model is **SSD MobileDet** (INT8 quantized).
| Name | IQ9100 Inference Time | QCS6490 Inference Time |
| ------------- | --------------------- | ---------------------- |
| ssd_mobiledet | ~ 0.8 ms | ~ 5.55 ms |
Detailed information is available [in the detector docs](/configuration/object_detectors#qualcomm).
## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version)
docs/docs/frigate/installation.md
+115
View File
@@ -472,6 +472,120 @@ If you are using `docker run`, add this option to your command `--device /dev/ax
#### Configuration
Finally, configure [hardware object detection](/configuration/object_detectors#axera) to complete the setup.
### Qualcomm
Hardware accelerated object detection is supported on the following Qualcomm SoCs:
| Board | SoC | CDI Config File |
| ----- | --- | --------------- |
| IQ-9075 EVK | IQ9100 | `cdi-hw-acc-9100.json` |
| RB3 Gen 2 Vision Kit / Rubik Pi 3 | QCS6490 | `cdi-hw-acc-6490.json` |
The Qualcomm integration uses [CDI (Container Device Interface)](https://docs.docker.com/build/building/cdi/) to provide the container with access to the NPU device nodes and the QNN delegate libraries from the host.
#### Prerequisites
- **QNN runtime on the host**: The QNN runtime libraries (including `libQnnTFLiteDelegate.so` and `libQnnHtp.so`), the Hexagon "skel" files, and the NPU device nodes (`/dev/fastrpc-cdsp`) must be present on the host; they are bind-mounted into the container via CDI. Both officially-supported images for these boards provide them:
- **Qualcomm Linux BSP** — ships them under the layout the bundled CDI descriptors target (`/usr/lib`, `/usr/lib/rfsa/adsp`).
- **Ubuntu for RB3 Gen 2 / Rubik Pi 3** — install the `qairt-libs` and `qairt-tools` packages from the Qualcomm IoT PPA. These board-select the Hexagon skels into a different directory (`/usr/lib/dsp/cdsp`), and `/usr/lib/rfsa/adsp` may contain self-referential symlink loops; `install_cdi.py` detects this and rewrites the affected mounts to the real files automatically (see below).
- **Docker 25.0+**: CDI device support requires Docker 25.0 or later. Check with `docker --version`.
- **arm64 architecture**: The Qualcomm Docker image is built for `linux/arm64` only. If building locally, you must run the build on the target board itself.
#### CDI Installation
1. Download or copy the CDI setup files from the [Frigate repository](https://github.com/blakeblackshear/frigate/tree/dev/docker/qualcomm/cdi).
2. Install the CDI configuration for your board:
```bash
# IQ9100 / IQ-9075 EVK
sudo python3 install_cdi.py --file cdi-hw-acc-9100.json
# QCS6490 / RB3 Gen 2 Vision Kit / Rubik Pi 3
sudo python3 install_cdi.py --file cdi-hw-acc-6490.json
```
This writes a CDI descriptor to `/etc/cdi/cdi-hw-acc.json`. For any mount whose default (BSP) path is absent or a broken symlink, the script first tries to resolve the file from the QAIRT layout (e.g. `/usr/lib/dsp/cdsp`) and rewrites the mount to the real file; paths it cannot resolve anywhere are skipped with a warning. If a **critical** QNN/HTP library is missing it prints a prominent warning — and, with `--strict`, exits non-zero — so a misconfigured host fails at install time instead of silently falling back to CPU at runtime.
3. Verify the descriptor was written and that it contains the QNN delegate and Hexagon skel mounts (their absence means the QNN runtime is not installed on the host):
```bash
ls -l /etc/cdi/cdi-hw-acc.json
grep -oE 'libQnn(TFLiteDelegate|HtpV[0-9]+Skel|System)\.so' /etc/cdi/cdi-hw-acc.json | sort -u
```
#### Setup
Follow Frigate's default installation instructions, but use a docker image with `-qualcomm` suffix, for example `ghcr.io/blakeblackshear/frigate:stable-qualcomm`.
:::note
The pre-built `stable-qualcomm` image is not yet published. Until it is available, you must build the image locally:
```bash
git clone https://github.com/blakeblackshear/frigate.git
cd frigate
make local-qualcomm
```
This builds `frigate:latest-qualcomm` on your device. Use `frigate:latest-qualcomm` as the image name in the examples below instead of the `ghcr.io` URL.
:::
Grant Docker access to your Qualcomm hardware by passing the CDI device. In your `docker-compose.yml`:
```yaml
services:
frigate:
container_name: frigate
restart: unless-stopped
image: ghcr.io/blakeblackshear/frigate:stable-qualcomm
devices:
- qualcomm.com/device=cdi-hw-acc
volumes:
- /etc/localtime:/etc/localtime:ro
- /path/to/your/config:/config
- /path/to/your/storage:/media/frigate
- type: tmpfs
target: /tmp/cache
tmpfs:
size: 1000000000
ports:
- "8971:8971"
- "8554:8554"
- "8555:8555/tcp"
- "8555:8555/udp"
```
If using `docker run`, pass the CDI device with:
```bash
docker run -d \
--name frigate \
--restart=unless-stopped \
--device qualcomm.com/device=cdi-hw-acc \
--mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \
--shm-size=256m \
-v /path/to/your/storage:/media/frigate \
-v /path/to/your/config:/config \
-v /etc/localtime:/etc/localtime:ro \
-e FRIGATE_RTSP_PASSWORD='password' \
-p 8971:8971 \
-p 8554:8554 \
-p 8555:8555/tcp \
-p 8555:8555/udp \
ghcr.io/blakeblackshear/frigate:stable-qualcomm
```
:::tip
Unlike other hardware integrations that pass individual `/dev/` device nodes, the Qualcomm CDI approach bundles all required device nodes, library bind mounts, and environment variables into a single `--device qualcomm.com/device=cdi-hw-acc` flag.
:::
#### Configuration
Next, you should configure [hardware object detection](/configuration/object_detectors#qualcomm).
## Docker
@@ -558,6 +672,7 @@ The community supported docker image tags for the current stable version are:
- `stable-tensorrt-jp6` - Frigate build optimized for Nvidia Jetson devices running Jetpack 6
- `stable-rk` - Frigate build for SBCs with Rockchip SoC
- `stable-qualcomm` - Frigate build for Qualcomm SoCs with Hexagon NPU (IQ9100, QCS6490)
## Home Assistant App