Feat/deepx npu detector (#24336)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s

* feat(deepx): add DEEPX NPU detector and runtime integration.

* feat(deepx): enforce model_format requirement when ppu is enabled and add integrity checks for driver installation

* Update frigate/detectors/plugins/deepx.py

Public method lacks docstring

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Refactor DEEPX detector tests, support SSD and DAMO-YOLO

* feat(deepx): add anchor-free output decoding and corresponding tests

* Add tests and updates for DEEPX detector and refactor DEEPX accelerator code structure.

* fix: enhance model type validation and update documentation for DEEPX detector

* fix: add support for customizable score and NMS thresholds

* refactor: infer YOLO layout from the model, drop per-detector options and the dxrtd placeholder

* fix: keep only the anchor-free PPU verdict, re-read anchor-based each frame

* Update latency data for DEEPX NPU

* Expanding PPU support for DEEPX and set yolo-generic as default.

* enhance scale count resolution logic

* Extend PPU layout handling and YOLOX support to DEEPX detector

* fix: assume the largest PPU anchor table when the .dxnn has no layout

* Improve PPU decoding and introduce strides handling

* Improve PPU scope and fix box format mismatch

* Fix unnamed node issue that breaks traversal

* fix: update object detection model type description to remove outdated architecture

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This commit is contained in:
A. Ahmet
2026-09-21 07:50:44 -05:00
committed by GitHub
co-authored by greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
parent 52f50a7396
commit af0ba19196
11 changed files with 2467 additions and 0 deletions
+59
View File
@@ -967,6 +967,65 @@ memryx:
# The .zip file must contain:
# ├── ssdlite_mobilenet.dfp (a file ending with .dfp)
# └── ssdlite_mobilenet_post.onnx (optional; only if the model includes a cropped post-processing network)
deepx:
title: DEEPX NPU
models:
- key: yolo
label: YOLO
recommended: true
download: No model is bundled with Frigate. Download a pre-compiled YOLO `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo) or compile your own with DX-COM, then bind-mount it into the container and point the model's `path` at it. The recommended model is `yolox-s_640x640_ppu.dxnn`. Its Post-Processing Unit (PPU) compile moves candidate selection onto the NPU, which makes it the fastest ModelZoo model measured through Frigate (about 13 ms on a DX-M1). The output layout is read from the compiled model, so anchor-based, anchor-free, NMS-in-head and PPU models (anchor-based or anchor-free) all need no extra configuration; prefer a `PPU` variant whenever the ModelZoo offers one. PPU models must be compiled with DX-COM 2.4.0 or later, which writes the head layout Frigate reads into the file.
ui: |-
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------------------------- |
| **Custom object detector model path** | `/config/model_cache/deepx/yolox-s_640x640_ppu.dxnn` |
| **Label map for custom object detector** | `/labelmap/coco-80.txt` |
| **Object detection model input width** | `640` |
| **Object detection model input height** | `640` |
| **Model Input Pixel Color Format** | `rgb` (Frigate's default value) |
| **Model Input Tensor Shape** | `nhwc` (Frigate's default value) |
| **Model Input D Type** | `int` (Frigate's default value) |
| **Object Detection Model Type** | `yolo-generic` |
Quantization is baked into the compiled model, so no normalization is applied on the host and the input defaults do not need to be overridden.
yaml: |-
models:
- devices:
- deepx:PCIe:0
path: /config/model_cache/deepx/yolox-s_640x640_ppu.dxnn
labelmap_path: /labelmap/coco-80.txt
model_type: yolo-generic
width: 640
height: 640
- key: yolox
label: YOLOX
recommended: false
download: No model is bundled with Frigate. Download a pre-compiled YOLOX `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo), then bind-mount it into the container and point the model's `path` at it. The `_ppu` variant is faster and also works with the `yolo-generic` model type; the plain export needs `yolox` so its raw head is decoded.
ui: |-
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------------- |
| **Custom object detector model path** | `/config/model_cache/deepx/yolox-s_640x640.dxnn`|
| **Label map for custom object detector** | `/labelmap/coco-80.txt` |
| **Object detection model input width** | `640` |
| **Object detection model input height** | `640` |
| **Model Input Pixel Color Format** | `rgb` (Frigate's default value) |
| **Model Input Tensor Shape** | `nhwc` (Frigate's default value) |
| **Model Input D Type** | `int` (Frigate's default value) |
| **Object Detection Model Type** | `yolox` |
The width and height must match the resolution the `.dxnn` file was compiled for.
yaml: |-
models:
- devices:
- deepx:PCIe:0
path: /config/model_cache/deepx/yolox-s_640x640.dxnn
labelmap_path: /labelmap/coco-80.txt
model_type: yolox
width: 640
height: 640
tensorrt:
title: TensorRT
models: