mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-09-28 02:08:57 +03:00
Feat/deepx npu detector (#24336)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* feat(deepx): add DEEPX NPU detector and runtime integration. * feat(deepx): enforce model_format requirement when ppu is enabled and add integrity checks for driver installation * Update frigate/detectors/plugins/deepx.py Public method lacks docstring Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Refactor DEEPX detector tests, support SSD and DAMO-YOLO * feat(deepx): add anchor-free output decoding and corresponding tests * Add tests and updates for DEEPX detector and refactor DEEPX accelerator code structure. * fix: enhance model type validation and update documentation for DEEPX detector * fix: add support for customizable score and NMS thresholds * refactor: infer YOLO layout from the model, drop per-detector options and the dxrtd placeholder * fix: keep only the anchor-free PPU verdict, re-read anchor-based each frame * Update latency data for DEEPX NPU * Expanding PPU support for DEEPX and set yolo-generic as default. * enhance scale count resolution logic * Extend PPU layout handling and YOLOX support to DEEPX detector * fix: assume the largest PPU anchor table when the .dxnn has no layout * Improve PPU decoding and introduce strides handling * Improve PPU scope and fix box format mismatch * Fix unnamed node issue that breaks traversal * fix: update object detection model type description to remove outdated architecture --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
parent
52f50a7396
commit
af0ba19196
@@ -967,6 +967,65 @@ memryx:
|
||||
# The .zip file must contain:
|
||||
# ├── ssdlite_mobilenet.dfp (a file ending with .dfp)
|
||||
# └── ssdlite_mobilenet_post.onnx (optional; only if the model includes a cropped post-processing network)
|
||||
deepx:
|
||||
title: DEEPX NPU
|
||||
models:
|
||||
- key: yolo
|
||||
label: YOLO
|
||||
recommended: true
|
||||
download: No model is bundled with Frigate. Download a pre-compiled YOLO `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo) or compile your own with DX-COM, then bind-mount it into the container and point the model's `path` at it. The recommended model is `yolox-s_640x640_ppu.dxnn`. Its Post-Processing Unit (PPU) compile moves candidate selection onto the NPU, which makes it the fastest ModelZoo model measured through Frigate (about 13 ms on a DX-M1). The output layout is read from the compiled model, so anchor-based, anchor-free, NMS-in-head and PPU models (anchor-based or anchor-free) all need no extra configuration; prefer a `PPU` variant whenever the ModelZoo offers one. PPU models must be compiled with DX-COM 2.4.0 or later, which writes the head layout Frigate reads into the file.
|
||||
ui: |-
|
||||
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
|
||||
|
||||
| Field | Value |
|
||||
| ---------------------------------------- | ---------------------------------------------------- |
|
||||
| **Custom object detector model path** | `/config/model_cache/deepx/yolox-s_640x640_ppu.dxnn` |
|
||||
| **Label map for custom object detector** | `/labelmap/coco-80.txt` |
|
||||
| **Object detection model input width** | `640` |
|
||||
| **Object detection model input height** | `640` |
|
||||
| **Model Input Pixel Color Format** | `rgb` (Frigate's default value) |
|
||||
| **Model Input Tensor Shape** | `nhwc` (Frigate's default value) |
|
||||
| **Model Input D Type** | `int` (Frigate's default value) |
|
||||
| **Object Detection Model Type** | `yolo-generic` |
|
||||
|
||||
Quantization is baked into the compiled model, so no normalization is applied on the host and the input defaults do not need to be overridden.
|
||||
yaml: |-
|
||||
models:
|
||||
- devices:
|
||||
- deepx:PCIe:0
|
||||
path: /config/model_cache/deepx/yolox-s_640x640_ppu.dxnn
|
||||
labelmap_path: /labelmap/coco-80.txt
|
||||
model_type: yolo-generic
|
||||
width: 640
|
||||
height: 640
|
||||
- key: yolox
|
||||
label: YOLOX
|
||||
recommended: false
|
||||
download: No model is bundled with Frigate. Download a pre-compiled YOLOX `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo), then bind-mount it into the container and point the model's `path` at it. The `_ppu` variant is faster and also works with the `yolo-generic` model type; the plain export needs `yolox` so its raw head is decoded.
|
||||
ui: |-
|
||||
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
|
||||
|
||||
| Field | Value |
|
||||
| ---------------------------------------- | ---------------------------------------- |
|
||||
| **Custom object detector model path** | `/config/model_cache/deepx/yolox-s_640x640.dxnn`|
|
||||
| **Label map for custom object detector** | `/labelmap/coco-80.txt` |
|
||||
| **Object detection model input width** | `640` |
|
||||
| **Object detection model input height** | `640` |
|
||||
| **Model Input Pixel Color Format** | `rgb` (Frigate's default value) |
|
||||
| **Model Input Tensor Shape** | `nhwc` (Frigate's default value) |
|
||||
| **Model Input D Type** | `int` (Frigate's default value) |
|
||||
| **Object Detection Model Type** | `yolox` |
|
||||
|
||||
The width and height must match the resolution the `.dxnn` file was compiled for.
|
||||
yaml: |-
|
||||
models:
|
||||
- devices:
|
||||
- deepx:PCIe:0
|
||||
path: /config/model_cache/deepx/yolox-s_640x640.dxnn
|
||||
labelmap_path: /labelmap/coco-80.txt
|
||||
model_type: yolox
|
||||
width: 640
|
||||
height: 640
|
||||
tensorrt:
|
||||
title: TensorRT
|
||||
models:
|
||||
|
||||
Reference in New Issue
Block a user