administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2025-12-06 21:44:13 +03:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Improve Intel Model (#20541)

Browse Source 
* Update supported models and inference times

* Fix d-fine inputs

* Improve d-fine
This commit is contained in:
Nicolas Mowen 2025-10-17 06:31:28 -06:00 committed by GitHub
parent d7275a3c1a
commit 4228861810
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 11 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/docs/configuration/object_detectors.md
View File

@ -281,7 +281,7 @@ detectors:
| ------------------------------------- | --- | --- | ------------------------------------------------------------ | | ------------------------------------- | --- | --- | ------------------------------------------------------------ |
| [YOLOv9](#yolo-v3-v4-v7-v9) | ✅ | ✅ | Recommended for GPU & NPU | | [YOLOv9](#yolo-v3-v4-v7-v9) | ✅ | ✅ | Recommended for GPU & NPU |
| [RF-DETR](#rf-detr) | ✅ | ✅ | Requires XE iGPU or Arc | | [RF-DETR](#rf-detr) | ✅ | ✅ | Requires XE iGPU or Arc |
| [YOLO-NAS](#yolo-nas) | ✅ | ⚠️ | YOLO-NAS only works on NPU in non-flat format | | [YOLO-NAS](#yolo-nas) | ✅ | ✅ | |
| [MobileNet v2](#ssdlite-mobilenet-v2) | ✅ | ✅ | Fast and lightweight model, less accurate than larger models | | [MobileNet v2](#ssdlite-mobilenet-v2) | ✅ | ✅ | Fast and lightweight model, less accurate than larger models |
| [YOLOX](#yolox) | ✅ | ? | | | [YOLOX](#yolox) | ✅ | ? | |
| [D-FINE](#d-fine) | ❌ | ❌ | | | [D-FINE](#d-fine) | ❌ | ❌ | |
@ -429,7 +429,7 @@ Currently D-FINE models only run on OpenVINO in CPU mode, GPUs currently fail to
::: :::
<details> <details>
<summary>YOLO-NAS Setup & Config</summary> <summary>D-FINE Setup & Config</summary>
After placing the downloaded onnx model in your config/model_cache folder, you can use the following configuration: After placing the downloaded onnx model in your config/model_cache folder, you can use the following configuration:
@ -445,7 +445,7 @@ model:
height: 640 height: 640
input_tensor: nchw input_tensor: nchw
input_dtype: float input_dtype: float
path: /config/model_cache/dfine_s_obj2coco.onnx path: /config/model_cache/dfine-s.onnx
labelmap_path: /labelmap/coco-80.txt labelmap_path: /labelmap/coco-80.txt
``` ```

4
docs/docs/frigate/hardware.md
View File

@ -167,8 +167,8 @@ Inference speeds vary greatly depending on the CPU or GPU used, some known examp
| Intel UHD 770 | ~ 15 ms | t-320: ~ 16 ms s-320: ~ 20 ms s-640: ~ 40 ms | 320: ~ 20 ms 640: ~ 46 ms | | | | Intel UHD 770 | ~ 15 ms | t-320: ~ 16 ms s-320: ~ 20 ms s-640: ~ 40 ms | 320: ~ 20 ms 640: ~ 46 ms | | |
| Intel N100 | ~ 15 ms | s-320: 30 ms | 320: ~ 25 ms | | Can only run one detector instance | | Intel N100 | ~ 15 ms | s-320: 30 ms | 320: ~ 25 ms | | Can only run one detector instance |
| Intel N150 | ~ 15 ms | t-320: 16 ms s-320: 24 ms | | | | | Intel N150 | ~ 15 ms | t-320: 16 ms s-320: 24 ms | | | |
| Intel Iris XE | ~ 10 ms | s-320: 8 ms s-640: 30 ms | 320: ~ 18 ms 640: ~ 50 ms | 320-n: 33 ms | | | Intel Iris XE | ~ 10 ms | t-320: 6 ms t-640: 14 ms s-320: 8 ms s-640: 16 ms | 320: ~ 10 ms 640: ~ 20 ms | 320-n: 33 ms | |
| Intel NPU | ~ 6 ms | s-320: 11 ms | | 320-n: 40 ms | | | Intel NPU | ~ 6 ms | s-320: 11 ms | 320: ~ 14 ms 640: ~ 34 ms | 320-n: 40 ms | |
| Intel Arc A310 | ~ 5 ms | t-320: 7 ms t-640: 11 ms s-320: 8 ms s-640: 15 ms | 320: ~ 8 ms 640: ~ 14 ms | | | | Intel Arc A310 | ~ 5 ms | t-320: 7 ms t-640: 11 ms s-320: 8 ms s-640: 15 ms | 320: ~ 8 ms 640: ~ 14 ms | | |
| Intel Arc A380 | ~ 6 ms | | 320: ~ 10 ms 640: ~ 22 ms | 336: 20 ms 448: 27 ms | | | Intel Arc A380 | ~ 6 ms | | 320: ~ 10 ms 640: ~ 22 ms | 336: 20 ms 448: 27 ms | |
| Intel Arc A750 | ~ 4 ms | | 320: ~ 8 ms | | | | Intel Arc A750 | ~ 4 ms | | 320: ~ 8 ms | | |

10
frigate/detectors/detection_runners.py
View File

@ -312,11 +312,13 @@ class OpenVINOModelRunner(BaseModelRunner):
# Multiple inputs case - set each input by name # Multiple inputs case - set each input by name
for input_name, input_data in inputs.items(): for input_name, input_data in inputs.items():
# Find the input by name # Find the input by name and its index
input_port = None input_port = None
for port in self.compiled_model.inputs: input_index = None
for idx, port in enumerate(self.compiled_model.inputs):
if port.get_any_name() == input_name: if port.get_any_name() == input_name:
input_port = port input_port = port
input_index = idx
break break
if input_port is None: if input_port is None:
@ -327,8 +329,8 @@ class OpenVINOModelRunner(BaseModelRunner):
input_tensor = ov.Tensor(input_element_type, input_data.shape) input_tensor = ov.Tensor(input_element_type, input_data.shape)
np.copyto(input_tensor.data, input_data) np.copyto(input_tensor.data, input_data)
# Set the input tensor # Set the input tensor for the specific port index
self.infer_request.set_input_tensor(input_tensor) self.infer_request.set_input_tensor(input_index, input_tensor)
# Run inference # Run inference
self.infer_request.infer() self.infer_request.infer()