mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-07-30 15:49:00 +03:00
Hailo Official integration (#16906)
* Adding Models * Final Async Update * Bug Fixing * Fix * Adding fixes * Working async infer * Final Documenatation and debug update * Removing some extra prints * Post-process correct label push * config docs fix * Review Fix * Review fix 2.0 * Fixing the ASYNC API to work from 30ms to 10ms * Fix for multi stream async infernce * Format * Fix #3 * Format#2 * Remove Unnessery includes * Sort Imports
This commit is contained in:
OmriAx
@@ -12,7 +12,7 @@ Frigate supports multiple different detectors that work on different types of ha
|
||||
**Most Hardware**
|
||||
|
||||
- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB and m.2 format allowing for a wide range of compatibility with devices.
|
||||
- [Hailo](#hailo-8l): The Hailo8 AI Acceleration module is available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
|
||||
- [Hailo](#hailo-8): The Hailo8 and Hailo8L AI Acceleration module is available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
|
||||
|
||||
**AMD**
|
||||
|
||||
@@ -129,15 +129,58 @@ detectors:
|
||||
type: edgetpu
|
||||
device: pci
|
||||
```
|
||||
---
|
||||
|
||||
## Hailo-8l
|
||||
|
||||
This detector is available for use with Hailo-8 AI Acceleration Module.
|
||||
## Hailo-8
|
||||
|
||||
See the [installation docs](../frigate/installation.md#hailo-8l) for information on configuring the hailo8.
|
||||
This detector is available for use with both Hailo-8 and Hailo-8L AI Acceleration Modules. The integration automatically detects your hardware architecture via the Hailo CLI and selects the appropriate default model if no custom model is specified.
|
||||
|
||||
See the [installation docs](../frigate/installation.md#hailo-8l) for information on configuring the Hailo hardware.
|
||||
|
||||
### Configuration
|
||||
|
||||
When configuring the Hailo detector, you have two options to specify the model: a local **path** or a **URL**.
|
||||
If both are provided, the detector will first check for the model at the given local path. If the file is not found, it will download the model from the specified URL. The model file is cached under `/config/model_cache/hailo`.
|
||||
|
||||
#### YOLO
|
||||
|
||||
Use this configuration for YOLO-based models. When no custom model path or URL is provided, the detector automatically downloads the default model based on the detected hardware:
|
||||
- **Hailo-8 hardware:** Uses **YOLOv6n** (default: `yolov6n.hef`)
|
||||
- **Hailo-8L hardware:** Uses **YOLOv6n** (default: `yolov6n.hef`)
|
||||
|
||||
```yaml
|
||||
detectors:
|
||||
hailo8l:
|
||||
type: hailo8l
|
||||
device: PCIe
|
||||
|
||||
model:
|
||||
width: 320
|
||||
height: 320
|
||||
input_tensor: nhwc
|
||||
input_pixel_format: rgb
|
||||
input_dtype: int
|
||||
model_type: yolo-generic
|
||||
|
||||
# The detector automatically selects the default model based on your hardware:
|
||||
# - For Hailo-8 hardware: YOLOv6n (default: yolov6n.hef)
|
||||
# - For Hailo-8L hardware: YOLOv6n (default: yolov6n.hef)
|
||||
#
|
||||
# Optionally, you can specify a local model path to override the default.
|
||||
# If a local path is provided and the file exists, it will be used instead of downloading.
|
||||
# Example:
|
||||
# path: /config/model_cache/hailo/yolov6n.hef
|
||||
#
|
||||
# You can also override using a custom URL:
|
||||
# path: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ModelZoo/Compiled/v2.14.0/hailo8/yolov6n.hef
|
||||
# just make sure to give it the write configuration based on the model
|
||||
```
|
||||
|
||||
#### SSD
|
||||
|
||||
For SSD-based models, provide either a model path or URL to your compiled SSD model. The integration will first check the local path before downloading if necessary.
|
||||
|
||||
```yaml
|
||||
detectors:
|
||||
hailo8l:
|
||||
@@ -148,11 +191,50 @@ model:
|
||||
width: 300
|
||||
height: 300
|
||||
input_tensor: nhwc
|
||||
input_pixel_format: bgr
|
||||
input_pixel_format: rgb
|
||||
model_type: ssd
|
||||
path: /config/model_cache/h8l_cache/ssd_mobilenet_v1.hef
|
||||
# Specify the local model path (if available) or URL for SSD MobileNet v1.
|
||||
# Example with a local path:
|
||||
# path: /config/model_cache/h8l_cache/ssd_mobilenet_v1.hef
|
||||
#
|
||||
# Or override using a custom URL:
|
||||
# path: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ModelZoo/Compiled/v2.14.0/hailo8l/ssd_mobilenet_v1.hef
|
||||
```
|
||||
|
||||
#### Custom Models
|
||||
|
||||
The Hailo detector supports all YOLO models compiled for Hailo hardware that include post-processing. You can specify a custom URL or a local path to download or use your model directly. If both are provided, the detector checks the local path first.
|
||||
|
||||
```yaml
|
||||
detectors:
|
||||
hailo8l:
|
||||
type: hailo8l
|
||||
device: PCIe
|
||||
|
||||
model:
|
||||
width: 640
|
||||
height: 640
|
||||
input_tensor: nhwc
|
||||
input_pixel_format: rgb
|
||||
input_dtype: int
|
||||
model_type: yolo-generic
|
||||
# Optional: Specify a local model path.
|
||||
# path: /config/model_cache/hailo/custom_model.hef
|
||||
#
|
||||
# Alternatively, or as a fallback, provide a custom URL:
|
||||
# path: https://custom-model-url.com/path/to/model.hef
|
||||
```
|
||||
For additional ready-to-use models, please visit: https://github.com/hailo-ai/hailo_model_zoo
|
||||
|
||||
Hailo8 supports all models in the Hailo Model Zoo that include HailoRT post-processing. You're welcome to choose any of these pre-configured models for your implementation.
|
||||
|
||||
> **Note:**
|
||||
> The config.path parameter can accept either a local file path or a URL ending with .hef. When provided, the detector will first check if the path is a local file path. If the file exists locally, it will use it directly. If the file is not found locally or if a URL was provided, it will attempt to download the model from the specified URL.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## OpenVINO Detector
|
||||
|
||||
The OpenVINO detector type runs an OpenVINO IR model on AMD and Intel CPUs, Intel GPUs and Intel VPU hardware. To configure an OpenVINO detector, set the `"type"` attribute to `"openvino"`.
|
||||
|
||||
@@ -92,11 +92,22 @@ Inference speeds will vary greatly depending on the GPU and the model used.
|
||||
|
||||
With the [rocm](../configuration/object_detectors.md#amdrocm-gpu-detector) detector Frigate can take advantage of many discrete AMD GPUs.
|
||||
|
||||
### Hailo-8l PCIe
|
||||
### Hailo-8
|
||||
|
||||
Frigate supports the Hailo-8l M.2 card on any hardware but currently it is only tested on the Raspberry Pi5 PCIe hat from the AI kit.
|
||||
| Name | Hailo‑8 Inference Time | Hailo‑8L Inference Time |
|
||||
| --------------- | ---------------------- | ----------------------- |
|
||||
| ssd mobilenet v1| ~ 6 ms | ~ 10 ms |
|
||||
| yolov6n | ~ 7 ms | ~ 11 ms |
|
||||
|
||||
|
||||
Frigate supports both the Hailo-8 and Hailo-8L AI Acceleration Modules on compatible hardware platforms—including the Raspberry Pi 5 with the PCIe hat from the AI kit. The Hailo detector integration in Frigate automatically identifies your hardware type and selects the appropriate default model when a custom model isn’t provided.
|
||||
|
||||
**Default Model Configuration:**
|
||||
- **Hailo-8L:** Default model is **YOLOv6n**.
|
||||
- **Hailo-8:** Default model is **YOLOv6n**.
|
||||
|
||||
In real-world deployments, even with multiple cameras running concurrently, Frigate has demonstrated consistent performance. Testing on x86 platforms—with dual PCIe lanes—yields further improvements in FPS, throughput, and latency compared to the Raspberry Pi setup.
|
||||
|
||||
The inference time for the Hailo-8L chip at time of writing is around 17-21 ms for the SSD MobileNet Version 1 model.
|
||||
|
||||
## Community Supported Detectors
|
||||
|
||||
|
||||
@@ -100,9 +100,9 @@ By default, the Raspberry Pi limits the amount of memory available to the GPU. I
|
||||
|
||||
Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with it's own power supply. Some have reported success with <a href="https://amzn.to/3a2mH0P" target="_blank" rel="nofollow noopener sponsored">this</a> (affiliate link).
|
||||
|
||||
### Hailo-8L
|
||||
### Hailo-8
|
||||
|
||||
The Hailo-8L is an M.2 card typically connected to a carrier board for PCIe, which then connects to the Raspberry Pi 5 as part of the AI Kit. However, it can also be used on other boards equipped with an M.2 M key edge connector.
|
||||
The Hailo-8 and Hailo-8L AI accelerators are available in both M.2 and HAT form factors for the Raspberry Pi. The M.2 version typically connects to a carrier board for PCIe, which then interfaces with the Raspberry Pi 5 as part of the AI Kit. The HAT version can be mounted directly onto compatible Raspberry Pi models. Both form factors have been successfully tested on x86 platforms as well, making them versatile options for various computing environments.
|
||||
|
||||
#### Installation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user