mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-08-02 17:12:16 +03:00
[Init] Initial commit for Synaptics SL1680 NPU (#19680)
* [Init] Initial commit for Synaptics SL1680 NPU * add a rough detector which is testing with yolov8 tflite model. * [Feat] Add dependencies installation in docker build - Add runtime library and wheels installation in main/Dockerfile - Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680 * [Update] Remove dependencies installation from main Dockerfile - remove deps installation from Dockerfile - add dependencies installation and split wheels, deps stage in synap1680 Dockerfile * Refactor synap detector to more closely match other implementations * [Update] Add model path configuration check * [Update] update ModelType to ssd * [Update] Remove unuse script - install_deps.sh has already been executing in deps download stage - Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain * [Update] update Synaptics SL1680 setup description * [Update] remove install_synap1680 - The deps download and installation is existed in synap1680 * [Fix] update document content * [Update] Update detector from synap1680 to synaptics This update is in order to make the synaptics SL-series NPU detector more general. - Fix detector `os` module not import bug - Update detector type `synap1680` to `synaptics` - Update document description `SL1680` to `Synaptics` only - Update docker build content `synap1680` to `synaptics` * [Fix] Update configuration document * Update docs/docs/configuration/object_detectors.md Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> * [Update] Update document content and detector default layout - Update object_detectors document - Update detector's default layout - Update default model name * [Update] Update object detector document content * [Fix] Fix InputTensorEnum not defined error - import InputTensorEnum from detector_config * [Update] Update detector script coding format * [Update] Update synaptics detector coding format * [Update] Add synaptics ci workflow * [Update] update synaptics runtime libs download path - Fork Synaptics astra sdk repo and put the runtime lib package on it - Frigate team can update this download path later --------- Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
This commit is contained in:
co-authored by
Nicolas Mowen
parent
082867447b
commit
b8b07ee6e1
@@ -427,3 +427,29 @@ cameras:
|
||||
```
|
||||
|
||||
:::
|
||||
|
||||
## Synaptics
|
||||
|
||||
Hardware accelerated video de-/encoding is supported on Synpatics SL-series SoC.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Make sure to follow the [Synaptics specific installation instructions](/frigate/installation#synaptics).
|
||||
|
||||
### Configuration
|
||||
|
||||
Add one of the following FFmpeg presets to your `config.yml` to enable hardware video processing:
|
||||
|
||||
```yaml
|
||||
ffmpeg:
|
||||
hwaccel_args: -c:v h264_v4l2m2m
|
||||
input_args: preset-rtsp-restream
|
||||
output_args:
|
||||
record: preset-record-generic-audio-aac
|
||||
```
|
||||
|
||||
:::warning
|
||||
|
||||
Make sure that your SoC supports hardware acceleration for your input stream and your input stream is h264 encoding. For example, if your camera streams with h264 encoding, your SoC must be able to de- and encode with it. If you are unsure whether your SoC meets the requirements, take a look at the datasheet.
|
||||
|
||||
:::
|
||||
|
||||
@@ -43,6 +43,10 @@ Frigate supports multiple different detectors that work on different types of ha
|
||||
|
||||
- [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs.
|
||||
|
||||
**Synaptics**
|
||||
|
||||
- [Synaptics](#synaptics): synap models can run on Synaptics devices(e.g astra machina) with included NPUs.
|
||||
|
||||
**For Testing**
|
||||
|
||||
- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results.
|
||||
@@ -1049,6 +1053,41 @@ model:
|
||||
height: 320 # MUST match the chosen model i.e yolov7-320 -> 320 yolov4-416 -> 416
|
||||
```
|
||||
|
||||
## Synaptics
|
||||
|
||||
Hardware accelerated object detection is supported on the following SoCs:
|
||||
|
||||
- SL1680
|
||||
|
||||
This implementation uses the [Synaptics model conversion](https://synaptics-synap.github.io/doc/v/latest/docs/manual/introduction.html#offline-model-conversion), version v3.1.0.
|
||||
|
||||
This implementation is based on sdk `v1.5.0`.
|
||||
|
||||
See the [installation docs](../frigate/installation.md#synaptics) for information on configuring the SL-series NPU hardware.
|
||||
|
||||
### Configuration
|
||||
|
||||
When configuring the Synap detector, you have to specify the model: a local **path**.
|
||||
|
||||
#### SSD Mobilenet
|
||||
|
||||
A synap model is provided in the container at /mobilenet.synap and is used by this detector type by default. The model comes from [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
|
||||
|
||||
Use the model configuration shown below when using the synaptics detector with the default synap model:
|
||||
|
||||
```yaml
|
||||
detectors: # required
|
||||
synap_npu: # required
|
||||
type: synaptics # required
|
||||
|
||||
model: # required
|
||||
path: /synaptics/mobilenet.synap # required
|
||||
width: 224 # required
|
||||
height: 224 # required
|
||||
tensor_format: nhwc # default value (optional. If you change the model, it is required)
|
||||
labelmap_path: /labelmap/coco-80.txt # required
|
||||
```
|
||||
|
||||
## Rockchip platform
|
||||
|
||||
Hardware accelerated object detection is supported on the following SoCs:
|
||||
|
||||
Reference in New Issue
Block a user