[Init] Initial commit for Synaptics SL1680 NPU (#19680)

* [Init] Initial commit for Synaptics SL1680 NPU

* add a rough detector which is testing with yolov8 tflite model.

* [Feat] Add dependencies installation in docker build

- Add runtime library and wheels installation in main/Dockerfile
- Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680

* [Update] Remove dependencies installation from main Dockerfile

- remove deps installation from Dockerfile
- add dependencies installation and split wheels, deps stage in synap1680 Dockerfile

* Refactor synap detector to more closely match other implementations

* [Update] Add model path configuration check

* [Update] update ModelType to ssd

* [Update] Remove unuse script

- install_deps.sh has already been executing in deps download stage
- Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain

* [Update] update Synaptics SL1680 setup description

* [Update] remove install_synap1680

- The deps download and installation is existed in synap1680

* [Fix] update document content

* [Update] Update detector from synap1680 to synaptics

This update is in order to make the synaptics SL-series NPU detector more general.

- Fix detector `os` module not import bug
- Update detector type `synap1680` to `synaptics`
- Update document description `SL1680` to `Synaptics` only
- Update docker build content `synap1680` to `synaptics`

* [Fix] Update configuration document

* Update docs/docs/configuration/object_detectors.md

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* [Update] Update document content and detector default layout

- Update object_detectors document
- Update detector's default layout
- Update default model name

* [Update] Update object detector document content

* [Fix] Fix InputTensorEnum not defined error

- import InputTensorEnum from detector_config

* [Update] Update detector script coding format

* [Update] Update synaptics detector coding format

* [Update] Add synaptics ci workflow

* [Update] update synaptics runtime libs download path

- Fork Synaptics astra sdk repo and put the runtime lib package on it
- Frigate team can update this download path later

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
This commit is contained in:
GaryHuang-ASUS
2025-09-26 07:07:12 -05:00
committed by GitHub
co-authored by Nicolas Mowen
parent 082867447b
commit b8b07ee6e1
9 changed files with 295 additions and 0 deletions
@@ -43,6 +43,10 @@ Frigate supports multiple different detectors that work on different types of ha
- [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs.
**Synaptics**
- [Synaptics](#synaptics): synap models can run on Synaptics devices(e.g astra machina) with included NPUs.
**For Testing**
- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results.
@@ -1049,6 +1053,41 @@ model:
height: 320 # MUST match the chosen model i.e yolov7-320 -> 320 yolov4-416 -> 416
```
## Synaptics
Hardware accelerated object detection is supported on the following SoCs:
- SL1680
This implementation uses the [Synaptics model conversion](https://synaptics-synap.github.io/doc/v/latest/docs/manual/introduction.html#offline-model-conversion), version v3.1.0.
This implementation is based on sdk `v1.5.0`.
See the [installation docs](../frigate/installation.md#synaptics) for information on configuring the SL-series NPU hardware.
### Configuration
When configuring the Synap detector, you have to specify the model: a local **path**.
#### SSD Mobilenet
A synap model is provided in the container at /mobilenet.synap and is used by this detector type by default. The model comes from [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
Use the model configuration shown below when using the synaptics detector with the default synap model:
```yaml
detectors: # required
synap_npu: # required
type: synaptics # required
model: # required
path: /synaptics/mobilenet.synap # required
width: 224 # required
height: 224 # required
tensor_format: nhwc # default value (optional. If you change the model, it is required)
labelmap_path: /labelmap/coco-80.txt # required
```
## Rockchip platform
Hardware accelerated object detection is supported on the following SoCs: