mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-10-18 04:32:47 +03:00
[Init] Initial commit for Synaptics SL1680 NPU (#19680)
* [Init] Initial commit for Synaptics SL1680 NPU * add a rough detector which is testing with yolov8 tflite model. * [Feat] Add dependencies installation in docker build - Add runtime library and wheels installation in main/Dockerfile - Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680 * [Update] Remove dependencies installation from main Dockerfile - remove deps installation from Dockerfile - add dependencies installation and split wheels, deps stage in synap1680 Dockerfile * Refactor synap detector to more closely match other implementations * [Update] Add model path configuration check * [Update] update ModelType to ssd * [Update] Remove unuse script - install_deps.sh has already been executing in deps download stage - Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain * [Update] update Synaptics SL1680 setup description * [Update] remove install_synap1680 - The deps download and installation is existed in synap1680 * [Fix] update document content * [Update] Update detector from synap1680 to synaptics This update is in order to make the synaptics SL-series NPU detector more general. - Fix detector `os` module not import bug - Update detector type `synap1680` to `synaptics` - Update document description `SL1680` to `Synaptics` only - Update docker build content `synap1680` to `synaptics` * [Fix] Update configuration document * Update docs/docs/configuration/object_detectors.md Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> * [Update] Update document content and detector default layout - Update object_detectors document - Update detector's default layout - Update default model name * [Update] Update object detector document content * [Fix] Fix InputTensorEnum not defined error - import InputTensorEnum from detector_config * [Update] Update detector script coding format * [Update] Update synaptics detector coding format * [Update] Add synaptics ci workflow * [Update] update synaptics runtime libs download path - Fork Synaptics astra sdk repo and put the runtime lib package on it - Frigate team can update this download path later --------- Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
This commit is contained in:
co-authored by
Nicolas Mowen
parent
082867447b
commit
b8b07ee6e1
@@ -43,6 +43,10 @@ Frigate supports multiple different detectors that work on different types of ha
|
||||
|
||||
- [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs.
|
||||
|
||||
**Synaptics**
|
||||
|
||||
- [Synaptics](#synaptics): synap models can run on Synaptics devices(e.g astra machina) with included NPUs.
|
||||
|
||||
**For Testing**
|
||||
|
||||
- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results.
|
||||
@@ -1049,6 +1053,41 @@ model:
|
||||
height: 320 # MUST match the chosen model i.e yolov7-320 -> 320 yolov4-416 -> 416
|
||||
```
|
||||
|
||||
## Synaptics
|
||||
|
||||
Hardware accelerated object detection is supported on the following SoCs:
|
||||
|
||||
- SL1680
|
||||
|
||||
This implementation uses the [Synaptics model conversion](https://synaptics-synap.github.io/doc/v/latest/docs/manual/introduction.html#offline-model-conversion), version v3.1.0.
|
||||
|
||||
This implementation is based on sdk `v1.5.0`.
|
||||
|
||||
See the [installation docs](../frigate/installation.md#synaptics) for information on configuring the SL-series NPU hardware.
|
||||
|
||||
### Configuration
|
||||
|
||||
When configuring the Synap detector, you have to specify the model: a local **path**.
|
||||
|
||||
#### SSD Mobilenet
|
||||
|
||||
A synap model is provided in the container at /mobilenet.synap and is used by this detector type by default. The model comes from [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
|
||||
|
||||
Use the model configuration shown below when using the synaptics detector with the default synap model:
|
||||
|
||||
```yaml
|
||||
detectors: # required
|
||||
synap_npu: # required
|
||||
type: synaptics # required
|
||||
|
||||
model: # required
|
||||
path: /synaptics/mobilenet.synap # required
|
||||
width: 224 # required
|
||||
height: 224 # required
|
||||
tensor_format: nhwc # default value (optional. If you change the model, it is required)
|
||||
labelmap_path: /labelmap/coco-80.txt # required
|
||||
```
|
||||
|
||||
## Rockchip platform
|
||||
|
||||
Hardware accelerated object detection is supported on the following SoCs:
|
||||
|
||||
Reference in New Issue
Block a user