Commit Graph

2038 Commits

Author SHA1 Message Date
Josh Hawkins
d818dbb6ba
Triggers tweaks (#20339)
* backend

* frontend

* use correct camera name param

* i18n

* change log message to debug level

* docs tweaks
2025-10-03 06:36:14 -06:00
Nicolas Mowen
2d45ea271e
Refactor object genai to be a post-processor (#20331)
Some checks failed
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* Refactor object genai to be a post-processor

* Include function correctly
2025-10-02 12:48:11 -06:00
Nicolas Mowen
37999abbe6
Improve review summary performance (#20328)
* Undo vite

* Balance the prompt

* Round duration

* Calculate context size to determine number of images

* Increase number of images
2025-10-02 10:17:25 -05:00
Nicolas Mowen
ed6b892200
Fix object genai prompt access (#20322) 2025-10-02 05:48:16 -06:00
Nicolas Mowen
41e5c12e5b
Don't use rknn if device is CPU (#20312)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
2025-10-01 19:14:04 -05:00
Nicolas Mowen
993459152b
fix stationary runtime error (#20309) 2025-10-01 09:17:30 -06:00
Nicolas Mowen
8f0be18422
Improve stationary classification (#20303)
* Improve stationary classification

* Cleanup for mypy
2025-10-01 07:39:11 -05:00
Nicolas Mowen
28e3aa39f0
Customizable GenAI Review prompt (#20296)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Add customizable prompt

* Update docs
2025-09-30 18:07:16 -05:00
Nicolas Mowen
923412ec1c
Improve Review Summary Prompt (#20289)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Improve prompt to have better discernment and logic based on detected objects

* Be more specific about the time of day

* Add re-inforcers for LLM to be accurate and not complete a narrative
2025-09-30 06:52:38 -06:00
Nicolas Mowen
9fdce80729
Handle case when no classification model exists (#20257)
Some checks failed
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
2025-09-28 16:03:44 -05:00
Josh Hawkins
12f8c3feac
Watchdog enhancements (#20237)
Some checks are pending
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
* refactor get_video_properties and use json output from ffprobe

* add zmq topic

* publish valid segment data in recording maintainer

* check for valid video data

- restart separate record ffmpeg process if no video data has been received in 120s
- refactor datetime import

* listen to correct topic in embeddings maintainer

* refactor to move get_latest_segment_datetime logic to recordings maintainer

* debug logging

* cleanup
2025-09-28 10:52:14 -06:00
Josh Hawkins
b6552987b0
Fixes (#20254)
* fix api async/await functions

* fix synaptics detector from throwing error when unused

* clean up
2025-09-28 07:08:52 -06:00
Nicolas Mowen
c207009d8a
Refactor AMD GPU support (#20239)
Some checks are pending
CI / ARM Extra Build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Update ROCm to 7.0.1

* Update ONNXRuntime

* Add back in

* Get basic detection working

* Use env vars

* Handle complex migraphx models

* Enable model caching

* Remove unused

* Add tip to docs
2025-09-27 14:43:11 -05:00
Nicolas Mowen
e6cbc93703
More stationary cleanup (#20229)
Some checks failed
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
* Always return false for active objects

* Cleanup
2025-09-26 07:23:29 -06:00
GaryHuang-ASUS
b8b07ee6e1
[Init] Initial commit for Synaptics SL1680 NPU (#19680)
* [Init] Initial commit for Synaptics SL1680 NPU

* add a rough detector which is testing with yolov8 tflite model.

* [Feat] Add dependencies installation in docker build

- Add runtime library and wheels installation in main/Dockerfile
- Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680

* [Update] Remove dependencies installation from main Dockerfile

- remove deps installation from Dockerfile
- add dependencies installation and split wheels, deps stage in synap1680 Dockerfile

* Refactor synap detector to more closely match other implementations

* [Update] Add model path configuration check

* [Update] update ModelType to ssd

* [Update] Remove unuse script

- install_deps.sh has already been executing in deps download stage
- Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain

* [Update] update Synaptics SL1680 setup description

* [Update] remove install_synap1680

- The deps download and installation is existed in synap1680

* [Fix] update document content

* [Update] Update detector from synap1680 to synaptics

This update is in order to make the synaptics SL-series NPU detector more general.

- Fix detector `os` module not import bug
- Update detector type `synap1680` to `synaptics`
- Update document description `SL1680` to `Synaptics` only
- Update docker build content `synap1680` to `synaptics`

* [Fix] Update configuration document

* Update docs/docs/configuration/object_detectors.md

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* [Update] Update document content and detector default layout

- Update object_detectors document
- Update detector's default layout
- Update default model name

* [Update] Update object detector document content

* [Fix] Fix InputTensorEnum not defined error

- import InputTensorEnum from detector_config

* [Update] Update detector script coding format

* [Update] Update synaptics detector coding format

* [Update] Add synaptics ci workflow

* [Update] update synaptics runtime libs download path

- Fork Synaptics astra sdk repo and put the runtime lib package on it
- Frigate team can update this download path later

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-09-26 07:07:12 -05:00
Nicolas Mowen
082867447b
Stationary bug fixes (#20225)
* Correctly only enable for car

* Fix limiting stationary objects history
2025-09-26 07:03:59 -05:00
Nicolas Mowen
8b293449f9
Improve review summary (#20216)
Some checks are pending
CI / ARM Extra Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
* Add debug logging for review summaries report

* Improve debug logging

* Improve review report prompt

* Cleanup

* Add date to report
2025-09-25 21:05:22 -05:00
Nicolas Mowen
2f209b2cf4
Implement stationary car classifier to improve parked car management (#20206)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Implement stationary car classifier to base stationary state on visual changes and not just bounding box stability

* Cleanup

* Fix mypy

* Move to new file and add config to disable if needed

* Cleanup

* Undo
2025-09-25 10:18:45 -05:00
Blake Blackshear
652fdc6a38 Merge remote-tracking branch 'origin/master' into dev 2025-09-24 06:57:50 -05:00
Nicolas Mowen
7e2f5a3017
Improve 640x640 model detection of small objects (#20190)
* Allow larger models to have smaller regions

* remove unnecessary hailo resize

* Update benchmark

* Fix table

* Update nvidia specs
2025-09-23 15:49:54 -05:00
Nicolas Mowen
318457113b
Add ability to transfer model via ZMQ Detector (#20161)
* Add ability to transfer model via ZMQ

* Cleanup
2025-09-22 07:02:55 -05:00
Nicolas Mowen
e4d5f1f94e
Tune OV for latency (#20160) 2025-09-21 18:52:04 -05:00
Nicolas Mowen
0e61d3f153
YOLOv9 LPR model is not compatible (#20159) 2025-09-21 18:51:45 -05:00
Andrew Marshall
a7bbca5014
Read secrets dir from CREDENTIALS_DIRECTORY (#19327)
This supports systemd credentials, see https://systemd.io/CREDENTIALS/.
Default to `/run/secrets` (the Docker Secrets dir) for backwards
compatibility.
2025-09-19 06:34:23 -06:00
iesad
dc96940eb9
Pull count of detection events by label into prometheus metrics (#20119)
* pull count of detection events by label into prometheus metrics

* format changes with ruff

* remove unneeded f-string

* fix imports format

---------

Co-authored-by: iesad <iesad>
2025-09-19 06:27:20 -06:00
Nicolas Mowen
b8fd0a2b31
Fix CUDA graph config (#20135) 2025-09-19 05:59:42 -05:00
Josh Hawkins
251b029d6e
LPR improvements (#20129)
* continue to use paddleocr v3 text detection model for large

v5 was not finding text on multi-line plates at all in testing

* implement clustering of plate variants per event

should reduce OCR inconsistencies and improve plate recognition stability by using string similarity to cluster similar variants (10 per event id) and choosing the highest confidence representative as the final plate

* pass camera

* prune number of variants based on detect fps

* implement replacement rules for cleaning up and normalizing plates

* docs

* docs
2025-09-18 15:12:17 -06:00
Nicolas Mowen
68f806bb61
Cleanup onnx detector (#20128)
* Cleanup onnx detector

* Fix

* Fix classification cropping

* Deprioritize openvino

* Send model type

* Use model type to decide if model can use full optimization

* Clenanup

* Cleanup
2025-09-18 15:12:09 -06:00
Nicolas Mowen
c05e260ae9
Update ROCm to not hang when running on complex RNN models (#20118)
* Update ROCm to not hang when running on complex RNN models

* Formatting
2025-09-17 19:26:32 -05:00
Nicolas Mowen
1efff67e32
Fix ov for LPR (#20117)
* Check complex model

* Reset state for complex models

* Send arg

* Fix

* Cleanup
2025-09-17 15:21:57 -06:00
Nicolas Mowen
26178444f3
Fixes (#20102)
* Catch bird classification resize error

* Improve openvino width detection

* Use auto by default

* Set type
2025-09-16 16:06:51 -06:00
Josh Hawkins
975c8485f9
Catch exception when regex in LPR format field is invalid (#20099) 2025-09-16 07:41:25 -05:00
Nicolas Mowen
5f34a18905
Dynamically adjust to configured attribute map for lpr (#20079) 2025-09-15 08:49:07 -05:00
Tim Wesley
6cd1d1f205
memryx: fix model download bug when using multiple detectors (#20030)
* Add locking for model download files

* ruff format

---------

Co-authored-by: Abinila Siva <abinila.siva@memryx.com>
2025-09-15 08:48:55 -05:00
Nicolas Mowen
03fe054078
OpenVINO Hardware Improvements (#20071)
* Use OpenVINO directly to detect if devices are available

* Cleanup

* Update OpenVINO

* Cleanup

* Don't try to use OpenVINO when CPU is set as device

* Catch case where input tensor can't be pre-defined

* Cleanup
2025-09-15 08:35:49 -05:00
Josh Hawkins
2d4a0cc584
false_positive endpoint needs to await send_to_plus call (#20069) 2025-09-14 11:31:07 -05:00
Nicolas Mowen
ff0430964c
Correctly calculate input data type for OV (#20066)
* Correctly calculate input data type for OV

* Formatting
2025-09-14 07:15:41 -06:00
GuoQing Liu
b08db4913f
feat: add github mirror download endpoint (#20007)
* feat: add github mirror download endpoint

* fix: fix face_embedding endpoint line

* fix: fix github raw endpoint

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-09-14 06:51:56 -06:00
Nicolas Mowen
81d7c47129
Optimize OpenVINO and ONNX Model Runners (#20063)
* Use re-usable inference request to reduce CPU usage

* Share tensor

* Don't count performance

* Create openvino runner class

* Break apart onnx runner

* Add specific note about inability to use CUDA graphs for some models

* Adjust rknn to use RKNNRunner

* Use optimized runner

* Add support for non-complex models for CudaExecutionProvider

* Use core mask for rknn

* Correctly handle cuda input

* Cleanup

* Sort imports
2025-09-14 06:22:22 -06:00
Josh Hawkins
41ed013cc4
Check cameras param and continue to split for query (#20048) 2025-09-12 09:58:47 -06:00
Nicolas Mowen
751678c845
Fix cuda graph fallback (#20039) 2025-09-12 06:41:26 -05:00
Josh Hawkins
ed1e3a7c9a
Enhance user roles to limit camera access (#20024)
* update config for roles and add validator

* ensure admin and viewer are never overridden

* add class method to user to retrieve all allowed cameras

* enforce config roles in auth api endpoints

* add camera access api dependency functions

* protect review endpoints

* protect preview endpoints

* rename param name for better fastapi injection matching

* remove unneeded

* protect export endpoints

* protect event endpoints

* protect media endpoints

* update auth hook for allowed cameras

* update default app view

* ensure anonymous user always returns all cameras

* limit cameras in explore

* cameras is already a list

* limit cameras in review/history

* limit cameras in live view

* limit cameras in camera groups

* only show face library and classification in sidebar for admin

* remove check in delete reviews

since admin role is required, no need to check camera access. fixes failing test

* pass request with camera access for tests

* more async

* camera access tests

* fix proxy auth tests

* allowed cameras for review tests

* combine event tests and refactor for camera access

* fix post validation for roles

* don't limit roles in create user dialog

* fix triggers endpoints

no need to run require camera access dep since the required role is admin

* fix type

* create and edit role dialogs

* delete role dialog

* fix role change dialog

* update settings view for roles

* i18n changes

* minor spacing tweaks

* docs

* use badges and camera name label component

* clarify docs

* display all cameras badge for admin and viewer

* i18n fix

* use validator to prevent reserved and empty roles from being assigned

* split users and roles into separate tabs in settings

* tweak docs

* clarify docs

* change icon

* don't memoize roles

always recalculate on component render
2025-09-12 05:19:29 -06:00
Nicolas Mowen
ba650af6f2
Correctly cast to enum when loading plus config (#20031) 2025-09-11 16:39:34 -06:00
Nicolas Mowen
fad28a764c
Use CUDA graphs for object detection on Nvidia GPUs (#20027)
* Use CUDA graphs to improve efficiency of object detection

* Cleanup comments and typing
2025-09-11 10:20:25 -06:00
Nicolas Mowen
15729e0f19
Provide model type in header for ZMQ detector (#20000)
* Provide model type in header

* Formatting
2025-09-09 17:53:36 -05:00
Nicolas Mowen
205fdf3ae3
Fixes (#19984)
* Always handle RKNN as NHWC in Frigate+ model loading

* Correct Intel stats

* Update inference time docs

* Update version

* Adjust inference speeds
2025-09-09 06:17:56 -06:00
Josh Hawkins
d7f7cd7be1
best thumbnail endpoint should pass correct extension param (#19930) 2025-09-05 06:33:57 -05:00
Blake Blackshear
a8b7e5dd24 Merge remote-tracking branch 'origin/master' into dev 2025-09-04 06:33:22 -05:00
Nicolas Mowen
6505ae5fb5
Optimize cuda execution to run in single stream (#19896) 2025-09-03 08:53:30 -05:00
Nicolas Mowen
e664cb2285
Set lower bound on retry interval (#19883) 2025-09-02 11:24:25 -05:00