Commit Graph

108 Commits

Author SHA1 Message Date
Nicolas Mowen
c136e5e8bd
Miscellaneous fixes (#21141)
Some checks failed
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* Remove source_type from API

* Don't require state classification models to select all classes

* Specifically validate provided end_time for manual events

* Remove yolov9 specification for warning

* Remove warning for coral

* clarify zone name tip

* clarify replace rules in lpr docs

* remove periods

* Add explanation for review report

* adjust HLS gap controller params

defaults to false, should help to recover from hangs and stalling in tracking details videos on chrome

* only redirect to login page once on 401

attempt to fix ios pwa safari redirect storm

* Use contextual information from other cameras to inform report summary

* Formatting and prompt improvements for review summary report

* More improvements to prompt

* Remove examples

* Don't show admin action buttons on export card

* fix redirect race condition

Coordinate 401 redirect logic between ApiProvider and ProtectedRoute using a shared flag to prevent multiple simultaneous redirects that caused UI flashing. Ensure both auth error paths check and set the redirect flag before navigating to login, eliminating race conditions where both mechanisms could trigger at once

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-12-04 12:19:07 -06:00
Dan Brown
9ab78f496c
Adds support for YOLO v9 models running on Google Coral (#21124)
Some checks failed
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* Adds support for YOLO v9 models running on Google Coral

* fix format by using ruff instead of black

* Remove comment

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* Remove log message

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* revert to hard-coded settings. use ModelTypeEnum directly

* remove log messages. detect invalid output tensor count

* remove 1-tensor processing. add pre_process() function

* check for valid model type

* fix formatting

* remove unused import and variable

* remove tip that indicates other YOLO models may be supported.

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-12-02 13:26:57 -07:00
Josh Hawkins
815303922d
Miscellaneous Fixes (#21005)
* update live view docs

* use swr as single source of truth for searchDetail

rather than maintaining a separate state, derive the selected item from swr cache. fixes websocket sync when regenerating descriptions or fetching transcriptions

* fix key warning in console

* don't try to fetch event from review item for audio events

* update audio transcription toast wording

* Add a community supported badge to specific detectors in the info summaries to better separate

* Make object classification publish to tracked object update and add examples for state classification

* Add item to advanced docs about tensorflow limiting

* Don't show submission for in progress objects

* fix for ios not reporting video dimensions on initial metadata load

in testing, polling with requestAnimationFrame finds the dimensions within 2 frames

* Catch jetson nvidia device tree

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-11-23 08:40:25 -07:00
Abinila Siva
3f9b153758
[MemryX] Update YOLOv9 post-processing (#20980)
* Update optimized YOLOv9 post-processing

* remove unused import
2025-11-21 14:24:17 -07:00
Josh Hawkins
01452e4c51
Miscellaneous Fixes (#20841)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* show id field when editing zone

* improve zone capitalization

* Update NPU models and docs

* fix mobilepage in tracked object details

* Use thread lock for openvino to avoid concurrent requests with JinaV2

* fix hashing function to avoid collisions

* remove extra flex div causing overflow

* ensure header stays on top of video controls

* don't smart capitalize friendly names

* Fix incorrect object classification crop

* don't display submit to plus if object doesn't have a snapshot

* check for snapshot and clip in actions menu

* frigate plus submission fix

still show frigate+ section if snapshot has already been submitted and run optimistic update, local state was being overridden

* Don't fail to show 0% when showing classification

* Don't fail on file system error

* Improve title and description for review genai

* fix overflowing truncated review item description in detail stream

* catch events with review items that start after the first timeline entry

review items may start later than events within them, so subtract a padding from the start time in the filter so the start of events are not incorrectly filtered out of the list in the detail stream

* also pad on review end_time

* fix

* change order of timeline zoom buttons on mobile

* use grid to ensure genai title does not cause overflow

* small tweaks

* Cleanup

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-11-08 05:44:30 -07:00
Blake Blackshear
32875fb4cc Merge remote-tracking branch 'origin/master' into dev
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
2025-10-25 11:16:09 +00:00
Nicolas Mowen
4228861810
Improve Intel Model (#20541)
* Update supported models and inference times

* Fix d-fine inputs

* Improve d-fine
2025-10-17 06:31:28 -06:00
Nicolas Mowen
0302db1c43
Fix model exports (#20540) 2025-10-17 07:16:30 -05:00
Nicolas Mowen
d7275a3c1a
Add support for Intel NPU (#20536) 2025-10-17 05:58:59 -05:00
Nicolas Mowen
a4764563a5
Fix YOLOv9 export script (#20514) 2025-10-16 07:56:37 -05:00
Nicolas Mowen
e0a8445bac
Improve rf-detr export (#20485) 2025-10-14 08:32:44 -05:00
Nicolas Mowen
c207009d8a
Refactor AMD GPU support (#20239)
Some checks are pending
CI / ARM Extra Build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Update ROCm to 7.0.1

* Update ONNXRuntime

* Add back in

* Get basic detection working

* Use env vars

* Handle complex migraphx models

* Enable model caching

* Remove unused

* Add tip to docs
2025-09-27 14:43:11 -05:00
GaryHuang-ASUS
b8b07ee6e1
[Init] Initial commit for Synaptics SL1680 NPU (#19680)
* [Init] Initial commit for Synaptics SL1680 NPU

* add a rough detector which is testing with yolov8 tflite model.

* [Feat] Add dependencies installation in docker build

- Add runtime library and wheels installation in main/Dockerfile
- Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680

* [Update] Remove dependencies installation from main Dockerfile

- remove deps installation from Dockerfile
- add dependencies installation and split wheels, deps stage in synap1680 Dockerfile

* Refactor synap detector to more closely match other implementations

* [Update] Add model path configuration check

* [Update] update ModelType to ssd

* [Update] Remove unuse script

- install_deps.sh has already been executing in deps download stage
- Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain

* [Update] update Synaptics SL1680 setup description

* [Update] remove install_synap1680

- The deps download and installation is existed in synap1680

* [Fix] update document content

* [Update] Update detector from synap1680 to synaptics

This update is in order to make the synaptics SL-series NPU detector more general.

- Fix detector `os` module not import bug
- Update detector type `synap1680` to `synaptics`
- Update document description `SL1680` to `Synaptics` only
- Update docker build content `synap1680` to `synaptics`

* [Fix] Update configuration document

* Update docs/docs/configuration/object_detectors.md

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* [Update] Update document content and detector default layout

- Update object_detectors document
- Update detector's default layout
- Update default model name

* [Update] Update object detector document content

* [Fix] Fix InputTensorEnum not defined error

- import InputTensorEnum from detector_config

* [Update] Update detector script coding format

* [Update] Update synaptics detector coding format

* [Update] Add synaptics ci workflow

* [Update] update synaptics runtime libs download path

- Fork Synaptics astra sdk repo and put the runtime lib package on it
- Frigate team can update this download path later

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-09-26 07:07:12 -05:00
Nicolas Mowen
b23355da53
Update apple silicon docs (#20204) 2025-09-25 06:12:35 -05:00
Blake Blackshear
652fdc6a38 Merge remote-tracking branch 'origin/master' into dev 2025-09-24 06:57:50 -05:00
Nicolas Mowen
7c7ff49b90
Improve d-fine model export docs (#20020) 2025-09-11 06:17:08 -05:00
laviddichterman
1613499218
Update object_detectors.md to document configuring image size in YOLO 9 (#19951)
* Update object_detectors.md for v16

* add configurability to IMG_SIZE for YOLOv9 export
* remove TensorRT detector as it's no longer supported in v16

* Revert removing NVIDIA TensorRT detector docs

Added documentation for NVidia TensorRT Detector, including model generation, configuration parameters, and example usage.

* Dumb copy/paste

* Enhance YOLOv9 export instructions in documentation

Updated YOLOv9 export command to include IMG_SIZE parameter and clarified model size options.
2025-09-09 14:27:30 -06:00
Josh Hawkins
880902cdd7
Add specific notes for frigate+ models in object detector docs (#19971) 2025-09-08 09:29:03 -05:00
Nicolas Mowen
c5ed95ec52
More inference speed updates (#19947)
* More inference speed updates

* Update hardware.md

* Update hardware.md

* Update index.md

* More inference speeds

* Update home-assistant.md

* Update object_detectors.md

* Update first_model.md
2025-09-08 07:43:04 -05:00
Blake Blackshear
a8b7e5dd24 Merge remote-tracking branch 'origin/master' into dev 2025-09-04 06:33:22 -05:00
Alone
9af7246b0b
Fix typo in Apple Silicon detector (#19854) 2025-08-31 22:10:52 -05:00
ChirayuRai
0febc4d456
DeGirum Detector for Frigate (#19111)
* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless

* Fixed broken link

* Made it so openvino prioritizes using GPU and NPU over CPU

* Version that detects model and can begin using @local

* Updating requirements to build dev container

* Added optimized version of degirum plugin + updated docs

* Added guard clause for empty inference reponse

* Updated DeGirum's docs

* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder

* Moved DeGirum section to correct place in community models

* Update ROCm to 6.4.0 (#18264)

* Update to rocm 6.4.0

* Update URL

* Remove old env var

* Dynamic Config Updates (#18353)

* Create classes to handle publishing and subscribing config updates

* Cleanup

* Use config updater

* Update handling for enabled config

* Cleanup

* Recording config updates

* Birdseye config updates

* Handle notifications

* handle review

* Update motion

* Dynamically update masks and zones for cameras (#18359)

* Include config publisher in api

* Call update topic for passed topics

* Update zones dynamically

* Update zones internally

* Support zone and mask reset

* Handle updating objects config

* Don't put status for needing to restart Frigate

* Cleanup http tests

* Fix tests

* Initial custom classification model config support (#18362)

* Add basic config for defining a teachable machine model

* Add model type

* Add basic config for teachable machine models

* Adjust config for state and object

* Use config to process

* Correctly check for objects

* Remove debug

* Rename to not be teachable machine specific

* Cleanup

* Implement support for no recordings indicator on timeline (#18363)

* Indicate no recordings on the history timeline with gray hash marks

This commit includes a new backend API endpoint and the frontend changes needed to support this functionality

* don't show slashes for now

* Update ROCm to 6.4.1 (#18364)

* Update rocm to 6.4.1

* Quick fix

* Add ability to configure when custom classification models run (#18380)

* Add config to control when classification models are run

* Cleanup

* Add basic config editor when Frigate can't startup (#18383)

* Start Frigate in safe mode when config does not validate

* Add safe mode page that is just the config editor

* Adjust Frigate config editor when in safe mode

* Cleanup

* Improve log message

* Fix incorrectly running lpr (#18390)

* Audio transcription support (#18398)

* install new packages for transcription support

* add config options

* audio maintainer modifications to support transcription

* pass main config to audio process

* embeddings support

* api and transcription post processor

* embeddings maintainer support for post processor

* live audio transcription with sherpa and faster-whisper

* update dispatcher with live transcription topic

* frontend websocket

* frontend live transcription

* frontend changes for speech events

* i18n changes

* docs

* mqtt docs

* fix linter

* use float16 and small model on gpu for real-time

* fix return value and use requestor to embed description instead of passing embeddings

* run real-time transcription in its own thread

* tweaks

* publish live transcriptions on their own topic instead of tracked_object_update

* config validator and docs

* clarify docs

* Implement API to train classification models (#18475)

* Intel updates (#18493)

* Update openvino and onnxruntime

* Install icd and level-zero-gpu deps from intel directly

* Install

* Add dep

* Fix package install

* Tiered recordings (#18492)

* Implement tiered recording

* Add migration for record config

* Update docs

* Update reference docs

* Fix preview query

* Fix incorrect accesses

* Fix

* Fix

* Fix

* Fix

* Upgrade PaddleOCR models to v4 (rec) and v5 (det) (#18505)

The PP_OCRv5 text detection models have greatly improved over v3. The v5 recognition model makes improvements to challenging handwriting and uncommon characters, which are not necessary for LPR, so using v4 seemed like a better choice to continue to keep inference time as low as possible. Also included is the full dictionary for Chinese character support.

* Audio transcription tweaks (#18540)

* use model runner

* unload whisper model when live transcription is complete

* Classification Model UI (#18571)

* Setup basic training structure

* Build out route

* Handle model configs

* Add image fetch APIs

* Implement model training screen with dataset selection

* Implement viewing of training images

* Adjust directories

* Implement viewing of images

* Add support for deleting images

* Implement full deletion

* Implement classification model training

* Improve naming

* More renaming

* Improve layout

* Reduce logging

* Cleanup

* Live classification model training (#18583)

* Implement model training via ZMQ and add model states to represent training

* Get model updates working

* Improve toasts and model state

* Clean up logging

* Add back in

* Classification Model Metrics (#18595)

* Add speed and rate metrics for custom classification models

* Use metrics for classification models

* Use keys

* Cast to list

* Add Mesa Teflon as a TFLite detector (#18310)

* Refactor common functions for tflite detector implementations

* Add detector using mesa teflon delegate

Non-EdgeTPU TFLite can use the standard .tflite format

* Add mesa-teflon-delegate from bookworm-backports to arm64 images

* feat: enable using GenAI for cameras with GenAI disabled from the API (#18616)

* fix: Initialize GenAI client if GenAI is enabled globally (#18623)

* Make Birdseye clickable (#18628)

* keep track of layout changes and publish on change

* websocket hook

* clickable overlay div to navigate to full camera view

* Refactor TensorRT (#18643)

* Combine base and arm trt detectors

* Remove unused deps for amd64 build

* Add missing packages and cleanup ldconfig

* Expand packages for tensorflow model training

* Cleanup

* Refactor training to not reserve memory

* Dynamic Management of Cameras (#18671)

* Add base class for global config updates

* Add or remove camera states

* Move camera process management to separate thread

* Move camera management fully to separate class

* Cleanup

* Stop camera processes when stop command is sent

* Start processes dynamically when needed

* Adjust

* Leave extra room in tracked object queue for two cameras

* Dynamically set extra config pieces

* Add some TODOs

* Fix type check

* Simplify config updates

* Improve typing

* Correctly handle indexed entries

* Cleanup

* Create out SHM

* Use ZMQ for signaling object detectoin is completed

* Get camera correctly created

* Cleanup for updating the cameras config

* Cleanup

* Don't enable audio if no cameras have audio transcription

* Use exact string so similar camera names don't interfere

* Add ability to update config via json body to config/set endpoint

Additionally, update the config in a single rather than multiple calls for each updated key

* fix autotracking calibration to support new config updater function

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Use Fork-Server As Spawn Method (#18682)

* Set runtime

* Use count correctly

* Don't assume camera sizes

* Use separate zmq proxy for object detection

* Correct order

* Use forkserver

* Only store PID instead of entire process reference

* Cleanup

* Catch correct errors

* Fix typing

* Remove before_run from process util

The before_run never actually ran because:

You're right to suspect an issue with before_run not being called and a potential deadlock. The way you've implemented the run_wrapper using __getattribute__ for the run method of BaseProcess is a common pitfall in Python's multiprocessing, especially when combined with how multiprocessing.Process works internally.

Here's a breakdown of why before_run isn't being called and why you might be experiencing a deadlock:

The Problem: __getattribute__ and Process Serialization
When you create a multiprocessing.Process object and call start(), the multiprocessing module needs to serialize the process object (or at least enough of it to re-create the process in the new interpreter). It then pickles this serialized object and sends it to the newly spawned process.

The issue with your __getattribute__ implementation for run is that:

run is retrieved during serialization: When multiprocessing tries to pickle your Process object to send to the new process, it will likely access the run attribute. This triggers your __getattribute__ wrapper, which then tries to bind run_wrapper to self.
run_wrapper is bound to the parent process's self: The run_wrapper closure, when created in the parent process, captures the self (the Process instance) from the parent's memory space.
Deserialization creates a new object: In the child process, a new Process object is created by deserializing the pickled data. However, the run_wrapper method that was pickled still holds a reference to the self from the parent process. This is a subtle but critical distinction.
The child's run is not your wrapped run: When the child process starts, it internally calls its own run method. Because of the serialization and deserialization process, the run method that's ultimately executed in the child process is the original multiprocessing.Process.run or the Process.run if you had directly overridden it. Your __getattribute__ magic, which wraps run, isn't correctly applied to the Process object within the child's context.

* Cleanup

* Logging bugfix (#18465)

* use mp Manager to handle logging queues

A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.

* consolidate

* fix typing

* Fix typing

* Use global log queue

* Move to using process for logging

* Convert camera tracking to process

* Add more processes

* Finalize process

* Cleanup

* Cleanup typing

* Formatting

* Remove daemon

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Add basic camera settings to UI for testing (#18690)

* add basic camera add/edit pane to the UI for testing

* only init model runner if transcription is enabled globally

* fix role checkboxes

* Ensure logging config is propagated to forked processes (#18704)

* Move log level initialization to log

* Use logger config

* Formatting

* Fix config order

* Set process names

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* Fix go2rtc init (#18708)

* Cleanup process handling

* Adjust process name

* Reduce tf initialization

* Don't use staticmethod

* Don't fail on unicode debug for config updates

* Catch unpickling error

* Fix birdseye crash when dynamically adding a camera (#18821)

* Catch invalid character index in lpr CTC decoder (#18825)

* Classification model cover images (#18843)

* Move to separate component

* Add cover images for clssification models

* Fix process name

* Handle SIGINT with forkserver (#18860)

* Pass stopevent from main start

* Share stop event across processes

* preload modules

* remove explicit os._exit call

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Don't try to close or join mp manager queues (#18866)

Multiprocessing Manager queues don't have a close() or join_thread() method, and the Manager will clean it up appropriately after we empty it. This prevents an infinite loop when an AttributeError exception fires for Manager AutoProxy queue objects.

* Improve logging (#18867)

* Ignore numpy get limits warning

* Add function wrapper to redirect stdout and stderr to logpipe

* Save stderr too

* Add more to catch

* run logpipe

* Use other logging redirect class

* Use other logging redirect class

* add decorator for redirecting c/c++ level output to logger

* fix typing

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Add ONVIF focus support (#18883)

* backend

* frontend and i18n

* 0.17 tweaks (#18892)

* Set version

* Cleanup more logs

* Don't log matplotlib

* Improve object classification (#18908)

* Ui improvements

* Improve image cropping and model saving

* Improve naming

* Add logs for training

* Improve model labeling

* Don't set sub label for none object classification

* Cleanup

* Remove TFLite init logs

* Improve classification UI (#18910)

* Move threhsold to base model config

* Improve score handling

* Add back button

* Classification improvements (#19020)

* Move classification training to full process

* Sort class images

* Semantic Search Triggers (#18969)

* semantic trigger test

* database and model

* config

* embeddings maintainer and trigger post-processor

* api to create, edit, delete triggers

* frontend and i18n keys

* use thumbnail and description for trigger types

* image picker tweaks

* initial sync

* thumbnail file management

* clean up logs and use saved thumbnail on frontend

* publish mqtt messages

* webpush changes to enable trigger notifications

* add enabled switch

* add triggers from explore

* renaming and deletion fixes

* fix typing

* UI updates and add last triggering event time and link

* log exception instead of return in endpoint

* highlight entry in UI when triggered

* save and delete thumbnails directly

* remove alert action for now and add descriptions

* tweaks

* clean up

* fix types

* docs

* docs tweaks

* docs

* reuse enum

* Optionally show tracked object paths in debug view (#19025)

* Dynamically enable/disable GenAI (#19139)

* config

* dispatcher and mqtt

* docs

* use config updater

* add switch to frontend

* Classification train updates (#19173)

* Improve model train button

* Add filters for classification

* Cleanup

* Don't run classification on false positives

* Cleanup filter

* Fix icon color

* Object attribute classification (#19205)

* Add enum for type of classification for objects

* Update recognized license plate topic to be used as attribute updater

* Update attribute for attribute type object classification

* Cleanup

* Require setting process priority for FrigateProcess (#19207)

* Add bookworm-backports to the rocm images and upgrade mesa/vaapi to support RDNA4 GPUs (#19312)

* Improve the tablet layout (#19320)

* Improve the tablet layout

* Update imports sort

* Fix more imports

* Implement start for review item description processor (#19352)

* Add review item data transmission

* Publish review updates

* Add review item subscriber

* Basic implementation for testing review processor

* Formatting

* Cleanup

* Improve comms typing (#18599)

* Enable mypy for comms

* Make zmq data types consistent

* Cleanup inter process typing issues

* Cleanup embeddings typing

* Cleanup config updater

* Cleanup recordings updator

* Make publisher have a generic type

* Cleanup event metadata updater

* Cleanup event metadata updater

* Cleanup detections updater

* Cleanup websocket

* Cleanup mqtt

* Cleanup webpush

* Cleanup dispatcher

* Formatting

* Remove unused

* Add return type

* Fix tests

* Fix semantic triggers config typing

* Cleanup

* Ensure alertVideos persistence is loaded before displaying thumb or preview (#19432)

The default value of true would cause previews to be loaded in the background even if the local storage value was false

* Adjust loitering behavior based on object type (#19433)

* Adjust loitering behavior based on object

* Update docs

* Grammar

* Enable mypy for DB and fix types (#19434)

* Install peewee type hints

* Models now have proper types

* Fix iterator type

* Enable debug builds with dev reqs installed

* Install as wheel

* Fix cast type

* Migrate object genai configuration (#19437)

* Move genAI object to objects section

* Adjust config propogation behavior

* Refactor genai config usage

* Automatic migration

* Always start the embeddings process

* Always init embeddings

* Config fixes

* Adjust reference config

* Adjust docs

* Formatting

* Fix

* Review Item GenAI metadata (#19442)

* Rename existing function

* Keep track of thumbnial updates

* Tinkering with genai prompt

* Adjust input format

* Create model for review description output

* testing prompt changes

* Prompt improvements and image saving

* Add config for review items genai

* Use genai review config

* Actual config usage

* Adjust debug image saving

* Fix

* Fix review creation

* Adjust prompt

* Prompt adjustment

* Run genai in thread

* Fix detections block

* Adjust prompt

* Prompt changes

* Save genai response to metadata model

* Handle metadata

* Send review update to dispatcher

* Save review metadata to DB

* Send review notification updates

* Quick fix

* Fix name

* Fix update type

* Correctly dump model

* Add card

* Add card

* Remove message

* Cleanup typing and UI

* Adjust prompt

* Formatting

* Add log

* Formatting

* Add inference speed and keep alive

* Review genai updates (#19448)

* Include extra level for normal activity

* Add dynamic toggling

* Update docs

* Add different threshold for genai

* Adjust webUI for object and review description feature

* Adjust config

* Send on startup

* Cleanup config setting

* Set config

* Fix config name

* Use preview frames for Review Descriptions (#19450)

* Use preview frames for genai

* Cleanup

* Adjust

* Add config for users to define additional concerns that GenAI should make note of in review summary (#19463)

* Don't default to openai

* Improve UI

* Allow configuring additional concerns that users may want the AI to note

* Formatting

* Add preferred language config

* Remove unused

* Added total camera fps, total processed fps, and total skipped fps to stats api (#19469)

Co-authored-by: Mark Francis <markfrancisonly@gmail.com>

* Genai review summaries (#19473)

* Generate review item summaries with requests

* Adjust logic to only send important items

* Don't mention ladder

* Adjust prompt to be more specific

* Add more relaxed nature for normal activity

* Cleanup summary

* Update ollama client

* Add more directions to analyze the frames in order

* Remove environment from prompt

* Add ability to pass additional args to Ollama (#19484)

* Call out recognized objects more specifically

* Cleanup

* Make keep_alive and options configurable

* Generalize

* Use for other providers

* Update GenAI docs for new review summaries feature (#19493)

* Remove old genai docs

* Separate existing genai docs to separate sections

* Add docs for genai features

* Update reference config

* Update link

* Move to bottom

* Improve natural language of prompt (#19515)

* Make sequence details human-readable so they are used in natural language response

* Cleanup

* Improve prompt and image selection

* Adjust

* Adjust sligtly

* Format time

* Adjust frame selection logic

* Debug save response

* Ignore extra fields

* Adjust docs

* Cleanup filename sanitization

* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless

* Fixed broken link

* Made it so openvino prioritizes using GPU and NPU over CPU

* Version that detects model and can begin using @local

* Added optimized version of degirum plugin + updated docs

* Updating requirements to build dev container

* Added guard clause for empty inference reponse

* Updated DeGirum's docs

* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder

* Moved DeGirum section to correct place in community models

* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless

* Fixed broken link

* Made it so openvino prioritizes using GPU and NPU over CPU

* Version that detects model and can begin using @local

* Added optimized version of degirum plugin + updated docs

* Updating requirements to build dev container

* Added guard clause for empty inference reponse

* Updated DeGirum's docs

* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder

* Moved DeGirum section to correct place in community models

* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless

* Fixed broken link

* Made it so openvino prioritizes using GPU and NPU over CPU

* Version that detects model and can begin using @local

* Added optimized version of degirum plugin + updated docs

* Updating requirements to build dev container

* Added guard clause for empty inference reponse

* Updated DeGirum's docs

* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder

* Moved DeGirum section to correct place in community models

* Reverted changes to classification and audio

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Co-authored-by: Jimmy <honj@alum.rpi.edu>
Co-authored-by: FL42 <46161216+fl42@users.noreply.github.com>
Co-authored-by: Steve Smith <tarkasteve@gmail.com>
Co-authored-by: markfrancisonly <12145270+markfrancisonly@users.noreply.github.com>
Co-authored-by: Mark Francis <markfrancisonly@gmail.com>
2025-08-26 16:38:34 -06:00
Nicolas Mowen
4347402fcc
Don't mention 9.0.0 GFX version (#19742) 2025-08-25 07:32:50 -05:00
Tim Wesley
dbceb4dcc7
MemryX MX3 detector integration (#17723)
* sdk_2.0_update

* memryx docs: minor reorg

* ran ruff

* whoops, more ruff fixes

* Fixes (#6)

* Fixes and custom model path updated

* ruff formatting

* removed apt install from main

* add comment about libgomp1 in install_deps

---------

Co-authored-by: Abinila Siva <abinila.siva@memryx.com>
Co-authored-by: Abinila Siva <163017635+abinila4@users.noreply.github.com>
2025-08-22 08:11:48 -05:00
On Freund
0309090852
Fix typo in Apple Silicon detector (#19595) 2025-08-18 10:02:06 -06:00
Nicolas Mowen
152d9ed4a0
Apple Silicon / ZMQ Detector (#19592)
* Add zmq detector

* Cleanup

* Logging

* Cleanup

* Cleanup

* Add to hardware docs

* Add apple silicon to docs

* Formatting
2025-08-18 09:51:12 -05:00
Nicolas Mowen
c3410cd13e
Revamp RF-DETR Export Docs (#19341)
* Revamp RFDETR docs

* Clarify model size

* Specify model size
2025-07-31 09:11:46 -05:00
Jack
23b32cbacf
Update object_detectors.md (#19328)
Minor documentation clarity
2025-07-30 07:08:44 -06:00
Felipe Santos
6d5fb65a68
Revamp YOLOv9 export guide (#19224)
* Revamp YOLOv9 export guide

* Make variant a build arg

* Change VARIANT to MODEL_SIZE

* Mention available models
2025-07-20 09:19:23 -06:00
Marc Altmann
60923ac1e3
RKNN: Fix broken models for Toolkit v2.3.2 (#19189)
* update models

* don't set input_dtype: float for yolov9 (rknn)
2025-07-18 06:26:41 -06:00
Nicolas Mowen
d69916694b
Remove TensorRT Detector from AMD64 Build (#19030)
* Don't support tensorrt detector for amd64 builds

* Add logs for directing users not to use tensorrt detector

* Rework docs

* Fix dockerfile index

* Don't undo jetson fix
2025-07-07 18:12:52 -05:00
Blake Blackshear
7238b3ea22 Merge remote-tracking branch 'origin/master' into dev 2025-06-27 06:53:02 -05:00
Nicolas Mowen
cbdac9ece5
Fixes (#18379)
* Don't return weighted name if it has the same number of results

* Remove link to incorrect format yolov9 models

* Fix command list from appearing when other inputs are focused

the description box in the tracked object details pane was causing the command input list to show when focused.

* clarify face docs

* Add note about python yolov9 export

* Check if hailort thread is still alive when timeout error is run into

* Reduce inference timeout

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-05-27 10:25:34 -05:00
Thomas A.
63f9689b0e
Improve documentation for YOLO-NAS notebook (#18381)
* Add readme file for notebooks
It took me quite some time to find out this workbook opens in Google Colab, so i though it may help someone else to have this visible in the folder aswell.

* Add explicit link to Google Colab for YOLO-NAS
2025-05-24 09:42:32 -06:00
Nicolas Mowen
3a69273f0c
revert onnx runtime update (#18074)
* revert onnx runtime update

* Fix docs
2025-05-06 09:02:34 -05:00
Nicolas Mowen
895afcdb0e
Various Fixes (#18035)
* Support multi and single core rknn npus

* Update docs config to be more clear
2025-05-04 09:33:27 -06:00
Josh Hawkins
08c087f221
Docs updates (#17986)
* docs updates

* revamp hwaccel

* remove

* clarify

* fix

* more clarity

* clean up
2025-05-01 08:17:35 -05:00
Nicolas Mowen
aa7899e9dc
Docs / typing updates (#17878)
* Mention support for multiple rknn detectors

* Improve typing

* Fix mypy change

* Fix typing
2025-04-23 17:06:06 -06:00
Marc Altmann
4c749e3004
update rknn toolkit version (#17877) 2025-04-23 14:30:44 -06:00
Nicolas Mowen
212784b68e
implement RKNN downloads for yolov9 and yolox models (#17875)
* Add other rockchip download models

* Specify newer release version

* Specify newer release version

* Update docs for rknn downloads

* Update hardware docs
2025-04-23 12:22:23 -05:00
Nicolas Mowen
89b54f19c8
Add YOLOv9 support to RKNN (#17791)
* Add yolov9

* Undo

* Update docs for rknn yolov9

* Update docs notes

* Add infernece times table
2025-04-18 16:51:04 -06:00
Nicolas Mowen
e8883a2a2e
Fix yolox docs (#17789) 2025-04-18 16:15:55 -05:00
Nicolas Mowen
1cdc9b6097
Implement YOLOx for RKNN (#17788)
* Implement yolox rknn inference and post processing

* rework docs
2025-04-18 14:44:02 -06:00
Nicolas Mowen
19aaa64fe9
Add support for yolox models to onnx detector (#17773) 2025-04-18 06:40:06 -05:00
Nicolas Mowen
8270967cdc
UI Fixes (#17762)
* Fix handling of non existing cameras in groups

* Update docs

* Improve typing
2025-04-17 13:48:09 -06:00
Nicolas Mowen
8744da3bf8
Fix yolov9 export script (#17744)
* Fix yolov9 export script

* Fix typo

* Update object_detectors.md
2025-04-16 14:52:21 -06:00
Nicolas Mowen
bd0ee86db9
Refactor yolov9 detector to support v3, v4, v7 as well (#17697)
* Implement blobbed yolov7 post processing and consolidate yolo implementation

* Update documentation

* Add repo

* fix name
2025-04-14 16:05:41 -05:00
Nicolas Mowen
e9787c5a88
Small Tweaks (#17652)
* Ensure that hailo uses correct labelmap

* Make whole button clickable

* Add weblate to readme

* Update docs for HEIC

* Fix explore chip icon logic

* Sort regardless of case

* Don't allow selection

* Fix image uploading
2025-04-11 08:21:01 -06:00
Nicolas Mowen
7cfa818e63
Update tensorrt model config to use coco-80 (#17640) 2025-04-10 17:28:57 -05:00
Nicolas Mowen
30ac868757
Quick fixes (#17639)
* Use mobile drawer for face selection

* Convert face selection to separate component

* Cleanup dialogs

* Add FAQ for record resolution

* Update image name

* Remove unused

* Cleanup
2025-04-10 16:33:51 -05:00