Commit Graph

4658 Commits

Author SHA1 Message Date
FL42
937459be47 fix: Initialize GenAI client if GenAI is enabled globally (#18623) 2025-08-16 10:20:33 -05:00
FL42
13b760346a feat: enable using GenAI for cameras with GenAI disabled from the API (#18616) 2025-08-16 10:20:33 -05:00
Jimmy
7ce26087f7 Add Mesa Teflon as a TFLite detector (#18310)
* Refactor common functions for tflite detector implementations

* Add detector using mesa teflon delegate

Non-EdgeTPU TFLite can use the standard .tflite format

* Add mesa-teflon-delegate from bookworm-backports to arm64 images
2025-08-16 10:20:33 -05:00
Nicolas Mowen
b1a65c88e8 Classification Model Metrics (#18595)
* Add speed and rate metrics for custom classification models

* Use metrics for classification models

* Use keys

* Cast to list
2025-08-16 10:20:33 -05:00
Nicolas Mowen
765a28d812 Live classification model training (#18583)
* Implement model training via ZMQ and add model states to represent training

* Get model updates working

* Improve toasts and model state

* Clean up logging

* Add back in
2025-08-16 10:20:33 -05:00
Nicolas Mowen
1c75ff59f1 Classification Model UI (#18571)
* Setup basic training structure

* Build out route

* Handle model configs

* Add image fetch APIs

* Implement model training screen with dataset selection

* Implement viewing of training images

* Adjust directories

* Implement viewing of images

* Add support for deleting images

* Implement full deletion

* Implement classification model training

* Improve naming

* More renaming

* Improve layout

* Reduce logging

* Cleanup
2025-08-16 10:20:33 -05:00
Josh Hawkins
ac7fb29b32 Audio transcription tweaks (#18540)
* use model runner

* unload whisper model when live transcription is complete
2025-08-16 10:20:33 -05:00
Josh Hawkins
b77e6f5ebc Upgrade PaddleOCR models to v4 (rec) and v5 (det) (#18505)
The PP_OCRv5 text detection models have greatly improved over v3. The v5 recognition model makes improvements to challenging handwriting and uncommon characters, which are not necessary for LPR, so using v4 seemed like a better choice to continue to keep inference time as low as possible. Also included is the full dictionary for Chinese character support.
2025-08-16 10:20:33 -05:00
Nicolas Mowen
3f8ec72336 Tiered recordings (#18492)
* Implement tiered recording

* Add migration for record config

* Update docs

* Update reference docs

* Fix preview query

* Fix incorrect accesses

* Fix

* Fix

* Fix

* Fix
2025-08-16 10:20:33 -05:00
Nicolas Mowen
0b9997015a Intel updates (#18493)
* Update openvino and onnxruntime

* Install icd and level-zero-gpu deps from intel directly

* Install

* Add dep

* Fix package install
2025-08-16 10:20:33 -05:00
Nicolas Mowen
2c7b71b16e Implement API to train classification models (#18475) 2025-08-16 10:20:33 -05:00
Josh Hawkins
6dc36fcbb4 Audio transcription support (#18398)
* install new packages for transcription support

* add config options

* audio maintainer modifications to support transcription

* pass main config to audio process

* embeddings support

* api and transcription post processor

* embeddings maintainer support for post processor

* live audio transcription with sherpa and faster-whisper

* update dispatcher with live transcription topic

* frontend websocket

* frontend live transcription

* frontend changes for speech events

* i18n changes

* docs

* mqtt docs

* fix linter

* use float16 and small model on gpu for real-time

* fix return value and use requestor to embed description instead of passing embeddings

* run real-time transcription in its own thread

* tweaks

* publish live transcriptions on their own topic instead of tracked_object_update

* config validator and docs

* clarify docs
2025-08-16 10:20:33 -05:00
Nicolas Mowen
2385c403ee Fix incorrectly running lpr (#18390) 2025-08-16 10:20:33 -05:00
Nicolas Mowen
cf1d50be30 Add basic config editor when Frigate can't startup (#18383)
* Start Frigate in safe mode when config does not validate

* Add safe mode page that is just the config editor

* Adjust Frigate config editor when in safe mode

* Cleanup

* Improve log message
2025-08-16 10:20:33 -05:00
Nicolas Mowen
723553edb7 Add ability to configure when custom classification models run (#18380)
* Add config to control when classification models are run

* Cleanup
2025-08-16 10:20:33 -05:00
Nicolas Mowen
53ff33135b Update ROCm to 6.4.1 (#18364)
* Update rocm to 6.4.1

* Quick fix
2025-08-16 10:20:33 -05:00
Josh Hawkins
4ebc4f6d21 Implement support for no recordings indicator on timeline (#18363)
* Indicate no recordings on the history timeline with gray hash marks

This commit includes a new backend API endpoint and the frontend changes needed to support this functionality

* don't show slashes for now
2025-08-16 10:20:33 -05:00
Nicolas Mowen
e1340443f5 Initial custom classification model config support (#18362)
* Add basic config for defining a teachable machine model

* Add model type

* Add basic config for teachable machine models

* Adjust config for state and object

* Use config to process

* Correctly check for objects

* Remove debug

* Rename to not be teachable machine specific

* Cleanup
2025-08-16 10:20:33 -05:00
Nicolas Mowen
4dc526761c Dynamically update masks and zones for cameras (#18359)
* Include config publisher in api

* Call update topic for passed topics

* Update zones dynamically

* Update zones internally

* Support zone and mask reset

* Handle updating objects config

* Don't put status for needing to restart Frigate

* Cleanup http tests

* Fix tests
2025-08-16 10:20:33 -05:00
Nicolas Mowen
dc187eee1c Dynamic Config Updates (#18353)
* Create classes to handle publishing and subscribing config updates

* Cleanup

* Use config updater

* Update handling for enabled config

* Cleanup

* Recording config updates

* Birdseye config updates

* Handle notifications

* handle review

* Update motion
2025-08-16 10:20:33 -05:00
Nicolas Mowen
b7dbcce6e5 Update ROCm to 6.4.0 (#18264)
* Update to rocm 6.4.0

* Update URL

* Remove old env var
2025-08-16 10:20:33 -05:00
Nicolas Mowen
5871ba397c Cleanup filename sanitization 2025-08-16 09:06:50 -06:00
Nicolas Mowen
f73424b307 Improve natural language of prompt (#19515)
* Make sequence details human-readable so they are used in natural language response

* Cleanup

* Improve prompt and image selection

* Adjust

* Adjust sligtly

* Format time

* Adjust frame selection logic

* Debug save response

* Ignore extra fields

* Adjust docs
2025-08-16 09:06:49 -06:00
Nicolas Mowen
19c0ee29bc Update GenAI docs for new review summaries feature (#19493)
* Remove old genai docs

* Separate existing genai docs to separate sections

* Add docs for genai features

* Update reference config

* Update link

* Move to bottom
2025-08-16 09:06:49 -06:00
Nicolas Mowen
d4732e1e1b Add ability to pass additional args to Ollama (#19484)
* Call out recognized objects more specifically

* Cleanup

* Make keep_alive and options configurable

* Generalize

* Use for other providers
2025-08-16 09:06:49 -06:00
Nicolas Mowen
d017f331b9 Genai review summaries (#19473)
* Generate review item summaries with requests

* Adjust logic to only send important items

* Don't mention ladder

* Adjust prompt to be more specific

* Add more relaxed nature for normal activity

* Cleanup summary

* Update ollama client

* Add more directions to analyze the frames in order

* Remove environment from prompt
2025-08-16 09:06:49 -06:00
markfrancisonly
83f0993d7d Added total camera fps, total processed fps, and total skipped fps to stats api (#19469)
Co-authored-by: Mark Francis <markfrancisonly@gmail.com>
2025-08-16 09:06:49 -06:00
Nicolas Mowen
175b9bcb71 Add config for users to define additional concerns that GenAI should make note of in review summary (#19463)
* Don't default to openai

* Improve UI

* Allow configuring additional concerns that users may want the AI to note

* Formatting

* Add preferred language config

* Remove unused
2025-08-16 09:06:49 -06:00
Nicolas Mowen
8b0f141c04 Use preview frames for Review Descriptions (#19450)
* Use preview frames for genai

* Cleanup

* Adjust
2025-08-16 09:06:49 -06:00
Nicolas Mowen
f0b6b0cd06 Review genai updates (#19448)
* Include extra level for normal activity

* Add dynamic toggling

* Update docs

* Add different threshold for genai

* Adjust webUI for object and review description feature

* Adjust config

* Send on startup

* Cleanup config setting

* Set config

* Fix config name
2025-08-16 09:06:49 -06:00
Nicolas Mowen
96b998f9b9 Review Item GenAI metadata (#19442)
* Rename existing function

* Keep track of thumbnial updates

* Tinkering with genai prompt

* Adjust input format

* Create model for review description output

* testing prompt changes

* Prompt improvements and image saving

* Add config for review items genai

* Use genai review config

* Actual config usage

* Adjust debug image saving

* Fix

* Fix review creation

* Adjust prompt

* Prompt adjustment

* Run genai in thread

* Fix detections block

* Adjust prompt

* Prompt changes

* Save genai response to metadata model

* Handle metadata

* Send review update to dispatcher

* Save review metadata to DB

* Send review notification updates

* Quick fix

* Fix name

* Fix update type

* Correctly dump model

* Add card

* Add card

* Remove message

* Cleanup typing and UI

* Adjust prompt

* Formatting

* Add log

* Formatting

* Add inference speed and keep alive
2025-08-16 09:06:49 -06:00
Nicolas Mowen
0428509c89 Migrate object genai configuration (#19437)
* Move genAI object to objects section

* Adjust config propogation behavior

* Refactor genai config usage

* Automatic migration

* Always start the embeddings process

* Always init embeddings

* Config fixes

* Adjust reference config

* Adjust docs

* Formatting

* Fix
2025-08-16 09:06:49 -06:00
Nicolas Mowen
b7c318241b Enable mypy for DB and fix types (#19434)
* Install peewee type hints

* Models now have proper types

* Fix iterator type

* Enable debug builds with dev reqs installed

* Install as wheel

* Fix cast type
2025-08-16 09:06:49 -06:00
Nicolas Mowen
6da5925f92 Adjust loitering behavior based on object type (#19433)
* Adjust loitering behavior based on object

* Update docs

* Grammar
2025-08-16 09:06:49 -06:00
Josh Hawkins
c6a48abc75 Ensure alertVideos persistence is loaded before displaying thumb or preview (#19432)
The default value of true would cause previews to be loaded in the background even if the local storage value was false
2025-08-16 09:06:49 -06:00
Nicolas Mowen
b7095028d0 Improve comms typing (#18599)
* Enable mypy for comms

* Make zmq data types consistent

* Cleanup inter process typing issues

* Cleanup embeddings typing

* Cleanup config updater

* Cleanup recordings updator

* Make publisher have a generic type

* Cleanup event metadata updater

* Cleanup event metadata updater

* Cleanup detections updater

* Cleanup websocket

* Cleanup mqtt

* Cleanup webpush

* Cleanup dispatcher

* Formatting

* Remove unused

* Add return type

* Fix tests

* Fix semantic triggers config typing

* Cleanup
2025-08-16 09:06:49 -06:00
Nicolas Mowen
6500c538e6 Cleanup 2025-08-16 09:06:49 -06:00
Nicolas Mowen
690cd6fa53 Implement start for review item description processor (#19352)
* Add review item data transmission

* Publish review updates

* Add review item subscriber

* Basic implementation for testing review processor

* Formatting
2025-08-16 09:06:49 -06:00
Nicolas Mowen
590ae02da4 Improve the tablet layout (#19320)
* Improve the tablet layout

* Update imports sort

* Fix more imports
2025-08-16 09:06:49 -06:00
Steve Smith
a13a6825ef Add bookworm-backports to the rocm images and upgrade mesa/vaapi to support RDNA4 GPUs (#19312) 2025-08-16 09:06:49 -06:00
Nicolas Mowen
3b7d9fe9ee Require setting process priority for FrigateProcess (#19207) 2025-08-16 09:06:49 -06:00
Nicolas Mowen
8fc78f4bc7 Object attribute classification (#19205)
* Add enum for type of classification for objects

* Update recognized license plate topic to be used as attribute updater

* Update attribute for attribute type object classification

* Cleanup
2025-08-16 09:06:49 -06:00
Nicolas Mowen
2be24e40de Classification train updates (#19173)
* Improve model train button

* Add filters for classification

* Cleanup

* Don't run classification on false positives

* Cleanup filter

* Fix icon color
2025-08-16 09:06:49 -06:00
Josh Hawkins
a19d9d44d4 Dynamically enable/disable GenAI (#19139)
* config

* dispatcher and mqtt

* docs

* use config updater

* add switch to frontend
2025-08-16 09:06:49 -06:00
Josh Hawkins
11ca401fff Optionally show tracked object paths in debug view (#19025) 2025-08-16 09:06:49 -06:00
Josh Hawkins
a209668c17 Semantic Search Triggers (#18969)
* semantic trigger test

* database and model

* config

* embeddings maintainer and trigger post-processor

* api to create, edit, delete triggers

* frontend and i18n keys

* use thumbnail and description for trigger types

* image picker tweaks

* initial sync

* thumbnail file management

* clean up logs and use saved thumbnail on frontend

* publish mqtt messages

* webpush changes to enable trigger notifications

* add enabled switch

* add triggers from explore

* renaming and deletion fixes

* fix typing

* UI updates and add last triggering event time and link

* log exception instead of return in endpoint

* highlight entry in UI when triggered

* save and delete thumbnails directly

* remove alert action for now and add descriptions

* tweaks

* clean up

* fix types

* docs

* docs tweaks

* docs

* reuse enum
2025-08-16 09:06:49 -06:00
Nicolas Mowen
3ed12918a1 Classification improvements (#19020)
* Move classification training to full process

* Sort class images
2025-08-16 09:06:49 -06:00
Nicolas Mowen
8139bc9bbd Improve classification UI (#18910)
* Move threhsold to base model config

* Improve score handling

* Add back button
2025-08-16 09:06:49 -06:00
Nicolas Mowen
2159d40ad5 Remove TFLite init logs 2025-08-16 09:06:49 -06:00
Nicolas Mowen
99bf29ab6b Improve object classification (#18908)
* Ui improvements

* Improve image cropping and model saving

* Improve naming

* Add logs for training

* Improve model labeling

* Don't set sub label for none object classification

* Cleanup
2025-08-16 09:06:49 -06:00