Commit Graph
20 Commits
Author SHA1 Message Date
Nicolas MowenandGitHub eccd10cd94 Implement annotated frames for GenAI Review (#24379)
CI / AMD64 Build (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* Implement annotated frames mode for GenAI reviews to improve models with lacking temporal understanding

* Updates

* Improve debug sharing

* Do not number objects

* Fix assumptions

* Remove unhelpful content

* Improve object data sent as part of prompt

* Cleanup ollama dumbness

* Bind db

* Fixes

* Cleanup
2026-09-17 10:28:37 -06:00
Nicolas Mowen 76708e7fa6 Add options for review prompt style (#24166)
* Add script for testing genai review prompts

* Add option for prompt styling

* Add tests

* Update docs
2026-09-12 07:30:04 -06:00
Nicolas MowenandGitHub 4ee12e6237 Increase ruff coverage (#23644)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / Jetson Jetpack 6 (push) Waiting to run
* Pin ruff

* Add python upgrade fixes

This enables python upgrade checks in ruff to look for deprecated types and patterns. This namely fixes:
- usage of deprecated `Typing` which is now built in
- some specific exceptions which are caught and have new aliases

Some specific UP checks were also ignored as they are stylistic / unimportant and likely to cause bugs

* Remove async blocking calls

Use asyncio.to_thread on two remaining blocking calls to fix hanging event thread loop. Enable this specific rule to block it in the future.

* Use proper logging mechanism

* Correctly format logs

* Raise with context

When raising an exception include the from context to improve debugging

* Cleanup
2026-07-06 12:28:02 -05:00
Josh HawkinsandGitHub c35cee2d2f Review labels widget (#22664)
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* add review labels widget

* register widget and add to review section

* i18n

* add border to switches widget

* padding tweaks

* don't show audio labels if audio is not enabled

* add docs links

* ability to add custom labels to review

* add hint for empty selection in review labels and SwitchesWidget

* language consistency
2026-03-27 08:45:50 -06:00
Josh HawkinsandGitHub e7250f24cb Full UI configuration (#22151)
* use react-jsonschema-form for UI config

* don't use properties wrapper when generating config i18n json

* configure for full i18n support

* section fields

* add descriptions to all fields for i18n

* motion i18n

* fix nullable fields

* sanitize internal fields

* add switches widgets and use friendly names

* fix nullable schema entries

* ensure update_topic is added to api calls

this needs further backend implementation to work correctly

* add global sections, camera config overrides, and reset button

* i18n

* add reset logic to global config view

* tweaks

* fix sections and live validation

* fix validation for schema objects that can be null

* generic and custom per-field validation

* improve generic error validation messages

* remove show advanced fields switch

* tweaks

* use shadcn theme

* fix array field template

* i18n tweaks

* remove collapsible around root section

* deep merge schema for advanced fields

* add array field item template and fix ffmpeg section

* add missing i18n keys

* tweaks

* comment out api call for testing

* add config groups as a separate i18n namespace

* add descriptions to all pydantic fields

* make titles more concise

* new titles as i18n

* update i18n config generation script to use json schema

* tweaks

* tweaks

* rebase

* clean up

* form tweaks

* add wildcards and fix object filter fields

* add field template for additionalproperties schema objects

* improve typing

* add section description from schema and clarify global vs camera level descriptions

* separate and consolidate global and camera i18n namespaces

* clean up now obsolete namespaces

* tweaks

* refactor sections and overrides

* add ability to render components before and after fields

* fix titles

* chore(sections): remove legacy single-section components replaced by template

* refactor configs to use individual files with a template

* fix review description

* apply hidden fields after ui schema

* move util

* remove unused i18n

* clean up error messages

* fix fast refresh

* add custom validation and use it for ffmpeg input roles

* update nav tree

* remove unused

* re-add override and modified indicators

* mark pending changes and add confirmation dialog for resets

* fix red unsaved dot

* tweaks

* add docs links, readonly keys, and restart required per field

* add special case and comments for global motion section

* add section form special cases

* combine review sections

* tweaks

* add audio labels endpoint

* add audio label switches and input to filter list

* fix type

* remove key from config when resetting to default/global

* don't show description for new key/val fields

* tweaks

* spacing tweaks

* add activity indicator and scrollbar tweaks

* add docs to filter fields

* wording changes

* fix global ffmpeg section

* add review classification zones to review form

* add backend endpoint and frontend widget for ffmpeg presets and manual args

* improve wording

* hide descriptions for additional properties arrays

* add warning log about incorrectly nested model config

* spacing and language tweaks

* fix i18n keys

* networking section docs and description

* small wording tweaks

* add layout grid field

* refactor with shared utilities

* field order

* add individual detectors to schema

add detector titles and descriptions (docstrings in pydantic are used for descriptions) and add i18n keys to globals

* clean up detectors section and i18n

* don't save model config back to yaml when saving detectors

* add full detectors config to api model dump

works around the way we use detector plugins so we can have the full detector config for the frontend

* add restart button to toast when restart is required

* add ui option to remove inner cards

* fix buttons

* section tweaks

* don't zoom into text on mobile

* make buttons sticky at bottom of sections

* small tweaks

* highlight label of changed fields

* add null to enum list when unwrapping

* refactor to shared utils and add save all button

* add undo all button

* add RJSF to dictionary

* consolidate utils

* preserve form data when changing cameras

* add mono fonts

* add popover to show what fields will be saved

* fix mobile menu not re-rendering with unsaved dots

* tweaks

* fix logger and env vars config section saving

use escaped periods in keys to retain them in the config file (eg "frigate.embeddings")

* add timezone widget

* role map field with validation

* fix validation for model section

* add another hidden field

* add footer message for required restart

* use rjsf for notifications view

* fix config saving

* add replace rules field

* default column layout and add field sizing

* clean up field template

* refactor profile settings to match rjsf forms

* tweaks

* refactor frigate+ view and make tweaks to sections

* show frigate+ model info in detection model settings when using a frigate+ model

* update restartRequired for all fields

* fix restart fields

* tweaks and add ability enable disabled cameras

more backend changes required

* require restart when enabling camera that is disabled in config

* disable save when form is invalid

* refactor ffmpeg section for readability

* change label

* clean up camera inputs fields

* misc tweaks to ffmpeg section

- add raw paths endpoint to ensure credentials get saved
- restart required tooltip

* maintenance settings tweaks

* don't mutate with lodash

* fix description re-rendering for nullable object fields

* hide reindex field

* update rjsf

* add frigate+ description to settings pane

* disable save all when any section is invalid

* show translated field name in validation error pane

* clean up

* remove unused

* fix genai merge

* fix genai
2026-02-27 08:55:36 -07:00
2d83992284 Miscellaneous fixes (0.17 beta) (#21867)
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
* Adjust title prompt to have less rigidity

* Improve motion boxes handling for features that don't require motion

* Improve handling of classes starting with digits

* Improve vehicle nuance

* tweak lpr docs

* Improve grammar

* Don't allow # in face name

* add password requirements to new user dialog

* change password requirements

* Clenaup

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2026-02-03 08:31:00 -06:00
Nicolas MowenandGitHub d41ee4ff88 Miscellaneous Fixes (#20848)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
* Fix filtering for classification

* Adjust prompt to account for response tokens

* Correctly return response for reprocess

* Use API response to update data instead of trying to re-parse all of the values

* Implement rename class api

* Fix model deletion / rename dialog

* Remove camera spatial context

* Catch error
2025-11-08 13:13:40 -07:00
Nicolas MowenandGitHub 35ce275071 Add ability to define Review Summary camera context (#20828)
* Add ability to define GenAI camera context

* Cleanup

* Only show example with list
2025-11-06 07:39:44 -07:00
Nicolas MowenandGitHub 971521cd8e Review description updates (#20723)
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* Update docs for review descriptions

* Add logging for context tokens used

* Incrase number of images due to lower than expected context usage

* Re-balance the suspicious activity checks

* Adjustments to context sizing

* optimize context usage

* Adjust context usage

* Make title more direct

* Update docs
2025-10-30 08:52:55 -06:00
Nicolas MowenandGitHub 29bc213c04 Various Tweaks (#20713)
* Adjust for commutes

* Tweaks

* Don't show no models view in grid

* Add text-md to inputs

* Adjust train title for mobile

* Cleanup prompt more

* Use i18n functions for tooltip

* Fix model complexity causing crash

* Cleanup
2025-10-29 09:40:50 -05:00
Nicolas MowenandGitHub 16e17e027d Review prompt adjustments (#20704)
* Make prompt more fair and reduce time extension

* Adjust naming of unrecognized objects

* Improve object naming behavior

* Add more context image levels
2025-10-28 08:28:36 -05:00
Nicolas MowenandGitHub 710a77679b Improve default genai review prompt structure (#20690) 2025-10-27 11:34:39 -05:00
Nicolas MowenandGitHub 094a0a6e05 Add ability to change source of images for review descriptions (#20676)
* Add ability to change source of images for review descriptions

* Undo
2025-10-26 08:40:38 -05:00
Nicolas MowenandGitHub 28e3aa39f0 Customizable GenAI Review prompt (#20296)
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
* Add customizable prompt

* Update docs
2025-09-30 18:07:16 -05:00
Nicolas MowenandGitHub 3a1e1d0841 Improve review segmentation behavior (#19850)
* Refactor active objects to class

* Keep segment going when detection is newer than end of alert

* Cleanup logic

* Fix

* Cleanup ending

* Adjust timing

* Improve detection saving

* Don't have padding at end for in progress reviews

* Add review config for cutoff times
2025-08-31 16:36:12 -05:00
Nicolas MowenandBlake Blackshear cc18d7f786 Add config for users to define additional concerns that GenAI should make note of in review summary (#19463)
* Don't default to openai

* Improve UI

* Allow configuring additional concerns that users may want the AI to note

* Formatting

* Add preferred language config

* Remove unused
2025-08-16 10:20:33 -05:00
Nicolas MowenandBlake Blackshear 92417a1b9c Review genai updates (#19448)
* Include extra level for normal activity

* Add dynamic toggling

* Update docs

* Add different threshold for genai

* Adjust webUI for object and review description feature

* Adjust config

* Send on startup

* Cleanup config setting

* Set config

* Fix config name
2025-08-16 10:20:33 -05:00
Nicolas MowenandBlake Blackshear 2cf8dd693c Review Item GenAI metadata (#19442)
* Rename existing function

* Keep track of thumbnial updates

* Tinkering with genai prompt

* Adjust input format

* Create model for review description output

* testing prompt changes

* Prompt improvements and image saving

* Add config for review items genai

* Use genai review config

* Actual config usage

* Adjust debug image saving

* Fix

* Fix review creation

* Adjust prompt

* Prompt adjustment

* Run genai in thread

* Fix detections block

* Adjust prompt

* Prompt changes

* Save genai response to metadata model

* Handle metadata

* Send review update to dispatcher

* Save review metadata to DB

* Send review notification updates

* Quick fix

* Fix name

* Fix update type

* Correctly dump model

* Add card

* Add card

* Remove message

* Cleanup typing and UI

* Adjust prompt

* Formatting

* Add log

* Formatting

* Add inference speed and keep alive
2025-08-16 10:20:33 -05:00
Josh HawkinsandGitHub 82f8694464 Toggle review alerts and detections (#16482)
* backend

* frontend

* docs

* fix topic name and initial websocket state

* update reference config

* fix mqtt docs

* fix initial topics

* don't apply max severity when alerts/detections are disabled

* fix ws merge

* tweaks
2025-02-11 07:46:25 -07:00
gtsiamandGitHub bbbb3b4a06 Split config.py into multiple files (#14038)
* Replace logging.warn with logging.warning

* Install config global state early

* Split config.py into more manageable pieces
2024-09-28 14:21:42 -05:00