* fix classification wizard finding no sample images on large databases
The wizard grouped tracked objects by camera and 6 hour block, oldest first, then kept the first 100. With more than 100 groups that was always the oldest objects, which often have no snapshot or thumbnail left on disk, so no examples were generated. Shuffle the selection before truncating, and keep extracting from the remaining tracked objects until 100 usable images are found.
* add notice and system ui message when a camera isn't using go2rtc
* add docs note about privacy masks
* match index-less device strings to hardware units in the detection models picker
A config with `edgetpu:usb` was reported as hardware that wasn't found, because the picker compared device strings exactly and the probe reports the unit as `edgetpu:usb:0`. A device without an index now resolves to the first unit of that kind, so the hardware dropdown, the unit checkboxes, and the model summary all recognize it.
* don't treat unknown audio as an audio-bearing stream
A recording row with NULL has_audio (ffprobe failed and the cv2 fallback can't report audio) marked the whole stream as audio-bearing, so every confirmed video-only row on it was dropped as a glitch. A stream now counts as audio-bearing only when a row is known to carry audio.
* stage main+sub exports on disk instead of /tmp/cache
A main+sub export stages a full copy of itself before the final file is written. That copy went to /tmp/cache, so a long export outgrew the tmpfs and failed with no space left on device. Staged runs are now written to the exports directory, and startup removes any left behind by a killed export.
* address review feedback
- stage main+sub export runs in a staging subfolder of the exports directory, so media sync can't delete them mid-export and startup cleanup can't remove a finished export with a matching name or fail on an unremovable file
- run object classification example collection off the event loop
- prefer an exact hardware unit match over an index-less one, so an AMD GPU's "onnx" no longer selects the NVIDIA entry
- add tests for the classification fallback and index-less hardware matching
* serialize example collection and clean up exports when staging dir creation fails
- overlapping object example requests could delete each other's images in the shared temp and train directories, so collection now runs under a lock
- a failed makedirs for the staging directory raised past the failed-export cleanup and left the export spinning until restart, so it now fails the run through the normal cleanup path
* revert
sanitize_filename leaves ".." intact and collapses variants like "..:" and "..*" to "..", so filesystem paths built from face names, classification model/category names, image ids, and trigger data could escape their base directory. Route every such site through new frigate/util/path.py helpers (safe_join, sanitize_path_component, sanitize_contained_path), which reject traversal and verify containment.
Worst case was DELETE /classification/{name}, which rmtree'd /media/frigate and /config while returning 200.
Important to note that all affected endpoints already require admin permission, so this sould be considered hardening rather than fixing exploitable code.
* stabilize chart options to stop ApexCharts updateOptions running on every stats tick
* constrain height of export dialog
* stop audio maintainer when deleting a camera
* run face register and recognize API handlers in threadpool
* use default stable api version for gemini genai client
* update gemini docs
* remove outdated genai.md and update correct file
* Classification fixes
* Mutate when a date is selected and marked as reviewed
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* remove footer messages and add update topic to motion tuner view
restart after changing values is no longer required
* add cache key and activity indicator for loading classification wizard images
* Always mark model as untrained when a classname is changed
* clarify object classification docs
* add debug logs for individual lpr replace_rules
* update memray docs
* memray tweaks
* Don't fail for audio transcription when semantic search is not enabled
* Fix incorrect mismatch for object vs sub label
* Check if the video is currently playing when deciding to seek due to misalignment
* Refactor timeline event handling to allow multiple timeline entries per update
* Check if zones have actually changed (not just count) for event state update
* show event icon on mobile
* move div inside conditional
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* attributes endpoint
* event endpoints
* add attributes to more filters
* add to suggestions and query in explore
* support attributes in search input
* i18n
* add object type filter to endpoint
* add attributes to tracked object details pane
* add generic multi select dialog
* save object attributes endpoint
* add group by param to fetch attributes endpoint
* add attribute editing to tracked object details
* docs
* fix docs
* update openapi spec to match python
* Fix saving zone friendly name when it wasn't set
* Fix UTF-8 handling for Onvif
* Don't remove none directory for classes
* Lookup all event IDs for review item immediately
* Cleanup typing
* Only fetch events when review group is open
* Cleanup
* disable debug paths switch for autotracking cameras
* fix clickable birdseye
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix history management failing when updating URL
* Handle case where user doesn't have images that represent all states
If a user selects all imags and can't proceed we show a warning that they can still proceed but the model won't be trained until they get at least one image for every state.
* Still create all classes
We stil need to create all classes even if the user didn't assign images to them.
* fix camera group access for non admin users
changes from previous PR wrongly included users from the standard viewer role (but excluded custom viewer roles)
* Adjust threat level interaction to be less strict
* use base path when fetching go2rtc data
* show config error message when starting in safe mode
* fix genai migration
* fix genai
* Fix genai migration
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* fix wording in reference config
* spacing tweaks
* make live view settings drawer scrollable
* clarify audio transcription docs
* change audio transcription icon to activity indicator when transcription is in progress
the backend doesn't implement any kind of queueing for speech event transcription
* tracking details tweaks
- Add attribute box overlay and area
- Add score
- Throttle swr revalidation during video component rerendering
* add mse codecs to console debug on errors
* add camera name
* Don't warn when event ids have expired for trigger sync
* Import faster_whisper conditinally to avoid illegal instruction
* Catch OpenVINO runtime error
* fix race condition in detail stream context
navigating between tracked objects in Explore would sometimes prevent the object track from appearing
* Handle case where classification images are deleted
* Adjust default rounded corners on larger screens
* Improve flow handling for classification state
* Remove images when wizard is cancelled
* Improve deletion handling for classes
* Set constraints on review buffers
* Update to support correct data format
* Set minimum duration for recording based review items
* Use friendly name in review genai prompt
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Fix filtering for classification
* Adjust prompt to account for response tokens
* Correctly return response for reprocess
* Use API response to update data instead of trying to re-parse all of the values
* Implement rename class api
* Fix model deletion / rename dialog
* Remove camera spatial context
* Catch error
* remove frigate+ icon from explore grid footer
* add margin
* pointer cursor on event menu items in detail stream
* don't show submit to plus for non-objects and if plus is disabled
* tweak spacing in annotation settings popover
* Fix deletion of classification images and library
* Ensure after creating a class that things are correct
* Fix dialog getting stuck
* Only show the genai summary popup on mobile when timeline is open
* fix audio transcription embedding
* spacing
* hide x icon on restart sheet to prevent closure issues
* prevent x overflow in detail stream on mobile safari
* ensure name is valid for search effect trigger
* add trigger to detail actions menu
* move find similar to actions menu
* Use a column layout for MobilePageContent in PlatformAwareSheet
This is so the header is outside the scrolling area and the content can grow/scroll independently. This now matches the way it's done in classification
* Skip azure execution provider
* add optional ref to always scroll to top
the more filters in explore was not scrolled to the top on open due to the use of framer motion
* fix title classes on desktop
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Correctly remove classification model from config
* Undo
* fix
* Use existing config update API and dynamically remove models that were running
* Set update message for face
* Implement extraction of images for classification state models
* Add object classification dataset preparation
* Add first step wizard
* Update i18n
* Add state classification image selection step
* Improve box handling
* Add object selector
* Improve object cropping implementation
* Fix state classification selection
* Finalize training and image selection step
* Cleanup
* Design optimizations
* Cleanup mobile styling
* Update no models screen
* Cleanups and fixes
* Fix bugs
* Improve model training and creation process
* Cleanup
* Dynamically add metrics for new model
* Add loading when hitting continue
* Improve image selection mechanism
* Remove unused translation keys
* Adjust wording
* Add retry button for image generation
* Make no models view more specific
* Adjust plus icon
* Adjust form label
* Start with correct type selected
* Cleanup sizing and more font colors
* Small tweaks
* Add tips and more info
* Cleanup dialog sizing
* Add cursor rule for frontend
* Cleanup
* remove underline
* Lazy loading
* Update classification API docs
* Add information to events api
* Fix tag
* Add exports
* Add generic response to model for classification apis
* Add preview API information
* Cleanup
* Cleanup
* Implement model training via ZMQ and add model states to represent training
* Get model updates working
* Improve toasts and model state
* Clean up logging
* Add back in
* Setup basic training structure
* Build out route
* Handle model configs
* Add image fetch APIs
* Implement model training screen with dataset selection
* Implement viewing of training images
* Adjust directories
* Implement viewing of images
* Add support for deleting images
* Implement full deletion
* Implement classification model training
* Improve naming
* More renaming
* Improve layout
* Reduce logging
* Cleanup
* install new packages for transcription support
* add config options
* audio maintainer modifications to support transcription
* pass main config to audio process
* embeddings support
* api and transcription post processor
* embeddings maintainer support for post processor
* live audio transcription with sherpa and faster-whisper
* update dispatcher with live transcription topic
* frontend websocket
* frontend live transcription
* frontend changes for speech events
* i18n changes
* docs
* mqtt docs
* fix linter
* use float16 and small model on gpu for real-time
* fix return value and use requestor to embed description instead of passing embeddings
* run real-time transcription in its own thread
* tweaks
* publish live transcriptions on their own topic instead of tracked_object_update
* config validator and docs
* clarify docs
* fix i18n keys
* hide disable from context menu for viewers
* Fix auto live check for default dashboard and camera groups
Disabling the Automatic Live View switch in Settings should prevent streaming from occurring. Overriding any settings in a camera group will override the global setting. The check here incorrectly always returned false instead of undefined.
* clarify hardware accelerated enrichments
* clarify
* add note about detect stream to face rec docs
* add note about low end Dahuas for autotracking
* Catch invalid face box / image
* Video tab tweaks
With the changes in https://github.com/blakeblackshear/frigate/pull/18220, the video tab in the Tracked Object Details pane now correctly trims the in-browser HLS video. Because of keyframes and record/detect stream differences, we can manually subtract a couple of seconds from the event start_time to ensure the first few frames aren't cut off from the video
* Clarify
* Don't use Migraphx by default
* Provide better support for running embeddings on GPU
* correctly join cameras
* Adjust blur confidence reduction
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Fix the `Any` typing hint treewide
There has been confusion between the Any type[1] and the any function[2]
in typing hints.
[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any
* Fix typing for various frame_shape members
Frame shapes are most likely defined by height and width, so a single int
cannot express that.
* Wrap gpu stats functions in Optional[]
These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.
* Fix return type in get_latest_segment_datetime
Returns a datetime object, not an integer.
* Make the return type of FrameManager.write optional
This is necessary since the SharedMemoryFrameManager.write function can
return None.
* Fix total_seconds() return type in get_tz_modifiers
The function returns a float, not an int.
https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds
* Account for floating point results in to_relative_box
Because the function uses division the return types may either be int or
float.
* Resolve ruff deprecation warning
The config has been split into formatter and linter, and the global
options are deprecated.
* face library i18n fixes
* face library i18n fixes
* add ability to use ctrl/cmd S to save in the config editor
* Use datetime as ID
* Update metrics inference speed to start with 0 ms
* fix android formatted thumbnail
* ensure role is comma separated and stripped correctly
* improve face library deletion
- add a confirmation dialog
- add ability to select all / delete faces in collections
* Implement lazy loading for video previews
* Force GPU for large embedding model
* GPU is required
* settings i18n fixes
* Don't delete train tab
* webpush debugging logs
* Fix incorrectly copying zones
* copy path data
* Ensure that cache dir exists for Frigate+
* face docs update
* Add description to upload image step to clarify the image
* Clean up
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Correctly ensure updates are more periodic when lpr or face detection is needed
* Cleanup
* Update api schema
* Don't update for stationary objects
* Simplify check
* Remove
* Section faces by event id
* Make score keeping more robust
* layout improvements
* Cleanup dialog
* Fix clicking behavior
* Add view in explore option
* math.round
* Don't require events
* Cleanup
* Remove selection
* Don't require
* Change dialog size with snapshot
* Use filename as key
* fix key
* Rework layout for mobile
* Handle mobile landscape
* Fix train issue
* Match logic
* Move deletion logic
* Fix reprocessing
* Support creating a new face
* Translations
* Do sorting in frontend
* Adjust unknown
* Cleanup
* Set max limit to faces to recognize
* Fix sorting
* Fix
* Don't assume landmark file is downloaded
* Rewrite build model task to be asynchronous so it doesn't block the pipeline
* Handle case where face recognition does not respond
* Cleanup
* Make daemon thread
* Ensure doesn't fail due to missing dir
* Remove redundant settings from tabs
* Adjust selection method for mobile
* Fix button descendent error
* Ensure train is option on mobile
* Cleanup face images
* Cleanup
* Add api to run face recognition on image
* Rework save attempts option
* Cleanup mobile object pane buttons
* Adjust api signature
* Remove param
* Cleanup
* Set maximum number of face images to be kept
* Fix vertical camera scaling
* adjust wording
* Add attributes to search data
* Add button to train face from event
* Handle event id saving in API
* db migration
* db model
* assign admin role on password reset
* add role to jwt and api responses
* don't restrict api access for admins yet
* use json response
* frontend auth context
* update auth form for profile endpoint
* add access denied page
* add protected routes
* auth hook
* dialogs
* user settings view
* restrict viewer access to settings
* restrict camera functions for viewer role
* add password dialog to account menu
* spacing tweak
* migrator default to admin
* escape quotes in migrator
* ui tweaks
* tweaks
* colors
* colors
* fix merge conflict
* fix icons
* add api layer enforcement
* ui tweaks
* fix error message
* debug
* clean up
* remove print
* guard apis for admin only
* fix tests
* fix review tests
* use correct error responses from api in toasts
* add role to account menu
* recordings data pub/sub
* function to process recording stream frames
* model runner
* lpr model runner
* refactor to mixin class and use model runner
* separate out realtime and post processors
* move model and mixin folders
* basic postprocessor
* clean up
* docs
* postprocessing logic
* clean up
* return none if recordings are disabled
* run postprocessor handle_requests too
* tweak expansion
* add put endpoint
* postprocessor tweaks with endpoint
* Actually send result to face registration
* Define postprocessing api and move face processing to fit
* Standardize request handling
* Standardize handling of processors
* Rename processing metrics
* Cleanup
* Standardize object end
* Update to newer formatting
* One more
* One more
