* fix wording in reference config
* spacing tweaks
* make live view settings drawer scrollable
* clarify audio transcription docs
* change audio transcription icon to activity indicator when transcription is in progress
the backend doesn't implement any kind of queueing for speech event transcription
* tracking details tweaks
- Add attribute box overlay and area
- Add score
- Throttle swr revalidation during video component rerendering
* add mse codecs to console debug on errors
* add camera name
* update live view docs
* use swr as single source of truth for searchDetail
rather than maintaining a separate state, derive the selected item from swr cache. fixes websocket sync when regenerating descriptions or fetching transcriptions
* fix key warning in console
* don't try to fetch event from review item for audio events
* update audio transcription toast wording
* Add a community supported badge to specific detectors in the info summaries to better separate
* Make object classification publish to tracked object update and add examples for state classification
* Add item to advanced docs about tensorflow limiting
* Don't show submission for in progress objects
* fix for ios not reporting video dimensions on initial metadata load
in testing, polling with requestAnimationFrame finds the dimensions within 2 frames
* Catch jetson nvidia device tree
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Include DB in safe mode config
Copy DB when going into safe mode to avoid creating a new one if a user has configured a separate location
* Fix documentation for example log module
* Set minimum duration for recording segments
Due to the inpoint logic, some recordings would get clipped on the end of the segment with a non-zero duration but not enough duration to include a frame. 100 ms is a safe value for any video that is 10fps or higher to have a frame
* Add docs to explain object assignment for classification
* Add warning for Intel GPU stats bug
Add warning with explanation on GPU stats page when all Intel GPU values are 0
* Update docs with creation instructions
* reset loading state when moving through events in tracking details
* disable pip on preview players
* Improve HLS handling for startPosition
The startPosition was incorrectly calculated assuming continuous recordings, when it needs to consider only some segments exist. This extracts that logic to a utility so all can use it.
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* ensure viewer roles are available in create user dialog
* admin-only endpoint to return unmaksed camera paths and go2rtc streams
* remove camera edit dropdown
pushing camera editing from the UI to 0.18
* clean up camera edit form
* rename component for clarity
CameraSettingsView is now CameraReviewSettingsView
* Catch case where user requsts clip for time that has no recordings
* ensure emergency cleanup also sets has_clip on overlapping events
improves https://github.com/blakeblackshear/frigate/discussions/20945
* use debug log instead of info
* update docs to recommend tmpfs
* improve display of in-progress events in explore tracking details
* improve seeking logic in tracking details
mimic the logic of DynamicVideoController
* only use ffprobe for duration to avoid blocking
fixes https://github.com/blakeblackshear/frigate/discussions/20737#discussioncomment-14999869
* Revert "only use ffprobe for duration to avoid blocking"
This reverts commit 8b15078005.
* update readme to link to object detector docs
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* don't flatten the search result cache when updating
this would cause an infinite swr fetch if something was mutated and then fetch was called again
* Properly sort keys for recording summary in StorageMetrics
* tracked object description box tweaks
* Remove ability to right click on elements inside of face popup
* Update reprocess message
* don't show object track until video metadata is loaded
* fix blue line height calc for in progress events
* Use timeline tab by default for notifications but add a query arg for customization
* Try and improve notification opening behavior
* Reduce review item buffering behavior
* ensure logging config is passed to camera capture and tracker processes
* ensure on demand recording stops when browser closes
* improve active line progress height with resize observer
* remove icons and duplicate find similar link in explore context menu
* fix for initial broken image when creating trigger from explore
* display friendly names for triggers in toasts
* lpr and triggers docs updates
* remove icons from dropdowns in face and classification
* fix comma dangle linter issue
* re-add incorrectly removed face library button icons
* fix sidebar nav links on < 768px desktop layout
* allow text to wrap on mark as reviewed button
* match exact pixels
* clarify LPR docs
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* digest auth backend
* frontend
* i18n
* update field description language to include note about onvif specific credentials
* mask util helper function
* language
* mask passwords in http-flv and others where a url param is password
* backend api endpoint
* don't add no-credentials version of streams to rtsp_candidates
* frontend types
* improve types
* add optional probe dialog to wizard step 1
* i18n
* form description and field change
* add onvif form description
* match onvif probe pane with other steps in the wizard
* refactor to add probe and snapshot as step 2
* consolidate probe dialog
* don't change dialog size
* radio button style
* refactor to select onvif urls via combobox in step 3
* i18n
* add scrollbar container
* i18n cleanup
* fix button activity indicator
* match test parsing in step 3 with step 2
* hide resolution if both width and height are zero
* use drawer for stream selection on mobile in step 3
* suppress double toasts
* api endpoint description
* Don't warn when event ids have expired for trigger sync
* Import faster_whisper conditinally to avoid illegal instruction
* Catch OpenVINO runtime error
* fix race condition in detail stream context
navigating between tracked objects in Explore would sometimes prevent the object track from appearing
* Handle case where classification images are deleted
* Adjust default rounded corners on larger screens
* Improve flow handling for classification state
* Remove images when wizard is cancelled
* Improve deletion handling for classes
* Set constraints on review buffers
* Update to support correct data format
* Set minimum duration for recording based review items
* Use friendly name in review genai prompt
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Fix filtering for classification
* Adjust prompt to account for response tokens
* Correctly return response for reprocess
* Use API response to update data instead of trying to re-parse all of the values
* Implement rename class api
* Fix model deletion / rename dialog
* Remove camera spatial context
* Catch error
* show id field when editing zone
* improve zone capitalization
* Update NPU models and docs
* fix mobilepage in tracked object details
* Use thread lock for openvino to avoid concurrent requests with JinaV2
* fix hashing function to avoid collisions
* remove extra flex div causing overflow
* ensure header stays on top of video controls
* don't smart capitalize friendly names
* Fix incorrect object classification crop
* don't display submit to plus if object doesn't have a snapshot
* check for snapshot and clip in actions menu
* frigate plus submission fix
still show frigate+ section if snapshot has already been submitted and run optimistic update, local state was being overridden
* Don't fail to show 0% when showing classification
* Don't fail on file system error
* Improve title and description for review genai
* fix overflowing truncated review item description in detail stream
* catch events with review items that start after the first timeline entry
review items may start later than events within them, so subtract a padding from the start time in the filter so the start of events are not incorrectly filtered out of the list in the detail stream
* also pad on review end_time
* fix
* change order of timeline zoom buttons on mobile
* use grid to ensure genai title does not cause overflow
* small tweaks
* Cleanup
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* feat: add zones friendly name
* fix: fix the issue where the input field was empty when there was no friendly_name
* chore: fix the issue where the friendly name would replace spaces with underscores
* docs: update zones docs
* Update web/src/components/settings/ZoneEditPane.tsx
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add friendly_name option for zone configuration
Added optional friendly name for zones in configuration.
* fix: fix the logical error in the null/empty check for the polygons parameter
* fix: remove the toast name for zones will use the friendly_name instead
* docs: remove emoji tips
* revert: revert zones doc ui tips
* Update docs/docs/configuration/zones.md
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Update docs/docs/configuration/zones.md
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Update docs/docs/configuration/zones.md
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* feat: add friendly zone names to tracking details and lifecycle item descriptions
* chore: lint fix
* refactor: add friendly zone names to timeline entries and clean up unused code
* refactor: add formatList
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* remove frigate+ icon from explore grid footer
* add margin
* pointer cursor on event menu items in detail stream
* don't show submit to plus for non-objects and if plus is disabled
* tweak spacing in annotation settings popover
* Fix deletion of classification images and library
* Ensure after creating a class that things are correct
* Fix dialog getting stuck
* Only show the genai summary popup on mobile when timeline is open
* fix audio transcription embedding
* spacing
* hide x icon on restart sheet to prevent closure issues
* prevent x overflow in detail stream on mobile safari
* ensure name is valid for search effect trigger
* add trigger to detail actions menu
* move find similar to actions menu
* Use a column layout for MobilePageContent in PlatformAwareSheet
This is so the header is outside the scrolling area and the content can grow/scroll independently. This now matches the way it's done in classification
* Skip azure execution provider
* add optional ref to always scroll to top
the more filters in explore was not scrolled to the top on open due to the use of framer motion
* fix title classes on desktop
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Correctly sort summary responses
* Consider JinaV2 as a complex model
* Subscribe to record updates in camera watchdog
* Cleanup score showing
* No need to sort review summary
* Add tests for recording summary
* Don't break existing format
* Sort event summary by day
* Don't show model selection or back button when in multi select mode
* Add dialog to edit classification models
* Fix header spacing
* Cleanup desktop
* Incrase max number of object classifications
* fix iOS mobile card
* Cleanup
* Correctly remove classification model from config
* Undo
* fix
* Use existing config update API and dynamically remove models that were running
* Set update message for face
* Pull context size from openai models
* Adjust wording based on type of model
* Instruct to not use parenthesis
* Simplify genai config
* Don't use GPU for training
* Update docs for review descriptions
* Add logging for context tokens used
* Incrase number of images due to lower than expected context usage
* Re-balance the suspicious activity checks
* Adjustments to context sizing
* optimize context usage
* Adjust context usage
* Make title more direct
* Update docs
* Adjust for commutes
* Tweaks
* Don't show no models view in grid
* Add text-md to inputs
* Adjust train title for mobile
* Cleanup prompt more
* Use i18n functions for tooltip
* Fix model complexity causing crash
* Cleanup
* black background
* fix backend logic
* fixes
* ensure data being sent to api is segment aligned
* tweak
* tweaks to keep motion review as-is
* fix for half segment fractional seconds when using zooming
* Make prompt more fair and reduce time extension
* Adjust naming of unrecognized objects
* Improve object naming behavior
* Add more context image levels
* tweak api to fetch multiple timelines
* support multiple selected objects in context
* rework context provider
* use toggle in detail stream
* use toggle in menu
* plot multiple object tracks
* verified icon, recognized plate, and clicking tweaks
* add plate to object lifecycle
* close menu before opening frigate+ dialog
* clean up
* normal text case for tooltip
* capitalization
* use flexbox for recording view
* Implement extraction of images for classification state models
* Add object classification dataset preparation
* Add first step wizard
* Update i18n
* Add state classification image selection step
* Improve box handling
* Add object selector
* Improve object cropping implementation
* Fix state classification selection
* Finalize training and image selection step
* Cleanup
* Design optimizations
* Cleanup mobile styling
* Update no models screen
* Cleanups and fixes
* Fix bugs
* Improve model training and creation process
* Cleanup
* Dynamically add metrics for new model
* Add loading when hitting continue
* Improve image selection mechanism
* Remove unused translation keys
* Adjust wording
* Add retry button for image generation
* Make no models view more specific
* Adjust plus icon
* Adjust form label
* Start with correct type selected
* Cleanup sizing and more font colors
* Small tweaks
* Add tips and more info
* Cleanup dialog sizing
* Add cursor rule for frontend
* Cleanup
* remove underline
* Lazy loading
* Add cutoff for object classification
* Add selector for classifiction model type
* Improve model selection view
* Clean up design of classification card
* Tweaks
* Adjust button colors
* Improvements to gradients and making face library consistent
* Add basic classification model wizard
* Use relative coordinates
* Properly get resolution
* Clean up exports
* Cleanup
* Cleanup
* Update to use pre-defined component for image shadow
* Refactor image grouping
* Clean up mobile
* Clean up decision logic
* Remove max check on classification objects
* Increase default number of faces shown
* Cleanup
* Improve mobile layout
* Clenaup
* Update vocabulary
* Fix layout
* Fix page
* Cleanup
* Choose last item for unknown objects
* Move explore button
* Cleanup grid
* Cleanup classification
* Cleanup grid
* Cleanup
* Set transparency
* Set unknown
* Don't filter all configs
* Check length
* Add optional idle heartbeat for Birdseye (periodic frame emission when idle)
birdseye: add optional idle heartbeat and FFmpeg tuning envs (default off)
This adds an optional configuration field `birdseye.idle_heartbeat_fps` to
enable a lightweight idle heartbeat mechanism in Birdseye. When set to a value
greater than 0, Birdseye periodically re-sends the last composed frame during
idle periods (no motion or active updates).
This helps downstream consumers such as go2rtc, Alexa, or Scrypted to attach
faster and maintain a low-latency RTSP stream when the system is idle.
Key details:
- Config-based (`birdseye.idle_heartbeat_fps`), default `0` (disabled).
- Uses existing Birdseye rendering pipeline; minimal performance impact.
- Does not alter behavior when unset.
Documentation: added tip section in docs/configuration/restream.md.
* Update docs/docs/configuration/restream.md
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Update docs/docs/configuration/reference.md
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Refactors Birdseye idle frame broadcasting
Simplifies the idle frame broadcasting logic by removing the dedicated thread.
The idle frame is now resent directly within the main loop,
improving efficiency and reducing complexity. Also, limits the idle
heartbeat FPS to a maximum of 10 since the framebuffer is limited to 10 anyway
* ruff fix
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
Co-authored-by: Francesco Durighetto <francesco.durighetto@subbyx.com>
Co-authored-by: duri <duri@homelabubuntu.durihome.unifi>
* save clean webp instead of png
* send clean webp to plus with fallback for old events
* manual event webp
* event cleanup
* api def
* convert png to webp if exists
* update reference config
* change quality
* Migrate camera APIs to separate tag
* Implement reolink detection to handle dynamic URL assignment
* Cleanup codec handling
* Use average framerate not relative framerate
* Add reolink rtsp warning
* Don't return exception
* Use avg_frame_rate in final info
* Clenaup
* Validate host
* Fix overlap
* fetch more from ffprobe
* add detailed param to ffprobe endpoint
* add dots variant to step indicator
* add classname
* tweak colors for dark mode to match figma
* add step 1 form
* add helper function for ffmpeg snapshot
* add go2rtc stream add and ffprobe snapshot endpoints
* add camera image and stream details on successful test
* step 1 tweaks
* step 2 and i18n
* types
* step 1 and 2 tweaks
* add wizard to camera settings view
* add data unit i18n keys
* restream tweak
* fix type
* implement rough idea for step 3
* add api endpoint to delete stream from go2rtc
* add main wizard dialog component
* extract logic for friendly_name and use in wizard
* add i18n and popover for brand url
* add camera name to top
* consolidate validation logic
* prevent dialog from closing when clicking outside
* center camera name on mobile
* add help/docs link popovers
* keep spaces in friendly name
* add stream details to overlay like stats in liveplayer
* add validation results pane to step 3
* ensure test is invalidated if stream is changed
* only display validation results and enable save button if all streams have been tested
* tweaks
* normalize camera name to lower case and improve hash generation
* move wizard to subfolder
* tweaks
* match look of camera edit form to wizard
* move wizard and edit form to its own component
* move enabled/disabled switch to management section
* clean up
* fixes
* fix mobile
* new body param
* use new body param in endpoint
* explicitly use new param in frontend endpoint
* use reviewsegment as type instead of list of strings
* add toggle function to mark as unreviewed when all selected are reviewed
* i18n
* fix tests
* Map verified objects to their sub label directly
* Simplify access
* Cleanup
* Add protection for mismatched object and index
* Keep track of verified objects separately
* camera level config
* set up model runner on thread start to avoid unpickling error
* ensure feature is enabled globally
* suppress info logs from faster_whisper
* fix incorrect event_type for api and audio timeline entries
* docs
* fix
* clean up
* Update classification API docs
* Add information to events api
* Fix tag
* Add exports
* Add generic response to model for classification apis
* Add preview API information
* Cleanup
* Cleanup
* Refactor face card into generic classification card
* Update classification data card to use classification card
* Refactor state training grid to use classification card
* Refactor grouped face card into generic component
* Combine classification objects by event
* Fixup
* Cleanup
* Cleanup
* Do not fail if a single event is not found
* Save original frame
* Cleanup
* Undo
* Improve prompt to have better discernment and logic based on detected objects
* Be more specific about the time of day
* Add re-inforcers for LLM to be accurate and not complete a narrative
* refactor get_video_properties and use json output from ffprobe
* add zmq topic
* publish valid segment data in recording maintainer
* check for valid video data
- restart separate record ffmpeg process if no video data has been received in 120s
- refactor datetime import
* listen to correct topic in embeddings maintainer
* refactor to move get_latest_segment_datetime logic to recordings maintainer
* debug logging
* cleanup
* Update ROCm to 7.0.1
* Update ONNXRuntime
* Add back in
* Get basic detection working
* Use env vars
* Handle complex migraphx models
* Enable model caching
* Remove unused
* Add tip to docs
* [Init] Initial commit for Synaptics SL1680 NPU
* add a rough detector which is testing with yolov8 tflite model.
* [Feat] Add dependencies installation in docker build
- Add runtime library and wheels installation in main/Dockerfile
- Add model.synap(default model, transfer from mobilenet_224full80) in docker/synap1680
* [Update] Remove dependencies installation from main Dockerfile
- remove deps installation from Dockerfile
- add dependencies installation and split wheels, deps stage in synap1680 Dockerfile
* Refactor synap detector to more closely match other implementations
* [Update] Add model path configuration check
* [Update] update ModelType to ssd
* [Update] Remove unuse script
- install_deps.sh has already been executing in deps download stage
- Dockerfile.toolchain is for testing to extract runtime libraries from Synaptics toolchain
* [Update] update Synaptics SL1680 setup description
* [Update] remove install_synap1680
- The deps download and installation is existed in synap1680
* [Fix] update document content
* [Update] Update detector from synap1680 to synaptics
This update is in order to make the synaptics SL-series NPU detector more general.
- Fix detector `os` module not import bug
- Update detector type `synap1680` to `synaptics`
- Update document description `SL1680` to `Synaptics` only
- Update docker build content `synap1680` to `synaptics`
* [Fix] Update configuration document
* Update docs/docs/configuration/object_detectors.md
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* [Update] Update document content and detector default layout
- Update object_detectors document
- Update detector's default layout
- Update default model name
* [Update] Update object detector document content
* [Fix] Fix InputTensorEnum not defined error
- import InputTensorEnum from detector_config
* [Update] Update detector script coding format
* [Update] Update synaptics detector coding format
* [Update] Add synaptics ci workflow
* [Update] update synaptics runtime libs download path
- Fork Synaptics astra sdk repo and put the runtime lib package on it
- Frigate team can update this download path later
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Implement stationary car classifier to base stationary state on visual changes and not just bounding box stability
* Cleanup
* Fix mypy
* Move to new file and add config to disable if needed
* Cleanup
* Undo
This supports systemd credentials, see https://systemd.io/CREDENTIALS/.
Default to `/run/secrets` (the Docker Secrets dir) for backwards
compatibility.
* pull count of detection events by label into prometheus metrics
* format changes with ruff
* remove unneeded f-string
* fix imports format
---------
Co-authored-by: iesad <iesad>
* continue to use paddleocr v3 text detection model for large
v5 was not finding text on multi-line plates at all in testing
* implement clustering of plate variants per event
should reduce OCR inconsistencies and improve plate recognition stability by using string similarity to cluster similar variants (10 per event id) and choosing the highest confidence representative as the final plate
* pass camera
* prune number of variants based on detect fps
* implement replacement rules for cleaning up and normalizing plates
* docs
* docs
* Cleanup onnx detector
* Fix
* Fix classification cropping
* Deprioritize openvino
* Send model type
* Use model type to decide if model can use full optimization
* Clenanup
* Cleanup
* Use OpenVINO directly to detect if devices are available
* Cleanup
* Update OpenVINO
* Cleanup
* Don't try to use OpenVINO when CPU is set as device
* Catch case where input tensor can't be pre-defined
* Cleanup
* Use re-usable inference request to reduce CPU usage
* Share tensor
* Don't count performance
* Create openvino runner class
* Break apart onnx runner
* Add specific note about inability to use CUDA graphs for some models
* Adjust rknn to use RKNNRunner
* Use optimized runner
* Add support for non-complex models for CudaExecutionProvider
* Use core mask for rknn
* Correctly handle cuda input
* Cleanup
* Sort imports
* update config for roles and add validator
* ensure admin and viewer are never overridden
* add class method to user to retrieve all allowed cameras
* enforce config roles in auth api endpoints
* add camera access api dependency functions
* protect review endpoints
* protect preview endpoints
* rename param name for better fastapi injection matching
* remove unneeded
* protect export endpoints
* protect event endpoints
* protect media endpoints
* update auth hook for allowed cameras
* update default app view
* ensure anonymous user always returns all cameras
* limit cameras in explore
* cameras is already a list
* limit cameras in review/history
* limit cameras in live view
* limit cameras in camera groups
* only show face library and classification in sidebar for admin
* remove check in delete reviews
since admin role is required, no need to check camera access. fixes failing test
* pass request with camera access for tests
* more async
* camera access tests
* fix proxy auth tests
* allowed cameras for review tests
* combine event tests and refactor for camera access
* fix post validation for roles
* don't limit roles in create user dialog
* fix triggers endpoints
no need to run require camera access dep since the required role is admin
* fix type
* create and edit role dialogs
* delete role dialog
* fix role change dialog
* update settings view for roles
* i18n changes
* minor spacing tweaks
* docs
* use badges and camera name label component
* clarify docs
* display all cameras badge for admin and viewer
* i18n fix
* use validator to prevent reserved and empty roles from being assigned
* split users and roles into separate tabs in settings
* tweak docs
* clarify docs
* change icon
* don't memoize roles
always recalculate on component render
* Use asyncio lock when checking camera status
get_camera_status() can be called during normal autotracking movement and from routine camera_maintenance(). Some cameras cause one of the status calls to hang, which then subsequently hangs autotracking. A lock serializes access and prevents the hang.
* use while loop in camera_maintenance for status check
some cameras seem to take a little bit to update their status, don't assume the first call shows the motor has stopped
* Refactor active objects to class
* Keep segment going when detection is newer than end of alert
* Cleanup logic
* Fix
* Cleanup ending
* Adjust timing
* Improve detection saving
* Don't have padding at end for in progress reviews
* Add review config for cutoff times
* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless
* Fixed broken link
* Made it so openvino prioritizes using GPU and NPU over CPU
* Version that detects model and can begin using @local
* Updating requirements to build dev container
* Added optimized version of degirum plugin + updated docs
* Added guard clause for empty inference reponse
* Updated DeGirum's docs
* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder
* Moved DeGirum section to correct place in community models
* Update ROCm to 6.4.0 (#18264)
* Update to rocm 6.4.0
* Update URL
* Remove old env var
* Dynamic Config Updates (#18353)
* Create classes to handle publishing and subscribing config updates
* Cleanup
* Use config updater
* Update handling for enabled config
* Cleanup
* Recording config updates
* Birdseye config updates
* Handle notifications
* handle review
* Update motion
* Dynamically update masks and zones for cameras (#18359)
* Include config publisher in api
* Call update topic for passed topics
* Update zones dynamically
* Update zones internally
* Support zone and mask reset
* Handle updating objects config
* Don't put status for needing to restart Frigate
* Cleanup http tests
* Fix tests
* Initial custom classification model config support (#18362)
* Add basic config for defining a teachable machine model
* Add model type
* Add basic config for teachable machine models
* Adjust config for state and object
* Use config to process
* Correctly check for objects
* Remove debug
* Rename to not be teachable machine specific
* Cleanup
* Implement support for no recordings indicator on timeline (#18363)
* Indicate no recordings on the history timeline with gray hash marks
This commit includes a new backend API endpoint and the frontend changes needed to support this functionality
* don't show slashes for now
* Update ROCm to 6.4.1 (#18364)
* Update rocm to 6.4.1
* Quick fix
* Add ability to configure when custom classification models run (#18380)
* Add config to control when classification models are run
* Cleanup
* Add basic config editor when Frigate can't startup (#18383)
* Start Frigate in safe mode when config does not validate
* Add safe mode page that is just the config editor
* Adjust Frigate config editor when in safe mode
* Cleanup
* Improve log message
* Fix incorrectly running lpr (#18390)
* Audio transcription support (#18398)
* install new packages for transcription support
* add config options
* audio maintainer modifications to support transcription
* pass main config to audio process
* embeddings support
* api and transcription post processor
* embeddings maintainer support for post processor
* live audio transcription with sherpa and faster-whisper
* update dispatcher with live transcription topic
* frontend websocket
* frontend live transcription
* frontend changes for speech events
* i18n changes
* docs
* mqtt docs
* fix linter
* use float16 and small model on gpu for real-time
* fix return value and use requestor to embed description instead of passing embeddings
* run real-time transcription in its own thread
* tweaks
* publish live transcriptions on their own topic instead of tracked_object_update
* config validator and docs
* clarify docs
* Implement API to train classification models (#18475)
* Intel updates (#18493)
* Update openvino and onnxruntime
* Install icd and level-zero-gpu deps from intel directly
* Install
* Add dep
* Fix package install
* Tiered recordings (#18492)
* Implement tiered recording
* Add migration for record config
* Update docs
* Update reference docs
* Fix preview query
* Fix incorrect accesses
* Fix
* Fix
* Fix
* Fix
* Upgrade PaddleOCR models to v4 (rec) and v5 (det) (#18505)
The PP_OCRv5 text detection models have greatly improved over v3. The v5 recognition model makes improvements to challenging handwriting and uncommon characters, which are not necessary for LPR, so using v4 seemed like a better choice to continue to keep inference time as low as possible. Also included is the full dictionary for Chinese character support.
* Audio transcription tweaks (#18540)
* use model runner
* unload whisper model when live transcription is complete
* Classification Model UI (#18571)
* Setup basic training structure
* Build out route
* Handle model configs
* Add image fetch APIs
* Implement model training screen with dataset selection
* Implement viewing of training images
* Adjust directories
* Implement viewing of images
* Add support for deleting images
* Implement full deletion
* Implement classification model training
* Improve naming
* More renaming
* Improve layout
* Reduce logging
* Cleanup
* Live classification model training (#18583)
* Implement model training via ZMQ and add model states to represent training
* Get model updates working
* Improve toasts and model state
* Clean up logging
* Add back in
* Classification Model Metrics (#18595)
* Add speed and rate metrics for custom classification models
* Use metrics for classification models
* Use keys
* Cast to list
* Add Mesa Teflon as a TFLite detector (#18310)
* Refactor common functions for tflite detector implementations
* Add detector using mesa teflon delegate
Non-EdgeTPU TFLite can use the standard .tflite format
* Add mesa-teflon-delegate from bookworm-backports to arm64 images
* feat: enable using GenAI for cameras with GenAI disabled from the API (#18616)
* fix: Initialize GenAI client if GenAI is enabled globally (#18623)
* Make Birdseye clickable (#18628)
* keep track of layout changes and publish on change
* websocket hook
* clickable overlay div to navigate to full camera view
* Refactor TensorRT (#18643)
* Combine base and arm trt detectors
* Remove unused deps for amd64 build
* Add missing packages and cleanup ldconfig
* Expand packages for tensorflow model training
* Cleanup
* Refactor training to not reserve memory
* Dynamic Management of Cameras (#18671)
* Add base class for global config updates
* Add or remove camera states
* Move camera process management to separate thread
* Move camera management fully to separate class
* Cleanup
* Stop camera processes when stop command is sent
* Start processes dynamically when needed
* Adjust
* Leave extra room in tracked object queue for two cameras
* Dynamically set extra config pieces
* Add some TODOs
* Fix type check
* Simplify config updates
* Improve typing
* Correctly handle indexed entries
* Cleanup
* Create out SHM
* Use ZMQ for signaling object detectoin is completed
* Get camera correctly created
* Cleanup for updating the cameras config
* Cleanup
* Don't enable audio if no cameras have audio transcription
* Use exact string so similar camera names don't interfere
* Add ability to update config via json body to config/set endpoint
Additionally, update the config in a single rather than multiple calls for each updated key
* fix autotracking calibration to support new config updater function
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Use Fork-Server As Spawn Method (#18682)
* Set runtime
* Use count correctly
* Don't assume camera sizes
* Use separate zmq proxy for object detection
* Correct order
* Use forkserver
* Only store PID instead of entire process reference
* Cleanup
* Catch correct errors
* Fix typing
* Remove before_run from process util
The before_run never actually ran because:
You're right to suspect an issue with before_run not being called and a potential deadlock. The way you've implemented the run_wrapper using __getattribute__ for the run method of BaseProcess is a common pitfall in Python's multiprocessing, especially when combined with how multiprocessing.Process works internally.
Here's a breakdown of why before_run isn't being called and why you might be experiencing a deadlock:
The Problem: __getattribute__ and Process Serialization
When you create a multiprocessing.Process object and call start(), the multiprocessing module needs to serialize the process object (or at least enough of it to re-create the process in the new interpreter). It then pickles this serialized object and sends it to the newly spawned process.
The issue with your __getattribute__ implementation for run is that:
run is retrieved during serialization: When multiprocessing tries to pickle your Process object to send to the new process, it will likely access the run attribute. This triggers your __getattribute__ wrapper, which then tries to bind run_wrapper to self.
run_wrapper is bound to the parent process's self: The run_wrapper closure, when created in the parent process, captures the self (the Process instance) from the parent's memory space.
Deserialization creates a new object: In the child process, a new Process object is created by deserializing the pickled data. However, the run_wrapper method that was pickled still holds a reference to the self from the parent process. This is a subtle but critical distinction.
The child's run is not your wrapped run: When the child process starts, it internally calls its own run method. Because of the serialization and deserialization process, the run method that's ultimately executed in the child process is the original multiprocessing.Process.run or the Process.run if you had directly overridden it. Your __getattribute__ magic, which wraps run, isn't correctly applied to the Process object within the child's context.
* Cleanup
* Logging bugfix (#18465)
* use mp Manager to handle logging queues
A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.
* consolidate
* fix typing
* Fix typing
* Use global log queue
* Move to using process for logging
* Convert camera tracking to process
* Add more processes
* Finalize process
* Cleanup
* Cleanup typing
* Formatting
* Remove daemon
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add basic camera settings to UI for testing (#18690)
* add basic camera add/edit pane to the UI for testing
* only init model runner if transcription is enabled globally
* fix role checkboxes
* Ensure logging config is propagated to forked processes (#18704)
* Move log level initialization to log
* Use logger config
* Formatting
* Fix config order
* Set process names
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Fix go2rtc init (#18708)
* Cleanup process handling
* Adjust process name
* Reduce tf initialization
* Don't use staticmethod
* Don't fail on unicode debug for config updates
* Catch unpickling error
* Fix birdseye crash when dynamically adding a camera (#18821)
* Catch invalid character index in lpr CTC decoder (#18825)
* Classification model cover images (#18843)
* Move to separate component
* Add cover images for clssification models
* Fix process name
* Handle SIGINT with forkserver (#18860)
* Pass stopevent from main start
* Share stop event across processes
* preload modules
* remove explicit os._exit call
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Don't try to close or join mp manager queues (#18866)
Multiprocessing Manager queues don't have a close() or join_thread() method, and the Manager will clean it up appropriately after we empty it. This prevents an infinite loop when an AttributeError exception fires for Manager AutoProxy queue objects.
* Improve logging (#18867)
* Ignore numpy get limits warning
* Add function wrapper to redirect stdout and stderr to logpipe
* Save stderr too
* Add more to catch
* run logpipe
* Use other logging redirect class
* Use other logging redirect class
* add decorator for redirecting c/c++ level output to logger
* fix typing
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add ONVIF focus support (#18883)
* backend
* frontend and i18n
* 0.17 tweaks (#18892)
* Set version
* Cleanup more logs
* Don't log matplotlib
* Improve object classification (#18908)
* Ui improvements
* Improve image cropping and model saving
* Improve naming
* Add logs for training
* Improve model labeling
* Don't set sub label for none object classification
* Cleanup
* Remove TFLite init logs
* Improve classification UI (#18910)
* Move threhsold to base model config
* Improve score handling
* Add back button
* Classification improvements (#19020)
* Move classification training to full process
* Sort class images
* Semantic Search Triggers (#18969)
* semantic trigger test
* database and model
* config
* embeddings maintainer and trigger post-processor
* api to create, edit, delete triggers
* frontend and i18n keys
* use thumbnail and description for trigger types
* image picker tweaks
* initial sync
* thumbnail file management
* clean up logs and use saved thumbnail on frontend
* publish mqtt messages
* webpush changes to enable trigger notifications
* add enabled switch
* add triggers from explore
* renaming and deletion fixes
* fix typing
* UI updates and add last triggering event time and link
* log exception instead of return in endpoint
* highlight entry in UI when triggered
* save and delete thumbnails directly
* remove alert action for now and add descriptions
* tweaks
* clean up
* fix types
* docs
* docs tweaks
* docs
* reuse enum
* Optionally show tracked object paths in debug view (#19025)
* Dynamically enable/disable GenAI (#19139)
* config
* dispatcher and mqtt
* docs
* use config updater
* add switch to frontend
* Classification train updates (#19173)
* Improve model train button
* Add filters for classification
* Cleanup
* Don't run classification on false positives
* Cleanup filter
* Fix icon color
* Object attribute classification (#19205)
* Add enum for type of classification for objects
* Update recognized license plate topic to be used as attribute updater
* Update attribute for attribute type object classification
* Cleanup
* Require setting process priority for FrigateProcess (#19207)
* Add bookworm-backports to the rocm images and upgrade mesa/vaapi to support RDNA4 GPUs (#19312)
* Improve the tablet layout (#19320)
* Improve the tablet layout
* Update imports sort
* Fix more imports
* Implement start for review item description processor (#19352)
* Add review item data transmission
* Publish review updates
* Add review item subscriber
* Basic implementation for testing review processor
* Formatting
* Cleanup
* Improve comms typing (#18599)
* Enable mypy for comms
* Make zmq data types consistent
* Cleanup inter process typing issues
* Cleanup embeddings typing
* Cleanup config updater
* Cleanup recordings updator
* Make publisher have a generic type
* Cleanup event metadata updater
* Cleanup event metadata updater
* Cleanup detections updater
* Cleanup websocket
* Cleanup mqtt
* Cleanup webpush
* Cleanup dispatcher
* Formatting
* Remove unused
* Add return type
* Fix tests
* Fix semantic triggers config typing
* Cleanup
* Ensure alertVideos persistence is loaded before displaying thumb or preview (#19432)
The default value of true would cause previews to be loaded in the background even if the local storage value was false
* Adjust loitering behavior based on object type (#19433)
* Adjust loitering behavior based on object
* Update docs
* Grammar
* Enable mypy for DB and fix types (#19434)
* Install peewee type hints
* Models now have proper types
* Fix iterator type
* Enable debug builds with dev reqs installed
* Install as wheel
* Fix cast type
* Migrate object genai configuration (#19437)
* Move genAI object to objects section
* Adjust config propogation behavior
* Refactor genai config usage
* Automatic migration
* Always start the embeddings process
* Always init embeddings
* Config fixes
* Adjust reference config
* Adjust docs
* Formatting
* Fix
* Review Item GenAI metadata (#19442)
* Rename existing function
* Keep track of thumbnial updates
* Tinkering with genai prompt
* Adjust input format
* Create model for review description output
* testing prompt changes
* Prompt improvements and image saving
* Add config for review items genai
* Use genai review config
* Actual config usage
* Adjust debug image saving
* Fix
* Fix review creation
* Adjust prompt
* Prompt adjustment
* Run genai in thread
* Fix detections block
* Adjust prompt
* Prompt changes
* Save genai response to metadata model
* Handle metadata
* Send review update to dispatcher
* Save review metadata to DB
* Send review notification updates
* Quick fix
* Fix name
* Fix update type
* Correctly dump model
* Add card
* Add card
* Remove message
* Cleanup typing and UI
* Adjust prompt
* Formatting
* Add log
* Formatting
* Add inference speed and keep alive
* Review genai updates (#19448)
* Include extra level for normal activity
* Add dynamic toggling
* Update docs
* Add different threshold for genai
* Adjust webUI for object and review description feature
* Adjust config
* Send on startup
* Cleanup config setting
* Set config
* Fix config name
* Use preview frames for Review Descriptions (#19450)
* Use preview frames for genai
* Cleanup
* Adjust
* Add config for users to define additional concerns that GenAI should make note of in review summary (#19463)
* Don't default to openai
* Improve UI
* Allow configuring additional concerns that users may want the AI to note
* Formatting
* Add preferred language config
* Remove unused
* Added total camera fps, total processed fps, and total skipped fps to stats api (#19469)
Co-authored-by: Mark Francis <markfrancisonly@gmail.com>
* Genai review summaries (#19473)
* Generate review item summaries with requests
* Adjust logic to only send important items
* Don't mention ladder
* Adjust prompt to be more specific
* Add more relaxed nature for normal activity
* Cleanup summary
* Update ollama client
* Add more directions to analyze the frames in order
* Remove environment from prompt
* Add ability to pass additional args to Ollama (#19484)
* Call out recognized objects more specifically
* Cleanup
* Make keep_alive and options configurable
* Generalize
* Use for other providers
* Update GenAI docs for new review summaries feature (#19493)
* Remove old genai docs
* Separate existing genai docs to separate sections
* Add docs for genai features
* Update reference config
* Update link
* Move to bottom
* Improve natural language of prompt (#19515)
* Make sequence details human-readable so they are used in natural language response
* Cleanup
* Improve prompt and image selection
* Adjust
* Adjust sligtly
* Format time
* Adjust frame selection logic
* Debug save response
* Ignore extra fields
* Adjust docs
* Cleanup filename sanitization
* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless
* Fixed broken link
* Made it so openvino prioritizes using GPU and NPU over CPU
* Version that detects model and can begin using @local
* Added optimized version of degirum plugin + updated docs
* Updating requirements to build dev container
* Added guard clause for empty inference reponse
* Updated DeGirum's docs
* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder
* Moved DeGirum section to correct place in community models
* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless
* Fixed broken link
* Made it so openvino prioritizes using GPU and NPU over CPU
* Version that detects model and can begin using @local
* Added optimized version of degirum plugin + updated docs
* Updating requirements to build dev container
* Added guard clause for empty inference reponse
* Updated DeGirum's docs
* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder
* Moved DeGirum section to correct place in community models
* Added degirum plugin, updated documentation for degirum detector usage, updated requirements with degirum_headless
* Fixed broken link
* Made it so openvino prioritizes using GPU and NPU over CPU
* Version that detects model and can begin using @local
* Added optimized version of degirum plugin + updated docs
* Updating requirements to build dev container
* Added guard clause for empty inference reponse
* Updated DeGirum's docs
* Moved DeGirum section to 'Community' detectors, fixed formatting of headers to be more consistent with the rest of the page, and removed uneeded 'models' folder
* Moved DeGirum section to correct place in community models
* Reverted changes to classification and audio
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Co-authored-by: Jimmy <honj@alum.rpi.edu>
Co-authored-by: FL42 <46161216+fl42@users.noreply.github.com>
Co-authored-by: Steve Smith <tarkasteve@gmail.com>
Co-authored-by: markfrancisonly <12145270+markfrancisonly@users.noreply.github.com>
Co-authored-by: Mark Francis <markfrancisonly@gmail.com>
* refactor: Refactor camera nickname
* fix: fix cameraNameLabel visually
* chore: The Explore search function also displays the Camera's nickname in English
* chore: add mobile page camera nickname
* feat: webpush support camera nickname
* fix: fix storage camera name is null
* chore: fix review detail and context menu camera nickname
* chore: fix use-stats and notification setting camera nickname
* fix: fix stats camera if not nickname need capitalize
* fix: fix debug page open camera web ui i18n and camera nickname support
* fix: fix camera metrics not use nickname
* refactor: refactor use-camera-nickname hook.
* add regex support to events api for recognized_license_plate
* frontend
add ability to use regexes in the plate search box and add select all/clear all links to quickly select all filtered plates
* use custom swr fetcher to check for audio support
The go2rtc API doesn't always return stream data for anything not being actively consumed, so audio support was not always being correctly deduced. So we can use a custom swr fetcher to call the endpoint that probes the streams, which returns the correct producers data.
* return correct mime type for thumbnail and latest frame endpoints
follow up to https://github.com/blakeblackshear/frigate/pull/19555
* Implement base rknn conversion
* Remove unused
* Formatting
* Add model conversion lock so it doesn't break when multiple detectors are defined
* Ignore unused impor
t
* Fix content type for latest image API endpoint
Extension is an enum and .value needed to be appended. Additionally, fastapi's Response() automatically sets the content type when media_type is specified, so a Content-Type in the headers was redundant.
* Remove another unneeded Content-Type
* Make sequence details human-readable so they are used in natural language response
* Cleanup
* Improve prompt and image selection
* Adjust
* Adjust sligtly
* Format time
* Adjust frame selection logic
* Debug save response
* Ignore extra fields
* Adjust docs
* Generate review item summaries with requests
* Adjust logic to only send important items
* Don't mention ladder
* Adjust prompt to be more specific
* Add more relaxed nature for normal activity
* Cleanup summary
* Update ollama client
* Add more directions to analyze the frames in order
* Remove environment from prompt
* Don't default to openai
* Improve UI
* Allow configuring additional concerns that users may want the AI to note
* Formatting
* Add preferred language config
* Remove unused
* Include extra level for normal activity
* Add dynamic toggling
* Update docs
* Add different threshold for genai
* Adjust webUI for object and review description feature
* Adjust config
* Send on startup
* Cleanup config setting
* Set config
* Fix config name
* Install peewee type hints
* Models now have proper types
* Fix iterator type
* Enable debug builds with dev reqs installed
* Install as wheel
* Fix cast type
* Add enum for type of classification for objects
* Update recognized license plate topic to be used as attribute updater
* Update attribute for attribute type object classification
* Cleanup
* semantic trigger test
* database and model
* config
* embeddings maintainer and trigger post-processor
* api to create, edit, delete triggers
* frontend and i18n keys
* use thumbnail and description for trigger types
* image picker tweaks
* initial sync
* thumbnail file management
* clean up logs and use saved thumbnail on frontend
* publish mqtt messages
* webpush changes to enable trigger notifications
* add enabled switch
* add triggers from explore
* renaming and deletion fixes
* fix typing
* UI updates and add last triggering event time and link
* log exception instead of return in endpoint
* highlight entry in UI when triggered
* save and delete thumbnails directly
* remove alert action for now and add descriptions
* tweaks
* clean up
* fix types
* docs
* docs tweaks
* docs
* reuse enum
* Ui improvements
* Improve image cropping and model saving
* Improve naming
* Add logs for training
* Improve model labeling
* Don't set sub label for none object classification
* Cleanup
* Ignore numpy get limits warning
* Add function wrapper to redirect stdout and stderr to logpipe
* Save stderr too
* Add more to catch
* run logpipe
* Use other logging redirect class
* Use other logging redirect class
* add decorator for redirecting c/c++ level output to logger
* fix typing
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Multiprocessing Manager queues don't have a close() or join_thread() method, and the Manager will clean it up appropriately after we empty it. This prevents an infinite loop when an AttributeError exception fires for Manager AutoProxy queue objects.
* Move log level initialization to log
* Use logger config
* Formatting
* Fix config order
* Set process names
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Set runtime
* Use count correctly
* Don't assume camera sizes
* Use separate zmq proxy for object detection
* Correct order
* Use forkserver
* Only store PID instead of entire process reference
* Cleanup
* Catch correct errors
* Fix typing
* Remove before_run from process util
The before_run never actually ran because:
You're right to suspect an issue with before_run not being called and a potential deadlock. The way you've implemented the run_wrapper using __getattribute__ for the run method of BaseProcess is a common pitfall in Python's multiprocessing, especially when combined with how multiprocessing.Process works internally.
Here's a breakdown of why before_run isn't being called and why you might be experiencing a deadlock:
The Problem: __getattribute__ and Process Serialization
When you create a multiprocessing.Process object and call start(), the multiprocessing module needs to serialize the process object (or at least enough of it to re-create the process in the new interpreter). It then pickles this serialized object and sends it to the newly spawned process.
The issue with your __getattribute__ implementation for run is that:
run is retrieved during serialization: When multiprocessing tries to pickle your Process object to send to the new process, it will likely access the run attribute. This triggers your __getattribute__ wrapper, which then tries to bind run_wrapper to self.
run_wrapper is bound to the parent process's self: The run_wrapper closure, when created in the parent process, captures the self (the Process instance) from the parent's memory space.
Deserialization creates a new object: In the child process, a new Process object is created by deserializing the pickled data. However, the run_wrapper method that was pickled still holds a reference to the self from the parent process. This is a subtle but critical distinction.
The child's run is not your wrapped run: When the child process starts, it internally calls its own run method. Because of the serialization and deserialization process, the run method that's ultimately executed in the child process is the original multiprocessing.Process.run or the Process.run if you had directly overridden it. Your __getattribute__ magic, which wraps run, isn't correctly applied to the Process object within the child's context.
* Cleanup
* Logging bugfix (#18465)
* use mp Manager to handle logging queues
A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.
* consolidate
* fix typing
* Fix typing
* Use global log queue
* Move to using process for logging
* Convert camera tracking to process
* Add more processes
* Finalize process
* Cleanup
* Cleanup typing
* Formatting
* Remove daemon
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add base class for global config updates
* Add or remove camera states
* Move camera process management to separate thread
* Move camera management fully to separate class
* Cleanup
* Stop camera processes when stop command is sent
* Start processes dynamically when needed
* Adjust
* Leave extra room in tracked object queue for two cameras
* Dynamically set extra config pieces
* Add some TODOs
* Fix type check
* Simplify config updates
* Improve typing
* Correctly handle indexed entries
* Cleanup
* Create out SHM
* Use ZMQ for signaling object detectoin is completed
* Get camera correctly created
* Cleanup for updating the cameras config
* Cleanup
* Don't enable audio if no cameras have audio transcription
* Use exact string so similar camera names don't interfere
* Add ability to update config via json body to config/set endpoint
Additionally, update the config in a single rather than multiple calls for each updated key
* fix autotracking calibration to support new config updater function
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Combine base and arm trt detectors
* Remove unused deps for amd64 build
* Add missing packages and cleanup ldconfig
* Expand packages for tensorflow model training
* Cleanup
* Refactor training to not reserve memory
* Refactor common functions for tflite detector implementations
* Add detector using mesa teflon delegate
Non-EdgeTPU TFLite can use the standard .tflite format
* Add mesa-teflon-delegate from bookworm-backports to arm64 images
* Implement model training via ZMQ and add model states to represent training
* Get model updates working
* Improve toasts and model state
* Clean up logging
* Add back in
* Setup basic training structure
* Build out route
* Handle model configs
* Add image fetch APIs
* Implement model training screen with dataset selection
* Implement viewing of training images
* Adjust directories
* Implement viewing of images
* Add support for deleting images
* Implement full deletion
* Implement classification model training
* Improve naming
* More renaming
* Improve layout
* Reduce logging
* Cleanup
The PP_OCRv5 text detection models have greatly improved over v3. The v5 recognition model makes improvements to challenging handwriting and uncommon characters, which are not necessary for LPR, so using v4 seemed like a better choice to continue to keep inference time as low as possible. Also included is the full dictionary for Chinese character support.
* install new packages for transcription support
* add config options
* audio maintainer modifications to support transcription
* pass main config to audio process
* embeddings support
* api and transcription post processor
* embeddings maintainer support for post processor
* live audio transcription with sherpa and faster-whisper
* update dispatcher with live transcription topic
* frontend websocket
* frontend live transcription
* frontend changes for speech events
* i18n changes
* docs
* mqtt docs
* fix linter
* use float16 and small model on gpu for real-time
* fix return value and use requestor to embed description instead of passing embeddings
* run real-time transcription in its own thread
* tweaks
* publish live transcriptions on their own topic instead of tracked_object_update
* config validator and docs
* clarify docs
* Start Frigate in safe mode when config does not validate
* Add safe mode page that is just the config editor
* Adjust Frigate config editor when in safe mode
* Cleanup
* Improve log message
* Indicate no recordings on the history timeline with gray hash marks
This commit includes a new backend API endpoint and the frontend changes needed to support this functionality
* don't show slashes for now
* Add basic config for defining a teachable machine model
* Add model type
* Add basic config for teachable machine models
* Adjust config for state and object
* Use config to process
* Correctly check for objects
* Remove debug
* Rename to not be teachable machine specific
* Cleanup
* Include config publisher in api
* Call update topic for passed topics
* Update zones dynamically
* Update zones internally
* Support zone and mask reset
* Handle updating objects config
* Don't put status for needing to restart Frigate
* Cleanup http tests
* Fix tests
* Fetch all license plates outside of filter component
If the swr call took a long time, the entire select component may not display. This change moves the fetch to the parent component (like sub labels).
* add loading indicator
* improve query
* Remove torch install
* notification fixes
the pubkey was not being returned if notifications was not enabled at the global level
* Put back
* single condition check for fetching and disabling button
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* fix embeddings reindex
- always increment processed objects to prevent division by zero
- ensure description still gets processed even if there is no thumbnail
* clean up
* Add newer labels to default attribute map
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
When an IPv6 address that doesn't map to an IPv4 address was checked
against an IPv4 trusted proxy, we'd hit an exception because
ip.ipv4_mapped was None. Fix this by verifying ipv4_mapped is not None
Co-authored-by: me <me@me>
* Fix Norwegian language code for date-fns module import
* Fix incorrect apple compatibility change
* Add other and unused as separate storage items
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Don't support tensorrt detector for amd64 builds
* Add logs for directing users not to use tensorrt detector
* Rework docs
* Fix dockerfile index
* Don't undo jetson fix
* Fix showing review items that span over multiple days
* Simplify
* Fix tests
* Fix unchanged value
* Allow admin as default role and viewer as passed header for proxy auth
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* i18n translated label fixes
* Fix frame cache race bug
Objects that were marked as false positives (that would later become true positives) would sometimes have their saved frame prematurely removed from the frame cache.
* Object labels with spaces should use correct i18n keys
* Add Hungarian
* Ensure onvif move request has a valid speed before removing
When autotracking zooming is set to `disabled` (or is left out of the config), move_request["Speed"] may not exist, depending on the camera
* Add another frame cache debug log
* Don't allow editing of sub label until object lifecycle has ended
* Update sub labels in ended review segments
When manually editing a sub label for a tracked object from the UI, any review segments containing that tracked object did not have their sub_labels and objects values altered
* simplify
* Additional onvif debug logs in get_camera_status
* Ensure that best object is only set when the snapshot is actually updated.
* Don't hide downlaod button when there is no review item
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
Model can be adjusted ahead of time to NHWC to avoid transpose on CPU,
for example. All the model information is already presented in the
configuration, and the stringent checks implemented in openvino are not
present on other backends anyway. OpenVINO will properly report issues
with mismatched layouts anyhow.
* Ensure config editor recalculates layout on error
* ensure empty lists are returned when lpr recognition model fails
* Add docs section for session_length
* clarify
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* clarify
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Catch missing file
* Improve graph axis colors
* Ensure playback rate controls are portaled to the video container in history view
On larger tablets in landscape view, the playback rate dropdown disappeared underneath the bottom bar. This small change ensures we use the correct container on the DropdownMenuContent so that the div is portaled correctly. The VideoControls are also used in motion review which does not pass in a container ref, so we can just fall back to the existing controlsContainer ref when it's undefined.
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix i18n keys
* hide disable from context menu for viewers
* Fix auto live check for default dashboard and camera groups
Disabling the Automatic Live View switch in Settings should prevent streaming from occurring. Overriding any settings in a camera group will override the global setting. The check here incorrectly always returned false instead of undefined.
* clarify hardware accelerated enrichments
* clarify
* add note about detect stream to face rec docs
* add note about low end Dahuas for autotracking
* Catch invalid face box / image
* Video tab tweaks
With the changes in https://github.com/blakeblackshear/frigate/pull/18220, the video tab in the Tracked Object Details pane now correctly trims the in-browser HLS video. Because of keyframes and record/detect stream differences, we can manually subtract a couple of seconds from the event start_time to ensure the first few frames aren't cut off from the video
* Clarify
* Don't use Migraphx by default
* Provide better support for running embeddings on GPU
* correctly join cameras
* Adjust blur confidence reduction
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* use mp Manager to handle logging queues
A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.
* consolidate
* fix typing
* Don't return weighted name if it has the same number of results
* Remove link to incorrect format yolov9 models
* Fix command list from appearing when other inputs are focused
the description box in the tracked object details pane was causing the command input list to show when focused.
* clarify face docs
* Add note about python yolov9 export
* Check if hailort thread is still alive when timeout error is run into
* Reduce inference timeout
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* improve spacing of face selection in mobile drawer
* fix spacing
* sort face names alphabetically
* Improve face selection dialog
* Use a state to track when face image loads
The naturalWidth and naturalHeight will always be 0 until the image loads. So we use onLoad and a state to track loading and then calculate the area after it has loaded
* Verify that a camera only tracks objects that are possible to track
* Fix test
* genai docs tweak
* Disable openvino model cache
* Clenaup
* Sanitize floats for estimated speed and angle
Users can configure speed zones in such a way that velocity estimates from Norfair cause a value of inf to be stored as an estimated speed. FastAPI doesn't serialize inf as a float, so trying to return this value would result in an API error. Sanitizing the value before storing should correct this.
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Add Thai (still need to merge weblate)
* Apply attribute logic to all label types
* Fix area check
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Don't use timezone in export dialog timestamps
Revert an unnecessary change made in https://github.com/blakeblackshear/frigate/pull/18257
* Ensure notifications register button is only disabled when both all cameras and every individual camera is disabled
* Send test notification if any cameras are enabled
* clarify docs about disabling cameras
* fix crash in autotracking zoom
* clean up
* masks and zones i18n fixes
* Check if camera is enabled in config
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Only check if an object is stationary to avoid mqtt snapshot
* docs heading tweak
* Add more API descriptions
* Add missing lib for new rocm onnxruntime whl
* Update inference times to reflect better rocm performance
* Cleanup resetting tracked object activity
* remove print
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Not everyone is using /dev/dri/card0. I recommend switching to these more generic parameters for intel_gpu_top_command (line 307 in the 0.16.0-beta2 version of services.py):
intel_gpu_top_command += ["-d", "sriov"]
* Add option to not trim clip
* Improve API
* Update snapshot for new best objects
* Fix missing strings
* Convert to separate key
* Always include bounding box on snapshots
* improve autotracking relative zooming time calculation
* update proxy docs to note the need for comma separated header roles
* Add count translation
* tracked object lifecycle i18n fix
* update speed estimation docs
* clarity
* Re-initialize onvif information when toggling camera on live view
* Move time ago to card info and add face area
* Clarify face recognition docs
* Increase minimum face recognition area
* use clipFrom to in vod module endpoint to start at the correct time
* Cleanup media api
* Don't change duration
* Use search detail dialog for face library
* Move to segment based
* Cleanup
* Add back duration modification
* clean up docs
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix the `Any` typing hint treewide
There has been confusion between the Any type[1] and the any function[2]
in typing hints.
[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any
* Fix typing for various frame_shape members
Frame shapes are most likely defined by height and width, so a single int
cannot express that.
* Wrap gpu stats functions in Optional[]
These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.
* Fix return type in get_latest_segment_datetime
Returns a datetime object, not an integer.
* Make the return type of FrameManager.write optional
This is necessary since the SharedMemoryFrameManager.write function can
return None.
* Fix total_seconds() return type in get_tz_modifiers
The function returns a float, not an int.
https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds
* Account for floating point results in to_relative_box
Because the function uses division the return types may either be int or
float.
* Resolve ruff deprecation warning
The config has been split into formatter and linter, and the global
options are deprecated.
* Add camera name tooltip to previews in recording view
* Apply face area check to cv2 face detection
* Delete review thumbnails
* Don't import hailo until it is used
* Add comment
* Clean up camera name
* Filter out empty keys when updating yaml config
HA ingress seems to randomly add an equal sign to the PUT urls for updating the config from the UI. This fix prevents empty keys from being processed, but still allows empty values.
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Catch error and show toast when failing to delete review items
* i18n keys
* add link to speed estimation docs in zone edit pane
* Implement reset of tracked object update for each camera
* Cleanup
* register mqtt callbacks for toggling alerts and detections
* clarify snapshots docs
* clarify semantic search reindexing
* add ukrainian
* adjust date granularity for last recording time
The api endpoint only returns granularity down to the day
* Add amd hardware
* fix crash in face library on initial start after enabling
* Fix recordings view for mobile landscape
The events view incorrectly was displaying two columns on landscape view and it only took up 20% of the screen width. Additionally, in landscape view the timeline was too wide (especially on iPads of various screen sizes) and would overlap the main video
* face rec overfitting instructions
* Clarify
* face docs
* clarify
* clarify
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* face library i18n fixes
* face library i18n fixes
* add ability to use ctrl/cmd S to save in the config editor
* Use datetime as ID
* Update metrics inference speed to start with 0 ms
* fix android formatted thumbnail
* ensure role is comma separated and stripped correctly
* improve face library deletion
- add a confirmation dialog
- add ability to select all / delete faces in collections
* Implement lazy loading for video previews
* Force GPU for large embedding model
* GPU is required
* settings i18n fixes
* Don't delete train tab
* webpush debugging logs
* Fix incorrectly copying zones
* copy path data
* Ensure that cache dir exists for Frigate+
* face docs update
* Add description to upload image step to clarify the image
* Clean up
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix onvif reinitialization
* api docs: clarify usage of clip.mp4 endpoint
* Always show train tab
* Add description to API
* catch lpr model inference exceptions
* always apply motion mask when using yolov9 plate detection
* lpr faq
* fix incorrect focus when reopening search detail dialog on video tab
* only use keyboard listener in face library when train tab is active
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* use async/await instead of asyncio.run()
* fix autotracking
* create cameras in same event loop that will use them
* more debug
* try using existing event loop instead of creating a new one
* merge dev
* fixes
* run get_camera_info onvifcontroller calls in dedicated loop
* move coroutine call with loop to api
* use asyncio for autotracking move queues
* clean up
* fix calibration
* improve exception logging
* frigate+ pane i18n fix
* catch more exceptions
* explore search result tooltip i18n fix
* i18n fix
* remove comments about deprecated strftime_fmt
* Catch producers exists but is None
* Formatting
* fix live camera view i18n
* Add default role config for proxy users
This allows users to specify a default role for users when using a proxy for auth. This can be useful for users who can't/don't want to define a header mapping for the remote-role header.
* update reference config and auth docs
* clarify face rec camera level config
* clarify auth docs
* Fix onnx not working with openvino
* Update openvino to fix failed npu plugin check
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
On startup, the object processing pipeline would be delayed for 10ms every iteration through the loop while the end event subscriber blocked. For users with large numbers of cameras and a fair amount of detected objects, this would cause the detected objects queue length to rise and frames to be eventually dropped, leading to processing delays, incorrect frame times applied to thumbnail_data, and other seemingly unexplainable things.
* add zoom time to movement predictions
* config migrator
* add space to face rename regex
* more debug
* only calculate zoom time of relative move
* fix test
* make migrated movement weight a zero
* check for str and bool for movestatus support
* Support automatic language selection
* Handle non-matching keys
* Cleanup
* Handle region specific language codes from browser
* Fix passing in requestor
* Send tracked object updates for face and license_plate objects
* Update docs
* Add to type enum
* Add camera to object description update
* Formatting
* Consolidate yue-Hant
* Add missing
* Only auto-populate some config fields down to the camera level
- Don't populate LPR and face global-only settings down to the camera level
- Ensure LPR mixin uses camera level min_area
- Explicitly forbid extra config values for LPR and face
* lpr docs tweak
* remove extra text already in i18n key
* consistency
* fix check for snapshot score
for manual events, snapScore is 0, so "0" gets displayed instead of a condition being evaluated.
* fix ongoing events from being returned for all review queries
The existing condition will include any record with a null end_time regardless of when it started, as long as the start_time is greater than the after param. With this fix, both the start time needs to be within the specified range, and for events that have already ended, their end time must be before the before param
* Base alert toggles on width not device class
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Don't update thumbnail/snapshot if frame is not in the frame cache
This follows up on changes originally made in https://github.com/blakeblackshear/frigate/pull/17671 - it is possible that is_better_thumbnail is true but the frame_time for the object is not the same as the current_frame_time. So rather than checking the frame cache, this change will only update the thumbnail_data when the two frame time values are the same.
* add camera name to debug log
* Don't limit review summary query
The /review/summary endpoint was being restricted to the last 30 days, so unreviewed alerts and detections weren't showing in the calendar before that point. This change removes the WHERE clause so that the endpoint returns all results.
* fix
* remove unneeded test
we are now returning all results so we don't need to test for > month_ago
* fix
FFmpeg adds an extra delay for the first start of the stream. For some cameras the stream start can be 1 second, for some cameras it can be more than 5 seconds. Frigate by default used a timeout of 5 seconds and drops the connection without waiting for a response from go2rtc. Originally suggested by @AlexxIT in https://github.com/AlexxIT/go2rtc/issues/1633
* Add npu usages as a statistic
* Support showing NPU stats in dashboard
* Add sys volume mount for npu usages
* Fix type
* Simplify check
* Cleanup
* Cleanup
* add support for multi-line plates
* config for model size
* default to small model
* add license plate as attribute to motorcycle
* use model size
* docs
* attribute map
* i18n key fix
* Save initial frame of new objects to frame cache
Objects that move quickly through the frame and are only seen briefly may not have the update() method called to save thumbnail_data, and may not have the initial frame saved to the tracked object frame cache. This caused a "Frame missing from frame cache" message that was patched by #7313 but this sometimes caused the wrong frame to be chosen for the thumb/snapshot.
* Tracking tweaks
- When registering new objects, use the past detections from Norfair to populate self.positions and self.stationary_box_history. This prevents the first call of update_position() from triggering a +1 on the object's stationary count (because the iou would be 1.0).
- Add a specific tracker for dedicated LPR cam license_plate objects using a lower R value and higher distance threshold to account for fast moving plates.
- Add helpful debug messages and keep them disabled with `if False:`
* Correctly ensure updates are more periodic when lpr or face detection is needed
* Cleanup
* Update api schema
* Don't update for stationary objects
* Simplify check
* Remove
* Merge nearby horizontal boxes
* only publish to recognized plate field if object already has a sub label
* don't overwrite sub labels in any situation
* always publish sub label if it's a known plate
* remove license plate from attributes for dedicated lpr cameras
* ensure we always have a color
* use frigate+ models with dedicated lpr cameras
* docs
* docs clarity
* docs enrichments
* use license_plate as object type
* Section faces by event id
* Make score keeping more robust
* layout improvements
* Cleanup dialog
* Fix clicking behavior
* Add view in explore option
* math.round
* Don't require events
* Cleanup
* Remove selection
* Don't require
* Change dialog size with snapshot
* Use filename as key
* fix key
* Rework layout for mobile
* Handle mobile landscape
* Fix train issue
* Match logic
* Move deletion logic
* Fix reprocessing
* Support creating a new face
* Translations
* Do sorting in frontend
* Adjust unknown
* Cleanup
* Set max limit to faces to recognize
* Fix sorting
* Fix
* add config validator for face and lpr
* more lpr docs tweaks
* fix object lifecycle point clicking for aspect ratios less than 16/9
* fix semantic search indexing i18n keys
* remove ability to set system language
* clarify debug output
