* bump backend and frontend dependencies
Bumps every outdated dependency that passed testing, with the code changes each one needs.
- peewee 4.5 and peewee_migrate 2.3: use SqliteDatabase in place of the removed SqliteExtDatabase, and fix types for peewee's bundled stubs
- mypy 2.4.0: remove unused type ignores and add the annotations mypy 2 needs
- pandas 3.0: convert the motion activity index with as_unit("s"), because pandas 3 infers second resolution for whole-second timestamps and the old // 10**9 returned 1
- transformers 5.19: load Jina tokenizers from the saved directory with trust_remote_code off and use CLIPImageProcessor, since v5 removed CLIPFeatureExtractor and remote code pulled in torch
- tensorflow-cpu 2.21: pin protobuf to 7.36 in the TensorRT amd64 image, because the old 3.20.3 pin was installed over TensorFlow and broke its import
- pydantic 2.13 and google-genai 2.29: regenerate the API spec
- ruff 0.16: set an explicit lint select, since 0.16 widened the default rules
- react-router-dom 7: set useTransitions={false} on BrowserRouter to keep v6 update timing, which useSearchEffect relies on
- typescript 6: drop baseUrl and add node types in tsconfig
- radix: update to the latest release and remove the slot patch, which upstream now includes
- apexcharts 7.8: guard the tooltip formatter against null values
- cryptography 50, joserfc 1.7, pywebpush 2.5, openai 3.26, aiohttp, uvicorn, onvif-zeep-async, scipy, framer-motion 14, react-day-picker 10 and other minor and patch bumps
* silence httpx2 request logging
* redownload an incomplete jina tokenizer
An interrupted tokenizer download left the Hugging Face cache directory without the saved tokenizer files, and since the download is skipped whenever that directory exists, every restart failed to load the tokenizer. The Jina constructors now remove a tokenizer directory that has no tokenizer_config.json so the download runs again.
* revert disable save buttons when there are no changes in config editor
* pass migrated config to each step in the config migration chain
* use 150 as the default max when typing a min speed in the search filter
* fix preview export outpoint to be relative to the start of the preview file
* don't crash on invalid trusted proxy entries or non-ip forwarded hops
* translate the camera count badge in the roles table
* run every batch through the lpr recognition model
* log rejected motion and notification mqtt payloads
* log invalid addresses in x-forwarded-for
* add test
* remove unused autotracked_object_region
* remove unreachable autotracker setup call in camera maintenance
* apply onvif retry limit when initialization fails
* fix reindex progress overcounting when there are fewer events than a batch
* match the register device button aria label to its text
* reset the add profile form on cancel
* ignore case when filtering search suggestions
* tweak comment
* fix semantic search reindex
sqlite-vec added the `_info` shadow table in 0.1.6 and drops it unconditionally when a vec0 table is destroyed, so `DROP TABLE` on a table written by 0.17 failed with "SQL logic error" once 0.18 moved to 0.1.9. `SqliteQueueDatabase` queues non-SELECT statements and stores the exception on the cursor it returns, and nothing read those cursors, so the failed drop and every write after it went unreported while reindex still logged "Embedded N thumbnails". `drop_embeddings_tables()` now recreates the missing `_info` stub before dropping, and writes go through `execute_write()`, which waits on the cursor so failures raise. `INSERT OR REPLACE` is gone too, since vec0 implements neither REPLACE nor UPSERT and it always failed on an id already in the table, including under the 0.1.3 build 0.17 shipped.
* use lock
* show reindex failure in status bar
* Add support for running transcription with GenAI
* Improve audio joining
* Fix GenAI model capability reporting
* Support language correctly
* Migrate existing users to keep english selected
* Fix models
* Fix tests
* Fix accepted null model
* Handle slwo providers
* Implement annotated frames mode for GenAI reviews to improve models with lacking temporal understanding
* Updates
* Improve debug sharing
* Do not number objects
* Fix assumptions
* Remove unhelpful content
* Improve object data sent as part of prompt
* Cleanup ollama dumbness
* Bind db
* Fixes
* Cleanup
* Rename face model to face recognizer
* Refactor face detection into own module
* Return face and face landmarks
* Align faces with 5 points instead of just the eyes
* Add landmark validation to throw out images which do not fit a landmark
* Fix circular import and lock face detector for concurrent runs across threads
* Fix mypy
* pin genai review frames to the main stream
* retain previews as long as either stream has recordings
* watch sub stream recording health separately from main
* reject record_sub on the same input as record and document the role
* derive recording paths from the cache segment timestamp
Recording paths carry one second of resolution, but since sub stream recording start times are resolved to fractional wall clock, anchored to the cache file mtime and chained to the previous segment's end. A stream cutting segments faster than once a second resolves consecutive segments into the same second, so two rows collide on the unique path index and the batch insert fails. The cache segment name is unique per camera stream and second by construction because ffmpeg names segments with strftime, so the recording path is now built from that timestamp while the row keeps the resolved start time. This also restores the path semantics from before sub stream recording, when start times came straight from the cache filename.
Nothing derives times from recording paths: playback offsets, stream switching, and export all use the row's start time, which is unchanged, and the recordings sync matches files by exact path string.
* keep the rest of a recording batch when one row conflicts
* only publish record_sub status when a sub stream is configured
* don't shadow camera_cfg when publishing empty cache streams
* back off restarts when a recording stream goes stale
* give the shared sub stream grace on any capture thread reset
* include segment details in recording discard warnings
* Guard object processor queue handlers against unknown cameras
* Skip embeddings post processing for removed cameras
* End review segments for removed cameras
* Drop queued autotracker moves for removed cameras
* Release tracked event thumbnails when skipping a removed camera
* Add locked accessors for camera states
* Read camera states through the processor accessors
* Guard output and recording paths against cameras not yet known
* Resolve camera state once in ONVIF, notification, and transcription paths
sanitize_filename leaves ".." intact and collapses variants like "..:" and "..*" to "..", so filesystem paths built from face names, classification model/category names, image ids, and trigger data could escape their base directory. Route every such site through new frigate/util/path.py helpers (safe_join, sanitize_path_component, sanitize_contained_path), which reject traversal and verify containment.
Worst case was DELETE /classification/{name}, which rmtree'd /media/frigate and /config while returning 200.
Important to note that all affected endpoints already require admin permission, so this sould be considered hardening rather than fixing exploitable code.
* Convert face crops to RGB before embedding
Face crops flow through the cv2 pipeline as BGR arrays, but
_process_image passes ndarrays to PIL without any channel conversion,
so the FaceNet and ArcFace embedders receive BGR input while both
models expect RGB. The error is symmetric between enrollment and
recognition so it partially cancels, but it still costs accuracy.
* Move BGR to RGB conversion into a shared helper
Deduplicate the channel swap from both _preprocess_inputs methods
into a BaseEmbedding._bgr_to_rgb static helper, as suggested in
review.
* add ability to edit enabled and save_attempts for classification models in the UI
* add state motion and interval configs to edit dialog
* fix preview playback rate for motion previews
* add docs note about environment vars and go2rtc
* update live view faq
* honor enabled flag for custom classification models
for both startup and dynamically, even though the UI doesn't currently have a way to toggle dynamically
* add test
* Pin ruff
* Add python upgrade fixes
This enables python upgrade checks in ruff to look for deprecated types and patterns. This namely fixes:
- usage of deprecated `Typing` which is now built in
- some specific exceptions which are caught and have new aliases
Some specific UP checks were also ignored as they are stylistic / unimportant and likely to cause bugs
* Remove async blocking calls
Use asyncio.to_thread on two remaining blocking calls to fix hanging event thread loop. Enable this specific rule to block it in the future.
* Use proper logging mechanism
* Correctly format logs
* Raise with context
When raising an exception include the from context to improve debugging
* Cleanup
* filter replay camera from camera selectors
* add face rec and lpr to replay configuration sheet
* add missing config topic subscriptions in embeddings maintainer
* pop replay camera from config object when stopping
* start audio transcription post processor when enabled on any camera
* Fetch embed key whenever an error occurs in case the llama server was restarted
* mypy
* add tooltips for colored dots in settings menu
* add ability to reorder cameras from management pane
* add ability to reorder birdseye
* add reordering save text to camera management view
* Include NPU in latency performance hint
* Implement turbo for NPU on object detection
* hide order fields
* drop auto-derived field paths from camera value when unset globally
* use correct field type for export hwaccel args
* add debug replay to detail actions menu
* clarify debug replay in docs
* guard get_current_frame_time against missing camera state
* Implement debug reply from export
* Refactor debug replay to use sources for dynamic playback
* Mypy
* fix debug export replay source timestamp handling
* skip replay cameras in stats immediately
* broadcast debug replay state over ws and buffer pre-OPEN sends
- push debug replay session state over the job_state ws topic so the status bar reacts instantly to start/stop without polling
- fix child-effect-before-parent-effect race in WsProvider that silently dropped initial snapshot requests on cold load
* fix debug replay test hang
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* add optional onClick to EmptyCard
* show EmptyCard in face rec when face library is empty
* add loading indicator
* add description to camera management pane
* Cleanup when use snapshot but can't load snapshot
* Migrate files
* fix birdseye color distortion when configured aspect ratio is unsupported
* Skip processing end for object descriptions
* don't crash if stats is null
* fix genai roles in migration
* frigate+ pane updates
- allow users to select a plus model from the select even when one was not previously loaded
- always show model summary card
- add model filter popover
- add restart button totast
* fix frigate+ pane layout and buttons to match other settings panes
* match button layout in go2rtc settings view
* make audio maintainer respond to dynamic config updates
* check correct zone name in publish state
* fix nested translation extraction for Optional dict and list fields
* mypy
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* ensure embeddings process restarts after maintainer thread crash
* add docs link to media sync settings
* fix color
Co-authored-by: Copilot <copilot@github.com>
* match link color with other sections
* ensure recording staleness threshold scales with segment_time
* docs tweak
* Fix llama.cpp media marker
* Fix gemini tools call
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* add ui to camera config update topics enum
* add mqtt to camera config update enum
* ensure cleanup runs when an event end skips post-processing
* end any in-progress audio events when audio detection is disabled
we already end in-progress audio events when we disable a camera, but this mirrors that logic for specifically disabling audio detection
* Improve GenAI metadata
* fix invalid recording segment topic being misrouted to the valid handler
* Add confidence default to avoid unnecessary field causing issues
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix mobile export crash by removing stale iOS non-modal drawer workaround
* Remove titlecase to avoid Gemma4 handling plain labels as proper nouns
* Improve titling:
* Make directions more clear
* Properly capitalize delivery services
* update dispatcher config reference on save
* subscribe to review topic so ReviewDescriptionProcessor knows genai is enabled
* auto-send ON genai review WS message when enabled_in_config transitions to true
* remove unused object level
* update docs to clarify pre/post capture settings
* add ui docs links
* improve known_plates field in settings UI
* only show save all when multiple sections are changed
or if the section being changed is not currently being viewed
* fix docs
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Switch to a feature-based roles so it is easier to choose models for different tasks
* Fallback and try llama-swap format
* List models supported by provider
* Cleanup
* Add frontend
* Improve model loading
* Make it possible to update genai without restarting
* Cleanup
* Cleanup
* Mypy
The name parameter was interpolated directly into the SQL query via
f-string, allowing SQL injection through crafted face name values.
Use a parameterized query with ? placeholder instead.
* enrichment updater and enum
* update_config stubs
* config updaters in enrichments
* update maintainer
* formatting
* simplify enrichment config updates to use single subscriber with topic-based routing
* Support GenAI for embeddings
* Add embed API support
* Add support for embedding via genai
* Basic docs
* undo
* Fix sending images
* Don't require download check
* Set model
* Handle emb correctly
* Clarification
* Cleanup
* Cleanup
* face recognition dynamic config
* lpr dynamic config
* safe changes for birdseye dynamic config
* bird classification dynamic config
* always assign new config to stats emitter to make telemetry fields dynamic
* add wildcard support for camera config updates in config_set
* update restart required fields for global sections
* add test
* fix rebase issue
* collapsible settings sidebar
use the preexisting control available with shadcn's sidebar (cmd/ctrl-B) to give users more space to set masks/zones on smaller screens
* dynamic ffmpeg
* ensure previews dir exists
when ffmpeg processes restart, there's a brief window where the preview frame generation pipeline is torn down and restarted. before these changes, ffmpeg only restarted on crash/stall recovery or full Frigate restart. Now that ffmpeg restarts happen on-demand via config changes, there's a higher chance a frontend request hits the preview_mp4 or preview_gif endpoints during that brief restart window when the directory might not exist yet. The existing os.listdir() call would throw FileNotFoundError without a directory existence check. this fix just checks if the directory exists and returns 404 if not, exactly how preview_thumbnail already handles the same scenario a few lines below
* global ffmpeg section
* clean up
* tweak
* fix test
* debug replay implementation
* fix masks after dev rebase
* fix squash merge issues
* fix
* fix
* fix
* no need to write debug replay camera to config
* camera and filter button and dropdown
* add filters
* add ability to edit motion and object config for debug replay
* add debug draw overlay to debug replay
* add guard to prevent crash when camera is no longer in camera_states
* fix overflow due to radix absolutely positioned elements
* increase number of messages
* ensure deep_merge replaces existing list values when override is true
* add back button
* add debug replay to explore and review menus
* clean up
* clean up
* update instructions to prevent exposing exception info
* fix typing
* refactor output logic
* refactor with helper function
* move init to function for consistency
* GenAI client manager
* Add config migration
* Convert to roles list
* Support getting client via manager
* Cleanup
* Fix import issues
* Set model in llama.cpp config
* Clenaup
* Use config update
* Clenaup
* Add new title and desc
The fallback to tensorflow was established back in 2023, because we could
not provide tflite-runtime downstream in nixpkgs.
By now we have ai-edge-litert available, which is the successor to the
tflite-runtime. It still provides the same entrypoints as tflite-runtime
and functionality has been verified in multiple deployments for the last
two weeks.
* Adjust title prompt to have less rigidity
* Improve motion boxes handling for features that don't require motion
* Improve handling of classes starting with digits
* Improve vehicle nuance
* tweak lpr docs
* Improve grammar
* Don't allow # in face name
* add password requirements to new user dialog
* change password requirements
* Clenaup
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix jetson stats reading
* Return result
* Avoid unknown class for cover image
* fix double encoding of passwords in camera wizard
* formatting
* empty homekit config fixes
* add locks to jina v1 embeddings
protect tokenizer and feature extractor in jina_v1_embedding with per-instance thread lock to avoid the "Already borrowed" RuntimeError during concurrent tokenization
* Capitalize correctly
* replace deprecated google-generativeai with google-genai
update gemini genai provider with new calls from SDK
provider_options specifies any http options
suppress unneeded info logging
* fix attribute area on detail stream hover
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Use thread lock for JinaV2 call as it sets multiple internal fields while being called
* fix audio label translation in explore filter
* Show event in all cases, even without non-none match
* improve i18n key fallback when translation files aren't loaded
just display a valid time now instead of "invalid time"
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add shortSummary field to review summary to be used for notifications
* pull in current config version into default config
* fix crash when dynamically adding cameras
depending on where we are in the update loop, camera configs might not be updated yet and we are receiving detections already
* add no tracked objects and icon to explore summary view
* reset add camera wizard when closing and saving
* don't flash no exports icon while loading
* Improve handling of homekit config
* Increase prompt tokens reservation
* Adjust
* Catch event not found object detection
* Use thread lock for JinaV2 in onnxruntime
* remove incorrect embeddings process from memray docs
* only show transcribe button if audio event has video
* apply aspect ratio and margin constraints to path overlay in detail stream on mobile
improves a specific case where the overlay was not aligned with 4:3 cameras on mobile phones
* show metadata title as tooltip on icon hover in detail stream
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* use fallback timeout for opening media source
covers the case where there is no active connection to the go2rtc stream and the camera takes a long time to start
* Add review thumbnail URL to integration docs
* fix weekday starting point on explore when set to monday in UI settings
* only show allowed cameras and groups in camera filter button
* Reset the wizard state after closing with model
* remove footnote about 0.17
* 0.17
* add triggers to note
* add slovak
* Ensure genai client exists
* Correctly catch JSONDecodeError
* clarify docs for none class
* version bump on updating page
* fix ExportRecordingsBody to allow optional name field
fixes https://github.com/blakeblackshear/frigate/discussions/21413 because of https://github.com/blakeblackshear/frigate-hass-integration/pull/1021
* Catch remote protocol error from ollama
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Send preferred language for report service
* make object lifecycle scrollable in tracking details
* fix info popovers in live camera drawer
* ensure metrics are initialized if genai is enabled
* docs
* ollama cloud model docs
* Ensure object descriptions get claened up
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix genai callbacks in MQTT
* Cleanup cursor pointer for classification cards
* Cleanup
* Handle unknown SOCs for RKNN converter by only using known SOCs
* don't allow "none" as a classification class name
* change internal port user to admin and default unspecified username to viewer
* keep 5000 as anonymous user
* suppress tensorflow logging during classification training
* Always apply base log level suppressions for noisy third-party libraries even if no specific logConfig is provided
* remove decorator and specifically suppress TFLite delegate creation messages
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Wait for config to load before evaluating route access
Fix race condition where custom role users are temporarily denied access after login while config is still loading. Defer route rendering in DefaultAppView until config is available so the complete role list is known before ProtectedRoute evaluates permissions
* Use batching for state classification generation
* Ignore incorrect scoring images if they make it through the deletion
* Delete unclassified images
* mitigate tensorflow atexit crash by pre-importing tflite/tensorflow on main thread
Pre-import Interpreter in embeddings maintainer and add defensive lazy imports in classification processors to avoid worker-thread tensorflow imports causing "can't register atexit after shutdown"
* don't require old password for users with admin role when changing passwords
* don't render actions menu if no options are available
* Remove hwaccel arg as it is not used for encoding
* change password button text
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
- don't look for event ids when trigger type is description
- don't try to delete thumbnail whe trigger type is description
- pass correct event ID into thumbnail deletion
* fix: Resolve deadlock and attribute error in embedding maintainer initialization
Updates the trigger embedding calculation to call embedding methods directly instead of using ZMQ. This prevents a deadlock during initialization where the ZMQ responder is not yet polling for requests.
Also updates sync_triggers to pass the camera name and trigger name to the calculation method, fixing an AttributeError where trigger.name was accessed on a TriggerConfig object.
* mocked repro
* Revert "mocked repro"
This reverts commit dea5b5d4db.
* fix formatting
* Update embeddings.py
new line
* update live view docs
* use swr as single source of truth for searchDetail
rather than maintaining a separate state, derive the selected item from swr cache. fixes websocket sync when regenerating descriptions or fetching transcriptions
* fix key warning in console
* don't try to fetch event from review item for audio events
* update audio transcription toast wording
* Add a community supported badge to specific detectors in the info summaries to better separate
* Make object classification publish to tracked object update and add examples for state classification
* Add item to advanced docs about tensorflow limiting
* Don't show submission for in progress objects
* fix for ios not reporting video dimensions on initial metadata load
in testing, polling with requestAnimationFrame finds the dimensions within 2 frames
* Catch jetson nvidia device tree
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Don't warn when event ids have expired for trigger sync
* Import faster_whisper conditinally to avoid illegal instruction
* Catch OpenVINO runtime error
* fix race condition in detail stream context
navigating between tracked objects in Explore would sometimes prevent the object track from appearing
* Handle case where classification images are deleted
* Adjust default rounded corners on larger screens
* Improve flow handling for classification state
* Remove images when wizard is cancelled
* Improve deletion handling for classes
* Set constraints on review buffers
* Update to support correct data format
* Set minimum duration for recording based review items
* Use friendly name in review genai prompt
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
