debug_save_thumbnails was only honored when a description was sent at the end of a tracked object, so requests triggered by after_significant_updates never wrote their images to genai-requests. The save now lives in a helper that both paths call.
The mypy cleanup in https://github.com/blakeblackshear/frigate/pull/22658 added a check on the tensor details to the top of process_frame for both state and object models. Those are unset until a model is trained, so untrained models returned early and never saved images to Recent Classifications. Remove the check from process_frame, the classify methods already check the tensor details before using them.
* run lpr on more vehicle types by default
before, a config change to attribute_map was required
* logging tweaks
* remove arg
* use attributes for frontend check
* docs
* only check thumbnail attributes the object can have
sanitize_filename leaves ".." intact and collapses variants like "..:" and "..*" to "..", so filesystem paths built from face names, classification model/category names, image ids, and trigger data could escape their base directory. Route every such site through new frigate/util/path.py helpers (safe_join, sanitize_path_component, sanitize_contained_path), which reject traversal and verify containment.
Worst case was DELETE /classification/{name}, which rmtree'd /media/frigate and /config while returning 200.
Important to note that all affected endpoints already require admin permission, so this sould be considered hardening rather than fixing exploitable code.
* Catch faces that become empty after cropping
* don't drop batched camera add/remove config updates
TrackedObjectProcessor drained all pending camera config updates at once but handled them in a mutually exclusive if/elif on enabled/add/remove, so only one topic was processed per drain. When an add arrived in the same batch as an enabled update, the add was skipped and the new camera never got a camera state. Adding a camera reliably produced that batch: config_set now re-applies runtime overrides, which republishes an enabled update for every previously toggled camera immediately before the add, in the same request. The dashboard and camera capture still saw the camera (the maintainer does not subscribe to enabled, so it got a clean add-only batch), but object_processing did not, and disabling the camera then crashed with a KeyError on the unguarded camera_states lookup.
Handle add and remove independently instead of as exclusive branches so a batched add is no longer dropped, and guard the remove lookup so a missing state is skipped rather than raising. Drop the enabled branch entirely: it only ever set prev_enabled when it was None, but prev_enabled is seeded to a bool at camera state creation and is never None (mypy flags the body as unreachable), and the actual enable/disable transition is already driven by the disabled-state loop from config.enabled.
* Don't stay on motion search page when user cancels flow
* fix notification test button being blocked by websocket auth
* fix overflowing model names in settings genai widget
* add note about auth debugging
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Pin ruff
* Add python upgrade fixes
This enables python upgrade checks in ruff to look for deprecated types and patterns. This namely fixes:
- usage of deprecated `Typing` which is now built in
- some specific exceptions which are caught and have new aliases
Some specific UP checks were also ignored as they are stylistic / unimportant and likely to cause bugs
* Remove async blocking calls
Use asyncio.to_thread on two remaining blocking calls to fix hanging event thread loop. Enable this specific rule to block it in the future.
* Use proper logging mechanism
* Correctly format logs
* Raise with context
When raising an exception include the from context to improve debugging
* Cleanup
* fix stale active object indicators on the live dashboard
The camera_activity/<camera> snapshot cache is only written when a client sends onConnect, and object "end" events only update the local state of mounted useCameraActivity hooks, never the cache. As a result, a hook that seeded from a stale cache or missed an "end" event while disconnected showed objects that had already left, with no path to correct itself short of a full page reload.
This change will re-request the snapshot on hook mount (collapsed to one onConnect per task across camera cards), and always re-notify camera_activity topics so hooks reconcile against their own local state instead of relying on snapshot-vs-snapshot comparison, and clear the payload dedup cache on reconnect and resync so byte-identical snapshots still apply.
* docs tweaks
* fix mqtt log message
* use consistent values for lpr debug frame filenames
with millisecond resolution
* apply object events through a functional updater to prevent lost updates
The events effect derived a new objects list from the value captured at render time and wrote the whole list back. When events arrived close
together, a run derived from a stale list erased a concurrent run's removal; the resurrected object then had no remaining "end" event to clear it, and the add branch could mint a duplicate entry that no splice could ever remove, leaving the live dashboard showing active objects the backend had already cleared, until a page reload.
The fix is to apply each event inside setObjects so it operates on the true current list exactly once. Unchanged results return the same reference so React bails out of re-rendering, and the label rewrite is hoisted so added objects get the sub_label/verified label directly instead of relying on the effect re-running against its own state update.
* add optional onClick to EmptyCard
* show EmptyCard in face rec when face library is empty
* add loading indicator
* add description to camera management pane
* Cleanup when use snapshot but can't load snapshot
* Migrate files
* fix birdseye color distortion when configured aspect ratio is unsupported
* Skip processing end for object descriptions
* don't crash if stats is null
* fix genai roles in migration
* frigate+ pane updates
- allow users to select a plus model from the select even when one was not previously loaded
- always show model summary card
- add model filter popover
- add restart button totast
* fix frigate+ pane layout and buttons to match other settings panes
* match button layout in go2rtc settings view
* make audio maintainer respond to dynamic config updates
* check correct zone name in publish state
* fix nested translation extraction for Optional dict and list fields
* mypy
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Change order
* Improve title
* add loading spinner to exports
* Simplify JSON since not all providers see or use this the same
* Add fields to primary prompt
* Adjust centering for no overrides
* Use GenAI title for exports when available
* detect form-root objects by field path instead of schema identity
* add bosnian
* Strip v1 if included in url
* prevent fast clicks in video controls from selecting text
* Use title for metadata chapters
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* fix(face_recognition): feed BGR (not RGB) to FaceDetectorYN in manual detection branch
Frigate's `requires_face_detection` branch in `FaceRealTimeProcessor.process_frame`
converts the YUV camera frame to RGB and passes it to `cv2.FaceDetectorYN`.
YuNet is trained on BGR — feeding it RGB silently degrades detection
confidence by ~10× on typical person crops, causing face_recognition to
emit no `sub_label` and produce no `train/` entries. There is no log signal
because the detector simply returns 0 faces; from outside the box it looks
like nobody is walking past any camera.
The same file already does the YUV→BGR conversion correctly in the
else-branch (was line 271, now line 285) — only the manual-detection
branch was missed.
## Reproduction
Verified in-pod against the running Frigate's models on identical
person crops (snapshot pulled from a real person event):
BGR (correct): cv2.FaceDetectorYN ← confidence 0.744 ✓
RGB (current): cv2.FaceDetectorYN ← confidence 0.047 ✗
The `score_threshold=0.5` set on `FaceDetectorYN.create()` filters anything
under 0.5 at the detector layer, so the RGB-degraded crops never reach
the user-configurable `detection_threshold`. Result: silent outage.
## Fix
Three changes in `frigate/data_processing/real_time/face.py`:
1. `cv2.COLOR_YUV2RGB_I420` → `cv2.COLOR_YUV2BGR_I420`
2. Variable rename `rgb` → `bgr` to match
3. Remove the now-redundant `cv2.cvtColor(face_frame, cv2.COLOR_RGB2BGR)`
block — `face_frame` is already BGR after the upstream conversion change
Net diff: +6 / -7. Pure Python, no new dependencies.
## How a deployment confirms the fix
After this change, walking past a camera produces:
- `data.attributes` with a `face` entry on the person event (currently empty)
- New entries in `/api/faces` `train/` array (currently frozen)
- `sub_label` populated on subsequent person events for trained faces
Signed-off-by: Vinnie Esposito <vespo21@gmail.com>
* Cleanup comment
---------
Signed-off-by: Vinnie Esposito <vespo21@gmail.com>
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* lpr fixes
- remove duplicate code
- fix min_area check for non frigate+ code path
- move log outside of non frigate+ code path
* only show chat link when a genai provider is configured with the chat role
* respect ui.timezone when generating fallback export names
* reapply radix pointer events fix to call sites that use navigate()
* formatting
* fall back to prior preview frame for short export thumbnails
* fix typing
* fix e2e test for chat navigation
* batch annotation offset to seek atomically and throttle slider drag
* add debug replay loading toast for explore actions
* Improve handling of webpush missing shortSummary
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Move openai specific workaround so it doesn't apply to other providers
* Fix gemini tool calling
* Improve efficiency of frame listing for previews
* debug replay fixes
- initial selection without changing the radio button in the dialog would select 1 hour (rather than 1 minute)
- use CLIPS_DIR instead of CACHE_DIR so that longer replay clips don't cause tmpfs cache overflows
* don't re-render the tracking details overlay on every video time tick
* change pinned to planned
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* add ui to camera config update topics enum
* add mqtt to camera config update enum
* ensure cleanup runs when an event end skips post-processing
* end any in-progress audio events when audio detection is disabled
we already end in-progress audio events when we disable a camera, but this mirrors that logic for specifically disabling audio detection
* Improve GenAI metadata
* fix invalid recording segment topic being misrouted to the valid handler
* Add confidence default to avoid unnecessary field causing issues
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Reduce max frames per second to 1
* Use pydantic but don't fail if some constraints are not met.
* Adjust limits
* Adjust limits
* Cleanup
* add unsaved changes icon/popover to individual settings section
* allow changing camera friendly_name from camera management pane
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Test for image token usage in llama.cpp so we can more appropriately decide how many frames to include
* Limit based on frames per second
* handle zone case sensitivity
* Improve formatting
* Add observations field so model can build CoT before outputting used fields
* ensure classification wizard dialog is scrollable on mobile too
* add chat and features group to mobile menu
Co-authored-by: Copilot <copilot@github.com>
* Set min length for summary too
* Don't use orange for review item
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix mobile export crash by removing stale iOS non-modal drawer workaround
* Remove titlecase to avoid Gemma4 handling plain labels as proper nouns
* Improve titling:
* Make directions more clear
* Properly capitalize delivery services
* update dispatcher config reference on save
* subscribe to review topic so ReviewDescriptionProcessor knows genai is enabled
* auto-send ON genai review WS message when enabled_in_config transitions to true
* remove unused object level
* update docs to clarify pre/post capture settings
* add ui docs links
* improve known_plates field in settings UI
* only show save all when multiple sections are changed
or if the section being changed is not currently being viewed
* fix docs
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Switch to a feature-based roles so it is easier to choose models for different tasks
* Fallback and try llama-swap format
* List models supported by provider
* Cleanup
* Add frontend
* Improve model loading
* Make it possible to update genai without restarting
* Cleanup
* Cleanup
* Mypy
* Use different association method
* Clarify
* Remove extra details from ollama schema
* Fix Gemini Chat
* Fix incorrect instructions
* Improve name handling
* Change order of information for llama.cpp
* Simplify prompt
* Fix formatting
* Improve title to better capture activity
* Improve efficiency of prompt
* Use json format for llama.cpp
* Cleanup prompt
* Add output format for other LLMs
* enrichment updater and enum
* update_config stubs
* config updaters in enrichments
* update maintainer
* formatting
* simplify enrichment config updates to use single subscriber with topic-based routing
apply length and format filters to the clustered representative plate rather than individual OCR readings, so noisy variants still contribute to clustering even when they don't pass on their own
* face recognition dynamic config
* lpr dynamic config
* safe changes for birdseye dynamic config
* bird classification dynamic config
* always assign new config to stats emitter to make telemetry fields dynamic
* add wildcard support for camera config updates in config_set
* update restart required fields for global sections
* add test
* fix rebase issue
* collapsible settings sidebar
use the preexisting control available with shadcn's sidebar (cmd/ctrl-B) to give users more space to set masks/zones on smaller screens
* dynamic ffmpeg
* ensure previews dir exists
when ffmpeg processes restart, there's a brief window where the preview frame generation pipeline is torn down and restarted. before these changes, ffmpeg only restarted on crash/stall recovery or full Frigate restart. Now that ffmpeg restarts happen on-demand via config changes, there's a higher chance a frontend request hits the preview_mp4 or preview_gif endpoints during that brief restart window when the directory might not exist yet. The existing os.listdir() call would throw FileNotFoundError without a directory existence check. this fix just checks if the directory exists and returns 404 if not, exactly how preview_thumbnail already handles the same scenario a few lines below
* global ffmpeg section
* clean up
* tweak
* fix test
* migrator and runtime config changes
* component changes to use rasterized_mask
* frontend
* convert none to empty string for config save
* i18n
* update tests
* add enabled config to zones
* zones frontend
* i18n
* docs
* tweaks
* use dashed stroke to indicate disabled
* allow toggle from icon
* use filelock to ensure atomic config updates from endpoint
* enforce atomic config update in the frontend
* toggle via mqtt
* fix global object masks
* correctly handle global object masks in dispatcher
* ws hooks
* render masks and zones based on ws enabled state
* use enabled_in_config for zones and masks
* frontend for enabled_in_config
* tweaks
* i18n
* publish websocket on config save
* i18n tweaks
* pydantic title and description
* i18n generation
* tweaks
* fix typing
The fallback to tensorflow was established back in 2023, because we could
not provide tflite-runtime downstream in nixpkgs.
By now we have ai-edge-litert available, which is the successor to the
tflite-runtime. It still provides the same entrypoints as tflite-runtime
and functionality has been verified in multiple deployments for the last
two weeks.
* Adjust title prompt to have less rigidity
* Improve motion boxes handling for features that don't require motion
* Improve handling of classes starting with digits
* Improve vehicle nuance
* tweak lpr docs
* Improve grammar
* Don't allow # in face name
* add password requirements to new user dialog
* change password requirements
* Clenaup
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add shortSummary field to review summary to be used for notifications
* pull in current config version into default config
* fix crash when dynamically adding cameras
depending on where we are in the update loop, camera configs might not be updated yet and we are receiving detections already
* add no tracked objects and icon to explore summary view
* reset add camera wizard when closing and saving
* don't flash no exports icon while loading
* Improve handling of homekit config
* Increase prompt tokens reservation
* Adjust
* Catch event not found object detection
* Use thread lock for JinaV2 in onnxruntime
* remove incorrect embeddings process from memray docs
* only show transcribe button if audio event has video
* apply aspect ratio and margin constraints to path overlay in detail stream on mobile
improves a specific case where the overlay was not aligned with 4:3 cameras on mobile phones
* show metadata title as tooltip on icon hover in detail stream
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Send preferred language for report service
* make object lifecycle scrollable in tracking details
* fix info popovers in live camera drawer
* ensure metrics are initialized if genai is enabled
* docs
* ollama cloud model docs
* Ensure object descriptions get claened up
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* remove footer messages and add update topic to motion tuner view
restart after changing values is no longer required
* add cache key and activity indicator for loading classification wizard images
* Always mark model as untrained when a classname is changed
* clarify object classification docs
* add debug logs for individual lpr replace_rules
* update memray docs
* memray tweaks
* Don't fail for audio transcription when semantic search is not enabled
* Fix incorrect mismatch for object vs sub label
* Check if the video is currently playing when deciding to seek due to misalignment
* Refactor timeline event handling to allow multiple timeline entries per update
* Check if zones have actually changed (not just count) for event state update
* show event icon on mobile
* move div inside conditional
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Fix genai callbacks in MQTT
* Cleanup cursor pointer for classification cards
* Cleanup
* Handle unknown SOCs for RKNN converter by only using known SOCs
* don't allow "none" as a classification class name
* change internal port user to admin and default unspecified username to viewer
* keep 5000 as anonymous user
* suppress tensorflow logging during classification training
* Always apply base log level suppressions for noisy third-party libraries even if no specific logConfig is provided
* remove decorator and specifically suppress TFLite delegate creation messages
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
