Commit Graph
15 Commits
Author SHA1 Message Date
Josh HawkinsandGitHub 7af29a5a53 only render same-origin images in chat replies (#24716)
Assistant markdown rendered any image URL, so a prompt injection in a GenAI description, transcription, or sub label could make the model emit an image pointing at an outside host and the browser would fetch it with chat context in the query string, without a click. Off-origin images now render as their alt text, and off-origin links open in a new tab with noopener noreferrer.
2026-10-19 04:47:05 -10:00
Josh HawkinsandGitHub 3d08bbe520 Miscellaneous fixes (#24402)
CI / AMD64 Build (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* check for a valid frame before using its shape

With the camera offline, no preview frame, and `camera-error.jpg` missing, `latest_frame` read `frame.shape` before its `frame is None` check, so it raised `AttributeError` and answered 500 with a traceback instead of the intended "Unable to get valid frame". The check now runs first.

* fix the has_clip self-heal for events with no recordings

`vod_event` looked for a `(body, 404)` tuple, but `vod_ts` returns a `JSONResponse`, so the check never matched and an old event whose recordings are gone kept offering a clip that can't play. It now checks the response status code.

* return 403 for a snapshot or thumbnail on another camera

The broad `except Exception` handlers in `event_snapshot` and `event_thumbnail` caught the `HTTPException` from `require_camera_access`, so a restricted user asking for another camera's snapshot got a 404 instead of a 403, and for an object still being tracked the snapshot was rendered before the check ran. Both endpoints now look up the event and check access in their own block, the way the other endpoints do, so a denial propagates.

* find DST transitions to the second

`get_dst_transitions` probed the offset once every 24 hours from the start time and reported a change at the first probe after it, up to a day late, so events, review items and recordings near a transition were grouped into days with the old offset. A transition after the last daily probe wasn't found at all. The end of the range is probed too now, and a probe that sees the offset change bisects the interval to the second of the transition.

* don't run page shortcuts for keys a dialog already handled

Radix dismisses a dialog on Escape from a capture-phase keydown listener and calls `preventDefault()` without stopping propagation, so `useKeyboardListener` still ran the page's Escape shortcut: cancelling the delete dialog in the face library or a classification model also cleared the whole selection. Keys another shortcut hook handled still get through, since their listener order changes with every render.

* fix train image filtering for a class with a dash

The backend writes a class with a `-` as `_` in train file names, since it splits those names on `-`, while a dataset folder keeps the dash. Filtering the Train grid by `half-open` compared it with `half_open` and hid every attempt. Both sides are normalized the same way now.

* don't edit a chat message while a reply streams

The edit button stayed active while a reply streamed. `submitConversation` returns early while loading, but the message bubble still closed its editor, so the edit was silently lost. The edit button is hidden while a reply streams, and an editor that's already open keeps its draft with send disabled until the reply ends.

* fix restart failing under non-root

restart_frigate() called psutil.Process(1).terminate() to signal s6-svscan, but s6-svscan runs as root while frigate runs as uid 1000, so the call raised AccessDenied. That exception escaped every caller: the UI restart button dropped its websocket client, MQTT restart and Save & Restart just logged and did nothing, and the watchdog crashed its own monitoring thread on a dead detector. This catches AccessDenied and falls through to the existing SIGINT branch, which exits the process for s6 to restart it.

* show runtime overrides in the settings form

The settings form read a camera section's saved config value, but its dependent warnings (audio transcription requiring audio detection, snapshots requiring detect, etc.) read the live config instead. A runtime toggle from the live view, MQTT, or an active profile can turn a section off without touching yaml, and that override persists across restarts, so the Enable switch showed on while the warning said the feature wasn't enabled. This adds an "Overridden (Live)" badge to any field whose live value differs from what's saved, and swaps the affected warnings to runtime-specific wording when a runtime override is the actual cause instead of the config.

* fix mobile overflowing icons in system due to new health pane

* fix genai settings keeping a stale model and dropping roles after save

Switching a GenAI entry's provider left the previous provider's model selected, so saving wrote a model the new provider doesn't serve. llama.cpp can't find that model in `/v1/models`, so the backend reported every capability as false for the entry, and once the save refetched `genai/models` the roles widget stripped `transcribe` from the form on its own. The section showed unsaved changes right after saving, and saving again would have dropped the role. Switching provider now clears the model, and the roles widget only strips a role for a model or provider picked in the form, since the entry-level capability flags only describe the saved model. A selected role stays visible when the provider can't confirm it, so it can still be switched off. The llama.cpp model list also no longer repeats a model whose alias matches its id, which is what `--alias` produces.

* close onvif sessions on shutdown

`OnvifController.close()` only stopped its event loop, so the aiohttp sessions each `ONVIFCamera` holds and the `_poll_config_updates` task were left to be garbage collected during interpreter shutdown, when their warnings can no longer be logged. Every restart ended with a run of `Unclosed client session` and `Task was destroyed but it is pending!` logging errors, which only became visible once restart started exiting the process itself under non-root. `close()` now closes each camera's client and cancels the tasks on the loop before stopping it.

* fixes

* fixes
2026-09-18 07:33:23 -06:00
Josh HawkinsandGitHub 4c648f8147 bump copy-to-clipboard to 4.0.2 (#24353) 2026-09-15 12:03:20 -05:00
Nicolas Mowen af60d2db48 GenAI Chat Improvements (#24173)
* Initial tool approval implementation

* Cleanups and fixes

* Improve robustness of loading case
2026-09-12 07:30:04 -06:00
Josh HawkinsandGitHub 6fdd65ddb5 UI tweaks (#23346)
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* remove redundant per-view toasters in settings

* add variants to standardize dialog footer button layouts

* remove text-md

this class name compiles to nothing in tailwind. we used to add it to prevent iOS from zooming when focusing on an input, but that is now solved via the viewport meta in index.html

* make wizard footers consistent with dialog footers

* consistent destructive button style

remove text-white from individual buttons and add it to the variant
2026-05-29 16:00:30 -06:00
Nicolas MowenandGitHub 66a2417229 Support Dynamic Thinking Models (#23281)
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Extra Build (push) Blocked by required conditions
* Add ability to toggle thinking

* Disable thinking for descriptions automatically

* mypy

* Cleanup
2026-05-21 12:54:23 -05:00
Nicolas MowenandGitHub b0b00fe1d0 GenAI Refactor (#23253)
* Ensure runtime options are passed

* Add attribute info to prompt when configured

* Move GenAI plugins to dedicated directory

* Migrate prompts to dedicated folder

* Move chat prompts to prompts

* Implement reasoning traces in the UI

* Cleanup

* Make azure a subclass of openai

* Implement reasoning for other providers

* mypy

* Cleanup
2026-05-19 13:03:57 -05:00
Josh HawkinsandGitHub 32daf6f494 Miscellaneous fixes (#23217)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* fix hardcoded leading-slash hrefs to respect FRIGATE_BASE_PATH

* update docs for default detector
2026-05-17 14:40:33 -06:00
Nicolas MowenandGitHub d9c1ea908d Chat improvements (#23195)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Support token streaming stats

* Propogate streaming token stats to chat calls

* Show token stats for each image

* Add settings to handle token stats and other options

* i18n

* Use select

* Improve mobile layout and spacing
2026-05-14 12:05:38 -05:00
ca75f06456 Miscellaneous fixes (#23186)
* improve scroll handling for non-modal DropdownMenu in classification and face selection dialogs

* clean up

* fix incorrect key capitalization

* fix profile array overrides not replacing base arrays

don't use lodash merge(), it does positional merging and an empty source array doesn't override the destination, and shorter arrays leak destination elements through.

backend is unaffected, so the saved config and actual backend functionality was right

* only show audio debug tab when audio is enabled in config

* move apple_compatibility out of advanced

* remove retry_interval from UI

99% of users should never be changing this

* hide switch in optionalfieldwidget if editing a profile

* add override badges for cameras and profiles

collect shared functions into the config util and separate hooks

* Use new models endpoint info to determine modalities

* clarify language

* fix linter

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2026-05-13 11:04:11 -05:00
Nicolas MowenandGitHub 0ea8924727 GenAI Optimizations (#23006)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Test for image token usage in llama.cpp so we can more appropriately decide how many frames to include

* Limit based on frames per second

* handle zone case sensitivity

* Improve formatting

* Add observations field so model can build CoT before outputting used fields
2026-04-25 17:38:18 -05:00
Josh HawkinsandGitHub 98c2fe00c1 Chat improvements (#22823)
* Add score fusion helpers for find_similar_objects chat tool

* Add candidate query builder for find_similar_objects chat tool

* register find_similar_objects chat tool definition

* implement _execute_find_similar_objects chat tool dispatcher

* Dispatch find_similar_objects in chat tool executor

* Teach chat system prompt when to use find_similar_objects

* Add i18n strings for find_similar_objects chat tool

* Add frontend extractor for find_similar_objects tool response

* Render anchor badge and similarity scores in chat results

* formatting

* filter similarity results in python, not sqlite-vec

* extract pure chat helpers to chat_util module

* Teach chat system prompt about attached_event marker

* Add parseAttachedEvent and prependAttachment helpers

* Add i18n strings for chat event attachments

* Add ChatAttachmentChip component

* Make chat thumbnails attach to composer on click

* Render attachment chip in user chat bubbles

* Add ChatQuickReplies pill row component

* Add ChatPaperclipButton with event picker popover

* Wire event attachments into chat composer and messages

* add ability to stop streaming

* tweak cursor to appear at the end of the same line of the streaming response

* use abort signal

* add tooltip

* display label and camera on attachment chip
2026-04-09 14:31:37 -06:00
Josh HawkinsandGitHub e24eb676a9 GenAI tweaks (#22756)
* add DictAsYamlField for genai provider and runtime options

* regenerate config translations

* chat tweaks

- add page title
- scroll if near bottom
- add tool call group that dynamically updates as tool calls are made
- add bouncing loading indicator and other UI polish

* tool call grouping
2026-04-04 06:54:51 -06:00
Nicolas MowenandGitHub e1245cb93d Improve profile state management and add recap tool (#22715)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Improve profile information

* Add chat tools

* Add quick links to new chats

* Improve usefulness

* Cleanup

* fix
2026-03-31 19:09:32 -05:00
Nicolas MowenandGitHub fa1f9a1fa4 Add GenAI Backend Streaming and Chat (#22152)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Add basic chat page with entry

* Add chat history

* processing

* Add markdown

* Improvements

* Adjust timing format

* Reduce fields in response

* More time parsing improvements

* Show tool calls separately from message

* Add title

* Improve UI handling

* Support streaming

* Full streaming support

* Fix tool calling

* Add copy button

* Improvements to UI

* Improve default behavior

* Implement message editing

* Add sub label to event tool filtering

* Cleanup

* Cleanup UI and prompt

* Cleanup UI bubbles

* Fix loading

* Add support for markdown tables

* Add thumbnail images to object results

* Add a starting state for chat

* Clenaup
2026-02-27 09:07:30 -07:00