Commit Graph

10 Commits

Author SHA1 Message Date
Nicolas Mowen
66a2417229
Support Dynamic Thinking Models (#23281)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Add ability to toggle thinking

* Disable thinking for descriptions automatically

* mypy

* Cleanup
2026-05-21 12:54:23 -05:00
Nicolas Mowen
b0b00fe1d0
GenAI Refactor (#23253)
* Ensure runtime options are passed

* Add attribute info to prompt when configured

* Move GenAI plugins to dedicated directory

* Migrate prompts to dedicated folder

* Move chat prompts to prompts

* Implement reasoning traces in the UI

* Cleanup

* Make azure a subclass of openai

* Implement reasoning for other providers

* mypy

* Cleanup
2026-05-19 13:03:57 -05:00
Josh Hawkins
32daf6f494
Miscellaneous fixes (#23217)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* fix hardcoded leading-slash hrefs to respect FRIGATE_BASE_PATH

* update docs for default detector
2026-05-17 14:40:33 -06:00
Nicolas Mowen
d9c1ea908d
Chat improvements (#23195)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Support token streaming stats

* Propogate streaming token stats to chat calls

* Show token stats for each image

* Add settings to handle token stats and other options

* i18n

* Use select

* Improve mobile layout and spacing
2026-05-14 12:05:38 -05:00
Josh Hawkins
ca75f06456
Miscellaneous fixes (#23186)
* improve scroll handling for non-modal DropdownMenu in classification and face selection dialogs

* clean up

* fix incorrect key capitalization

* fix profile array overrides not replacing base arrays

don't use lodash merge(), it does positional merging and an empty source array doesn't override the destination, and shorter arrays leak destination elements through.

backend is unaffected, so the saved config and actual backend functionality was right

* only show audio debug tab when audio is enabled in config

* move apple_compatibility out of advanced

* remove retry_interval from UI

99% of users should never be changing this

* hide switch in optionalfieldwidget if editing a profile

* add override badges for cameras and profiles

collect shared functions into the config util and separate hooks

* Use new models endpoint info to determine modalities

* clarify language

* fix linter

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2026-05-13 11:04:11 -05:00
Nicolas Mowen
0ea8924727
GenAI Optimizations (#23006)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Test for image token usage in llama.cpp so we can more appropriately decide how many frames to include

* Limit based on frames per second

* handle zone case sensitivity

* Improve formatting

* Add observations field so model can build CoT before outputting used fields
2026-04-25 17:38:18 -05:00
Josh Hawkins
98c2fe00c1
Chat improvements (#22823)
* Add score fusion helpers for find_similar_objects chat tool

* Add candidate query builder for find_similar_objects chat tool

* register find_similar_objects chat tool definition

* implement _execute_find_similar_objects chat tool dispatcher

* Dispatch find_similar_objects in chat tool executor

* Teach chat system prompt when to use find_similar_objects

* Add i18n strings for find_similar_objects chat tool

* Add frontend extractor for find_similar_objects tool response

* Render anchor badge and similarity scores in chat results

* formatting

* filter similarity results in python, not sqlite-vec

* extract pure chat helpers to chat_util module

* Teach chat system prompt about attached_event marker

* Add parseAttachedEvent and prependAttachment helpers

* Add i18n strings for chat event attachments

* Add ChatAttachmentChip component

* Make chat thumbnails attach to composer on click

* Render attachment chip in user chat bubbles

* Add ChatQuickReplies pill row component

* Add ChatPaperclipButton with event picker popover

* Wire event attachments into chat composer and messages

* add ability to stop streaming

* tweak cursor to appear at the end of the same line of the streaming response

* use abort signal

* add tooltip

* display label and camera on attachment chip
2026-04-09 14:31:37 -06:00
Josh Hawkins
e24eb676a9
GenAI tweaks (#22756)
* add DictAsYamlField for genai provider and runtime options

* regenerate config translations

* chat tweaks

- add page title
- scroll if near bottom
- add tool call group that dynamically updates as tool calls are made
- add bouncing loading indicator and other UI polish

* tool call grouping
2026-04-04 06:54:51 -06:00
Nicolas Mowen
e1245cb93d
Improve profile state management and add recap tool (#22715)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Improve profile information

* Add chat tools

* Add quick links to new chats

* Improve usefulness

* Cleanup

* fix
2026-03-31 19:09:32 -05:00
Nicolas Mowen
fa1f9a1fa4
Add GenAI Backend Streaming and Chat (#22152)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / Assemble and push default build (push) Blocked by required conditions
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
* Add basic chat page with entry

* Add chat history

* processing

* Add markdown

* Improvements

* Adjust timing format

* Reduce fields in response

* More time parsing improvements

* Show tool calls separately from message

* Add title

* Improve UI handling

* Support streaming

* Full streaming support

* Fix tool calling

* Add copy button

* Improvements to UI

* Improve default behavior

* Implement message editing

* Add sub label to event tool filtering

* Cleanup

* Cleanup UI and prompt

* Cleanup UI bubbles

* Fix loading

* Add support for markdown tables

* Add thumbnail images to object results

* Add a starting state for chat

* Clenaup
2026-02-27 09:07:30 -07:00