administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-10-07 15:22:48 +03:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Implement annotated frames for GenAI Review (#24379)
CI / AMD64 Build (push) Canceled after 0s
Details
CI / AMD64 Smoke Test (push) Canceled after 0s
Details
CI / ARM Build (push) Canceled after 0s
Details
CI / Jetson Jetpack 6 (push) Canceled after 0s
Details
CI / AMD64 Extra Build (push) Canceled after 0s
Details
CI / ARM Extra Build (push) Canceled after 0s
Details
CI / Synaptics Build (push) Canceled after 0s
Details
CI / Assemble and push default build (push) Canceled after 0s
Details

Browse Source 
* Implement annotated frames mode for GenAI reviews to improve models with lacking temporal understanding

* Updates

* Improve debug sharing

* Do not number objects

* Fix assumptions

* Remove unhelpful content

* Improve object data sent as part of prompt

* Cleanup ollama dumbness

* Bind db

* Fixes

* Cleanup
This commit is contained in:
Nicolas Mowen
2026-09-17 10:28:37 -06:00
committed by GitHub
parent 10a0d5ea37
commit eccd10cd94
19 changed files with 1128 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/configuration/genai/config.md
+5 -5
View File
@@ -63,11 +63,11 @@ Running Generative AI models on CPU is not recommended, as high inference times
You must use a vision-capable model with Frigate. The following models are recommended for local deployment of the `descriptions` and `chat` roles:
| Model | Notes |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. |
| `qwen3.6`/`qwen3.8` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. |
| Model | Review [frame mode](/configuration/genai/genai_review#frame-mode) | Notes |
| ------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | `frames` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. Follows a sequence of frames on its own. |
| `qwen3.6`/`qwen3.8` | `frames` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | `annotated_frames` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. Loses track of activity that repeats or reverses, so it benefits from annotated frames. |
#### Embedding models