mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-10-29 09:54:41 +03:00
Enable ability to toggle thinking for local models genai review
This commit is contained in:
@@ -504,6 +504,9 @@ review:
|
||||
# Presets adjust the tone and level of detail of the user-facing title,
|
||||
# summary, and scene description; "default" leaves the built-in prompt unchanged.
|
||||
response_style: default
|
||||
# Optional: Let the model think before writing the description (default: shown below)
|
||||
# Only applies to models with a per-request thinking toggle (ollama and llamacpp providers).
|
||||
enable_thinking: False
|
||||
# Optional: Save thumbnails sent to the GenAI provider for review/debugging purposes (default: shown below)
|
||||
debug_save_thumbnails: False
|
||||
|
||||
|
||||
@@ -119,7 +119,7 @@ Vision-language models come in **instruct** variants (fine-tuned to follow instr
|
||||
|
||||
Frigate manages reasoning per task automatically:
|
||||
|
||||
- **Description tasks** (object descriptions, review descriptions, review summaries) are synthesis-only and benefit from concise, direct output, so Frigate disables thinking for these calls when the model exposes a per-request toggle.
|
||||
- **Description tasks** (object descriptions, review descriptions, review summaries) are synthesis-only and benefit from concise, direct output, so Frigate disables thinking for these calls when the model exposes a per-request toggle. Review descriptions can opt back in with [`enable_thinking`](/configuration/genai/genai_review#thinking) for models trained to reason over video.
|
||||
- **Chat** lets you toggle thinking on or off from the composer when the configured model supports it.
|
||||
|
||||
You can use a pure instruct, hybrid, or thinking-capable model with Frigate. No extra configuration is required to disable thinking for descriptions.
|
||||
@@ -510,7 +510,7 @@ Frigate's Generative AI features are configured and enabled separately. [Review
|
||||
|
||||
5. Save the exact images and prompts that were sent to your provider.
|
||||
- Turn on **Save thumbnails** for the feature you are debugging (`review.genai.debug_save_thumbnails` or `objects.genai.debug_save_thumbnails`). Both features write to `/media/frigate/clips/genai-requests/`, and these files are admin-only.
|
||||
- Review descriptions write `genai-requests/<review_id>/` containing the numbered frames that were sent, plus `prompt.txt` and `response.txt` with the exact prompt and the raw, unparsed model response.
|
||||
- Review descriptions write `genai-requests/<review_id>/` containing the numbered frames that were sent, plus `prompt.txt` and `response.txt` with the exact prompt and the raw, unparsed model response. When the provider returns a reasoning trace (for example with [thinking](/configuration/genai/genai_review#thinking) enabled), it is saved to `reasoning.txt`.
|
||||
- Review summary reports write `genai-requests/<start_ts>-<end_ts>/prompt.txt` and `response.txt`. No images are involved, since a report summarizes existing review descriptions.
|
||||
- Object descriptions write `genai-requests/<event_id>/` containing the numbered thumbnails. The prompt for object descriptions is not written to a file, it is only visible in the debug logs from step 4.
|
||||
- Look at the saved images before blaming the model. If the object is small, blurry, or out of frame, no prompt will fix the result. For object descriptions, consider turning on **Use snapshots** (`objects.genai.use_snapshot`) to send a higher quality image. For review items, consider setting **Review image source** (`review.genai.image_source`) to `recordings` for 480p frames instead of the lower resolution preview frames.
|
||||
|
||||
@@ -264,6 +264,36 @@ review:
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
### Thinking
|
||||
|
||||
By default, Frigate turns thinking off for review descriptions, since most models write better descriptions without spending tokens on a reasoning trace. Some models are trained to reason over video, and may follow complex scenes more accurately when allowed to think first. The `enable_thinking` option lets you turn thinking on for review descriptions to evaluate this with your model.
|
||||
|
||||
`enable_thinking` only applies to models that expose a per-request thinking toggle with the `ollama` or `llamacpp` providers. For llama.cpp, the model's chat template must use the `enable_thinking` variable. Other models keep their default behavior. With `openai`, use `runtime_options` (for example `reasoning_effort`) to control reasoning instead.
|
||||
|
||||
Thinking makes each review description take longer and use more tokens. If responses come back empty, the model may be using its whole output budget on reasoning.
|
||||
|
||||
When **Save thumbnails** (`debug_save_thumbnails`) is enabled, the reasoning trace is saved to `reasoning.txt` alongside `prompt.txt` and `response.txt`, so you can see what the model considered before answering. The trace is saved even when the model returns no answer. To compare a saved example with thinking on and off, use `testing-scripts/genai_review_tester.py` in the Frigate repository.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > Global configuration > Review" />.
|
||||
|
||||
- Enable **GenAI config > Enable thinking**. This option is only shown when the provider assigned the descriptions role is `ollama` or `llamacpp`.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml {4}
|
||||
review:
|
||||
genai:
|
||||
enabled: true
|
||||
enable_thinking: true
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
## Review Reports
|
||||
|
||||
Along with individual review item summaries, Generative AI can also produce a single report of review items from all cameras marked "suspicious" over a specified time period (for example, a daily summary of suspicious activity while you're on vacation).
|
||||
|
||||
Reference in New Issue
Block a user