administrator/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-10-20 13:42:47 +03:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Docs refactor (#22703)
CI / AMD64 Build (push) Waiting to run
Details
CI / ARM Build (push) Waiting to run
Details
CI / Jetson Jetpack 6 (push) Waiting to run
Details
CI / AMD64 Extra Build (push) Blocked by required conditions
Details
CI / ARM Extra Build (push) Blocked by required conditions
Details
CI / Synaptics Build (push) Blocked by required conditions
Details
CI / Assemble and push default build (push) Blocked by required conditions
Details

Browse Source 
* add generation script

a script to read yaml code blocks from docs markdown files and generate corresponding "Frigate UI" tab instructions based on the json schema, i18n, section configs (hidden fields), and nav mappings

* first pass

* components

* add to gitignore

* second pass

* fix broken anchors

* fixes

* clean up tabs

* version bump

* tweaks

* remove role mapping config from ui
This commit is contained in:
Josh Hawkins
2026-03-30 10:36:45 -06:00
committed by GitHub
parent a5e3dfd107
commit 5a5d23b503
53 changed files with 5310 additions and 276 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/configuration/genai/config.md
+106
View File
@@ -3,6 +3,10 @@ id: genai_config
title: Configuring Generative AI
---
import ConfigTabs from "@site/src/components/ConfigTabs";
import TabItem from "@theme/TabItem";
import NavPath from "@site/src/components/NavPath";
## Configuration
A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently 4 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI-Compatible section below.
@@ -69,6 +73,18 @@ You must use a vision capable model with Frigate. The llama.cpp server supports
All llama.cpp native options can be passed through `provider_options`, including `temperature`, `top_k`, `top_p`, `min_p`, `repeat_penalty`, `repeat_last_n`, `seed`, `grammar`, and more. See the [llama.cpp server documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) for a complete list of available parameters.
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `llamacpp`
- Set **Base URL** to your llama.cpp server address (e.g., `http://localhost:8080`)
- Set **Model** to the name of your model
- Under **Provider Options**, set `context_size` to tell Frigate your context size so it can send the appropriate amount of information
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: llamacpp
@@ -78,6 +94,9 @@ genai:
context_size: 16000 # Tell Frigate your context size so it can send the appropriate amount of information.
```
</TabItem>
</ConfigTabs>
### Ollama
[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
@@ -96,6 +115,18 @@ Note that Frigate will not automatically download the model you specify in your
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `ollama`
- Set **Base URL** to your Ollama server address (e.g., `http://localhost:11434`)
- Set **Model** to the model tag (e.g., `qwen3-vl:4b`)
- Under **Provider Options**, set `keep_alive` (e.g., `-1`) and `options.num_ctx` to match your desired context size
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: ollama
@@ -107,6 +138,9 @@ genai:
num_ctx: 8192 # make sure the context matches other services that are using ollama
```
</TabItem>
</ConfigTabs>
### OpenAI-Compatible
Frigate supports any provider that implements the OpenAI API standard. This includes self-hosted solutions like [vLLM](https://docs.vllm.ai/), [LocalAI](https://localai.io/), and other OpenAI-compatible servers.
@@ -130,6 +164,18 @@ This ensures Frigate uses the correct context window size when generating prompt
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `openai`
- Set **Base URL** to your server address (e.g., `http://your-server:port`)
- Set **API key** if required by your server
- Set **Model** to the model name
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: openai
@@ -138,6 +184,9 @@ genai:
model: your-model-name
```
</TabItem>
</ConfigTabs>
To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
## Cloud Providers
@@ -150,6 +199,17 @@ Ollama also supports [cloud models](https://ollama.com/cloud), where your local
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `ollama`
- Set **Base URL** to your local Ollama address (e.g., `http://localhost:11434`)
- Set **Model** to the cloud model name
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: ollama
@@ -157,6 +217,9 @@ genai:
model: cloud-model-name
```
</TabItem>
</ConfigTabs>
### Google Gemini
Google Gemini has a [free tier](https://ai.google.dev/pricing) for the API, however the limits may not be sufficient for standard Frigate usage. Choose a plan appropriate for your installation.
@@ -176,6 +239,17 @@ To start using Gemini, you must first get an API key from [Google AI Studio](htt
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `gemini`
- Set **API key** to your Gemini API key (or use an environment variable such as `{FRIGATE_GEMINI_API_KEY}`)
- Set **Model** to the desired model (e.g., `gemini-2.5-flash`)
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: gemini
@@ -183,6 +257,9 @@ genai:
model: gemini-2.5-flash
```
</TabItem>
</ConfigTabs>
:::note
To use a different Gemini-compatible API endpoint, set the `provider_options` with the `base_url` key to your provider's API URL. For example:
@@ -213,6 +290,17 @@ To start using OpenAI, you must first [create an API key](https://platform.opena
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `openai`
- Set **API key** to your OpenAI API key (or use an environment variable such as `{FRIGATE_OPENAI_API_KEY}`)
- Set **Model** to the desired model (e.g., `gpt-4o`)
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: openai
@@ -220,6 +308,9 @@ genai:
model: gpt-4o
```
</TabItem>
</ConfigTabs>
:::note
To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
@@ -257,6 +348,18 @@ To start using Azure OpenAI, you must first [create a resource](https://learn.mi
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `azure_openai`
- Set **Base URL** to your Azure resource URL including the `api-version` parameter (e.g., `https://instance.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview`)
- Set **Model** to your deployed model name (e.g., `gpt-5-mini`)
- Set **API key** to your Azure OpenAI API key (or use an environment variable such as `{FRIGATE_OPENAI_API_KEY}`)
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: azure_openai
@@ -264,3 +367,6 @@ genai:
model: gpt-5-mini
api_key: "{FRIGATE_OPENAI_API_KEY}"
```
</TabItem>
</ConfigTabs>
docs/docs/configuration/genai/objects.md
+40 -4
View File
@@ -3,6 +3,10 @@ id: genai_objects
title: Object Descriptions
---
import ConfigTabs from "@site/src/components/ConfigTabs";
import TabItem from "@theme/TabItem";
import NavPath from "@site/src/components/NavPath";
Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail.
Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle, or can optionally be sent earlier after a number of significantly changed frames, for example in use in more real-time notifications. Descriptions can also be regenerated manually via the Frigate UI. Note that if you are manually entering a description for tracked objects prior to its end, this will be overwritten by the generated response.
@@ -15,9 +19,9 @@ Generative AI object descriptions can also be toggled dynamically for a camera v
## Usage and Best Practices
Frigate's thumbnail search excels at identifying specific details about tracked objects – for example, using an "image caption" approach to find a "person wearing a yellow vest," "a white dog running across the lawn," or "a red car on a residential street." To enhance this further, Frigate’s default prompts are designed to ask your AI provider about the intent behind the object's actions, rather than just describing its appearance.
Frigate's thumbnail search excels at identifying specific details about tracked objects -- for example, using an "image caption" approach to find a "person wearing a yellow vest," "a white dog running across the lawn," or "a red car on a residential street." To enhance this further, Frigate's default prompts are designed to ask your AI provider about the intent behind the object's actions, rather than just describing its appearance.
While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate’s default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what’s happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they’re moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation’s context.
While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate's default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what's happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they're moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation's context.
## Custom Prompts
@@ -33,7 +37,18 @@ Prompts can use variable replacements `{label}`, `{sub_label}`, and `{camera}` t
:::
You are also able to define custom prompts in your configuration.
You can define custom prompts at the global level and per-object type. To configure custom prompts:
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Global configuration > Objects" />.
- Expand the **GenAI object config** section
- Set **Caption prompt** to your custom prompt text
- Under **Object prompts**, add entries keyed by object type (e.g., `person`, `car`) with custom prompts for each
</TabItem>
<TabItem value="yaml">
```yaml
genai:
@@ -49,7 +64,25 @@ objects:
car: "Observe the primary vehicle in these images. Focus on its movement, direction, or purpose (e.g., parking, approaching, circling). If it's a delivery vehicle, mention the company."
```
Prompts can also be overridden at the camera level to provide a more detailed prompt to the model about your specific camera, if you desire.
</TabItem>
</ConfigTabs>
Prompts can also be overridden at the camera level to provide a more detailed prompt to the model about your specific camera. To configure camera-level overrides:
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Camera configuration > Objects" /> for the desired camera.
- Expand the **GenAI object config** section
- Set **Enable GenAI** to on
- Set **Use snapshots** to on if desired
- Set **Caption prompt** to a camera-specific prompt
- Under **Object prompts**, add entries keyed by object type with camera-specific prompts
- Set **GenAI objects** to the list of object types that should receive descriptions (e.g., `person`, `cat`)
- Set **Required zones** to limit descriptions to objects in specific zones (e.g., `steps`)
</TabItem>
<TabItem value="yaml">
```yaml
cameras:
@@ -69,6 +102,9 @@ cameras:
- steps
```
</TabItem>
</ConfigTabs>
### Experiment with prompts
Many providers also have a public facing chat interface for their models. Download a couple of different thumbnails or snapshots from Frigate and try new things in the playground to get descriptions to your liking before updating the prompt in Frigate.
docs/docs/configuration/genai/review_summaries.md
+71 -3
View File
@@ -3,6 +3,10 @@ id: genai_review
title: Review Summaries
---
import ConfigTabs from "@site/src/components/ConfigTabs";
import TabItem from "@theme/TabItem";
import NavPath from "@site/src/components/NavPath";
Generative AI can be used to automatically generate structured summaries of review items. These summaries will show up in Frigate's native notifications as well as in the UI. Generative AI can also be used to take a collection of summaries over a period of time and provide a report, which may be useful to get a quick report of everything that happened while out for some amount of time.
Requests for a summary are requested automatically to your AI provider for alert review items when the activity has ended, they can also be optionally enabled for detections as well.
@@ -28,6 +32,30 @@ This will show in multiple places in the UI to give additional context about eac
Each installation and even camera can have different parameters for what is considered suspicious activity. Frigate allows the `activity_context_prompt` to be defined globally and at the camera level, which allows you to define more specifically what should be considered normal activity. It is important that this is not overly specific as it can sway the output of the response.
To configure the activity context prompt:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Global configuration > Review" />.
- Set **GenAI config > Activity context prompt** to your custom activity context text
</TabItem>
<TabItem value="yaml">
```yaml
review:
genai:
activity_context_prompt: |
### Normal Activity Indicators (Level 0)
- Known/verified people in any zone at any time
...
```
</TabItem>
</ConfigTabs>
<details>
<summary>Default Activity Context Prompt</summary>
@@ -74,7 +102,18 @@ review:
### Image Source
By default, review summaries use preview images (cached preview frames) which have a lower resolution but use fewer tokens per image. For better image quality and more detailed analysis, you can configure Frigate to extract frames directly from recordings at a higher resolution:
By default, review summaries use preview images (cached preview frames) which have a lower resolution but use fewer tokens per image. For better image quality and more detailed analysis, configure Frigate to extract frames directly from recordings at a higher resolution.
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Global configuration > Review" />.
- Set **GenAI config > Enable GenAI descriptions** to on
- Set **GenAI config > Review image source** to `recordings` (default is `preview`)
</TabItem>
<TabItem value="yaml">
```yaml
review:
@@ -84,6 +123,9 @@ review:
image_source: recordings # Options: "preview" (default) or "recordings"
```
</TabItem>
</ConfigTabs>
When using `recordings`, frames are extracted at 480px height while maintaining the camera's original aspect ratio, providing better detail for the LLM while being mindful of context window size. This is particularly useful for scenarios where fine details matter, such as identifying license plates, reading text, or analyzing distant objects.
The number of frames sent to the LLM is dynamically calculated based on:
@@ -103,7 +145,17 @@ If recordings are not available for a given time period, the system will automat
### Additional Concerns
Along with the concern of suspicious activity or immediate threat, you may have concerns such as animals in your garden or a gate being left open. These concerns can be configured so that the review summaries will make note of them if the activity requires additional review. For example:
Along with the concern of suspicious activity or immediate threat, you may have concerns such as animals in your garden or a gate being left open. Configure these concerns so that review summaries will make note of them if the activity requires additional review.
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Global configuration > Review" />.
- Set **GenAI config > Additional concerns** to a list of your concerns (e.g., `animals in the garden`)
</TabItem>
<TabItem value="yaml">
```yaml {4,5}
review:
@@ -113,9 +165,22 @@ review:
- animals in the garden
```
</TabItem>
</ConfigTabs>
### Preferred Language
By default, review summaries are generated in English. You can configure Frigate to generate summaries in your preferred language by setting the `preferred_language` option:
By default, review summaries are generated in English. Configure Frigate to generate summaries in your preferred language by setting the `preferred_language` option.
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Global configuration > Review" />.
- Set **GenAI config > Preferred language** to the desired language (e.g., `Spanish`)
</TabItem>
<TabItem value="yaml">
```yaml {4}
review:
@@ -124,6 +189,9 @@ review:
preferred_language: Spanish
```
</TabItem>
</ConfigTabs>
## Review Reports
Along with individual review item summaries, Generative AI can also produce a single report of review items from all cameras marked "suspicious" over a specified time period (for example, a daily summary of suspicious activity while you're on vacation).