mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-10-13 10:12:47 +03:00
Docs refactor (#22703)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* add generation script a script to read yaml code blocks from docs markdown files and generate corresponding "Frigate UI" tab instructions based on the json schema, i18n, section configs (hidden fields), and nav mappings * first pass * components * add to gitignore * second pass * fix broken anchors * fixes * clean up tabs * version bump * tweaks * remove role mapping config from ui
This commit is contained in:
Josh Hawkins
@@ -3,13 +3,17 @@ id: object_classification
|
||||
title: Object Classification
|
||||
---
|
||||
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
|
||||
Object classification allows you to train a custom MobileNetV2 classification model to run on tracked objects (persons, cars, animals, etc.) to identify a finer category or attribute for that object. Classification results are visible in the Tracked Object Details pane in Explore, through the `frigate/tracked_object_details` MQTT topic, in Home Assistant sensors via the official Frigate integration, or through the event endpoints in the HTTP API.
|
||||
|
||||
## Minimum System Requirements
|
||||
|
||||
Object classification models are lightweight and run very fast on CPU.
|
||||
|
||||
Training the model does briefly use a high amount of system resources for about 1–3 minutes per training run. On lower-power devices, training may take longer.
|
||||
Training the model does briefly use a high amount of system resources for about 1-3 minutes per training run. On lower-power devices, training may take longer.
|
||||
|
||||
A CPU with AVX + AVX2 instructions is required for training and inference.
|
||||
|
||||
@@ -27,7 +31,7 @@ For object classification:
|
||||
### Classification Type
|
||||
|
||||
- **Sub label**:
|
||||
- Applied to the object’s `sub_label` field.
|
||||
- Applied to the object's `sub_label` field.
|
||||
- Ideal for a single, more specific identity or type.
|
||||
- Example: `cat` → `Leo`, `Charlie`, `None`.
|
||||
|
||||
@@ -55,7 +59,7 @@ This two-step verification prevents false positives by requiring consistent pred
|
||||
|
||||
### Sub label
|
||||
|
||||
- **Known pet vs unknown**: For `dog` objects, set sub label to your pet’s name (e.g., `buddy`) or `none` for others.
|
||||
- **Known pet vs unknown**: For `dog` objects, set sub label to your pet's name (e.g., `buddy`) or `none` for others.
|
||||
- **Mail truck vs normal car**: For `car`, classify as `mail_truck` vs `car` to filter important arrivals.
|
||||
- **Delivery vs non-delivery person**: For `person`, classify `delivery` vs `visitor` based on uniform/props.
|
||||
|
||||
@@ -68,7 +72,27 @@ This two-step verification prevents false positives by requiring consistent pred
|
||||
|
||||
## Configuration
|
||||
|
||||
Object classification is configured as a custom classification model. Each model has its own name and settings. You must list which object labels should be classified.
|
||||
Object classification is configured as a custom classification model. Each model has its own name and settings. Specify which object labels should be classified.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to the **Classification** page from the main navigation sidebar, then click **Add Classification**.
|
||||
|
||||
In the **Create New Classification** dialog:
|
||||
|
||||
| Field | Description |
|
||||
| ----------------------- | ------------------------------------------------------------- |
|
||||
| **Name** | A name for your classification model (e.g., `dog`) |
|
||||
| **Type** | Select **Object** for object classification |
|
||||
| **Object Label** | The object label to classify (e.g., `dog`, `person`, `car`) |
|
||||
| **Classification Type** | Whether to assign results as a **Sub Label** or **Attribute** |
|
||||
| **Classes** | The class names the model will learn to distinguish between |
|
||||
|
||||
The `threshold` (default: `0.8`) can be adjusted in the YAML configuration.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
classification:
|
||||
@@ -82,6 +106,9 @@ classification:
|
||||
|
||||
An optional config, `save_attempts`, can be set as a key under the model name. This defines the number of classification attempts to save in the Recent Classifications tab. For object classification models, the default is 200.
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
## Training the model
|
||||
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of two steps:
|
||||
@@ -104,18 +131,18 @@ If examples for some of your classes do not appear in the grid, you can continue
|
||||
|
||||
:::tip Diversity matters far more than volume
|
||||
|
||||
Selecting dozens of nearly identical images is one of the fastest ways to degrade model performance. MobileNetV2 can overfit quickly when trained on homogeneous data — the model learns what *that exact moment* looked like rather than what actually defines the class. **This is why Frigate does not implement bulk training in the UI.**
|
||||
Selecting dozens of nearly identical images is one of the fastest ways to degrade model performance. MobileNetV2 can overfit quickly when trained on homogeneous data — the model learns what _that exact moment_ looked like rather than what actually defines the class. **This is why Frigate does not implement bulk training in the UI.**
|
||||
|
||||
For more detail, see [Frigate Tip: Best Practices for Training Face and Custom Classification Models](https://github.com/blakeblackshear/frigate/discussions/21374).
|
||||
|
||||
:::
|
||||
|
||||
- **Start small and iterate**: Begin with a small, representative set of images per class. Models often begin working well with surprisingly few examples and improve naturally over time.
|
||||
- **Favor hard examples**: When images appear in the Recent Classifications tab, prioritize images scoring below 90–100% or those captured under new lighting, weather, or distance conditions.
|
||||
- **Favor hard examples**: When images appear in the Recent Classifications tab, prioritize images scoring below 90-100% or those captured under new lighting, weather, or distance conditions.
|
||||
- **Avoid bulk training similar images**: Training large batches of images that already score 100% (or close) adds little new information and increases the risk of overfitting.
|
||||
- **The wizard is just the starting point**: You don’t need to find and label every class upfront. Missing classes will naturally appear in Recent Classifications, and those images tend to be more valuable because they represent new conditions and edge cases.
|
||||
- **The wizard is just the starting point**: You don't need to find and label every class upfront. Missing classes will naturally appear in Recent Classifications, and those images tend to be more valuable because they represent new conditions and edge cases.
|
||||
- **Problem framing**: Keep classes visually distinct and relevant to the chosen object types.
|
||||
- **Preprocessing**: Ensure examples reflect object crops similar to Frigate’s boxes; keep the subject centered.
|
||||
- **Preprocessing**: Ensure examples reflect object crops similar to Frigate's boxes; keep the subject centered.
|
||||
- **Labels**: Keep label names short and consistent; include a `none` class if you plan to ignore uncertain predictions for sub labels.
|
||||
- **Threshold**: Tune `threshold` per model to reduce false assignments. Start at `0.8` and adjust based on validation.
|
||||
|
||||
@@ -125,6 +152,17 @@ To troubleshoot issues with object classification models, enable debug logging t
|
||||
|
||||
Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > System > Logging" />.
|
||||
|
||||
- Set **Logging level** to `debug`
|
||||
- Set **Per-process log level > Frigate.Data Processing.Real Time.Custom Classification** to `debug` for verbose classification logging
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
logger:
|
||||
default: info
|
||||
@@ -133,6 +171,9 @@ logger:
|
||||
frigate.data_processing.real_time.custom_classification: debug
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
The debug logs will show:
|
||||
|
||||
- Classification probabilities for each attempt
|
||||
|
||||
@@ -3,13 +3,17 @@ id: state_classification
|
||||
title: State Classification
|
||||
---
|
||||
|
||||
import ConfigTabs from "@site/src/components/ConfigTabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import NavPath from "@site/src/components/NavPath";
|
||||
|
||||
State classification allows you to train a custom MobileNetV2 classification model on a fixed region of your camera frame(s) to determine a current state. The model can be configured to run on a schedule and/or when motion is detected in that region. Classification results are available through the `frigate/<camera_name>/classification/<model_name>` MQTT topic and in Home Assistant sensors via the official Frigate integration.
|
||||
|
||||
## Minimum System Requirements
|
||||
|
||||
State classification models are lightweight and run very fast on CPU.
|
||||
|
||||
Training the model does briefly use a high amount of system resources for about 1–3 minutes per training run. On lower-power devices, training may take longer.
|
||||
Training the model does briefly use a high amount of system resources for about 1-3 minutes per training run. On lower-power devices, training may take longer.
|
||||
|
||||
A CPU with AVX + AVX2 instructions is required for training and inference.
|
||||
|
||||
@@ -33,7 +37,25 @@ For state classification:
|
||||
|
||||
## Configuration
|
||||
|
||||
State classification is configured as a custom classification model. Each model has its own name and settings. You must provide at least one camera crop under `state_config.cameras`.
|
||||
State classification is configured as a custom classification model. Each model has its own name and settings. Provide at least one camera crop under `state_config.cameras`.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to the **Classification** page from the main navigation sidebar, select the **States** tab, then click **Add Classification**.
|
||||
|
||||
In the **Create New Classification** dialog:
|
||||
|
||||
| Field | Description |
|
||||
| ----------- | ------------------------------------------------------------------------------------ |
|
||||
| **Name** | A name for your state classification model (e.g., `front_door`) |
|
||||
| **Type** | Select **State** for state classification |
|
||||
| **Classes** | The state names the model will learn to distinguish between (e.g., `open`, `closed`) |
|
||||
|
||||
After creating the model, the wizard will guide you through selecting the camera crop area and assigning training examples. The `threshold` (default: `0.8`), `motion`, and `interval` settings can be adjusted in the YAML configuration.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
classification:
|
||||
@@ -50,6 +72,9 @@ classification:
|
||||
|
||||
An optional config, `save_attempts`, can be set as a key under the model name. This defines the number of classification attempts to save in the Recent Classifications tab. For state classification models, the default is 100.
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
## Training the model
|
||||
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of three steps:
|
||||
@@ -72,7 +97,7 @@ Once some images are assigned, training will begin automatically.
|
||||
|
||||
:::tip Diversity matters far more than volume
|
||||
|
||||
Selecting dozens of nearly identical images is one of the fastest ways to degrade model performance. MobileNetV2 can overfit quickly when trained on homogeneous data — the model learns what *that exact moment* looked like rather than what actually defines the state. This often leads to models that work perfectly under the original conditions but become unstable when day turns to night, weather changes, or seasonal lighting shifts. **This is why Frigate does not implement bulk training in the UI.**
|
||||
Selecting dozens of nearly identical images is one of the fastest ways to degrade model performance. MobileNetV2 can overfit quickly when trained on homogeneous data — the model learns what _that exact moment_ looked like rather than what actually defines the state. This often leads to models that work perfectly under the original conditions but become unstable when day turns to night, weather changes, or seasonal lighting shifts. **This is why Frigate does not implement bulk training in the UI.**
|
||||
|
||||
For more detail, see [Frigate Tip: Best Practices for Training Face and Custom Classification Models](https://github.com/blakeblackshear/frigate/discussions/21374).
|
||||
|
||||
@@ -82,7 +107,7 @@ For more detail, see [Frigate Tip: Best Practices for Training Face and Custom C
|
||||
- **Problem framing**: Keep classes visually distinct and state-focused (e.g., `open`, `closed`, `unknown`). Avoid combining object identity with state in a single model unless necessary.
|
||||
- **Data collection**: Use the model's Recent Classifications tab to gather balanced examples across times of day and weather.
|
||||
- **When to train**: Focus on cases where the model is entirely incorrect or flips between states when it should not. There's no need to train additional images when the model is already working consistently.
|
||||
- **Favor hard examples**: When images appear in the Recent Classifications tab, prioritize images scoring below 90–100% or those captured under new conditions (e.g., first snow of the year, seasonal changes, objects temporarily in view, insects at night). These represent scenarios different from the default state and help prevent overfitting.
|
||||
- **Favor hard examples**: When images appear in the Recent Classifications tab, prioritize images scoring below 90-100% or those captured under new conditions (e.g., first snow of the year, seasonal changes, objects temporarily in view, insects at night). These represent scenarios different from the default state and help prevent overfitting.
|
||||
- **Avoid bulk training similar images**: Training large batches of images that already score 100% (or close) adds little new information and increases the risk of overfitting.
|
||||
- **The wizard is just the starting point**: You don't need to find and label every state upfront. Missing states will naturally appear in Recent Classifications, and those images tend to be more valuable because they represent new conditions and edge cases.
|
||||
|
||||
@@ -92,6 +117,17 @@ To troubleshoot issues with state classification models, enable debug logging to
|
||||
|
||||
Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
|
||||
|
||||
<ConfigTabs>
|
||||
<TabItem value="ui">
|
||||
|
||||
Navigate to <NavPath path="Settings > System > Logging" />.
|
||||
|
||||
- Set **Logging level** to `debug`
|
||||
- Set **Per-process log level > `frigate.data_processing.real_time.custom_classification`** to `debug` for verbose classification logging
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="yaml">
|
||||
|
||||
```yaml
|
||||
logger:
|
||||
default: info
|
||||
@@ -100,6 +136,9 @@ logger:
|
||||
frigate.data_processing.real_time.custom_classification: debug
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</ConfigTabs>
|
||||
|
||||
The debug logs will show:
|
||||
|
||||
- Classification probabilities for each attempt
|
||||
|
||||
Reference in New Issue
Block a user