Compare commits

...
23 Commits
Author SHA1 Message Date
Josh Hawkins 26d4f5f0ab remove decorator and specifically suppress TFLite delegate creation messages 2025-12-18 15:43:36 -06:00
Josh Hawkins 2f9308e0b1 Always apply base log level suppressions for noisy third-party libraries even if no specific logConfig is provided 2025-12-18 15:16:26 -06:00
Josh Hawkins 886d9c4a55 suppress tensorflow logging during classification training 2025-12-18 15:07:41 -06:00
Josh Hawkins ba4f4304ec keep 5000 as anonymous user 2025-12-18 09:34:44 -06:00
Josh Hawkins e84ef53536 change internal port user to admin and default unspecified username to viewer 2025-12-18 09:12:54 -06:00
Josh Hawkins b052035046 don't allow "none" as a classification class name 2025-12-18 08:41:47 -06:00
Nicolas Mowen 657fb6da62 Handle unknown SOCs for RKNN converter by only using known SOCs 2025-12-18 07:37:23 -07:00
Nicolas Mowen 63d3227457 Cleanup 2025-12-18 07:30:19 -07:00
Nicolas Mowen eefa0977e3 Cleanup cursor pointer for classification cards 2025-12-18 07:27:24 -07:00
Nicolas Mowen 86f3244b6a Fix genai callbacks in MQTT 2025-12-18 07:20:01 -07:00
GuoQing LiuandGitHub 074b060e9c fix: temp directory is only created when there are review_items. (#21344) 2025-12-18 07:08:45 -07:00
ae009b9861 Miscellaneous Fixes (0.17 beta) (#21336)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* fix coral docs

* add note about sub label object classification with person

* Catch OSError for deleting classification image

* add docs for dummy camera debugging

* add to sidebar

* fix formatting

* fix

* avx instructions are required for classification

* break text on classification card to prevent button overflow

* Ensure there is no NameError when processing

* Don't use region for state classification models

* fix spelling

* Handle attribute based models

* Catch case of non-trained model that doesn't add infinite number of classification images

* Actually train object classification models automatically

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-12-17 16:52:27 -07:00
GuoQing LiuandGitHub 13957fec00 classification i18n fix (#21331)
* fix: fix classification pages none label i18n

* fix: fix README_CN formatting issue
2025-12-17 15:26:11 -07:00
Blake BlackshearandGitHub 3edfd905de consider anonymous user authenticated (#21335)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* consider anonymous user authenticated

* simplify and update comments
2025-12-17 08:01:20 -06:00
78eace258e Miscellaneous Fixes (0.17 Beta) (#21320)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
* Exclude D-FINE from using CUDA Graphs

* fix objects count in detail stream

* Add debugging for classification models

* validate idb stored stream name and reset if invalid

fixes https://github.com/blakeblackshear/frigate/discussions/21311

* ensure jina loading takes place in the main thread to prevent lazily importing tensorflow in another thread later

reverts atexit changes in https://github.com/blakeblackshear/frigate/pull/21301 and fixes https://github.com/blakeblackshear/frigate/discussions/21306

* revert old atexit change in bird too

* revert types

* ensure we bail in the live mode hook for empty camera groups

prevent infinite rendering on camera groups with no cameras

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-12-16 22:35:43 -06:00
Issy SzemetiandGitHub c292cd207d Align node versions used in GHA PR workflow (#21302)
* Add node/npm version config to package.json

* Bump npm version/fix node version format

* Version range

* Use package.json for github actions node version

* Unification

* Move it all to the bottom

* Remove this

* Bump versions in docs

* Add volta config here too

* Revert changes

* Revert this
2025-12-16 20:28:35 -07:00
e7d047715d Miscellaneous Fixes (0.17 beta) (#21301)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
* Wait for config to load before evaluating route access

Fix race condition where custom role users are temporarily denied access after login while config is still loading. Defer route rendering in DefaultAppView until config is available so the complete role list is known before ProtectedRoute evaluates permissions

* Use batching for state classification generation

* Ignore incorrect scoring images if they make it through the deletion

* Delete unclassified images

* mitigate tensorflow atexit crash by pre-importing tflite/tensorflow on main thread

Pre-import Interpreter in embeddings maintainer and add defensive lazy imports in classification processors to avoid worker-thread tensorflow imports causing "can't register atexit after shutdown"

* don't require old password for users with admin role when changing passwords

* don't render actions menu if no options are available

* Remove hwaccel arg as it is not used for encoding

* change password button text

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-12-16 08:11:53 -06:00
Issy SzemetiandGitHub 818cccb2e3 Settings page layout shift - follow up (#21300)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Fix layout shift with camera filter

* Move min height
2025-12-15 11:42:11 -07:00
Issy SzemetiandGitHub f543d0ab31 Fix layout shift with camera filter (#21298) 2025-12-15 11:18:41 -07:00
GuoQing LiuandGitHub 39af85625e feat: add train classification download weights file endpoint (#21294)
* feat: add train classification download weights file endpoint: "TF_KERAS_MOBILENET_V2_ENDPOINT"

* refactor: custom weights file url
2025-12-15 08:59:13 -07:00
fa16539429 Miscellaneous Fixes (#21289)
* Exclude yolov9 license plate from migraphx runner

* clarify auth endpoint return in openapi schema

* Clarify ROCm enrichments

* fix object mask creation

* Consider audio activity when deciding if recording segments should be kept due to motion

* ensure python defs match openapi spec for auth endpoints

* Fix check for audio activity to keep a segemnt

* fix calendar popover modal bug on export dialog

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-12-15 09:32:11 -06:00
e1545a8db8 Miscellaneous Fixes (0.17 beta) (#21279)
CI / Synaptics Build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
* Fix Safari popover issue in classification wizard

* use name for key instead of title

prevents duplicate key warnings when users mix vaapi and qsv

* update auth api endpoint descriptions and docs

* tweak headings

* fix note

* clarify classification docs

* Fix cuda birdseye

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-12-14 16:41:38 -07:00
Josh HawkinsandGitHub 51ee6f26e6 Fix yolov9 coral docs labelmap path (#21278) 2025-12-14 11:00:48 -07:00
50 changed files with 916 additions and 269 deletions
+4 -4
View File
@@ -19,9 +19,9 @@ jobs:
- uses: actions/checkout@v6
with:
persist-credentials: false
- uses: actions/setup-node@master
- uses: actions/setup-node@v6
with:
node-version: 16.x
node-version: 20.x
- run: npm install
working-directory: ./web
- name: Lint
@@ -35,7 +35,7 @@ jobs:
- uses: actions/checkout@v6
with:
persist-credentials: false
- uses: actions/setup-node@master
- uses: actions/setup-node@v6
with:
node-version: 20.x
- run: npm install
@@ -78,7 +78,7 @@ jobs:
uses: actions/checkout@v6
with:
persist-credentials: false
- uses: actions/setup-node@master
- uses: actions/setup-node@v6
with:
node-version: 20.x
- name: Install devcontainer cli
+5 -4
View File
@@ -4,14 +4,14 @@
# Frigate NVR™ - 一个具有实时目标检测的本地 NVR
[English](https://github.com/blakeblackshear/frigate) | \[简体中文\]
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
<a href="https://hosted.weblate.org/engage/frigate-nvr/-/zh_Hans/">
<img src="https://hosted.weblate.org/widget/frigate-nvr/-/zh_Hans/svg-badge.svg" alt="翻译状态" />
</a>
[English](https://github.com/blakeblackshear/frigate) | \[简体中文\]
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
一个完整的本地网络视频录像机(NVR),专为[Home Assistant](https://www.home-assistant.io)设计,具备 AI 目标/物体检测功能。使用 OpenCV 和 TensorFlow 在本地为 IP 摄像头执行实时物体检测。
强烈推荐使用 GPU 或者 AI 加速器(例如[Google Coral 加速器](https://coral.ai/products/) 或者 [Hailo](https://hailo.ai/)等)。它们的运行效率远远高于现在的顶级 CPU,并且功耗也极低。
@@ -38,6 +38,7 @@
## 协议
本项目采用 **MIT 许可证**授权。
**代码部分**:本代码库中的源代码、配置文件和文档均遵循 [MIT 许可证](LICENSE)。您可以自由使用、修改和分发这些代码,但必须保留原始版权声明。
**商标部分**:“Frigate”名称、“Frigate NVR”品牌以及 Frigate 的 Logo 为 **Frigate LLC 的商标**,**不在** MIT 许可证覆盖范围内。
+11 -1
View File
@@ -237,8 +237,18 @@ ENV PYTHONWARNINGS="ignore:::numpy.core.getlimits"
# Set HailoRT to disable logging
ENV HAILORT_LOGGER_PATH=NONE
# TensorFlow error only
# TensorFlow C++ logging suppression (must be set before import)
# TF_CPP_MIN_LOG_LEVEL: 0=all, 1=INFO+, 2=WARNING+, 3=ERROR+ (we use 3 for errors only)
ENV TF_CPP_MIN_LOG_LEVEL=3
# Suppress verbose logging from TensorFlow C++ code
ENV TF_CPP_MIN_VLOG_LEVEL=3
# Disable oneDNN optimization messages ("optimized with oneDNN...")
ENV TF_ENABLE_ONEDNN_OPTS=0
# Suppress AutoGraph verbosity during conversion
ENV AUTOGRAPH_VERBOSITY=0
# Google Logging (GLOG) suppression for TensorFlow components
ENV GLOG_minloglevel=3
ENV GLOG_logtostderr=0
ENV PATH="/usr/local/go2rtc/bin:/usr/local/tempio/bin:/usr/local/nginx/sbin:${PATH}"
+39
View File
@@ -270,3 +270,42 @@ To use role-based access control, you must connect to Frigate via the **authenti
1. Log in as an **admin** user via port `8971`.
2. Navigate to **Settings > Users**.
3. Edit a users role by selecting **admin** or **viewer**.
## API Authentication Guide
### Getting a Bearer Token
To use the Frigate API, you need to authenticate first. Follow these steps to obtain a Bearer token:
#### 1. Login
Make a POST request to `/login` with your credentials:
```bash
curl -i -X POST https://frigate_ip:8971/api/login \
-H "Content-Type: application/json" \
-d '{"user": "admin", "password": "your_password"}'
```
:::note
You may need to include `-k` in the argument list in these steps (eg: `curl -k -i -X POST ...`) if your Frigate instance is using a self-signed certificate.
:::
The response will contain a cookie with the JWT token.
#### 2. Using the Bearer Token
Once you have the token, include it in the Authorization header for subsequent requests:
```bash
curl -H "Authorization: Bearer <your_token>" https://frigate_ip:8971/api/profile
```
#### 3. Token Lifecycle
- Tokens are valid for the configured session length
- Tokens are automatically refreshed when you visit the `/auth` endpoint
- Tokens are invalidated when the user's password is changed
- Use `/logout` to clear your session cookie
@@ -11,6 +11,8 @@ Object classification models are lightweight and run very fast on CPU. Inference
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
A CPU with AVX instructions is required for training and inference.
## Classes
Classes are the categories your model will learn to distinguish between. Each class represents a distinct visual category that the model will predict.
@@ -35,6 +37,12 @@ For object classification:
- Ideal when multiple attributes can coexist independently.
- Example: Detecting if a `person` in a construction yard is wearing a helmet or not.
:::note
A tracked object can only have a single sub label. If you are using Face Recognition and you configure an object classification model for `person` using the sub label type, your sub label may not be assigned correctly as it depends on which enrichment completes its analysis first. Consider using the `attribute` type instead.
:::
## Assignment Requirements
Sub labels and attributes are only assigned when both conditions are met:
@@ -94,3 +102,23 @@ When choosing which objects to classify, start with a small number of visually d
- **Preprocessing**: Ensure examples reflect object crops similar to Frigates boxes; keep the subject centered.
- **Labels**: Keep label names short and consistent; include a `none` class if you plan to ignore uncertain predictions for sub labels.
- **Threshold**: Tune `threshold` per model to reduce false assignments. Start at `0.8` and adjust based on validation.
## Debugging Classification Models
To troubleshoot issues with object classification models, enable debug logging to see detailed information about classification attempts, scores, and consensus calculations.
Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
```yaml
logger:
default: info
logs:
frigate.data_processing.real_time.custom_classification: debug
```
The debug logs will show:
- Classification probabilities for each attempt
- Whether scores meet the threshold requirement
- Consensus calculations and when assignments are made
- Object classification history and weighted scores
@@ -11,6 +11,8 @@ State classification models are lightweight and run very fast on CPU. Inference
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
A CPU with AVX instructions is required for training and inference.
## Classes
Classes are the different states an area on your camera can be in. Each class represents a distinct visual state that the model will learn to recognize.
@@ -60,11 +62,9 @@ Choose one or more cameras and draw a rectangle over the area of interest for ea
### Step 3: Assign Training Examples
The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state.
The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state. It's not strictly required to select all images you see. If a state is missing from the samples, you can train it from the Recent tab later.
**Important**: All images must be assigned to a state before training can begin. This includes images that may not be optimal, such as when people temporarily block the view, sun glare is present, or other distractions occur. Assign these images to the state that is actually present (based on what you know the state to be), not based on the distraction. This training helps the model correctly identify the state even when such conditions occur during inference.
Once all images are assigned, training will begin automatically.
Once some images are assigned, training will begin automatically.
### Improving the Model
@@ -72,3 +72,34 @@ Once all images are assigned, training will begin automatically.
- **Data collection**: Use the model's Recent Classifications tab to gather balanced examples across times of day and weather.
- **When to train**: Focus on cases where the model is entirely incorrect or flips between states when it should not. There's no need to train additional images when the model is already working consistently.
- **Selecting training images**: Images scoring below 100% due to new conditions (e.g., first snow of the year, seasonal changes) or variations (e.g., objects temporarily in view, insects at night) are good candidates for training, as they represent scenarios different from the default state. Training these lower-scoring images that differ from existing training data helps prevent overfitting. Avoid training large quantities of images that look very similar, especially if they already score 100% as this can lead to overfitting.
## Debugging Classification Models
To troubleshoot issues with state classification models, enable debug logging to see detailed information about classification attempts, scores, and state verification.
Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
```yaml
logger:
default: info
logs:
frigate.data_processing.real_time.custom_classification: debug
```
The debug logs will show:
- Classification probabilities for each attempt
- Whether scores meet the threshold requirement
- State verification progress (consecutive detections needed)
- When state changes are published
### Recent Classifications
For state classification, images are only added to recent classifications under specific circumstances:
- **First detection**: The first classification attempt for a camera is always saved
- **State changes**: Images are saved when the detected state differs from the current verified state
- **Pending verification**: Images are saved when there's a pending state change being verified (requires 3 consecutive identical states)
- **Low confidence**: Images with scores below 100% are saved even if the state matches the current state (useful for training)
Images are **not** saved when the state is stable (detected state matches current state) **and** the score is 100%. This prevents unnecessary storage of redundant high-confidence classifications.
@@ -13,7 +13,7 @@ Object detection and enrichments (like Semantic Search, Face Recognition, and Li
- **AMD**
- ROCm will automatically be detected and used for enrichments in the `-rocm` Frigate image.
- ROCm support in the `-rocm` Frigate image is automatically detected for enrichments, but only some enrichment models are available due to ROCm's focus on LLMs and limited stability with certain neural network models. Frigate disables models that perform poorly or are unstable to ensure reliable operation, so only compatible enrichments may be active.
- **Intel**
+7 -7
View File
@@ -146,16 +146,16 @@ detectors:
### EdgeTPU Supported Models
| Model | Notes |
| ------------------------------------- | ------------------------------------------- |
| [MobileNet v2](#ssdlite-mobilenet-v2) | Default model |
| [YOLOv9](#yolo-v9) | More accurate but slower than default model |
| Model | Notes |
| ----------------------- | ------------------------------------------- |
| [Mobiledet](#mobiledet) | Default model |
| [YOLOv9](#yolov9) | More accurate but slower than default model |
#### SSDLite MobileNet v2
#### Mobiledet
A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`.
#### YOLO v9
#### YOLOv9
[YOLOv9](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite) models that are compiled for Tensorflow Lite and properly quantized are supported, but not included by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`. Note that the model may require a custom label file (eg. [use this 17 label file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) for the model linked above.)
@@ -175,7 +175,7 @@ model:
width: 320 # <--- should match the imgsize of the model, typically 320
height: 320 # <--- should match the imgsize of the model, typically 320
path: /config/model_cache/yolov9-s-relu6-best_320_int8_edgetpu.tflite
labelmap_path: /labelmap/labels-coco-17.txt
labelmap_path: /config/labels-coco17.txt
```
Note that the labelmap uses a subset of the complete COCO label set that has only 17 objects.
+60
View File
@@ -0,0 +1,60 @@
---
id: dummy-camera
title: Troubleshooting Detection
---
When investigating object detection or tracking problems, it can be helpful to replay an exported video as a temporary "dummy" camera. This lets you reproduce issues locally, iterate on configuration (detections, zones, enrichment settings), and capture logs and clips for analysis.
## When to use
- Replaying an exported clip to reproduce incorrect detections
- Testing configuration changes (model settings, trackers, filters) against a known clip
- Gathering deterministic logs and recordings for debugging or issue reports
## Example Config
Place the clip you want to replay in a location accessible to Frigate (for example `/media/frigate/` or the repository `debug/` folder when developing). Then add a temporary camera to your `config/config.yml` like this:
```yaml
cameras:
test:
ffmpeg:
inputs:
- path: /media/frigate/car-stopping.mp4
input_args: -re -stream_loop -1 -fflags +genpts
roles:
- detect
detect:
enabled: true
record:
enabled: false
snapshots:
enabled: false
```
- `-re -stream_loop -1` tells `ffmpeg` to play the file in realtime and loop indefinitely, which is useful for long debugging sessions.
- `-fflags +genpts` helps generate presentation timestamps when they are missing in the file.
## Steps
1. Export or copy the clip you want to replay to the Frigate host (e.g., `/media/frigate/` or `debug/clips/`).
2. Add the temporary camera to `config/config.yml` (example above). Use a unique name such as `test` or `replay_camera` so it's easy to remove later.
- If you're debugging a specific camera, copy the settings from that camera (frame rate, model/enrichment settings, zones, etc.) into the temporary camera so the replay closely matches the original environment. Leave `record` and `snapshots` disabled unless you are specifically debugging recording or snapshot behavior.
3. Restart Frigate.
4. Observe the Debug view in the UI and logs as the clip is replayed. Watch detections, zones, or any feature you're looking to debug, and note any errors in the logs to reproduce the issue.
5. Iterate on camera or enrichment settings (model, fps, zones, filters) and re-check the replay until the behavior is resolved.
6. Remove the temporary camera from your config after debugging to avoid spurious telemetry or recordings.
## Variables to consider in object tracking
- The exported video will not always line up exactly with how it originally ran through Frigate (or even with the last loop). Different frames may be used on replay, which can change detections and tracking.
- Motion detection depends on the frames used; small frame shifts can change motion regions and therefore what gets passed to the detector.
- Object detection is not deterministic: models and post-processing can yield different results across runs, so you may not get identical detections or track IDs every time.
When debugging, treat the replay as a close approximation rather than a byte-for-byte replay. Capture multiple runs, enable recording if helpful, and examine logs and saved event clips to understand variability.
## Troubleshooting
- No video: verify the path is correct and accessible from the Frigate process/container.
- FFmpeg errors: check the log output for ffmpeg-specific flags and adjust `input_args` accordingly for your file/container. You may also need to disable hardware acceleration (`hwaccel_args: ""`) for the dummy camera.
- No detections: confirm the camera `roles` include `detect`, and model/detector configuration is enabled.
+1
View File
@@ -132,6 +132,7 @@ const sidebars: SidebarsConfig = {
"troubleshooting/gpu",
"troubleshooting/edgetpu",
"troubleshooting/memory",
"troubleshooting/dummy-camera",
],
Development: [
"development/contributing",
+106 -14
View File
@@ -14,19 +14,38 @@ paths:
get:
tags:
- Auth
summary: Auth
summary: Authenticate request
description: |-
Authenticates the current request based on proxy headers or JWT token.
This endpoint verifies authentication credentials and manages JWT token refresh.
On success, no JSON body is returned; authentication state is communicated via response headers and cookies.
operationId: auth_auth_get
responses:
"200":
description: Successful Response
content:
application/json:
schema: {}
"202":
description: Authentication Accepted (no response body, different headers depending on auth method)
headers:
remote-user:
description: Authenticated username or "viewer" in proxy-only mode
schema:
type: string
remote-role:
description: Resolved role (e.g., admin, viewer, or custom)
schema:
type: string
Set-Cookie:
description: May include refreshed JWT cookie ("frigate-token") when applicable
schema:
type: string
"401":
description: Authentication Failed
/profile:
get:
tags:
- Auth
summary: Profile
summary: Get user profile
description: |-
Returns the current authenticated user's profile including username, role, and allowed cameras.
This endpoint requires authentication and returns information about the user's permissions.
operationId: profile_profile_get
responses:
"200":
@@ -34,11 +53,16 @@ paths:
content:
application/json:
schema: {}
"401":
description: Unauthorized
/logout:
get:
tags:
- Auth
summary: Logout
summary: Logout user
description: |-
Logs out the current user by clearing the session cookie.
After logout, subsequent requests will require re-authentication.
operationId: logout_logout_get
responses:
"200":
@@ -46,11 +70,22 @@ paths:
content:
application/json:
schema: {}
"303":
description: See Other (redirects to login page)
/login:
post:
tags:
- Auth
summary: Login
summary: Login with credentials
description: |-
Authenticates a user with username and password.
Returns a JWT token as a secure HTTP-only cookie that can be used for subsequent API requests.
The JWT token can also be retrieved from the response and used as a Bearer token in the Authorization header.
Example using Bearer token:
```
curl -H "Authorization: Bearer <token_value>" https://frigate_ip:8971/api/profile
```
operationId: login_login_post
requestBody:
required: true
@@ -64,6 +99,11 @@ paths:
content:
application/json:
schema: {}
"401":
description: Login Failed - Invalid credentials
content:
application/json:
schema: {}
"422":
description: Validation Error
content:
@@ -74,7 +114,10 @@ paths:
get:
tags:
- Auth
summary: Get Users
summary: Get all users
description: |-
Returns a list of all users with their usernames and roles.
Requires admin role. Each user object contains the username and assigned role.
operationId: get_users_users_get
responses:
"200":
@@ -82,10 +125,19 @@ paths:
content:
application/json:
schema: {}
"403":
description: Forbidden - Admin role required
post:
tags:
- Auth
summary: Create User
summary: Create new user
description: |-
Creates a new user with the specified username, password, and role.
Requires admin role. Password must meet strength requirements:
- Minimum 8 characters
- At least one uppercase letter
- At least one digit
- At least one special character (!@#$%^&*(),.?":{}\|<>)
operationId: create_user_users_post
requestBody:
required: true
@@ -99,6 +151,13 @@ paths:
content:
application/json:
schema: {}
"400":
description: Bad Request - Invalid username or role
content:
application/json:
schema: {}
"403":
description: Forbidden - Admin role required
"422":
description: Validation Error
content:
@@ -109,7 +168,10 @@ paths:
delete:
tags:
- Auth
summary: Delete User
summary: Delete user
description: |-
Deletes a user by username. The built-in admin user cannot be deleted.
Requires admin role. Returns success message or error if user not found.
operationId: delete_user_users__username__delete
parameters:
- name: username
@@ -118,12 +180,15 @@ paths:
schema:
type: string
title: Username
description: The username of the user to delete
responses:
"200":
description: Successful Response
content:
application/json:
schema: {}
"403":
description: Forbidden - Cannot delete admin user or admin role required
"422":
description: Validation Error
content:
@@ -134,7 +199,17 @@ paths:
put:
tags:
- Auth
summary: Update Password
summary: Update user password
description: |-
Updates a user's password. Users can only change their own password unless they have admin role.
Requires the current password to verify identity for non-admin users.
Password must meet strength requirements:
- Minimum 8 characters
- At least one uppercase letter
- At least one digit
- At least one special character (!@#$%^&*(),.?":{}\|<>)
If user changes their own password, a new JWT cookie is automatically issued.
operationId: update_password_users__username__password_put
parameters:
- name: username
@@ -143,6 +218,7 @@ paths:
schema:
type: string
title: Username
description: The username of the user whose password to update
requestBody:
required: true
content:
@@ -155,6 +231,14 @@ paths:
content:
application/json:
schema: {}
"400":
description: Bad Request - Current password required or password doesn't meet requirements
"401":
description: Unauthorized - Current password is incorrect
"403":
description: Forbidden - Viewers can only update their own password
"404":
description: Not Found - User not found
"422":
description: Validation Error
content:
@@ -165,7 +249,10 @@ paths:
put:
tags:
- Auth
summary: Update Role
summary: Update user role
description: |-
Updates a user's role. The built-in admin user's role cannot be modified.
Requires admin role. Valid roles are defined in the configuration.
operationId: update_role_users__username__role_put
parameters:
- name: username
@@ -174,6 +261,7 @@ paths:
schema:
type: string
title: Username
description: The username of the user whose role to update
requestBody:
required: true
content:
@@ -186,6 +274,10 @@ paths:
content:
application/json:
schema: {}
"400":
description: Bad Request - Invalid role
"403":
description: Forbidden - Cannot modify admin user's role or admin role required
"422":
description: Validation Error
content:
+88 -41
View File
@@ -143,17 +143,6 @@ def require_admin_by_default():
return admin_checker
def _is_authenticated(request: Request) -> bool:
"""
Helper to determine if a request is from an authenticated user.
Returns True if the request has a valid authenticated user (not anonymous).
Port 5000 internal requests are considered anonymous despite having admin role.
"""
username = request.headers.get("remote-user")
return username is not None and username != "anonymous"
def allow_public():
"""
Override dependency to allow unauthenticated access to an endpoint.
@@ -173,27 +162,24 @@ def allow_public():
def allow_any_authenticated():
"""
Override dependency to allow any authenticated user (bypass admin requirement).
Override dependency to allow any request that passed through the /auth endpoint.
Allows:
- Port 5000 internal requests (have admin role despite anonymous user)
- Any authenticated user with a real username (not "anonymous")
- Port 5000 internal requests (remote-user: "anonymous", remote-role: "admin")
- Authenticated users with JWT tokens (remote-user: username)
- Unauthenticated requests when auth is disabled (remote-user: "viewer")
Rejects:
- Port 8971 requests with anonymous user (auth disabled, no proxy auth)
- Requests with no remote-user header (did not pass through /auth endpoint)
Example:
@router.get("/authenticated-endpoint", dependencies=[Depends(allow_any_authenticated())])
"""
async def auth_checker(request: Request):
# Port 5000 requests have admin role and should be allowed
role = request.headers.get("remote-role")
if role == "admin":
return
# Otherwise require a real authenticated user (not anonymous)
if not _is_authenticated(request):
# Ensure a remote-user has been set by the /auth endpoint
username = request.headers.get("remote-user")
if username is None:
raise HTTPException(status_code=401, detail="Authentication required")
return
@@ -549,7 +535,37 @@ def resolve_role(
# Endpoints
@router.get("/auth", dependencies=[Depends(allow_public())])
@router.get(
"/auth",
dependencies=[Depends(allow_public())],
summary="Authenticate request",
description=(
"Authenticates the current request based on proxy headers or JWT token. "
"This endpoint verifies authentication credentials and manages JWT token refresh. "
"On success, no JSON body is returned; authentication state is communicated via response headers and cookies."
),
status_code=202,
responses={
202: {
"description": "Authentication Accepted (no response body)",
"headers": {
"remote-user": {
"description": 'Authenticated username or "viewer" in proxy-only mode',
"schema": {"type": "string"},
},
"remote-role": {
"description": "Resolved role (e.g., admin, viewer, or custom)",
"schema": {"type": "string"},
},
"Set-Cookie": {
"description": "May include refreshed JWT cookie when applicable",
"schema": {"type": "string"},
},
},
},
401: {"description": "Authentication Failed"},
},
)
def auth(request: Request):
auth_config: AuthConfig = request.app.frigate_config.auth
proxy_config: ProxyConfig = request.app.frigate_config.proxy
@@ -576,12 +592,12 @@ def auth(request: Request):
# if auth is disabled, just apply the proxy header map and return success
if not auth_config.enabled:
# pass the user header value from the upstream proxy if a mapping is specified
# or use anonymous if none are specified
# or use viewer if none are specified
user_header = proxy_config.header_map.user
success_response.headers["remote-user"] = (
request.headers.get(user_header, default="anonymous")
request.headers.get(user_header, default="viewer")
if user_header
else "anonymous"
else "viewer"
)
# parse header and resolve a valid role
@@ -689,9 +705,14 @@ def auth(request: Request):
return fail_response
@router.get("/profile", dependencies=[Depends(allow_any_authenticated())])
@router.get(
"/profile",
dependencies=[Depends(allow_any_authenticated())],
summary="Get user profile",
description="Returns the current authenticated user's profile including username, role, and allowed cameras. This endpoint requires authentication and returns information about the user's permissions.",
)
def profile(request: Request):
username = request.headers.get("remote-user", "anonymous")
username = request.headers.get("remote-user", "viewer")
role = request.headers.get("remote-role", "viewer")
all_camera_names = set(request.app.frigate_config.cameras.keys())
@@ -703,7 +724,12 @@ def profile(request: Request):
)
@router.get("/logout", dependencies=[Depends(allow_public())])
@router.get(
"/logout",
dependencies=[Depends(allow_public())],
summary="Logout user",
description="Logs out the current user by clearing the session cookie. After logout, subsequent requests will require re-authentication.",
)
def logout(request: Request):
auth_config: AuthConfig = request.app.frigate_config.auth
response = RedirectResponse("/login", status_code=303)
@@ -714,7 +740,12 @@ def logout(request: Request):
limiter = Limiter(key_func=get_remote_addr)
@router.post("/login", dependencies=[Depends(allow_public())])
@router.post(
"/login",
dependencies=[Depends(allow_public())],
summary="Login with credentials",
description='Authenticates a user with username and password. Returns a JWT token as a secure HTTP-only cookie that can be used for subsequent API requests. The JWT token can also be retrieved from the response and used as a Bearer token in the Authorization header.\n\nExample using Bearer token:\n```\ncurl -H "Authorization: Bearer <token_value>" https://frigate_ip:8971/api/profile\n```',
)
@limiter.limit(limit_value=rateLimiter.get_limit)
def login(request: Request, body: AppPostLoginBody):
JWT_COOKIE_NAME = request.app.frigate_config.auth.cookie_name
@@ -752,7 +783,12 @@ def login(request: Request, body: AppPostLoginBody):
return JSONResponse(content={"message": "Login failed"}, status_code=401)
@router.get("/users", dependencies=[Depends(require_role(["admin"]))])
@router.get(
"/users",
dependencies=[Depends(require_role(["admin"]))],
summary="Get all users",
description="Returns a list of all users with their usernames and roles. Requires admin role. Each user object contains the username and assigned role.",
)
def get_users():
exports = (
User.select(User.username, User.role).order_by(User.username).dicts().iterator()
@@ -760,7 +796,12 @@ def get_users():
return JSONResponse([e for e in exports])
@router.post("/users", dependencies=[Depends(require_role(["admin"]))])
@router.post(
"/users",
dependencies=[Depends(require_role(["admin"]))],
summary="Create new user",
description='Creates a new user with the specified username, password, and role. Requires admin role. Password must meet strength requirements: minimum 8 characters, at least one uppercase letter, at least one digit, and at least one special character (!@#$%^&*(),.?":{} |<>).',
)
def create_user(
request: Request,
body: AppPostUsersBody,
@@ -789,7 +830,12 @@ def create_user(
return JSONResponse(content={"username": body.username})
@router.delete("/users/{username}", dependencies=[Depends(require_role(["admin"]))])
@router.delete(
"/users/{username}",
dependencies=[Depends(require_role(["admin"]))],
summary="Delete user",
description="Deletes a user by username. The built-in admin user cannot be deleted. Requires admin role. Returns success message or error if user not found.",
)
def delete_user(request: Request, username: str):
# Prevent deletion of the built-in admin user
if username == "admin":
@@ -802,7 +848,10 @@ def delete_user(request: Request, username: str):
@router.put(
"/users/{username}/password", dependencies=[Depends(allow_any_authenticated())]
"/users/{username}/password",
dependencies=[Depends(allow_any_authenticated())],
summary="Update user password",
description="Updates a user's password. Users can only change their own password unless they have admin role. Requires the current password to verify identity for non-admin users. Password must meet strength requirements: minimum 8 characters, at least one uppercase letter, at least one digit, and at least one special character (!@#$%^&*(),.?\":{} |<>). If user changes their own password, a new JWT cookie is automatically issued.",
)
async def update_password(
request: Request,
@@ -830,13 +879,9 @@ async def update_password(
except DoesNotExist:
return JSONResponse(content={"message": "User not found"}, status_code=404)
# Require old_password when:
# 1. Non-admin user is changing another user's password (admin only action)
# 2. Any user is changing their own password
is_changing_own_password = current_username == username
is_non_admin = current_role != "admin"
if is_changing_own_password or is_non_admin:
# Require old_password when non-admin user is changing any password
# Admin users changing passwords do NOT need to provide the current password
if current_role != "admin":
if not body.old_password:
return JSONResponse(
content={"message": "Current password is required"},
@@ -887,6 +932,8 @@ async def update_password(
@router.put(
"/users/{username}/role",
dependencies=[Depends(require_role(["admin"]))],
summary="Update user role",
description="Updates a user's role. The built-in admin user's role cannot be modified. Requires admin role. Valid roles are defined in the configuration.",
)
async def update_role(
request: Request,
+2 -1
View File
@@ -225,7 +225,8 @@ class MqttClient(Communicator):
"birdseye_mode",
"review_alerts",
"review_detections",
"genai",
"object_descriptions",
"review_descriptions",
]
for name in self.config.cameras.keys():
+3
View File
@@ -77,6 +77,9 @@ FFMPEG_HWACCEL_RKMPP = "preset-rkmpp"
FFMPEG_HWACCEL_AMF = "preset-amd-amf"
FFMPEG_HVC1_ARGS = ["-tag:v", "hvc1"]
# RKNN constants
SUPPORTED_RK_SOCS = ["rk3562", "rk3566", "rk3568", "rk3576", "rk3588"]
# Regex constants
REGEX_CAMERA_NAME = r"^[a-zA-Z0-9_-]+$"
+8 -7
View File
@@ -13,7 +13,7 @@ from frigate.comms.event_metadata_updater import (
)
from frigate.config import FrigateConfig
from frigate.const import MODEL_CACHE_DIR
from frigate.log import redirect_output_to_logger
from frigate.log import suppress_stderr_during
from frigate.util.object import calculate_region
from ..types import DataProcessorMetrics
@@ -80,13 +80,14 @@ class BirdRealTimeProcessor(RealTimeProcessorApi):
except Exception as e:
logger.error(f"Failed to download {path}: {e}")
@redirect_output_to_logger(logger, logging.DEBUG)
def __build_detector(self) -> None:
self.interpreter = Interpreter(
model_path=os.path.join(MODEL_CACHE_DIR, "bird/bird.tflite"),
num_threads=2,
)
self.interpreter.allocate_tensors()
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
self.interpreter = Interpreter(
model_path=os.path.join(MODEL_CACHE_DIR, "bird/bird.tflite"),
num_threads=2,
)
self.interpreter.allocate_tensors()
self.tensor_input_details = self.interpreter.get_input_details()
self.tensor_output_details = self.interpreter.get_output_details()
@@ -21,7 +21,7 @@ from frigate.config.classification import (
ObjectClassificationType,
)
from frigate.const import CLIPS_DIR, MODEL_CACHE_DIR
from frigate.log import redirect_output_to_logger
from frigate.log import suppress_stderr_during
from frigate.types import TrackedObjectUpdateTypesEnum
from frigate.util.builtin import EventsPerSecond, InferenceSpeed, load_labels
from frigate.util.object import box_overlaps, calculate_region
@@ -52,7 +52,7 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
self.requestor = requestor
self.model_dir = os.path.join(MODEL_CACHE_DIR, self.model_config.name)
self.train_dir = os.path.join(CLIPS_DIR, self.model_config.name, "train")
self.interpreter: Interpreter | None = None
self.interpreter: Interpreter = None
self.tensor_input_details: dict[str, Any] | None = None
self.tensor_output_details: dict[str, Any] | None = None
self.labelmap: dict[int, str] = {}
@@ -72,8 +72,12 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
self.last_run = datetime.datetime.now().timestamp()
self.__build_detector()
@redirect_output_to_logger(logger, logging.DEBUG)
def __build_detector(self) -> None:
try:
from tflite_runtime.interpreter import Interpreter
except ModuleNotFoundError:
from tensorflow.lite.python.interpreter import Interpreter
model_path = os.path.join(self.model_dir, "model.tflite")
labelmap_path = os.path.join(self.model_dir, "labelmap.txt")
@@ -84,11 +88,13 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
self.labelmap = {}
return
self.interpreter = Interpreter(
model_path=model_path,
num_threads=2,
)
self.interpreter.allocate_tensors()
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
self.interpreter = Interpreter(
model_path=model_path,
num_threads=2,
)
self.interpreter.allocate_tensors()
self.tensor_input_details = self.interpreter.get_input_details()
self.tensor_output_details = self.interpreter.get_output_details()
self.labelmap = load_labels(labelmap_path, prefill=0)
@@ -224,28 +230,34 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
if not should_run:
return
x, y, x2, y2 = calculate_region(
frame.shape,
crop[0],
crop[1],
crop[2],
crop[3],
224,
1.0,
)
rgb = cv2.cvtColor(frame, cv2.COLOR_YUV2RGB_I420)
frame = rgb[
y:y2,
x:x2,
]
height, width = rgb.shape[:2]
if frame.shape != (224, 224):
try:
resized_frame = cv2.resize(frame, (224, 224))
except Exception:
logger.warning("Failed to resize image for state classification")
return
# Convert normalized crop coordinates to pixel values
x1 = int(camera_config.crop[0] * width)
y1 = int(camera_config.crop[1] * height)
x2 = int(camera_config.crop[2] * width)
y2 = int(camera_config.crop[3] * height)
# Clip coordinates to frame boundaries
x1 = max(0, min(x1, width))
y1 = max(0, min(y1, height))
x2 = max(0, min(x2, width))
y2 = max(0, min(y2, height))
if x2 <= x1 or y2 <= y1:
logger.warning(
f"Invalid crop coordinates for {camera}: [{x1}, {y1}, {x2}, {y2}]"
)
return
frame = rgb[y1:y2, x1:x2]
try:
resized_frame = cv2.resize(frame, (224, 224))
except Exception:
logger.warning("Failed to resize image for state classification")
return
if self.interpreter is None:
# When interpreter is None, always save (score is 0.0, which is < 1.0)
@@ -345,7 +357,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
self.model_config = model_config
self.model_dir = os.path.join(MODEL_CACHE_DIR, self.model_config.name)
self.train_dir = os.path.join(CLIPS_DIR, self.model_config.name, "train")
self.interpreter: Interpreter | None = None
self.interpreter: Interpreter = None
self.sub_label_publisher = sub_label_publisher
self.requestor = requestor
self.tensor_input_details: dict[str, Any] | None = None
@@ -366,7 +378,6 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
self.__build_detector()
@redirect_output_to_logger(logger, logging.DEBUG)
def __build_detector(self) -> None:
model_path = os.path.join(self.model_dir, "model.tflite")
labelmap_path = os.path.join(self.model_dir, "labelmap.txt")
@@ -378,11 +389,13 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
self.labelmap = {}
return
self.interpreter = Interpreter(
model_path=model_path,
num_threads=2,
)
self.interpreter.allocate_tensors()
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
self.interpreter = Interpreter(
model_path=model_path,
num_threads=2,
)
self.interpreter.allocate_tensors()
self.tensor_input_details = self.interpreter.get_input_details()
self.tensor_output_details = self.interpreter.get_output_details()
self.labelmap = load_labels(labelmap_path, prefill=0)
@@ -508,6 +521,13 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
0.0,
max_files=save_attempts,
)
# Still track history even when model doesn't exist to respect MAX_OBJECT_CLASSIFICATIONS
# Add an entry with "unknown" label so the history limit is enforced
if object_id not in self.classification_history:
self.classification_history[object_id] = []
self.classification_history[object_id].append(("unknown", 0.0, now))
return
input = np.expand_dims(resized_crop, axis=0)
@@ -649,5 +669,5 @@ def write_classification_attempt(
if len(files) > max_files:
os.unlink(os.path.join(folder, files[-1]))
except FileNotFoundError:
except (FileNotFoundError, OSError):
pass
+2
View File
@@ -131,6 +131,7 @@ class ONNXModelRunner(BaseModelRunner):
return model_type in [
EnrichmentModelTypeEnum.paddleocr.value,
EnrichmentModelTypeEnum.yolov9_license_plate.value,
EnrichmentModelTypeEnum.jina_v1.value,
EnrichmentModelTypeEnum.jina_v2.value,
EnrichmentModelTypeEnum.facenet.value,
@@ -169,6 +170,7 @@ class CudaGraphRunner(BaseModelRunner):
return model_type not in [
ModelTypeEnum.yolonas.value,
ModelTypeEnum.dfine.value,
EnrichmentModelTypeEnum.paddleocr.value,
EnrichmentModelTypeEnum.jina_v1.value,
EnrichmentModelTypeEnum.jina_v2.value,
+7 -6
View File
@@ -5,7 +5,7 @@ from typing_extensions import Literal
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detector_config import BaseDetectorConfig
from frigate.log import redirect_output_to_logger
from frigate.log import suppress_stderr_during
from ..detector_utils import tflite_detect_raw, tflite_init
@@ -28,12 +28,13 @@ class CpuDetectorConfig(BaseDetectorConfig):
class CpuTfl(DetectionApi):
type_key = DETECTOR_KEY
@redirect_output_to_logger(logger, logging.DEBUG)
def __init__(self, detector_config: CpuDetectorConfig):
interpreter = Interpreter(
model_path=detector_config.model.path,
num_threads=detector_config.num_threads or 3,
)
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
interpreter = Interpreter(
model_path=detector_config.model.path,
num_threads=detector_config.num_threads or 3,
)
tflite_init(self, interpreter)
+3 -5
View File
@@ -8,7 +8,7 @@ import cv2
import numpy as np
from pydantic import Field
from frigate.const import MODEL_CACHE_DIR
from frigate.const import MODEL_CACHE_DIR, SUPPORTED_RK_SOCS
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detection_runners import RKNNModelRunner
from frigate.detectors.detector_config import BaseDetectorConfig, ModelTypeEnum
@@ -19,8 +19,6 @@ logger = logging.getLogger(__name__)
DETECTOR_KEY = "rknn"
supported_socs = ["rk3562", "rk3566", "rk3568", "rk3576", "rk3588"]
supported_models = {
ModelTypeEnum.yologeneric: "^frigate-fp16-yolov9-[cemst]$",
ModelTypeEnum.yolonas: "^deci-fp16-yolonas_[sml]$",
@@ -82,9 +80,9 @@ class Rknn(DetectionApi):
except FileNotFoundError:
raise Exception("Make sure to run docker in privileged mode.")
if soc not in supported_socs:
if soc not in SUPPORTED_RK_SOCS:
raise Exception(
f"Your SoC is not supported. Your SoC is: {soc}. Currently these SoCs are supported: {supported_socs}."
f"Your SoC is not supported. Your SoC is: {soc}. Currently these SoCs are supported: {SUPPORTED_RK_SOCS}."
)
return soc
+8 -7
View File
@@ -8,7 +8,7 @@ import numpy as np
from frigate.const import MODEL_CACHE_DIR
from frigate.detectors.detection_runners import get_optimized_runner
from frigate.embeddings.types import EnrichmentModelTypeEnum
from frigate.log import redirect_output_to_logger
from frigate.log import suppress_stderr_during
from frigate.util.downloader import ModelDownloader
from ...config import FaceRecognitionConfig
@@ -57,17 +57,18 @@ class FaceNetEmbedding(BaseEmbedding):
self._load_model_and_utils()
logger.debug(f"models are already downloaded for {self.model_name}")
@redirect_output_to_logger(logger, logging.DEBUG)
def _load_model_and_utils(self):
if self.runner is None:
if self.downloader:
self.downloader.wait_for_download()
self.runner = Interpreter(
model_path=os.path.join(MODEL_CACHE_DIR, "facedet/facenet.tflite"),
num_threads=2,
)
self.runner.allocate_tensors()
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
self.runner = Interpreter(
model_path=os.path.join(MODEL_CACHE_DIR, "facedet/facenet.tflite"),
num_threads=2,
)
self.runner.allocate_tensors()
self.tensor_input_details = self.runner.get_input_details()
self.tensor_output_details = self.runner.get_output_details()
@@ -186,6 +186,9 @@ class JinaV1ImageEmbedding(BaseEmbedding):
download_func=self._download_model,
)
self.downloader.ensure_model_files()
# Avoid lazy loading in worker threads: block until downloads complete
# and load the model on the main thread during initialization.
self._load_model_and_utils()
else:
self.downloader = None
ModelDownloader.mark_files_state(
@@ -65,6 +65,9 @@ class JinaV2Embedding(BaseEmbedding):
download_func=self._download_model,
)
self.downloader.ensure_model_files()
# Avoid lazy loading in worker threads: block until downloads complete
# and load the model on the main thread during initialization.
self._load_model_and_utils()
else:
self.downloader = None
ModelDownloader.mark_files_state(
+8 -8
View File
@@ -34,7 +34,7 @@ from frigate.data_processing.real_time.audio_transcription import (
AudioTranscriptionRealTimeProcessor,
)
from frigate.ffmpeg_presets import parse_preset_input
from frigate.log import LogPipe, redirect_output_to_logger
from frigate.log import LogPipe, suppress_stderr_during
from frigate.object_detection.base import load_labels
from frigate.util.builtin import get_ffmpeg_arg_list
from frigate.util.process import FrigateProcess
@@ -367,17 +367,17 @@ class AudioEventMaintainer(threading.Thread):
class AudioTfl:
@redirect_output_to_logger(logger, logging.DEBUG)
def __init__(self, stop_event: threading.Event, num_threads=2):
self.stop_event = stop_event
self.num_threads = num_threads
self.labels = load_labels("/audio-labelmap.txt", prefill=521)
self.interpreter = Interpreter(
model_path="/cpu_audio_model.tflite",
num_threads=self.num_threads,
)
self.interpreter.allocate_tensors()
# Suppress TFLite delegate creation messages that bypass Python logging
with suppress_stderr_during("tflite_interpreter_init"):
self.interpreter = Interpreter(
model_path="/cpu_audio_model.tflite",
num_threads=self.num_threads,
)
self.interpreter.allocate_tensors()
self.tensor_input_details = self.interpreter.get_input_details()
self.tensor_output_details = self.interpreter.get_output_details()
+1 -1
View File
@@ -153,7 +153,7 @@ PRESETS_HW_ACCEL_ENCODE_BIRDSEYE = {
FFMPEG_HWACCEL_VAAPI: "{0} -hide_banner -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device {3} {1} -c:v h264_vaapi -g 50 -bf 0 -profile:v high -level:v 4.1 -sei:v 0 -an -vf format=vaapi|nv12,hwupload {2}",
"preset-intel-qsv-h264": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v high -level:v 4.1 -async_depth:v 1 {2}",
"preset-intel-qsv-h265": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v main -level:v 4.1 -async_depth:v 1 {2}",
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} -hwaccel device {3} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
"preset-jetson-h264": "{0} -hide_banner {1} -c:v h264_nvmpi -profile high {2}",
"preset-jetson-h265": "{0} -hide_banner {1} -c:v h264_nvmpi -profile main {2}",
FFMPEG_HWACCEL_RKMPP: "{0} -hide_banner {1} -c:v h264_rkmpp -profile:v high {2}",
+33
View File
@@ -80,10 +80,15 @@ def apply_log_levels(default: str, log_levels: dict[str, LogLevel]) -> None:
log_levels = {
"absl": LogLevel.error,
"httpx": LogLevel.error,
"h5py": LogLevel.error,
"keras": LogLevel.error,
"matplotlib": LogLevel.error,
"tensorflow": LogLevel.error,
"tensorflow.python": LogLevel.error,
"werkzeug": LogLevel.error,
"ws4py": LogLevel.error,
"PIL": LogLevel.warning,
"numba": LogLevel.warning,
**log_levels,
}
@@ -318,3 +323,31 @@ def suppress_os_output(func: Callable) -> Callable:
return result
return wrapper
@contextmanager
def suppress_stderr_during(operation_name: str) -> Generator[None, None, None]:
"""
Context manager to suppress stderr output during a specific operation.
Useful for silencing LLVM debug output, CUDA messages, and other native
library logging that cannot be controlled via Python logging or environment
variables. Completely redirects file descriptor 2 (stderr) to /dev/null.
Usage:
with suppress_stderr_during("model_conversion"):
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
Args:
operation_name: Name of the operation for debugging purposes
"""
original_stderr_fd = os.dup(2)
devnull = os.open(os.devnull, os.O_WRONLY)
try:
os.dup2(devnull, 2)
yield
finally:
os.dup2(original_stderr_fd, 2)
os.close(devnull)
os.close(original_stderr_fd)
+3
View File
@@ -119,6 +119,7 @@ class RecordingCleanup(threading.Thread):
Recordings.path,
Recordings.objects,
Recordings.motion,
Recordings.dBFS,
)
.where(
(Recordings.camera == config.name)
@@ -126,6 +127,7 @@ class RecordingCleanup(threading.Thread):
(
(Recordings.end_time < continuous_expire_date)
& (Recordings.motion == 0)
& (Recordings.dBFS == 0)
)
| (Recordings.end_time < motion_expire_date)
)
@@ -185,6 +187,7 @@ class RecordingCleanup(threading.Thread):
mode == RetainModeEnum.motion
and recording.motion == 0
and recording.objects == 0
and recording.dBFS == 0
)
or (mode == RetainModeEnum.active_objects and recording.objects == 0)
):
+1 -1
View File
@@ -67,7 +67,7 @@ class SegmentInfo:
if (
not keep
and retain_mode == RetainModeEnum.motion
and (self.motion_count > 0 or self.average_dBFS > 0)
and (self.motion_count > 0 or self.average_dBFS != 0)
):
keep = True
+91 -38
View File
@@ -19,9 +19,10 @@ from frigate.const import (
PROCESS_PRIORITY_LOW,
UPDATE_MODEL_STATE,
)
from frigate.log import redirect_output_to_logger
from frigate.log import redirect_output_to_logger, suppress_stderr_during
from frigate.models import Event, Recordings, ReviewSegment
from frigate.types import ModelStatusTypesEnum
from frigate.util.downloader import ModelDownloader
from frigate.util.file import get_event_thumbnail_bytes
from frigate.util.image import get_image_from_recording
from frigate.util.process import FrigateProcess
@@ -121,6 +122,10 @@ def get_dataset_image_count(model_name: str) -> int:
class ClassificationTrainingProcess(FrigateProcess):
def __init__(self, model_name: str) -> None:
self.BASE_WEIGHT_URL = os.environ.get(
"TF_KERAS_MOBILENET_V2_WEIGHTS_URL",
"",
)
super().__init__(
stop_event=None,
priority=PROCESS_PRIORITY_LOW,
@@ -179,11 +184,23 @@ class ClassificationTrainingProcess(FrigateProcess):
)
return False
weights_path = "imagenet"
# Download MobileNetV2 weights if not present
if self.BASE_WEIGHT_URL:
weights_path = os.path.join(
MODEL_CACHE_DIR, "MobileNet", "mobilenet_v2_weights.h5"
)
if not os.path.exists(weights_path):
logger.info("Downloading MobileNet V2 weights file")
ModelDownloader.download_from_url(
self.BASE_WEIGHT_URL, weights_path
)
# Start with imagenet base model with 35% of channels in each layer
base_model = MobileNetV2(
input_shape=(224, 224, 3),
include_top=False,
weights="imagenet",
weights=weights_path,
alpha=0.35,
)
base_model.trainable = False # Freeze pre-trained layers
@@ -233,15 +250,20 @@ class ClassificationTrainingProcess(FrigateProcess):
logger.debug(f"Converting {self.model_name} to TFLite...")
# convert model to tflite
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = (
self.__generate_representative_dataset_factory(dataset_dir)
)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()
# Suppress stderr during conversion to avoid LLVM debug output
# (fully_quantize, inference_type, MLIR optimization messages, etc)
with suppress_stderr_during("tflite_conversion"):
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = (
self.__generate_representative_dataset_factory(dataset_dir)
)
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS_INT8
]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()
# write model
model_path = os.path.join(model_dir, "model.tflite")
@@ -338,8 +360,6 @@ def collect_state_classification_examples(
cameras: Dict mapping camera names to normalized crop coordinates [x1, y1, x2, y2] (0-1)
"""
dataset_dir = os.path.join(CLIPS_DIR, model_name, "dataset")
temp_dir = os.path.join(dataset_dir, "temp")
os.makedirs(temp_dir, exist_ok=True)
# Step 1: Get review items for the cameras
camera_names = list(cameras.keys())
@@ -354,6 +374,10 @@ def collect_state_classification_examples(
logger.warning(f"No review items found for cameras: {camera_names}")
return
# The temp directory is only created when there are review_items.
temp_dir = os.path.join(dataset_dir, "temp")
os.makedirs(temp_dir, exist_ok=True)
# Step 2: Create balanced timestamp selection (100 samples)
timestamps = _select_balanced_timestamps(review_items, target_count=100)
@@ -482,6 +506,10 @@ def _extract_keyframes(
"""
Extract keyframes from recordings at specified timestamps and crop to specified regions.
This implementation batches work by running multiple ffmpeg snapshot commands
concurrently, which significantly reduces total runtime compared to
processing each timestamp serially.
Args:
ffmpeg_path: Path to ffmpeg binary
timestamps: List of timestamp dicts from _select_balanced_timestamps
@@ -491,15 +519,21 @@ def _extract_keyframes(
Returns:
List of paths to successfully extracted and cropped keyframe images
"""
keyframe_paths = []
from concurrent.futures import ThreadPoolExecutor, as_completed
for idx, ts_info in enumerate(timestamps):
if not timestamps:
return []
# Limit the number of concurrent ffmpeg processes so we don't overload the host.
max_workers = min(5, len(timestamps))
def _process_timestamp(idx: int, ts_info: dict) -> tuple[int, str | None]:
camera = ts_info["camera"]
timestamp = ts_info["timestamp"]
if camera not in camera_crops:
logger.warning(f"No crop coordinates for camera {camera}")
continue
return idx, None
norm_x1, norm_y1, norm_x2, norm_y2 = camera_crops[camera]
@@ -516,7 +550,7 @@ def _extract_keyframes(
.get()
)
except Exception:
continue
return idx, None
relative_time = timestamp - recording.start_time
@@ -530,38 +564,57 @@ def _extract_keyframes(
height=None,
)
if image_data:
nparr = np.frombuffer(image_data, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
if not image_data:
return idx, None
if img is not None:
height, width = img.shape[:2]
nparr = np.frombuffer(image_data, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
x1 = int(norm_x1 * width)
y1 = int(norm_y1 * height)
x2 = int(norm_x2 * width)
y2 = int(norm_y2 * height)
if img is None:
return idx, None
x1_clipped = max(0, min(x1, width))
y1_clipped = max(0, min(y1, height))
x2_clipped = max(0, min(x2, width))
y2_clipped = max(0, min(y2, height))
height, width = img.shape[:2]
if x2_clipped > x1_clipped and y2_clipped > y1_clipped:
cropped = img[y1_clipped:y2_clipped, x1_clipped:x2_clipped]
resized = cv2.resize(cropped, (224, 224))
x1 = int(norm_x1 * width)
y1 = int(norm_y1 * height)
x2 = int(norm_x2 * width)
y2 = int(norm_y2 * height)
output_path = os.path.join(output_dir, f"frame_{idx:04d}.jpg")
cv2.imwrite(output_path, resized)
keyframe_paths.append(output_path)
x1_clipped = max(0, min(x1, width))
y1_clipped = max(0, min(y1, height))
x2_clipped = max(0, min(x2, width))
y2_clipped = max(0, min(y2, height))
if x2_clipped <= x1_clipped or y2_clipped <= y1_clipped:
return idx, None
cropped = img[y1_clipped:y2_clipped, x1_clipped:x2_clipped]
resized = cv2.resize(cropped, (224, 224))
output_path = os.path.join(output_dir, f"frame_{idx:04d}.jpg")
cv2.imwrite(output_path, resized)
return idx, output_path
except Exception as e:
logger.debug(
f"Failed to extract frame from {recording.path} at {relative_time}s: {e}"
)
continue
return idx, None
return keyframe_paths
keyframes_with_index: list[tuple[int, str]] = []
with ThreadPoolExecutor(max_workers=max_workers) as executor:
future_to_idx = {
executor.submit(_process_timestamp, idx, ts_info): idx
for idx, ts_info in enumerate(timestamps)
}
for future in as_completed(future_to_idx):
_, path = future.result()
if path:
keyframes_with_index.append((future_to_idx[future], path))
keyframes_with_index.sort(key=lambda item: item[0])
return [path for _, path in keyframes_with_index]
def _select_distinct_images(
+5
View File
@@ -65,10 +65,15 @@ class FrigateProcess(BaseProcess):
logging.basicConfig(handlers=[], force=True)
logging.getLogger().addHandler(QueueHandler(self.__log_queue))
# Always apply base log level suppressions for noisy third-party libraries
# even if no specific logConfig is provided
if logConfig:
frigate.log.apply_log_levels(
logConfig.default.value.upper(), logConfig.logs
)
else:
# Apply default INFO level with standard library suppressions
frigate.log.apply_log_levels("INFO", {})
self._setup_memray()
+12
View File
@@ -8,6 +8,7 @@ import time
from pathlib import Path
from typing import Optional
from frigate.const import SUPPORTED_RK_SOCS
from frigate.util.file import FileLock
logger = logging.getLogger(__name__)
@@ -68,9 +69,20 @@ def is_rknn_compatible(model_path: str, model_type: str | None = None) -> bool:
True if the model is RKNN-compatible, False otherwise
"""
soc = get_soc_type()
if soc is None:
return False
# Check if the SoC is actually a supported RK device
# This prevents false positives on non-RK devices (e.g., macOS Docker)
# where /proc/device-tree/compatible might exist but contain non-RK content
if soc not in SUPPORTED_RK_SOCS:
logger.debug(
f"SoC '{soc}' is not a supported RK device for RKNN conversion. "
f"Supported SoCs: {SUPPORTED_RK_SOCS}"
)
return False
if not model_type:
model_type = get_rknn_model_type(model_path)
@@ -1,7 +1,9 @@
{
"documentTitle": "Classification Models - Frigate",
"details": {
"scoreInfo": "Score represents the average classification confidence across all detections of this object."
"scoreInfo": "Score represents the average classification confidence across all detections of this object.",
"none": "None",
"unknown": "Unknown"
},
"button": {
"deleteClassificationAttempts": "Delete Classification Images",
@@ -72,7 +74,7 @@
},
"renameCategory": {
"title": "Rename Class",
"desc": "Enter a new name for {{name}}. You will be required to retrain the model for the name change to take affect."
"desc": "Enter a new name for {{name}}. You will be required to retrain the model for the name change to take effect."
},
"description": {
"invalidName": "Invalid name. Names can only include letters, numbers, spaces, apostrophes, underscores, and hyphens."
@@ -83,7 +85,6 @@
"aria": "Select Recent Classifications"
},
"categories": "Classes",
"none": "None",
"createCategory": {
"new": "Create New Class"
},
@@ -138,6 +139,7 @@
"nameOnlyNumbers": "Model name cannot contain only numbers",
"classRequired": "At least 1 class is required",
"classesUnique": "Class names must be unique",
"noneNotAllowed": "The class 'none' is not allowed",
"stateRequiresTwoClasses": "State models require at least 2 classes",
"objectLabelRequired": "Please select an object label",
"objectTypeRequired": "Please select a classification type"
+2 -2
View File
@@ -679,7 +679,7 @@
"desc": "Manage this Frigate instance's user accounts."
},
"addUser": "Add User",
"updatePassword": "Update Password",
"updatePassword": "Reset Password",
"toast": {
"success": {
"createUser": "User {{user}} created successfully",
@@ -700,7 +700,7 @@
"role": "Role",
"noUsers": "No users found.",
"changeRole": "Change user role",
"password": "Password",
"password": "Reset Password",
"deleteUser": "Delete user"
},
"dialog": {
+13 -7
View File
@@ -14,6 +14,7 @@ import ProtectedRoute from "@/components/auth/ProtectedRoute";
import { AuthProvider } from "@/context/auth-context";
import useSWR from "swr";
import { FrigateConfig } from "./types/frigateConfig";
import ActivityIndicator from "@/components/indicators/activity-indicator";
const Live = lazy(() => import("@/pages/Live"));
const Events = lazy(() => import("@/pages/Events"));
@@ -50,6 +51,13 @@ function DefaultAppView() {
const { data: config } = useSWR<FrigateConfig>("config", {
revalidateOnFocus: false,
});
// Compute required roles for main routes, ensuring we have config first
// to prevent race condition where custom roles are temporarily unavailable
const mainRouteRoles = config?.auth?.roles
? Object.keys(config.auth.roles)
: undefined;
return (
<div className="size-full overflow-hidden">
{isDesktop && <Sidebar />}
@@ -68,13 +76,11 @@ function DefaultAppView() {
<Routes>
<Route
element={
<ProtectedRoute
requiredRoles={
config?.auth.roles
? Object.keys(config.auth.roles)
: ["admin", "viewer"]
}
/>
mainRouteRoles ? (
<ProtectedRoute requiredRoles={mainRouteRoles} />
) : (
<ActivityIndicator className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2" />
)
}
>
<Route index element={<Live />} />
+2 -2
View File
@@ -116,10 +116,10 @@ export default function Statusbar() {
}
return (
<Link key={gpuTitle} to="/system#general">
<Link key={name} to="/system#general">
{" "}
<div
key={gpuTitle}
key={name}
className="flex cursor-pointer items-center gap-2 text-sm hover:underline"
>
<MdCircle
+56 -44
View File
@@ -4,8 +4,8 @@ import { cn } from "@/lib/utils";
import {
ClassificationItemData,
ClassificationThreshold,
ClassifiedEvent,
} from "@/types/classification";
import { Event } from "@/types/event";
import { forwardRef, useMemo, useRef, useState } from "react";
import { isDesktop, isIOS, isMobile, isMobileOnly } from "react-device-detect";
import { useTranslation } from "react-i18next";
@@ -40,6 +40,7 @@ type ClassificationCardProps = {
data: ClassificationItemData;
threshold?: ClassificationThreshold;
selected: boolean;
clickable: boolean;
i18nLibrary: string;
showArea?: boolean;
count?: number;
@@ -56,6 +57,7 @@ export const ClassificationCard = forwardRef<
data,
threshold,
selected,
clickable,
i18nLibrary,
showArea = true,
count,
@@ -101,11 +103,12 @@ export const ClassificationCard = forwardRef<
<div
ref={ref}
className={cn(
"relative flex size-full cursor-pointer flex-col overflow-hidden rounded-lg outline outline-[3px]",
"relative flex size-full flex-col overflow-hidden rounded-lg outline outline-[3px]",
className,
selected
? "shadow-selected outline-selected"
: "outline-transparent duration-500",
clickable && "cursor-pointer",
)}
onClick={(e) => {
const isMeta = e.metaKey || e.ctrlKey;
@@ -160,8 +163,12 @@ export const ClassificationCard = forwardRef<
data.score != undefined ? "text-xs" : "text-sm",
)}
>
<div className="smart-capitalize">
{data.name == "unknown" ? t("details.unknown") : data.name}
<div className="break-all smart-capitalize">
{data.name == "unknown"
? t("details.unknown")
: data.name == "none"
? t("details.none")
: data.name}
</div>
{data.score != undefined && (
<div
@@ -186,7 +193,7 @@ export const ClassificationCard = forwardRef<
type GroupedClassificationCardProps = {
group: ClassificationItemData[];
event?: Event;
classifiedEvent?: ClassifiedEvent;
threshold?: ClassificationThreshold;
selectedItems: string[];
i18nLibrary: string;
@@ -197,7 +204,7 @@ type GroupedClassificationCardProps = {
};
export function GroupedClassificationCard({
group,
event,
classifiedEvent,
threshold,
selectedItems,
i18nLibrary,
@@ -232,14 +239,15 @@ export function GroupedClassificationCard({
const bestTyped: ClassificationItemData = best;
return {
...bestTyped,
name: event
? event.sub_label && event.sub_label !== "none"
? event.sub_label
: t(noClassificationLabel)
: bestTyped.name,
score: event?.data?.sub_label_score,
name:
classifiedEvent?.label && classifiedEvent.label !== "none"
? classifiedEvent.label
: classifiedEvent
? t(noClassificationLabel)
: bestTyped.name,
score: classifiedEvent?.score,
};
}, [group, event, noClassificationLabel, t]);
}, [group, classifiedEvent, noClassificationLabel, t]);
const bestScoreStatus = useMemo(() => {
if (!bestItem?.score || !threshold) {
@@ -284,6 +292,7 @@ export function GroupedClassificationCard({
data={bestItem}
threshold={threshold}
selected={selectedItems.includes(bestItem.filename)}
clickable={true}
i18nLibrary={i18nLibrary}
count={group.length}
onClick={(_, meta) => {
@@ -325,36 +334,38 @@ export function GroupedClassificationCard({
)}
>
<ContentTitle className="flex items-center gap-2 font-normal capitalize">
{event?.sub_label && event.sub_label !== "none"
? event.sub_label
{classifiedEvent?.label && classifiedEvent.label !== "none"
? classifiedEvent.label
: t(noClassificationLabel)}
{event?.sub_label && event.sub_label !== "none" && (
<div className="flex items-center gap-1">
<div
className={cn(
"",
bestScoreStatus == "match" && "text-success",
bestScoreStatus == "potential" && "text-orange-400",
bestScoreStatus == "unknown" && "text-danger",
)}
>{`${Math.round((event.data.sub_label_score || 0) * 100)}%`}</div>
<Popover>
<PopoverTrigger asChild>
<button
className="focus:outline-none"
aria-label={t("details.scoreInfo", {
ns: i18nLibrary,
})}
>
<LuInfo className="size-3" />
</button>
</PopoverTrigger>
<PopoverContent className="w-80 text-sm">
{t("details.scoreInfo", { ns: i18nLibrary })}
</PopoverContent>
</Popover>
</div>
)}
{classifiedEvent?.label &&
classifiedEvent.label !== "none" &&
classifiedEvent.score !== undefined && (
<div className="flex items-center gap-1">
<div
className={cn(
"",
bestScoreStatus == "match" && "text-success",
bestScoreStatus == "potential" && "text-orange-400",
bestScoreStatus == "unknown" && "text-danger",
)}
>{`${Math.round((classifiedEvent.score || 0) * 100)}%`}</div>
<Popover>
<PopoverTrigger asChild>
<button
className="focus:outline-none"
aria-label={t("details.scoreInfo", {
ns: i18nLibrary,
})}
>
<LuInfo className="size-3" />
</button>
</PopoverTrigger>
<PopoverContent className="w-80 text-sm">
{t("details.scoreInfo", { ns: i18nLibrary })}
</PopoverContent>
</Popover>
</div>
)}
</ContentTitle>
<ContentDescription className={cn("", isMobile && "px-2")}>
{time && (
@@ -368,14 +379,14 @@ export function GroupedClassificationCard({
</div>
{isDesktop && (
<div className="flex flex-row justify-between">
{event && (
{classifiedEvent && (
<Tooltip>
<TooltipTrigger asChild>
<div
className="cursor-pointer"
tabIndex={-1}
onClick={() => {
navigate(`/explore?event_id=${event.id}`);
navigate(`/explore?event_id=${classifiedEvent.id}`);
}}
>
<LuSearch className="size-4 text-secondary-foreground" />
@@ -406,6 +417,7 @@ export function GroupedClassificationCard({
data={data}
threshold={threshold}
selected={false}
clickable={false}
i18nLibrary={i18nLibrary}
onClick={() => {}}
>
@@ -94,7 +94,14 @@ export default function Step1NameAndDefine({
objectLabel: z.string().optional(),
objectType: z.enum(["sub_label", "attribute"]).optional(),
classes: z
.array(z.string())
.array(
z
.string()
.refine(
(val) => val.trim().toLowerCase() !== "none",
t("wizard.step1.errors.noneNotAllowed"),
),
)
.min(1, t("wizard.step1.errors.classRequired"))
.refine(
(classes) => {
@@ -315,7 +322,7 @@ export default function Step1NameAndDefine({
<FormLabel className="text-primary-variant">
{t("wizard.step1.classificationType")}
</FormLabel>
<Popover>
<Popover modal={true}>
<PopoverTrigger asChild>
<Button
variant="ghost"
@@ -398,7 +405,7 @@ export default function Step1NameAndDefine({
? t("wizard.step1.states")
: t("wizard.step1.classes")}
</FormLabel>
<Popover>
<Popover modal={true}>
<PopoverTrigger asChild>
<Button variant="ghost" size="sm" className="h-4 w-4 p-0">
<LuInfo className="size-3" />
@@ -467,6 +474,7 @@ export default function Step1NameAndDefine({
)}
</div>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
@@ -141,7 +141,37 @@ export default function Step3ChooseExamples({
);
await Promise.all(categorizePromises);
// Step 2.5: Create empty folders for classes that don't have any images
// Step 2.5: Delete any unselected images from train folder
// For state models, all images must be classified, so unselected images should be removed
// For object models, unselected images are assigned to "none" so they're already categorized
if (step1Data.modelType === "state") {
try {
// Fetch current train images to see what's left after categorization
const trainImagesResponse = await axios.get<string[]>(
`/classification/${step1Data.modelName}/train`,
);
const remainingTrainImages = trainImagesResponse.data || [];
const categorizedImageNames = new Set(Object.keys(classifications));
const unselectedImages = remainingTrainImages.filter(
(imageName) => !categorizedImageNames.has(imageName),
);
if (unselectedImages.length > 0) {
await axios.post(
`/classification/${step1Data.modelName}/train/delete`,
{
ids: unselectedImages,
},
);
}
} catch (error) {
// Silently fail - unselected images will remain but won't cause issues
// since the frontend filters out images that don't match expected format
}
}
// Step 2.6: Create empty folders for classes that don't have any images
// This ensures all classes are available in the dataset view later
const classesWithImages = new Set(
Object.values(classifications).filter((c) => c && c !== "none"),
@@ -156,15 +186,17 @@ export default function Step3ChooseExamples({
await Promise.all(emptyFolderPromises);
// Step 3: Determine if we should train
// For state models, we need ALL states to have examples
// For object models, we need at least 2 classes with images
// For state models, we need ALL states to have examples (at least 2 states)
// For object models, we need at least 1 class with images (the rest go to "none")
const allStatesHaveExamplesForTraining =
step1Data.modelType !== "state" ||
step1Data.classes.every((className) =>
classesWithImages.has(className),
);
const shouldTrain =
allStatesHaveExamplesForTraining && classesWithImages.size >= 2;
step1Data.modelType === "object"
? classesWithImages.size >= 1
: allStatesHaveExamplesForTraining && classesWithImages.size >= 2;
// Step 4: Kick off training only if we have enough classes with images
if (shouldTrain) {
@@ -132,7 +132,7 @@ export default function ClassificationSelectionDialog({
onClick={() => onCategorizeImage(category)}
>
{category === "none"
? t("none")
? t("details.none")
: category.replaceAll("_", " ")}
</SelectorItem>
))}
+11 -3
View File
@@ -440,6 +440,7 @@ function CustomTimeSelector({
<FaCalendarAlt />
<div className="flex flex-wrap items-center">
<Popover
modal={false}
open={startOpen}
onOpenChange={(open) => {
if (!open) {
@@ -461,7 +462,10 @@ function CustomTimeSelector({
{formattedStart}
</Button>
</PopoverTrigger>
<PopoverContent className="flex flex-col items-center">
<PopoverContent
disablePortal={isDesktop}
className="flex flex-col items-center"
>
<TimezoneAwareCalendar
timezone={config?.ui.timezone}
selectedDay={new Date(startTime * 1000)}
@@ -506,6 +510,7 @@ function CustomTimeSelector({
</Popover>
<FaArrowRight className="size-4 text-primary" />
<Popover
modal={false}
open={endOpen}
onOpenChange={(open) => {
if (!open) {
@@ -527,7 +532,10 @@ function CustomTimeSelector({
{formattedEnd}
</Button>
</PopoverTrigger>
<PopoverContent className="flex flex-col items-center">
<PopoverContent
disablePortal={isDesktop}
className="flex flex-col items-center"
>
<TimezoneAwareCalendar
timezone={config?.ui.timezone}
selectedDay={new Date(endTime * 1000)}
@@ -545,7 +553,7 @@ function CustomTimeSelector({
<SelectSeparator className="bg-secondary" />
<input
className="text-md mx-4 w-full border border-input bg-background p-1 text-secondary-foreground hover:bg-accent hover:text-accent-foreground dark:[color-scheme:dark]"
id="startTime"
id="endTime"
type="time"
value={endClock}
step={isIOS ? "60" : "1"}
@@ -49,6 +49,29 @@ export default function DetailActionsMenu({
search.data?.type === "audio" ? null : [`review/event/${search.id}`],
);
// don't render menu at all if no options are available
const hasSemanticSearchOption =
config?.semantic_search.enabled &&
setSimilarity !== undefined &&
search.data?.type === "object";
const hasReviewItem = !!(reviewItem && reviewItem.id);
const hasAdminTriggerOption =
isAdmin &&
config?.semantic_search.enabled &&
search.data?.type === "object";
if (
!search.has_snapshot &&
!search.has_clip &&
!hasSemanticSearchOption &&
!hasReviewItem &&
!hasAdminTriggerOption
) {
return null;
}
return (
<DropdownMenu open={isOpen} onOpenChange={setIsOpen}>
<DropdownMenuTrigger>
@@ -170,7 +170,9 @@ export function ClassFilterContent({
<FilterSwitch
key={item}
label={
item === "none" ? t("none") : item.replaceAll("_", " ")
item === "none"
? t("details.none", { ns: "views/classificationModel" })
: item.replaceAll("_", " ")
}
isChecked={classes?.includes(item) ?? false}
onCheckedChange={(isChecked) => {
@@ -178,6 +178,19 @@ export default function ObjectMaskEditPane({
filteredMask.splice(index, 0, coordinates);
}
// prevent duplicating global masks under specific object filters
if (!globalMask) {
const globalObjectMasksArray = Array.isArray(cameraConfig.objects.mask)
? cameraConfig.objects.mask
: cameraConfig.objects.mask
? [cameraConfig.objects.mask]
: [];
filteredMask = filteredMask.filter(
(mask) => !globalObjectMasksArray.includes(mask),
);
}
queryString = filteredMask
.map((pointsArray) => {
const coordinates = flattenPoints(parseCoordinates(pointsArray)).join(
+3 -3
View File
@@ -345,9 +345,9 @@ function ReviewGroup({
}
const reviewInfo = useMemo(() => {
const objectCount = fetchedEvents
? fetchedEvents.length
: (review.data.objects ?? []).length;
const detectionsCount =
review.data?.detections?.length ?? (review.data?.objects ?? []).length;
const objectCount = fetchedEvents ? fetchedEvents.length : detectionsCount;
return `${t("detail.trackedObject", { count: objectCount })}`;
}, [review, t, fetchedEvents]);
+1 -1
View File
@@ -54,7 +54,7 @@ export default function useCameraLiveMode(
}>({});
useEffect(() => {
if (!cameras) return;
if (!cameras || cameras.length === 0) return;
const mseSupported =
"MediaSource" in window || "ManagedMediaSource" in window;
+18 -2
View File
@@ -68,7 +68,10 @@ import {
ClassificationCard,
GroupedClassificationCard,
} from "@/components/card/ClassificationCard";
import { ClassificationItemData } from "@/types/classification";
import {
ClassificationItemData,
ClassifiedEvent,
} from "@/types/classification";
export default function FaceLibrary() {
const { t } = useTranslation(["views/faceLibrary"]);
@@ -922,10 +925,22 @@ function FaceAttemptGroup({
[onRefresh, t],
);
// Create ClassifiedEvent from Event (face recognition uses sub_label)
const classifiedEvent: ClassifiedEvent | undefined = useMemo(() => {
if (!event || !event.sub_label || event.sub_label === "none") {
return undefined;
}
return {
id: event.id,
label: event.sub_label,
score: event.data?.sub_label_score,
};
}, [event]);
return (
<GroupedClassificationCard
group={group}
event={event}
classifiedEvent={classifiedEvent}
threshold={threshold}
selectedItems={selectedFaces}
i18nLibrary="views/faceLibrary"
@@ -1011,6 +1026,7 @@ function FaceGrid({
filepath: `clips/faces/${pageToggle}/${image}`,
}}
selected={selectedFaces.includes(image)}
clickable={selectedFaces.length > 0}
i18nLibrary="views/faceLibrary"
onClick={(data, meta) => onClickFaces([data.filename], meta)}
>
+1 -1
View File
@@ -437,7 +437,7 @@ export default function Settings() {
return (
<div className="flex h-full flex-col">
<div className="flex items-center justify-between border-b border-secondary p-3">
<div className="flex min-h-16 items-center justify-between border-b border-secondary p-3">
<Heading as="h3" className="mb-0">
{t("menu.settings", { ns: "common" })}
</Heading>
+6
View File
@@ -21,6 +21,12 @@ export type ClassificationThreshold = {
unknown: number;
};
export type ClassifiedEvent = {
id: string;
label?: string;
score?: number;
};
export type ClassificationDatasetResponse = {
categories: {
[id: string]: string[];
+7
View File
@@ -24,5 +24,12 @@ export interface Event {
type: "object" | "audio" | "manual";
recognized_license_plate?: string;
path_data: [number[], number][];
// Allow arbitrary keys for attributes (e.g., model_name, model_name_score)
[key: string]:
| number
| number[]
| string
| [number[], number][]
| undefined;
};
}
@@ -62,6 +62,7 @@ import useApiFilter from "@/hooks/use-api-filter";
import {
ClassificationDatasetResponse,
ClassificationItemData,
ClassifiedEvent,
TrainFilter,
} from "@/types/classification";
import {
@@ -707,7 +708,7 @@ function LibrarySelector({
className="flex-grow cursor-pointer capitalize"
onClick={() => setPageToggle(id)}
>
{id === "none" ? t("none") : id.replaceAll("_", " ")}
{id === "none" ? t("details.none") : id.replaceAll("_", " ")}
<span className="ml-2 text-muted-foreground">
({dataset?.[id].length})
</span>
@@ -803,6 +804,7 @@ function DatasetGrid({
name: "",
}}
showArea={false}
clickable={selectedImages.length > 0}
selected={selectedImages.includes(image)}
i18nLibrary="views/classificationModel"
onClick={(data, _) => onClickImages([data.filename], true)}
@@ -866,6 +868,12 @@ function TrainGrid({
};
})
.filter((data) => {
// Ignore images that don't match the expected format (event-camera-timestamp-state-score.webp)
// Expected format has 5 parts when split by "-", and score should be a valid number
if (data.score === undefined || isNaN(data.score) || !data.name) {
return false;
}
if (!trainFilter) {
return true;
}
@@ -955,6 +963,7 @@ function StateTrainGrid({
data={data}
threshold={threshold}
selected={selectedImages.includes(data.filename)}
clickable={selectedImages.length > 0}
i18nLibrary="views/classificationModel"
showArea={false}
onClick={(data, meta) => onClickImages([data.filename], meta)}
@@ -1027,6 +1036,45 @@ function ObjectTrainGrid({
};
}, [model]);
// Helper function to create ClassifiedEvent from Event
const createClassifiedEvent = useCallback(
(event: Event | undefined): ClassifiedEvent | undefined => {
if (!event || !model.object_config) {
return undefined;
}
const classificationType = model.object_config.classification_type;
if (classificationType === "attribute") {
// For attribute type, look at event.data[model.name]
const attributeValue = event.data[model.name] as string | undefined;
const attributeScore = event.data[`${model.name}_score`] as
| number
| undefined;
if (attributeValue && attributeValue !== "none") {
return {
id: event.id,
label: attributeValue,
score: attributeScore,
};
}
} else {
// For sub_label type, use event.sub_label
if (event.sub_label && event.sub_label !== "none") {
return {
id: event.id,
label: event.sub_label,
score: event.data?.sub_label_score,
};
}
}
return undefined;
},
[model],
);
// selection
const [selectedEvent, setSelectedEvent] = useState<Event>();
@@ -1089,11 +1137,13 @@ function ObjectTrainGrid({
>
{Object.entries(groups).map(([key, group]) => {
const event = events?.find((ev) => ev.id == key);
const classifiedEvent = createClassifiedEvent(event);
return (
<div key={key} className="aspect-square w-full">
<GroupedClassificationCard
group={group}
event={event}
classifiedEvent={classifiedEvent}
threshold={threshold}
selectedItems={selectedImages}
i18nLibrary="views/classificationModel"
+18 -4
View File
@@ -147,10 +147,11 @@ export default function LiveCameraView({
// supported features
const [streamName, setStreamName] = useUserPersistence<string>(
`${camera.name}-stream`,
Object.values(camera.live.streams)[0],
);
const [streamName, setStreamName, streamNameLoaded] =
useUserPersistence<string>(
`${camera.name}-stream`,
Object.values(camera.live.streams)[0],
);
const isRestreamed = useMemo(
() =>
@@ -159,6 +160,19 @@ export default function LiveCameraView({
[config, streamName],
);
// validate stored stream name and reset if now invalid
useEffect(() => {
if (!streamNameLoaded) return;
const available = Object.values(camera.live.streams || {});
if (available.length === 0) return;
if (streamName != null && !available.includes(streamName)) {
setStreamName(available[0]);
}
}, [streamNameLoaded, camera.live.streams, streamName, setStreamName]);
const { data: cameraMetadata } = useSWR<LiveStreamMetadata>(
isRestreamed ? `go2rtc/streams/${streamName}` : null,
{