mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-07-28 14:49:01 +03:00
Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
35d91f5b24 | ||
|
|
538ecc03fe | ||
|
|
011ee32595 | ||
|
|
918373cb69 | ||
|
|
ae0c1ca941 | ||
|
|
29a747ca83 | ||
|
|
2d0ad54661 | ||
|
|
603d9f7d27 | ||
|
|
0f36422b35 | ||
|
|
f543d0ab31 | ||
|
|
39af85625e | ||
|
|
fa16539429 | ||
|
|
e1545a8db8 | ||
|
|
51ee6f26e6 |
@@ -270,3 +270,42 @@ To use role-based access control, you must connect to Frigate via the **authenti
|
||||
1. Log in as an **admin** user via port `8971`.
|
||||
2. Navigate to **Settings > Users**.
|
||||
3. Edit a user’s role by selecting **admin** or **viewer**.
|
||||
|
||||
## API Authentication Guide
|
||||
|
||||
### Getting a Bearer Token
|
||||
|
||||
To use the Frigate API, you need to authenticate first. Follow these steps to obtain a Bearer token:
|
||||
|
||||
#### 1. Login
|
||||
|
||||
Make a POST request to `/login` with your credentials:
|
||||
|
||||
```bash
|
||||
curl -i -X POST https://frigate_ip:8971/api/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"user": "admin", "password": "your_password"}'
|
||||
```
|
||||
|
||||
:::note
|
||||
|
||||
You may need to include `-k` in the argument list in these steps (eg: `curl -k -i -X POST ...`) if your Frigate instance is using a self-signed certificate.
|
||||
|
||||
:::
|
||||
|
||||
The response will contain a cookie with the JWT token.
|
||||
|
||||
#### 2. Using the Bearer Token
|
||||
|
||||
Once you have the token, include it in the Authorization header for subsequent requests:
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer <your_token>" https://frigate_ip:8971/api/profile
|
||||
```
|
||||
|
||||
#### 3. Token Lifecycle
|
||||
|
||||
- Tokens are valid for the configured session length
|
||||
- Tokens are automatically refreshed when you visit the `/auth` endpoint
|
||||
- Tokens are invalidated when the user's password is changed
|
||||
- Use `/logout` to clear your session cookie
|
||||
|
||||
@@ -60,11 +60,9 @@ Choose one or more cameras and draw a rectangle over the area of interest for ea
|
||||
|
||||
### Step 3: Assign Training Examples
|
||||
|
||||
The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state.
|
||||
The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state. It's not strictly required to select all images you see. If a state is missing from the samples, you can train it from the Recent tab later.
|
||||
|
||||
**Important**: All images must be assigned to a state before training can begin. This includes images that may not be optimal, such as when people temporarily block the view, sun glare is present, or other distractions occur. Assign these images to the state that is actually present (based on what you know the state to be), not based on the distraction. This training helps the model correctly identify the state even when such conditions occur during inference.
|
||||
|
||||
Once all images are assigned, training will begin automatically.
|
||||
Once some images are assigned, training will begin automatically.
|
||||
|
||||
### Improving the Model
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ Object detection and enrichments (like Semantic Search, Face Recognition, and Li
|
||||
|
||||
- **AMD**
|
||||
|
||||
- ROCm will automatically be detected and used for enrichments in the `-rocm` Frigate image.
|
||||
- ROCm support in the `-rocm` Frigate image is automatically detected for enrichments, but only some enrichment models are available due to ROCm's focus on LLMs and limited stability with certain neural network models. Frigate disables models that perform poorly or are unstable to ensure reliable operation, so only compatible enrichments may be active.
|
||||
|
||||
- **Intel**
|
||||
|
||||
|
||||
@@ -175,7 +175,7 @@ model:
|
||||
width: 320 # <--- should match the imgsize of the model, typically 320
|
||||
height: 320 # <--- should match the imgsize of the model, typically 320
|
||||
path: /config/model_cache/yolov9-s-relu6-best_320_int8_edgetpu.tflite
|
||||
labelmap_path: /labelmap/labels-coco-17.txt
|
||||
labelmap_path: /config/labels-coco17.txt
|
||||
```
|
||||
|
||||
Note that the labelmap uses a subset of the complete COCO label set that has only 17 objects.
|
||||
|
||||
Vendored
+106
-14
@@ -14,19 +14,38 @@ paths:
|
||||
get:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Auth
|
||||
summary: Authenticate request
|
||||
description: |-
|
||||
Authenticates the current request based on proxy headers or JWT token.
|
||||
This endpoint verifies authentication credentials and manages JWT token refresh.
|
||||
On success, no JSON body is returned; authentication state is communicated via response headers and cookies.
|
||||
operationId: auth_auth_get
|
||||
responses:
|
||||
"200":
|
||||
description: Successful Response
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"202":
|
||||
description: Authentication Accepted (no response body, different headers depending on auth method)
|
||||
headers:
|
||||
remote-user:
|
||||
description: Authenticated username or "anonymous" in proxy-only mode
|
||||
schema:
|
||||
type: string
|
||||
remote-role:
|
||||
description: Resolved role (e.g., admin, viewer, or custom)
|
||||
schema:
|
||||
type: string
|
||||
Set-Cookie:
|
||||
description: May include refreshed JWT cookie ("frigate-token") when applicable
|
||||
schema:
|
||||
type: string
|
||||
"401":
|
||||
description: Authentication Failed
|
||||
/profile:
|
||||
get:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Profile
|
||||
summary: Get user profile
|
||||
description: |-
|
||||
Returns the current authenticated user's profile including username, role, and allowed cameras.
|
||||
This endpoint requires authentication and returns information about the user's permissions.
|
||||
operationId: profile_profile_get
|
||||
responses:
|
||||
"200":
|
||||
@@ -34,11 +53,16 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"401":
|
||||
description: Unauthorized
|
||||
/logout:
|
||||
get:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Logout
|
||||
summary: Logout user
|
||||
description: |-
|
||||
Logs out the current user by clearing the session cookie.
|
||||
After logout, subsequent requests will require re-authentication.
|
||||
operationId: logout_logout_get
|
||||
responses:
|
||||
"200":
|
||||
@@ -46,11 +70,22 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"303":
|
||||
description: See Other (redirects to login page)
|
||||
/login:
|
||||
post:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Login
|
||||
summary: Login with credentials
|
||||
description: |-
|
||||
Authenticates a user with username and password.
|
||||
Returns a JWT token as a secure HTTP-only cookie that can be used for subsequent API requests.
|
||||
The JWT token can also be retrieved from the response and used as a Bearer token in the Authorization header.
|
||||
|
||||
Example using Bearer token:
|
||||
```
|
||||
curl -H "Authorization: Bearer <token_value>" https://frigate_ip:8971/api/profile
|
||||
```
|
||||
operationId: login_login_post
|
||||
requestBody:
|
||||
required: true
|
||||
@@ -64,6 +99,11 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"401":
|
||||
description: Login Failed - Invalid credentials
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"422":
|
||||
description: Validation Error
|
||||
content:
|
||||
@@ -74,7 +114,10 @@ paths:
|
||||
get:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Get Users
|
||||
summary: Get all users
|
||||
description: |-
|
||||
Returns a list of all users with their usernames and roles.
|
||||
Requires admin role. Each user object contains the username and assigned role.
|
||||
operationId: get_users_users_get
|
||||
responses:
|
||||
"200":
|
||||
@@ -82,10 +125,19 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"403":
|
||||
description: Forbidden - Admin role required
|
||||
post:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Create User
|
||||
summary: Create new user
|
||||
description: |-
|
||||
Creates a new user with the specified username, password, and role.
|
||||
Requires admin role. Password must meet strength requirements:
|
||||
- Minimum 8 characters
|
||||
- At least one uppercase letter
|
||||
- At least one digit
|
||||
- At least one special character (!@#$%^&*(),.?":{}\|<>)
|
||||
operationId: create_user_users_post
|
||||
requestBody:
|
||||
required: true
|
||||
@@ -99,6 +151,13 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"400":
|
||||
description: Bad Request - Invalid username or role
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"403":
|
||||
description: Forbidden - Admin role required
|
||||
"422":
|
||||
description: Validation Error
|
||||
content:
|
||||
@@ -109,7 +168,10 @@ paths:
|
||||
delete:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Delete User
|
||||
summary: Delete user
|
||||
description: |-
|
||||
Deletes a user by username. The built-in admin user cannot be deleted.
|
||||
Requires admin role. Returns success message or error if user not found.
|
||||
operationId: delete_user_users__username__delete
|
||||
parameters:
|
||||
- name: username
|
||||
@@ -118,12 +180,15 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
title: Username
|
||||
description: The username of the user to delete
|
||||
responses:
|
||||
"200":
|
||||
description: Successful Response
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"403":
|
||||
description: Forbidden - Cannot delete admin user or admin role required
|
||||
"422":
|
||||
description: Validation Error
|
||||
content:
|
||||
@@ -134,7 +199,17 @@ paths:
|
||||
put:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Update Password
|
||||
summary: Update user password
|
||||
description: |-
|
||||
Updates a user's password. Users can only change their own password unless they have admin role.
|
||||
Requires the current password to verify identity for non-admin users.
|
||||
Password must meet strength requirements:
|
||||
- Minimum 8 characters
|
||||
- At least one uppercase letter
|
||||
- At least one digit
|
||||
- At least one special character (!@#$%^&*(),.?":{}\|<>)
|
||||
|
||||
If user changes their own password, a new JWT cookie is automatically issued.
|
||||
operationId: update_password_users__username__password_put
|
||||
parameters:
|
||||
- name: username
|
||||
@@ -143,6 +218,7 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
title: Username
|
||||
description: The username of the user whose password to update
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
@@ -155,6 +231,14 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"400":
|
||||
description: Bad Request - Current password required or password doesn't meet requirements
|
||||
"401":
|
||||
description: Unauthorized - Current password is incorrect
|
||||
"403":
|
||||
description: Forbidden - Viewers can only update their own password
|
||||
"404":
|
||||
description: Not Found - User not found
|
||||
"422":
|
||||
description: Validation Error
|
||||
content:
|
||||
@@ -165,7 +249,10 @@ paths:
|
||||
put:
|
||||
tags:
|
||||
- Auth
|
||||
summary: Update Role
|
||||
summary: Update user role
|
||||
description: |-
|
||||
Updates a user's role. The built-in admin user's role cannot be modified.
|
||||
Requires admin role. Valid roles are defined in the configuration.
|
||||
operationId: update_role_users__username__role_put
|
||||
parameters:
|
||||
- name: username
|
||||
@@ -174,6 +261,7 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
title: Username
|
||||
description: The username of the user whose role to update
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
@@ -186,6 +274,10 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {}
|
||||
"400":
|
||||
description: Bad Request - Invalid role
|
||||
"403":
|
||||
description: Forbidden - Cannot modify admin user's role or admin role required
|
||||
"422":
|
||||
description: Validation Error
|
||||
content:
|
||||
|
||||
+76
-15
@@ -549,7 +549,37 @@ def resolve_role(
|
||||
|
||||
|
||||
# Endpoints
|
||||
@router.get("/auth", dependencies=[Depends(allow_public())])
|
||||
@router.get(
|
||||
"/auth",
|
||||
dependencies=[Depends(allow_public())],
|
||||
summary="Authenticate request",
|
||||
description=(
|
||||
"Authenticates the current request based on proxy headers or JWT token. "
|
||||
"This endpoint verifies authentication credentials and manages JWT token refresh. "
|
||||
"On success, no JSON body is returned; authentication state is communicated via response headers and cookies."
|
||||
),
|
||||
status_code=202,
|
||||
responses={
|
||||
202: {
|
||||
"description": "Authentication Accepted (no response body)",
|
||||
"headers": {
|
||||
"remote-user": {
|
||||
"description": 'Authenticated username or "anonymous" in proxy-only mode',
|
||||
"schema": {"type": "string"},
|
||||
},
|
||||
"remote-role": {
|
||||
"description": "Resolved role (e.g., admin, viewer, or custom)",
|
||||
"schema": {"type": "string"},
|
||||
},
|
||||
"Set-Cookie": {
|
||||
"description": "May include refreshed JWT cookie when applicable",
|
||||
"schema": {"type": "string"},
|
||||
},
|
||||
},
|
||||
},
|
||||
401: {"description": "Authentication Failed"},
|
||||
},
|
||||
)
|
||||
def auth(request: Request):
|
||||
auth_config: AuthConfig = request.app.frigate_config.auth
|
||||
proxy_config: ProxyConfig = request.app.frigate_config.proxy
|
||||
@@ -689,7 +719,12 @@ def auth(request: Request):
|
||||
return fail_response
|
||||
|
||||
|
||||
@router.get("/profile", dependencies=[Depends(allow_any_authenticated())])
|
||||
@router.get(
|
||||
"/profile",
|
||||
dependencies=[Depends(allow_any_authenticated())],
|
||||
summary="Get user profile",
|
||||
description="Returns the current authenticated user's profile including username, role, and allowed cameras. This endpoint requires authentication and returns information about the user's permissions.",
|
||||
)
|
||||
def profile(request: Request):
|
||||
username = request.headers.get("remote-user", "anonymous")
|
||||
role = request.headers.get("remote-role", "viewer")
|
||||
@@ -703,7 +738,12 @@ def profile(request: Request):
|
||||
)
|
||||
|
||||
|
||||
@router.get("/logout", dependencies=[Depends(allow_public())])
|
||||
@router.get(
|
||||
"/logout",
|
||||
dependencies=[Depends(allow_public())],
|
||||
summary="Logout user",
|
||||
description="Logs out the current user by clearing the session cookie. After logout, subsequent requests will require re-authentication.",
|
||||
)
|
||||
def logout(request: Request):
|
||||
auth_config: AuthConfig = request.app.frigate_config.auth
|
||||
response = RedirectResponse("/login", status_code=303)
|
||||
@@ -714,7 +754,12 @@ def logout(request: Request):
|
||||
limiter = Limiter(key_func=get_remote_addr)
|
||||
|
||||
|
||||
@router.post("/login", dependencies=[Depends(allow_public())])
|
||||
@router.post(
|
||||
"/login",
|
||||
dependencies=[Depends(allow_public())],
|
||||
summary="Login with credentials",
|
||||
description='Authenticates a user with username and password. Returns a JWT token as a secure HTTP-only cookie that can be used for subsequent API requests. The JWT token can also be retrieved from the response and used as a Bearer token in the Authorization header.\n\nExample using Bearer token:\n```\ncurl -H "Authorization: Bearer <token_value>" https://frigate_ip:8971/api/profile\n```',
|
||||
)
|
||||
@limiter.limit(limit_value=rateLimiter.get_limit)
|
||||
def login(request: Request, body: AppPostLoginBody):
|
||||
JWT_COOKIE_NAME = request.app.frigate_config.auth.cookie_name
|
||||
@@ -752,7 +797,12 @@ def login(request: Request, body: AppPostLoginBody):
|
||||
return JSONResponse(content={"message": "Login failed"}, status_code=401)
|
||||
|
||||
|
||||
@router.get("/users", dependencies=[Depends(require_role(["admin"]))])
|
||||
@router.get(
|
||||
"/users",
|
||||
dependencies=[Depends(require_role(["admin"]))],
|
||||
summary="Get all users",
|
||||
description="Returns a list of all users with their usernames and roles. Requires admin role. Each user object contains the username and assigned role.",
|
||||
)
|
||||
def get_users():
|
||||
exports = (
|
||||
User.select(User.username, User.role).order_by(User.username).dicts().iterator()
|
||||
@@ -760,7 +810,12 @@ def get_users():
|
||||
return JSONResponse([e for e in exports])
|
||||
|
||||
|
||||
@router.post("/users", dependencies=[Depends(require_role(["admin"]))])
|
||||
@router.post(
|
||||
"/users",
|
||||
dependencies=[Depends(require_role(["admin"]))],
|
||||
summary="Create new user",
|
||||
description='Creates a new user with the specified username, password, and role. Requires admin role. Password must meet strength requirements: minimum 8 characters, at least one uppercase letter, at least one digit, and at least one special character (!@#$%^&*(),.?":{} |<>).',
|
||||
)
|
||||
def create_user(
|
||||
request: Request,
|
||||
body: AppPostUsersBody,
|
||||
@@ -789,7 +844,12 @@ def create_user(
|
||||
return JSONResponse(content={"username": body.username})
|
||||
|
||||
|
||||
@router.delete("/users/{username}", dependencies=[Depends(require_role(["admin"]))])
|
||||
@router.delete(
|
||||
"/users/{username}",
|
||||
dependencies=[Depends(require_role(["admin"]))],
|
||||
summary="Delete user",
|
||||
description="Deletes a user by username. The built-in admin user cannot be deleted. Requires admin role. Returns success message or error if user not found.",
|
||||
)
|
||||
def delete_user(request: Request, username: str):
|
||||
# Prevent deletion of the built-in admin user
|
||||
if username == "admin":
|
||||
@@ -802,7 +862,10 @@ def delete_user(request: Request, username: str):
|
||||
|
||||
|
||||
@router.put(
|
||||
"/users/{username}/password", dependencies=[Depends(allow_any_authenticated())]
|
||||
"/users/{username}/password",
|
||||
dependencies=[Depends(allow_any_authenticated())],
|
||||
summary="Update user password",
|
||||
description="Updates a user's password. Users can only change their own password unless they have admin role. Requires the current password to verify identity for non-admin users. Password must meet strength requirements: minimum 8 characters, at least one uppercase letter, at least one digit, and at least one special character (!@#$%^&*(),.?\":{} |<>). If user changes their own password, a new JWT cookie is automatically issued.",
|
||||
)
|
||||
async def update_password(
|
||||
request: Request,
|
||||
@@ -830,13 +893,9 @@ async def update_password(
|
||||
except DoesNotExist:
|
||||
return JSONResponse(content={"message": "User not found"}, status_code=404)
|
||||
|
||||
# Require old_password when:
|
||||
# 1. Non-admin user is changing another user's password (admin only action)
|
||||
# 2. Any user is changing their own password
|
||||
is_changing_own_password = current_username == username
|
||||
is_non_admin = current_role != "admin"
|
||||
|
||||
if is_changing_own_password or is_non_admin:
|
||||
# Require old_password when non-admin user is changing any password
|
||||
# Admin users changing passwords do NOT need to provide the current password
|
||||
if current_role != "admin":
|
||||
if not body.old_password:
|
||||
return JSONResponse(
|
||||
content={"message": "Current password is required"},
|
||||
@@ -887,6 +946,8 @@ async def update_password(
|
||||
@router.put(
|
||||
"/users/{username}/role",
|
||||
dependencies=[Depends(require_role(["admin"]))],
|
||||
summary="Update user role",
|
||||
description="Updates a user's role. The built-in admin user's role cannot be modified. Requires admin role. Valid roles are defined in the configuration.",
|
||||
)
|
||||
async def update_role(
|
||||
request: Request,
|
||||
|
||||
@@ -19,11 +19,6 @@ from frigate.util.object import calculate_region
|
||||
from ..types import DataProcessorMetrics
|
||||
from .api import RealTimeProcessorApi
|
||||
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter
|
||||
except ModuleNotFoundError:
|
||||
from tensorflow.lite.python.interpreter import Interpreter
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -35,7 +30,7 @@ class BirdRealTimeProcessor(RealTimeProcessorApi):
|
||||
metrics: DataProcessorMetrics,
|
||||
):
|
||||
super().__init__(config, metrics)
|
||||
self.interpreter: Interpreter = None
|
||||
self.interpreter: Any | None = None
|
||||
self.sub_label_publisher = sub_label_publisher
|
||||
self.tensor_input_details: dict[str, Any] = None
|
||||
self.tensor_output_details: dict[str, Any] = None
|
||||
@@ -82,6 +77,11 @@ class BirdRealTimeProcessor(RealTimeProcessorApi):
|
||||
|
||||
@redirect_output_to_logger(logger, logging.DEBUG)
|
||||
def __build_detector(self) -> None:
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter
|
||||
except ModuleNotFoundError:
|
||||
from tensorflow.lite.python.interpreter import Interpreter
|
||||
|
||||
self.interpreter = Interpreter(
|
||||
model_path=os.path.join(MODEL_CACHE_DIR, "bird/bird.tflite"),
|
||||
num_threads=2,
|
||||
|
||||
@@ -29,11 +29,6 @@ from frigate.util.object import box_overlaps, calculate_region
|
||||
from ..types import DataProcessorMetrics
|
||||
from .api import RealTimeProcessorApi
|
||||
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter
|
||||
except ModuleNotFoundError:
|
||||
from tensorflow.lite.python.interpreter import Interpreter
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
MAX_OBJECT_CLASSIFICATIONS = 16
|
||||
@@ -52,7 +47,7 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
|
||||
self.requestor = requestor
|
||||
self.model_dir = os.path.join(MODEL_CACHE_DIR, self.model_config.name)
|
||||
self.train_dir = os.path.join(CLIPS_DIR, self.model_config.name, "train")
|
||||
self.interpreter: Interpreter | None = None
|
||||
self.interpreter: Any | None = None
|
||||
self.tensor_input_details: dict[str, Any] | None = None
|
||||
self.tensor_output_details: dict[str, Any] | None = None
|
||||
self.labelmap: dict[int, str] = {}
|
||||
@@ -74,6 +69,11 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
|
||||
|
||||
@redirect_output_to_logger(logger, logging.DEBUG)
|
||||
def __build_detector(self) -> None:
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter
|
||||
except ModuleNotFoundError:
|
||||
from tensorflow.lite.python.interpreter import Interpreter
|
||||
|
||||
model_path = os.path.join(self.model_dir, "model.tflite")
|
||||
labelmap_path = os.path.join(self.model_dir, "labelmap.txt")
|
||||
|
||||
@@ -345,7 +345,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
|
||||
self.model_config = model_config
|
||||
self.model_dir = os.path.join(MODEL_CACHE_DIR, self.model_config.name)
|
||||
self.train_dir = os.path.join(CLIPS_DIR, self.model_config.name, "train")
|
||||
self.interpreter: Interpreter | None = None
|
||||
self.interpreter: Any | None = None
|
||||
self.sub_label_publisher = sub_label_publisher
|
||||
self.requestor = requestor
|
||||
self.tensor_input_details: dict[str, Any] | None = None
|
||||
@@ -368,6 +368,11 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
|
||||
|
||||
@redirect_output_to_logger(logger, logging.DEBUG)
|
||||
def __build_detector(self) -> None:
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter
|
||||
except ModuleNotFoundError:
|
||||
from tensorflow.lite.python.interpreter import Interpreter
|
||||
|
||||
model_path = os.path.join(self.model_dir, "model.tflite")
|
||||
labelmap_path = os.path.join(self.model_dir, "labelmap.txt")
|
||||
|
||||
|
||||
@@ -131,6 +131,7 @@ class ONNXModelRunner(BaseModelRunner):
|
||||
|
||||
return model_type in [
|
||||
EnrichmentModelTypeEnum.paddleocr.value,
|
||||
EnrichmentModelTypeEnum.yolov9_license_plate.value,
|
||||
EnrichmentModelTypeEnum.jina_v1.value,
|
||||
EnrichmentModelTypeEnum.jina_v2.value,
|
||||
EnrichmentModelTypeEnum.facenet.value,
|
||||
|
||||
@@ -146,6 +146,29 @@ class EmbeddingMaintainer(threading.Thread):
|
||||
self.detected_license_plates: dict[str, dict[str, Any]] = {}
|
||||
self.genai_client = get_genai_client(config)
|
||||
|
||||
# Pre-import TensorFlow/tflite on main thread to avoid atexit registration issues
|
||||
# when importing from worker threads later (e.g., during dynamic config updates)
|
||||
if (
|
||||
self.config.classification.bird.enabled
|
||||
or len(self.config.classification.custom) > 0
|
||||
):
|
||||
try:
|
||||
from tflite_runtime.interpreter import Interpreter # noqa: F401
|
||||
except ModuleNotFoundError:
|
||||
try:
|
||||
from tensorflow.lite.python.interpreter import ( # noqa: F401
|
||||
Interpreter,
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
"Pre-imported TensorFlow Interpreter on main thread for classification models"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Failed to pre-import TensorFlow Interpreter: {e}. "
|
||||
"Classification models may fail to load if added dynamically."
|
||||
)
|
||||
|
||||
# model runners to share between realtime and post processors
|
||||
if self.config.lpr.enabled:
|
||||
lpr_model_runner = LicensePlateModelRunner(
|
||||
|
||||
@@ -153,7 +153,7 @@ PRESETS_HW_ACCEL_ENCODE_BIRDSEYE = {
|
||||
FFMPEG_HWACCEL_VAAPI: "{0} -hide_banner -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device {3} {1} -c:v h264_vaapi -g 50 -bf 0 -profile:v high -level:v 4.1 -sei:v 0 -an -vf format=vaapi|nv12,hwupload {2}",
|
||||
"preset-intel-qsv-h264": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v high -level:v 4.1 -async_depth:v 1 {2}",
|
||||
"preset-intel-qsv-h265": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v main -level:v 4.1 -async_depth:v 1 {2}",
|
||||
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} -hwaccel device {3} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
|
||||
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
|
||||
"preset-jetson-h264": "{0} -hide_banner {1} -c:v h264_nvmpi -profile high {2}",
|
||||
"preset-jetson-h265": "{0} -hide_banner {1} -c:v h264_nvmpi -profile main {2}",
|
||||
FFMPEG_HWACCEL_RKMPP: "{0} -hide_banner {1} -c:v h264_rkmpp -profile:v high {2}",
|
||||
|
||||
@@ -119,6 +119,7 @@ class RecordingCleanup(threading.Thread):
|
||||
Recordings.path,
|
||||
Recordings.objects,
|
||||
Recordings.motion,
|
||||
Recordings.dBFS,
|
||||
)
|
||||
.where(
|
||||
(Recordings.camera == config.name)
|
||||
@@ -126,6 +127,7 @@ class RecordingCleanup(threading.Thread):
|
||||
(
|
||||
(Recordings.end_time < continuous_expire_date)
|
||||
& (Recordings.motion == 0)
|
||||
& (Recordings.dBFS == 0)
|
||||
)
|
||||
| (Recordings.end_time < motion_expire_date)
|
||||
)
|
||||
@@ -185,6 +187,7 @@ class RecordingCleanup(threading.Thread):
|
||||
mode == RetainModeEnum.motion
|
||||
and recording.motion == 0
|
||||
and recording.objects == 0
|
||||
and recording.dBFS == 0
|
||||
)
|
||||
or (mode == RetainModeEnum.active_objects and recording.objects == 0)
|
||||
):
|
||||
|
||||
@@ -67,7 +67,7 @@ class SegmentInfo:
|
||||
if (
|
||||
not keep
|
||||
and retain_mode == RetainModeEnum.motion
|
||||
and (self.motion_count > 0 or self.average_dBFS > 0)
|
||||
and (self.motion_count > 0 or self.average_dBFS != 0)
|
||||
):
|
||||
keep = True
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@ from frigate.const import (
|
||||
from frigate.log import redirect_output_to_logger
|
||||
from frigate.models import Event, Recordings, ReviewSegment
|
||||
from frigate.types import ModelStatusTypesEnum
|
||||
from frigate.util.downloader import ModelDownloader
|
||||
from frigate.util.file import get_event_thumbnail_bytes
|
||||
from frigate.util.image import get_image_from_recording
|
||||
from frigate.util.process import FrigateProcess
|
||||
@@ -121,6 +122,10 @@ def get_dataset_image_count(model_name: str) -> int:
|
||||
|
||||
class ClassificationTrainingProcess(FrigateProcess):
|
||||
def __init__(self, model_name: str) -> None:
|
||||
self.BASE_WEIGHT_URL = os.environ.get(
|
||||
"TF_KERAS_MOBILENET_V2_WEIGHTS_URL",
|
||||
"",
|
||||
)
|
||||
super().__init__(
|
||||
stop_event=None,
|
||||
priority=PROCESS_PRIORITY_LOW,
|
||||
@@ -179,11 +184,23 @@ class ClassificationTrainingProcess(FrigateProcess):
|
||||
)
|
||||
return False
|
||||
|
||||
weights_path = "imagenet"
|
||||
# Download MobileNetV2 weights if not present
|
||||
if self.BASE_WEIGHT_URL:
|
||||
weights_path = os.path.join(
|
||||
MODEL_CACHE_DIR, "MobileNet", "mobilenet_v2_weights.h5"
|
||||
)
|
||||
if not os.path.exists(weights_path):
|
||||
logger.info("Downloading MobileNet V2 weights file")
|
||||
ModelDownloader.download_from_url(
|
||||
self.BASE_WEIGHT_URL, weights_path
|
||||
)
|
||||
|
||||
# Start with imagenet base model with 35% of channels in each layer
|
||||
base_model = MobileNetV2(
|
||||
input_shape=(224, 224, 3),
|
||||
include_top=False,
|
||||
weights="imagenet",
|
||||
weights=weights_path,
|
||||
alpha=0.35,
|
||||
)
|
||||
base_model.trainable = False # Freeze pre-trained layers
|
||||
@@ -482,6 +499,10 @@ def _extract_keyframes(
|
||||
"""
|
||||
Extract keyframes from recordings at specified timestamps and crop to specified regions.
|
||||
|
||||
This implementation batches work by running multiple ffmpeg snapshot commands
|
||||
concurrently, which significantly reduces total runtime compared to
|
||||
processing each timestamp serially.
|
||||
|
||||
Args:
|
||||
ffmpeg_path: Path to ffmpeg binary
|
||||
timestamps: List of timestamp dicts from _select_balanced_timestamps
|
||||
@@ -491,15 +512,21 @@ def _extract_keyframes(
|
||||
Returns:
|
||||
List of paths to successfully extracted and cropped keyframe images
|
||||
"""
|
||||
keyframe_paths = []
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
for idx, ts_info in enumerate(timestamps):
|
||||
if not timestamps:
|
||||
return []
|
||||
|
||||
# Limit the number of concurrent ffmpeg processes so we don't overload the host.
|
||||
max_workers = min(5, len(timestamps))
|
||||
|
||||
def _process_timestamp(idx: int, ts_info: dict) -> tuple[int, str | None]:
|
||||
camera = ts_info["camera"]
|
||||
timestamp = ts_info["timestamp"]
|
||||
|
||||
if camera not in camera_crops:
|
||||
logger.warning(f"No crop coordinates for camera {camera}")
|
||||
continue
|
||||
return idx, None
|
||||
|
||||
norm_x1, norm_y1, norm_x2, norm_y2 = camera_crops[camera]
|
||||
|
||||
@@ -516,7 +543,7 @@ def _extract_keyframes(
|
||||
.get()
|
||||
)
|
||||
except Exception:
|
||||
continue
|
||||
return idx, None
|
||||
|
||||
relative_time = timestamp - recording.start_time
|
||||
|
||||
@@ -530,38 +557,57 @@ def _extract_keyframes(
|
||||
height=None,
|
||||
)
|
||||
|
||||
if image_data:
|
||||
nparr = np.frombuffer(image_data, np.uint8)
|
||||
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
|
||||
if not image_data:
|
||||
return idx, None
|
||||
|
||||
if img is not None:
|
||||
height, width = img.shape[:2]
|
||||
nparr = np.frombuffer(image_data, np.uint8)
|
||||
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
|
||||
|
||||
x1 = int(norm_x1 * width)
|
||||
y1 = int(norm_y1 * height)
|
||||
x2 = int(norm_x2 * width)
|
||||
y2 = int(norm_y2 * height)
|
||||
if img is None:
|
||||
return idx, None
|
||||
|
||||
x1_clipped = max(0, min(x1, width))
|
||||
y1_clipped = max(0, min(y1, height))
|
||||
x2_clipped = max(0, min(x2, width))
|
||||
y2_clipped = max(0, min(y2, height))
|
||||
height, width = img.shape[:2]
|
||||
|
||||
if x2_clipped > x1_clipped and y2_clipped > y1_clipped:
|
||||
cropped = img[y1_clipped:y2_clipped, x1_clipped:x2_clipped]
|
||||
resized = cv2.resize(cropped, (224, 224))
|
||||
x1 = int(norm_x1 * width)
|
||||
y1 = int(norm_y1 * height)
|
||||
x2 = int(norm_x2 * width)
|
||||
y2 = int(norm_y2 * height)
|
||||
|
||||
output_path = os.path.join(output_dir, f"frame_{idx:04d}.jpg")
|
||||
cv2.imwrite(output_path, resized)
|
||||
keyframe_paths.append(output_path)
|
||||
x1_clipped = max(0, min(x1, width))
|
||||
y1_clipped = max(0, min(y1, height))
|
||||
x2_clipped = max(0, min(x2, width))
|
||||
y2_clipped = max(0, min(y2, height))
|
||||
|
||||
if x2_clipped <= x1_clipped or y2_clipped <= y1_clipped:
|
||||
return idx, None
|
||||
|
||||
cropped = img[y1_clipped:y2_clipped, x1_clipped:x2_clipped]
|
||||
resized = cv2.resize(cropped, (224, 224))
|
||||
|
||||
output_path = os.path.join(output_dir, f"frame_{idx:04d}.jpg")
|
||||
cv2.imwrite(output_path, resized)
|
||||
return idx, output_path
|
||||
except Exception as e:
|
||||
logger.debug(
|
||||
f"Failed to extract frame from {recording.path} at {relative_time}s: {e}"
|
||||
)
|
||||
continue
|
||||
return idx, None
|
||||
|
||||
return keyframe_paths
|
||||
keyframes_with_index: list[tuple[int, str]] = []
|
||||
|
||||
with ThreadPoolExecutor(max_workers=max_workers) as executor:
|
||||
future_to_idx = {
|
||||
executor.submit(_process_timestamp, idx, ts_info): idx
|
||||
for idx, ts_info in enumerate(timestamps)
|
||||
}
|
||||
|
||||
for future in as_completed(future_to_idx):
|
||||
_, path = future.result()
|
||||
if path:
|
||||
keyframes_with_index.append((future_to_idx[future], path))
|
||||
|
||||
keyframes_with_index.sort(key=lambda item: item[0])
|
||||
return [path for _, path in keyframes_with_index]
|
||||
|
||||
|
||||
def _select_distinct_images(
|
||||
|
||||
@@ -679,7 +679,7 @@
|
||||
"desc": "Manage this Frigate instance's user accounts."
|
||||
},
|
||||
"addUser": "Add User",
|
||||
"updatePassword": "Update Password",
|
||||
"updatePassword": "Reset Password",
|
||||
"toast": {
|
||||
"success": {
|
||||
"createUser": "User {{user}} created successfully",
|
||||
@@ -700,7 +700,7 @@
|
||||
"role": "Role",
|
||||
"noUsers": "No users found.",
|
||||
"changeRole": "Change user role",
|
||||
"password": "Password",
|
||||
"password": "Reset Password",
|
||||
"deleteUser": "Delete user"
|
||||
},
|
||||
"dialog": {
|
||||
|
||||
+13
-7
@@ -14,6 +14,7 @@ import ProtectedRoute from "@/components/auth/ProtectedRoute";
|
||||
import { AuthProvider } from "@/context/auth-context";
|
||||
import useSWR from "swr";
|
||||
import { FrigateConfig } from "./types/frigateConfig";
|
||||
import ActivityIndicator from "@/components/indicators/activity-indicator";
|
||||
|
||||
const Live = lazy(() => import("@/pages/Live"));
|
||||
const Events = lazy(() => import("@/pages/Events"));
|
||||
@@ -50,6 +51,13 @@ function DefaultAppView() {
|
||||
const { data: config } = useSWR<FrigateConfig>("config", {
|
||||
revalidateOnFocus: false,
|
||||
});
|
||||
|
||||
// Compute required roles for main routes, ensuring we have config first
|
||||
// to prevent race condition where custom roles are temporarily unavailable
|
||||
const mainRouteRoles = config?.auth?.roles
|
||||
? Object.keys(config.auth.roles)
|
||||
: undefined;
|
||||
|
||||
return (
|
||||
<div className="size-full overflow-hidden">
|
||||
{isDesktop && <Sidebar />}
|
||||
@@ -68,13 +76,11 @@ function DefaultAppView() {
|
||||
<Routes>
|
||||
<Route
|
||||
element={
|
||||
<ProtectedRoute
|
||||
requiredRoles={
|
||||
config?.auth.roles
|
||||
? Object.keys(config.auth.roles)
|
||||
: ["admin", "viewer"]
|
||||
}
|
||||
/>
|
||||
mainRouteRoles ? (
|
||||
<ProtectedRoute requiredRoles={mainRouteRoles} />
|
||||
) : (
|
||||
<ActivityIndicator className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2" />
|
||||
)
|
||||
}
|
||||
>
|
||||
<Route index element={<Live />} />
|
||||
|
||||
@@ -116,10 +116,10 @@ export default function Statusbar() {
|
||||
}
|
||||
|
||||
return (
|
||||
<Link key={gpuTitle} to="/system#general">
|
||||
<Link key={name} to="/system#general">
|
||||
{" "}
|
||||
<div
|
||||
key={gpuTitle}
|
||||
key={name}
|
||||
className="flex cursor-pointer items-center gap-2 text-sm hover:underline"
|
||||
>
|
||||
<MdCircle
|
||||
|
||||
@@ -315,7 +315,7 @@ export default function Step1NameAndDefine({
|
||||
<FormLabel className="text-primary-variant">
|
||||
{t("wizard.step1.classificationType")}
|
||||
</FormLabel>
|
||||
<Popover>
|
||||
<Popover modal={true}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
@@ -398,7 +398,7 @@ export default function Step1NameAndDefine({
|
||||
? t("wizard.step1.states")
|
||||
: t("wizard.step1.classes")}
|
||||
</FormLabel>
|
||||
<Popover>
|
||||
<Popover modal={true}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button variant="ghost" size="sm" className="h-4 w-4 p-0">
|
||||
<LuInfo className="size-3" />
|
||||
|
||||
@@ -141,7 +141,37 @@ export default function Step3ChooseExamples({
|
||||
);
|
||||
await Promise.all(categorizePromises);
|
||||
|
||||
// Step 2.5: Create empty folders for classes that don't have any images
|
||||
// Step 2.5: Delete any unselected images from train folder
|
||||
// For state models, all images must be classified, so unselected images should be removed
|
||||
// For object models, unselected images are assigned to "none" so they're already categorized
|
||||
if (step1Data.modelType === "state") {
|
||||
try {
|
||||
// Fetch current train images to see what's left after categorization
|
||||
const trainImagesResponse = await axios.get<string[]>(
|
||||
`/classification/${step1Data.modelName}/train`,
|
||||
);
|
||||
const remainingTrainImages = trainImagesResponse.data || [];
|
||||
|
||||
const categorizedImageNames = new Set(Object.keys(classifications));
|
||||
const unselectedImages = remainingTrainImages.filter(
|
||||
(imageName) => !categorizedImageNames.has(imageName),
|
||||
);
|
||||
|
||||
if (unselectedImages.length > 0) {
|
||||
await axios.post(
|
||||
`/classification/${step1Data.modelName}/train/delete`,
|
||||
{
|
||||
ids: unselectedImages,
|
||||
},
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
// Silently fail - unselected images will remain but won't cause issues
|
||||
// since the frontend filters out images that don't match expected format
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2.6: Create empty folders for classes that don't have any images
|
||||
// This ensures all classes are available in the dataset view later
|
||||
const classesWithImages = new Set(
|
||||
Object.values(classifications).filter((c) => c && c !== "none"),
|
||||
|
||||
@@ -440,6 +440,7 @@ function CustomTimeSelector({
|
||||
<FaCalendarAlt />
|
||||
<div className="flex flex-wrap items-center">
|
||||
<Popover
|
||||
modal={false}
|
||||
open={startOpen}
|
||||
onOpenChange={(open) => {
|
||||
if (!open) {
|
||||
@@ -461,7 +462,10 @@ function CustomTimeSelector({
|
||||
{formattedStart}
|
||||
</Button>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent className="flex flex-col items-center">
|
||||
<PopoverContent
|
||||
disablePortal={isDesktop}
|
||||
className="flex flex-col items-center"
|
||||
>
|
||||
<TimezoneAwareCalendar
|
||||
timezone={config?.ui.timezone}
|
||||
selectedDay={new Date(startTime * 1000)}
|
||||
@@ -506,6 +510,7 @@ function CustomTimeSelector({
|
||||
</Popover>
|
||||
<FaArrowRight className="size-4 text-primary" />
|
||||
<Popover
|
||||
modal={false}
|
||||
open={endOpen}
|
||||
onOpenChange={(open) => {
|
||||
if (!open) {
|
||||
@@ -527,7 +532,10 @@ function CustomTimeSelector({
|
||||
{formattedEnd}
|
||||
</Button>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent className="flex flex-col items-center">
|
||||
<PopoverContent
|
||||
disablePortal={isDesktop}
|
||||
className="flex flex-col items-center"
|
||||
>
|
||||
<TimezoneAwareCalendar
|
||||
timezone={config?.ui.timezone}
|
||||
selectedDay={new Date(endTime * 1000)}
|
||||
@@ -545,7 +553,7 @@ function CustomTimeSelector({
|
||||
<SelectSeparator className="bg-secondary" />
|
||||
<input
|
||||
className="text-md mx-4 w-full border border-input bg-background p-1 text-secondary-foreground hover:bg-accent hover:text-accent-foreground dark:[color-scheme:dark]"
|
||||
id="startTime"
|
||||
id="endTime"
|
||||
type="time"
|
||||
value={endClock}
|
||||
step={isIOS ? "60" : "1"}
|
||||
|
||||
@@ -49,6 +49,29 @@ export default function DetailActionsMenu({
|
||||
search.data?.type === "audio" ? null : [`review/event/${search.id}`],
|
||||
);
|
||||
|
||||
// don't render menu at all if no options are available
|
||||
const hasSemanticSearchOption =
|
||||
config?.semantic_search.enabled &&
|
||||
setSimilarity !== undefined &&
|
||||
search.data?.type === "object";
|
||||
|
||||
const hasReviewItem = !!(reviewItem && reviewItem.id);
|
||||
|
||||
const hasAdminTriggerOption =
|
||||
isAdmin &&
|
||||
config?.semantic_search.enabled &&
|
||||
search.data?.type === "object";
|
||||
|
||||
if (
|
||||
!search.has_snapshot &&
|
||||
!search.has_clip &&
|
||||
!hasSemanticSearchOption &&
|
||||
!hasReviewItem &&
|
||||
!hasAdminTriggerOption
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<DropdownMenu open={isOpen} onOpenChange={setIsOpen}>
|
||||
<DropdownMenuTrigger>
|
||||
|
||||
@@ -178,6 +178,19 @@ export default function ObjectMaskEditPane({
|
||||
filteredMask.splice(index, 0, coordinates);
|
||||
}
|
||||
|
||||
// prevent duplicating global masks under specific object filters
|
||||
if (!globalMask) {
|
||||
const globalObjectMasksArray = Array.isArray(cameraConfig.objects.mask)
|
||||
? cameraConfig.objects.mask
|
||||
: cameraConfig.objects.mask
|
||||
? [cameraConfig.objects.mask]
|
||||
: [];
|
||||
|
||||
filteredMask = filteredMask.filter(
|
||||
(mask) => !globalObjectMasksArray.includes(mask),
|
||||
);
|
||||
}
|
||||
|
||||
queryString = filteredMask
|
||||
.map((pointsArray) => {
|
||||
const coordinates = flattenPoints(parseCoordinates(pointsArray)).join(
|
||||
|
||||
@@ -438,7 +438,7 @@ export default function Settings() {
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<div className="flex items-center justify-between border-b border-secondary p-3">
|
||||
<Heading as="h3" className="mb-0">
|
||||
<Heading as="h3" className="mb-0 min-h-9">
|
||||
{t("menu.settings", { ns: "common" })}
|
||||
</Heading>
|
||||
{CAMERA_SELECT_BUTTON_PAGES.includes(page) && (
|
||||
|
||||
@@ -866,6 +866,12 @@ function TrainGrid({
|
||||
};
|
||||
})
|
||||
.filter((data) => {
|
||||
// Ignore images that don't match the expected format (event-camera-timestamp-state-score.webp)
|
||||
// Expected format has 5 parts when split by "-", and score should be a valid number
|
||||
if (data.score === undefined || isNaN(data.score) || !data.name) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!trainFilter) {
|
||||
return true;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user