Compare commits

..
Author SHA1 Message Date
dependabot[bot]andGitHub 7962733c82 Bump axios from 1.13.6 to 1.15.2 in /web
Bumps [axios](https://github.com/axios/axios) from 1.13.6 to 1.15.2.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.13.6...v1.15.2)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.15.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-01 13:18:00 +00:00
611 changed files with 4857 additions and 40474 deletions
-1
View File
@@ -1 +0,0 @@
AGENTS.md
+401
View File
@@ -0,0 +1,401 @@
# GitHub Copilot Instructions for Frigate NVR
This document provides coding guidelines and best practices for contributing to Frigate NVR, a complete and local NVR designed for Home Assistant with AI object detection.
## Project Overview
Frigate NVR is a realtime object detection system for IP cameras that uses:
- **Backend**: Python 3.13+ with FastAPI, OpenCV, TensorFlow/ONNX
- **Frontend**: React with TypeScript, Vite, TailwindCSS
- **Architecture**: Multiprocessing design with ZMQ and MQTT communication
- **Focus**: Minimal resource usage with maximum performance
## Code Review Guidelines
When reviewing code, do NOT comment on:
- Missing imports - Static analysis tooling catches these
- Code formatting - Ruff (Python) and Prettier (TypeScript/React) handle formatting
- Minor style inconsistencies already enforced by linters
## Python Backend Standards
### Python Requirements
- **Compatibility**: Python 3.13+
- **Language Features**: Use modern Python features:
- Pattern matching
- Type hints (comprehensive typing preferred)
- f-strings (preferred over `%` or `.format()`)
- Dataclasses
- Async/await patterns
### Code Quality Standards
- **Formatting**: Ruff (configured in `pyproject.toml`)
- **Linting**: Ruff with rules defined in project config
- **Type Checking**: Use type hints consistently
- **Testing**: unittest framework - use `python3 -u -m unittest` to run tests
- **Language**: American English for all code, comments, and documentation
### Logging Standards
- **Logger Pattern**: Use module-level logger
```python
import logging
logger = logging.getLogger(__name__)
```
- **Format Guidelines**:
- No periods at end of log messages
- No sensitive data (keys, tokens, passwords)
- Use lazy logging: `logger.debug("Message with %s", variable)`
- **Log Levels**:
- `debug`: Development and troubleshooting information
- `info`: Important runtime events (startup, shutdown, state changes)
- `warning`: Recoverable issues that should be addressed
- `error`: Errors that affect functionality but don't crash the app
- `exception`: Use in except blocks to include traceback
### Error Handling
- **Exception Types**: Choose most specific exception available
- **Try/Catch Best Practices**:
- Only wrap code that can throw exceptions
- Keep try blocks minimal - process data after the try/except
- Avoid bare exceptions except in background tasks
Bad pattern:
```python
try:
data = await device.get_data() # Can throw
# ❌ Don't process data inside try block
processed = data.get("value", 0) * 100
result = processed
except DeviceError:
logger.error("Failed to get data")
```
Good pattern:
```python
try:
data = await device.get_data() # Can throw
except DeviceError:
logger.error("Failed to get data")
return
# ✅ Process data outside try block
processed = data.get("value", 0) * 100
result = processed
```
### Async Programming
- **External I/O**: All external I/O operations must be async
- **Best Practices**:
- Avoid sleeping in loops - use `asyncio.sleep()` not `time.sleep()`
- Avoid awaiting in loops - use `asyncio.gather()` instead
- No blocking calls in async functions
- Use `asyncio.create_task()` for background operations
- **Thread Safety**: Use proper synchronization for shared state
### Documentation Standards
- **Module Docstrings**: Concise descriptions at top of files
```python
"""Utilities for motion detection and analysis."""
```
- **Function Docstrings**: Required for public functions and methods
```python
async def process_frame(frame: ndarray, config: Config) -> Detection:
"""Process a video frame for object detection.
Args:
frame: The video frame as numpy array
config: Detection configuration
Returns:
Detection results with bounding boxes
"""
```
- **Comment Style**:
- Explain the "why" not just the "what"
- Keep lines under 88 characters when possible
- Use clear, descriptive comments
### File Organization
- **API Endpoints**: `frigate/api/` - FastAPI route handlers
- **Configuration**: `frigate/config/` - Configuration parsing and validation
- **Detectors**: `frigate/detectors/` - Object detection backends
- **Events**: `frigate/events/` - Event management and storage
- **Utilities**: `frigate/util/` - Shared utility functions
## Frontend (React/TypeScript) Standards
### Internationalization (i18n)
- **CRITICAL**: Never write user-facing strings directly in components
- **Always use react-i18next**: Import and use the `t()` function
```tsx
import { useTranslation } from "react-i18next";
function MyComponent() {
const { t } = useTranslation(["views/live"]);
return <div>{t("camera_not_found")}</div>;
}
```
- **Translation Files**: Add English strings to the appropriate json files in `web/public/locales/en`
- **Namespaces**: Organize translations by feature/view (e.g., `views/live`, `common`, `views/system`)
### Code Quality
- **Linting**: ESLint (see `web/.eslintrc.cjs`)
- **Formatting**: Prettier with Tailwind CSS plugin
- **Type Safety**: TypeScript strict mode enabled
- **Testing**: Vitest for unit tests
### Component Patterns
- **UI Components**: Use Radix UI primitives (in `web/src/components/ui/`)
- **Styling**: TailwindCSS with `cn()` utility for class merging
- **State Management**: React hooks (useState, useEffect, useCallback, useMemo)
- **Data Fetching**: Custom hooks with proper loading and error states
### ESLint Rules
Key rules enforced:
- `react-hooks/rules-of-hooks`: error
- `react-hooks/exhaustive-deps`: error
- `no-console`: error (use proper logging or remove)
- `@typescript-eslint/no-explicit-any`: warn (always use proper types instead of `any`)
- Unused variables must be prefixed with `_`
- Comma dangles required for multiline objects/arrays
### File Organization
- **Pages**: `web/src/pages/` - Route components
- **Views**: `web/src/views/` - Complex view components
- **Components**: `web/src/components/` - Reusable components
- **Hooks**: `web/src/hooks/` - Custom React hooks
- **API**: `web/src/api/` - API client functions
- **Types**: `web/src/types/` - TypeScript type definitions
## Testing Requirements
### Backend Testing
- **Framework**: Python unittest
- **Run Command**: `python3 -u -m unittest`
- **Location**: `frigate/test/`
- **Coverage**: Aim for comprehensive test coverage of core functionality
- **Pattern**: Use `TestCase` classes with descriptive test method names
```python
class TestMotionDetection(unittest.TestCase):
def test_detects_motion_above_threshold(self):
# Test implementation
```
### Test Best Practices
- Always have a way to test your work and confirm your changes
- Write tests for bug fixes to prevent regressions
- Test edge cases and error conditions
- Mock external dependencies (cameras, APIs, hardware)
- Use fixtures for test data
## Development Commands
### Python Backend
```bash
# Run all tests
python3 -u -m unittest
# Run specific test file
python3 -u -m unittest frigate.test.test_ffmpeg_presets
# Check formatting (Ruff)
ruff format --check frigate/
# Apply formatting
ruff format frigate/
# Run linter
ruff check frigate/
```
### Frontend (from web/ directory)
```bash
# Start dev server (AI agents should never run this directly unless asked)
npm run dev
# Build for production
npm run build
# Run linter
npm run lint
# Fix linting issues
npm run lint:fix
# Format code
npm run prettier:write
```
### Docker Development
AI agents should never run these commands directly unless instructed.
```bash
# Build local image
make local
# Build debug image
make debug
```
## Common Patterns
### API Endpoint Pattern
```python
from fastapi import APIRouter, Request
from frigate.api.defs.tags import Tags
router = APIRouter(tags=[Tags.Events])
@router.get("/events")
async def get_events(request: Request, limit: int = 100):
"""Retrieve events from the database."""
# Implementation
```
### Configuration Access
```python
# Access Frigate configuration
config: FrigateConfig = request.app.frigate_config
camera_config = config.cameras["front_door"]
```
### Database Queries
```python
from frigate.models import Event
# Use Peewee ORM for database access
events = (
Event.select()
.where(Event.camera == camera_name)
.order_by(Event.start_time.desc())
.limit(limit)
)
```
## Common Anti-Patterns to Avoid
### ❌ Avoid These
```python
# Blocking operations in async functions
data = requests.get(url) # ❌ Use async HTTP client
time.sleep(5) # ❌ Use asyncio.sleep()
# Hardcoded strings in React components
<div>Camera not found</div> # ❌ Use t("camera_not_found")
# Missing error handling
data = await api.get_data() # ❌ No exception handling
# Bare exceptions in regular code
try:
value = await sensor.read()
except Exception: # ❌ Too broad
logger.error("Failed")
# Returning exceptions in JSON responses
except ValueError as e:
return JSONResponse(
content={"success": False, "message": str(e)},
)
```
### ✅ Use These Instead
```python
# Async operations
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
data = await response.json()
await asyncio.sleep(5) # ✅ Non-blocking
# Translatable strings in React
const { t } = useTranslation();
<div>{t("camera_not_found")}</div> # ✅ Translatable
# Proper error handling
try:
data = await api.get_data()
except ApiException as err:
logger.error("API error: %s", err)
raise
# Specific exceptions
try:
value = await sensor.read()
except SensorException as err: # ✅ Specific
logger.exception("Failed to read sensor")
# Safe error responses
except ValueError:
logger.exception("Invalid parameters for API request")
return JSONResponse(
content={
"success": False,
"message": "Invalid request parameters",
},
)
```
## Project-Specific Conventions
### Configuration Files
- Main config: `config/config.yml`
### Directory Structure
- Backend code: `frigate/`
- Frontend code: `web/`
- Docker files: `docker/`
- Documentation: `docs/`
- Database migrations: `migrations/`
### Code Style Conformance
Always conform new and refactored code to the existing coding style in the project:
- Follow established patterns in similar files
- Match indentation and formatting of surrounding code
- Use consistent naming conventions (snake_case for Python, camelCase for TypeScript)
- Maintain the same level of verbosity in comments and docstrings
## Additional Resources
- Documentation: https://docs.frigate.video
- Main Repository: https://github.com/blakeblackshear/frigate
- Home Assistant Integration: https://github.com/blakeblackshear/frigate-hass-integration
-5
View File
@@ -22,8 +22,3 @@ core
!/web/**/*.ts
.idea/*
.ipynb_checkpoints
# Auto-generated Docker Compose Generator config files
docs/src/components/DockerComposeGenerator/config/devices.ts
docs/src/components/DockerComposeGenerator/config/hardware.ts
docs/src/components/DockerComposeGenerator/config/ports.ts
-439
View File
@@ -1,439 +0,0 @@
# Agent Instructions for Frigate NVR
This document provides coding guidelines and best practices for contributing to Frigate NVR, a complete and local NVR designed for Home Assistant with AI object detection.
## Project Overview
Frigate NVR is a realtime object detection system for IP cameras that uses:
- **Backend**: Python 3.13+ with FastAPI, OpenCV, TensorFlow/ONNX
- **Frontend**: React with TypeScript, Vite, TailwindCSS
- **Architecture**: Multiprocessing design with ZMQ and MQTT communication
- **Focus**: Minimal resource usage with maximum performance
## Code Review Guidelines
When reviewing code, do NOT comment on:
- Missing imports - Static analysis tooling catches these
- Code formatting - Ruff (Python) and Prettier (TypeScript/React) handle formatting
- Minor style inconsistencies already enforced by linters
## Python Backend Standards
### Python Requirements
- **Compatibility**: Python 3.13+
- **Language Features**: Use modern Python features:
- Pattern matching
- Type hints (comprehensive typing preferred)
- f-strings (preferred over `%` or `.format()`)
- Dataclasses
- Async/await patterns
### Code Quality Standards
- **Formatting**: Ruff (configured in `pyproject.toml`)
- **Linting**: Ruff with rules defined in project config
- **Type Checking**: Use type hints consistently
- **Testing**: unittest framework - use `python3 -u -m unittest` to run tests
- **Language**: American English for all code, comments, and documentation
### Logging Standards
- **Logger Pattern**: Use module-level logger
```python
import logging
logger = logging.getLogger(__name__)
```
- **Format Guidelines**:
- No periods at end of log messages
- No sensitive data (keys, tokens, passwords)
- Use lazy logging: `logger.debug("Message with %s", variable)`
- **Log Levels**:
- `debug`: Development and troubleshooting information
- `info`: Important runtime events (startup, shutdown, state changes)
- `warning`: Recoverable issues that should be addressed
- `error`: Errors that affect functionality but don't crash the app
- `exception`: Use in except blocks to include traceback
### Error Handling
- **Exception Types**: Choose most specific exception available
- **Try/Catch Best Practices**:
- Only wrap code that can throw exceptions
- Keep try blocks minimal - process data after the try/except
- Avoid bare exceptions except in background tasks
Bad pattern:
```python
try:
data = await device.get_data() # Can throw
# ❌ Don't process data inside try block
processed = data.get("value", 0) * 100
result = processed
except DeviceError:
logger.error("Failed to get data")
```
Good pattern:
```python
try:
data = await device.get_data() # Can throw
except DeviceError:
logger.error("Failed to get data")
return
# ✅ Process data outside try block
processed = data.get("value", 0) * 100
result = processed
```
### Async Programming
- **External I/O**: All external I/O operations must be async
- **Best Practices**:
- Avoid sleeping in loops - use `asyncio.sleep()` not `time.sleep()`
- Avoid awaiting in loops - use `asyncio.gather()` instead
- No blocking calls in async functions
- Use `asyncio.create_task()` for background operations
- **Thread Safety**: Use proper synchronization for shared state
### Documentation Standards
- **Module Docstrings**: Concise descriptions at top of files
```python
"""Utilities for motion detection and analysis."""
```
- **Function Docstrings**: Required for public functions and methods
```python
async def process_frame(frame: ndarray, config: Config) -> Detection:
"""Process a video frame for object detection.
Args:
frame: The video frame as numpy array
config: Detection configuration
Returns:
Detection results with bounding boxes
"""
```
- **Comment Style**:
- Explain the "why" not just the "what"
- Keep lines under 88 characters when possible
- Use clear, descriptive comments
### File Organization
- **API Endpoints**: `frigate/api/` - FastAPI route handlers
- **Configuration**: `frigate/config/` - Configuration parsing and validation
- **Detectors**: `frigate/detectors/` - Object detection backends
- **Events**: `frigate/events/` - Event management and storage
- **Utilities**: `frigate/util/` - Shared utility functions
## Frontend (React/TypeScript) Standards
### Internationalization (i18n)
- **CRITICAL**: Never write user-facing strings directly in components
- **Always use react-i18next**: Import and use the `t()` function
```tsx
import { useTranslation } from "react-i18next";
function MyComponent() {
const { t } = useTranslation(["views/live"]);
return <div>{t("camera_not_found")}</div>;
}
```
- **Translation Files**: Add English strings to the appropriate json files in `web/public/locales/en`
- **Namespaces**: Organize translations by feature/view (e.g., `views/live`, `common`, `views/system`)
### Code Quality
- **Linting**: ESLint (see `web/.eslintrc.cjs`)
- **Formatting**: Prettier with Tailwind CSS plugin
- **Type Safety**: TypeScript strict mode enabled
### Component Patterns
- **UI Components**: Use Radix UI primitives (in `web/src/components/ui/`)
- **Styling**: TailwindCSS with `cn()` utility for class merging
- **State Management**: React hooks (useState, useEffect, useCallback, useMemo)
- **Data Fetching**: Custom hooks with proper loading and error states
### ESLint Rules
Key rules enforced:
- `react-hooks/rules-of-hooks`: error
- `react-hooks/exhaustive-deps`: error
- `no-console`: error (use proper logging or remove)
- `@typescript-eslint/no-explicit-any`: warn (always use proper types instead of `any`)
- Unused variables must be prefixed with `_`
- Comma dangles required for multiline objects/arrays
### File Organization
- **Pages**: `web/src/pages/` - Route components
- **Views**: `web/src/views/` - Complex view components
- **Components**: `web/src/components/` - Reusable components
- **Hooks**: `web/src/hooks/` - Custom React hooks
- **API**: `web/src/api/` - API client functions
- **Types**: `web/src/types/` - TypeScript type definitions
## Testing Requirements
### Backend Testing
- **Framework**: Python unittest
- **Run Command**: `python3 -u -m unittest`
- **Location**: `frigate/test/`
- **Coverage**: Aim for comprehensive test coverage of core functionality
- **Pattern**: Use `TestCase` classes with descriptive test method names
```python
class TestMotionDetection(unittest.TestCase):
def test_detects_motion_above_threshold(self):
# Test implementation
```
### Test Best Practices
- Always have a way to test your work and confirm your changes
- Write tests for bug fixes to prevent regressions
- Test edge cases and error conditions
- Mock external dependencies (cameras, APIs, hardware)
- Use fixtures for test data
## Development Commands
### Python Backend
```bash
# Run all tests
python3 -u -m unittest
# Run specific test file
python3 -u -m unittest frigate.test.test_ffmpeg_presets
# Check formatting (Ruff)
ruff format --check frigate/
# Apply formatting
ruff format frigate/
# Run linter
ruff check frigate/
# Type check
python3 -u -m mypy --config-file frigate/mypy.ini frigate
```
### Frontend (from web/ directory)
```bash
# Start dev server (AI agents should never run this directly unless asked)
npm run dev
# Build for production
npm run build
# Run linter
npm run lint
# Fix linting issues
npm run lint:fix
# Format code
npm run prettier:write
# E2E: first-time setup
npm install
npx playwright install chromium
# E2E: build the app and run all tests
npm run e2e:build && npm run e2e
# E2E: interactive UI for debugging
npm run e2e:ui
# E2E: run a specific spec
npx playwright test --config e2e/playwright.config.ts e2e/specs/live.spec.ts
# E2E: filter by name, or run only desktop/mobile
npx playwright test --config e2e/playwright.config.ts --grep="severity tab"
npx playwright test --config e2e/playwright.config.ts --project=desktop
# E2E: regenerate mock data after backend model changes (from repo root)
PYTHONPATH=. python3 web/e2e/fixtures/mock-data/generate-mock-data.py
# Regenerate config translations from Pydantic models — outputs to
# web/public/locales/en/config/{global,cameras}.json. NEVER edit those
# JSON files by hand; change the Pydantic field title/description and
# re-run this script. (from repo root)
python3 generate_config_translations.py
# Extract i18n keys from source into the locale files after adding
# new t() calls. Use the :ci variant to verify the locale files are
# in sync with source (fails if extraction would change anything).
npm run i18n:extract
npm run i18n:extract:ci
```
### Docker Development
AI agents should never run these commands directly unless instructed.
```bash
# Build local image
make local
# Build debug image
make debug
```
## Common Patterns
### API Endpoint Pattern
```python
from fastapi import APIRouter, Request
from frigate.api.defs.tags import Tags
router = APIRouter(tags=[Tags.Events])
@router.get("/events")
async def get_events(request: Request, limit: int = 100):
"""Retrieve events from the database."""
# Implementation
```
### Configuration Access
```python
# Access Frigate configuration
config: FrigateConfig = request.app.frigate_config
camera_config = config.cameras["front_door"]
```
### Database Queries
```python
from frigate.models import Event
# Use Peewee ORM for database access
events = (
Event.select()
.where(Event.camera == camera_name)
.order_by(Event.start_time.desc())
.limit(limit)
)
```
## Common Anti-Patterns to Avoid
### ❌ Avoid These
```python
# Blocking operations in async functions
data = requests.get(url) # ❌ Use async HTTP client
time.sleep(5) # ❌ Use asyncio.sleep()
# Hardcoded strings in React components
<div>Camera not found</div> # ❌ Use t("camera_not_found")
# Missing error handling
data = await api.get_data() # ❌ No exception handling
# Bare exceptions in regular code
try:
value = await sensor.read()
except Exception: # ❌ Too broad
logger.error("Failed")
# Returning exceptions in JSON responses
except ValueError as e:
return JSONResponse(
content={"success": False, "message": str(e)},
)
```
### ✅ Use These Instead
```python
# Async operations
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
data = await response.json()
await asyncio.sleep(5) # ✅ Non-blocking
# Translatable strings in React
const { t } = useTranslation();
<div>{t("camera_not_found")}</div> # ✅ Translatable
# Proper error handling
try:
data = await api.get_data()
except ApiException as err:
logger.error("API error: %s", err)
raise
# Specific exceptions
try:
value = await sensor.read()
except SensorException as err: # ✅ Specific
logger.exception("Failed to read sensor")
# Safe error responses
except ValueError:
logger.exception("Invalid parameters for API request")
return JSONResponse(
content={
"success": False,
"message": "Invalid request parameters",
},
)
```
## WebSocket Broadcasts
Outbound WebSocket broadcasts go through a per-recipient classifier in `frigate/comms/ws.py` that enforces camera-level access. **The classifier is fail-closed: any topic it doesn't recognize is dropped for every client.** New outbound topics must be classified there or they'll silently disappear.
## Project-Specific Conventions
### Configuration Files
- Main config: `config/config.yml`
### Directory Structure
- Backend code: `frigate/`
- Frontend code: `web/`
- Docker files: `docker/`
- Documentation: `docs/`
- Database migrations: `migrations/`
### Code Style Conformance
Always conform new and refactored code to the existing coding style in the project:
- Follow established patterns in similar files
- Match indentation and formatting of surrounding code
- Use consistent naming conventions (snake_case for Python, camelCase for TypeScript)
- Maintain the same level of verbosity in comments and docstrings
## Additional Resources
- Documentation: https://docs.frigate.video
- Main Repository: https://github.com/blakeblackshear/frigate
- Home Assistant Integration: https://github.com/blakeblackshear/frigate-hass-integration
-1
View File
@@ -1 +0,0 @@
AGENTS.md
+2 -7
View File
@@ -10,14 +10,11 @@ If you've found a bug and want to fix it, go for it. Link to the relevant issue
### New features
A pull request is more than just code — it's a request for the maintainers to review, integrate, and support the change long-term. We're selective about what we take on, and prioritize changes that align with the project's direction and can be responsibly maintained in the long term.
**Large or highly-requested features** raise the bar even higher. Popularity signals demand, but it doesn't pre-approve any particular implementation. The bigger the change, the higher the long-term cost, and the more important it is that we're aligned on scope and approach before any code is written. A large PR that lands without prior discussion is unlikely to be merged as-is, no matter how well it's implemented.
Before writing code for a new feature:
Every new feature adds scope that the maintainers must test, maintain, and support long-term. Before writing code for a new feature:
1. **Check for existing discussion.** Search [feature requests](https://github.com/blakeblackshear/frigate/issues) and [discussions](https://github.com/blakeblackshear/frigate/discussions) to see if it's been proposed or discussed. Feature requests tagged with "planned" are on our radar — we plan to get to them, but we don't maintain a public roadmap or timeline. Check in with us first if you have interest in contributing to one.
2. **Start a discussion or feature request first.** This helps ensure your idea aligns with Frigate's direction before you invest time building it. Community interest in a feature request helps us gauge demand, though a great idea is a great idea even without a crowd behind it.
3. **Be open to "no".** We try to be thoughtful about what we take on, and sometimes that means saying no to good code if the feature isn't the right fit for the project. These calls are sometimes subjective, and we won't always get them right. We're happy to discuss and reconsider.
## AI usage policy
@@ -42,8 +39,6 @@ We're not trying to gatekeep how you write code. Use whatever tools make you pro
Some honest context: when we review a PR, we're not just evaluating whether the code works today. We're evaluating whether we can maintain it, debug it, and extend it long-term — often without the original author's involvement. Code that the author doesn't deeply understand is code that nobody understands, and that's a liability.
One more thing worth saying directly: most maintainers already have access to the same AI tools you do. A PR that's entirely AI-generated — where the author can't explain the design, debug issues independently, or engage substantively in design discussions — doesn't offer something we couldn't produce ourselves. What makes a contribution genuinely valuable is the human judgment and domain understanding behind it, as well as the engagement during review that shapes it into something we can confidently take on long-term.
## Pull request guidelines
### Before submitting
@@ -3,6 +3,7 @@
import json
import os
import sys
from pathlib import Path
from typing import Any
from ruamel.yaml import YAML
@@ -17,12 +18,37 @@ from frigate.const import (
)
from frigate.ffmpeg_presets import parse_preset_hardware_acceleration_encode
from frigate.util.config import find_config_file
from frigate.util.services import is_restricted_go2rtc_source
sys.path.remove("/opt/frigate")
yaml = YAML()
# Check if arbitrary exec sources are allowed (defaults to False for security)
allow_arbitrary_exec = None
if "GO2RTC_ALLOW_ARBITRARY_EXEC" in os.environ:
allow_arbitrary_exec = os.environ.get("GO2RTC_ALLOW_ARBITRARY_EXEC")
elif (
os.path.isdir("/run/secrets")
and os.access("/run/secrets", os.R_OK)
and "GO2RTC_ALLOW_ARBITRARY_EXEC" in os.listdir("/run/secrets")
):
allow_arbitrary_exec = (
Path(os.path.join("/run/secrets", "GO2RTC_ALLOW_ARBITRARY_EXEC"))
.read_text()
.strip()
)
# check for the add-on options file
elif os.path.isfile("/data/options.json"):
with open("/data/options.json") as f:
raw_options = f.read()
options = json.loads(raw_options)
allow_arbitrary_exec = options.get("go2rtc_allow_arbitrary_exec")
ALLOW_ARBITRARY_EXEC = allow_arbitrary_exec is not None and str(
allow_arbitrary_exec
).lower() in ("true", "1", "yes")
config_file = find_config_file()
try:
@@ -102,13 +128,18 @@ if LIBAVFORMAT_VERSION_MAJOR < 59:
go2rtc_config["ffmpeg"]["rtsp"] = rtsp_args
def is_restricted_source(stream_source: str) -> bool:
"""Check if a stream source is restricted (echo, expr, or exec)."""
return stream_source.strip().startswith(("echo:", "expr:", "exec:"))
for name in list(go2rtc_config.get("streams", {})):
stream = go2rtc_config["streams"][name]
if isinstance(stream, str):
try:
formatted_stream = substitute_frigate_vars(stream)
if is_restricted_go2rtc_source(formatted_stream):
if not ALLOW_ARBITRARY_EXEC and is_restricted_source(formatted_stream):
print(
f"[ERROR] Stream '{name}' uses a restricted source (echo/expr/exec) which is disabled by default for security. "
f"Set GO2RTC_ALLOW_ARBITRARY_EXEC=true to enable arbitrary exec sources."
@@ -127,7 +158,7 @@ for name in list(go2rtc_config.get("streams", {})):
for i, stream_item in enumerate(stream):
try:
formatted_stream = substitute_frigate_vars(stream_item)
if is_restricted_go2rtc_source(formatted_stream):
if not ALLOW_ARBITRARY_EXEC and is_restricted_source(formatted_stream):
print(
f"[ERROR] Stream '{name}' item {i + 1} uses a restricted source (echo/expr/exec) which is disabled by default for security. "
f"Set GO2RTC_ALLOW_ARBITRARY_EXEC=true to enable arbitrary exec sources."
+1 -5
View File
@@ -13,7 +13,7 @@ ARG ROCM
RUN apt update -qq && \
apt install -y wget gpg && \
wget -O rocm.deb https://repo.radeon.com/amdgpu-install/7.2.3/ubuntu/jammy/amdgpu-install_7.2.3.70203-1_all.deb && \
wget -O rocm.deb https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb && \
apt install -y ./rocm.deb && \
apt update && \
apt install -qq -y rocm
@@ -78,10 +78,6 @@ ENV MIGRAPHX_DISABLE_MIOPEN_FUSION=1
ENV MIGRAPHX_DISABLE_SCHEDULE_PASS=1
ENV MIGRAPHX_DISABLE_REDUCE_FUSION=1
ENV MIGRAPHX_ENABLE_HIPRTC_WORKAROUNDS=1
ENV MIOPEN_CUSTOM_CACHE_DIR=/config/model_cache/migraphx
ENV MIOPEN_USER_DB_PATH=/config/model_cache/migraphx
ENV AMD_COMGR_CACHE=1
ENV AMD_COMGR_CACHE_DIR=/config/model_cache/migraphx
COPY --from=rocm-dist / /
+1 -1
View File
@@ -1 +1 @@
onnxruntime-migraphx @ https://github.com/NickM-27/frigate-onnxruntime-rocm/releases/download/v7.2.3-1/onnxruntime_migraphx-1.24.4-cp311-cp311-linux_x86_64.whl
onnxruntime-migraphx @ https://github.com/NickM-27/frigate-onnxruntime-rocm/releases/download/v7.2.0/onnxruntime_migraphx-1.23.1-cp311-cp311-linux_x86_64.whl
+1 -1
View File
@@ -1,5 +1,5 @@
variable "ROCM" {
default = "7.2.3"
default = "7.2.0"
}
variable "HSA_OVERRIDE_GFX_VERSION" {
default = ""
+1 -1
View File
@@ -172,7 +172,7 @@ Custom models may also require different input tensor formats. The colorspace co
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and open the **Custom Model** tab to configure the model path, dimensions, and input format.
Navigate to <NavPath path="Settings > System > Detection model" /> to configure the model path, dimensions, and input format.
| Field | Description |
| --------------------------------------------- | ------------------------------------ |
+1 -1
View File
@@ -19,7 +19,7 @@ Face recognition requires a one-time internet connection to download detection a
### Face Detection
When running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/index.md#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
When running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
When running a default COCO model or another model that does not include `face` as a detectable label, face detection will run via CV2 using a lightweight DNN model that runs on the CPU. In this case, you should _not_ define `face` in your list of objects to track.
+2 -13
View File
@@ -201,7 +201,7 @@ Cloud Generative AI providers require an active internet connection to send imag
### Ollama Cloud
Ollama also supports [cloud models](https://ollama.com/cloud), where model inference is performed in the cloud. You can connect directly to Ollama Cloud by setting `base_url` to `https://ollama.com` and providing an API key. Alternatively, you can run Ollama locally and use a cloud model name so your local instance forwards requests to the cloud. For more details, see the Ollama cloud model [docs](https://docs.ollama.com/cloud).
Ollama also supports [cloud models](https://ollama.com/cloud), where your local Ollama instance handles requests from Frigate, but model inference is performed in the cloud. Set up Ollama locally, sign in with your Ollama account, and specify the cloud model name in your Frigate config. For more details, see the Ollama cloud model [docs](https://docs.ollama.com/cloud).
#### Configuration
@@ -210,8 +210,7 @@ Ollama also supports [cloud models](https://ollama.com/cloud), where model infer
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `ollama`
- Set **Base URL** to your local Ollama address (e.g., `http://localhost:11434`) or `https://ollama.com` for direct cloud inference
- Set **API key** if required by your endpoint (e.g., when using `https://ollama.com`)
- Set **Base URL** to your local Ollama address (e.g., `http://localhost:11434`)
- Set **Model** to the cloud model name
</TabItem>
@@ -224,16 +223,6 @@ genai:
model: cloud-model-name
```
or when using Ollama Cloud directly
```yaml
genai:
provider: ollama
base_url: https://ollama.com
model: cloud-model-name
api_key: your-api-key
```
</TabItem>
</ConfigTabs>
@@ -136,32 +136,90 @@ ffmpeg:
</TabItem>
</ConfigTabs>
### Configuring Intel GPU Stats
### Configuring Intel GPU Stats in Docker
Frigate reads Intel GPU utilization directly from the kernel's per-client DRM usage counters exposed at `/proc/<pid>/fdinfo/<fd>`. This requires:
Additional configuration is needed for the Docker container to be able to access the `intel_gpu_top` command for GPU stats. There are two options:
- Linux kernel **5.19 or newer** for the `i915` driver, or any release of the `xe` driver.
- Frigate running with permission to read other processes' fdinfo. Running as root inside the container (the default) satisfies this; non-root setups may need `CAP_SYS_PTRACE`.
1. Run the container as privileged.
2. Add the `CAP_PERFMON` capability (note: you might need to set the `perf_event_paranoid` low enough to allow access to the performance event system.)
No `intel_gpu_top` binary, `CAP_PERFMON`, privileged mode, or `perf_event_paranoid` tuning is required.
#### Run as privileged
#### Stats for SR-IOV or specific devices
This method works, but it gives more permissions to the container than are actually needed.
If the host has more than one Intel GPU (e.g. an iGPU plus a discrete GPU, or SR-IOV virtual functions), pin stats collection to a specific device by setting `intel_gpu_device` to either its PCI bus address or a DRM card/render-node path:
##### Docker Compose - Privileged
```yaml
services:
frigate:
...
image: ghcr.io/blakeblackshear/frigate:stable
# highlight-next-line
privileged: true
```
##### Docker Run CLI - Privileged
```bash {4}
docker run -d \
--name frigate \
...
--privileged \
ghcr.io/blakeblackshear/frigate:stable
```
#### CAP_PERFMON
Only recent versions of Docker support the `CAP_PERFMON` capability. You can test to see if yours supports it by running: `docker run --cap-add=CAP_PERFMON hello-world`
##### Docker Compose - CAP_PERFMON
```yaml {5,6}
services:
frigate:
...
image: ghcr.io/blakeblackshear/frigate:stable
cap_add:
- CAP_PERFMON
```
##### Docker Run CLI - CAP_PERFMON
```bash {4}
docker run -d \
--name frigate \
...
--cap-add=CAP_PERFMON \
ghcr.io/blakeblackshear/frigate:stable
```
#### perf_event_paranoid
_Note: This setting must be changed for the entire system._
For more information on the various values across different distributions, see https://askubuntu.com/questions/1400874/what-does-perf-paranoia-level-four-do.
Depending on your OS and kernel configuration, you may need to change the `/proc/sys/kernel/perf_event_paranoid` kernel tunable. You can test the change by running `sudo sh -c 'echo 2 >/proc/sys/kernel/perf_event_paranoid'` which will persist until a reboot. Make it permanent by running `sudo sh -c 'echo kernel.perf_event_paranoid=2 >> /etc/sysctl.d/local.conf'`
#### Stats for SR-IOV or other devices
When using virtualized GPUs via SR-IOV, you need to specify the device path to use to gather stats from `intel_gpu_top`. This example may work for some systems using SR-IOV:
```yaml
telemetry:
stats:
intel_gpu_device: "0000:00:02.0"
intel_gpu_device: "sriov"
```
For other virtualized GPUs, try specifying the direct path to the device instead:
```yaml
telemetry:
stats:
intel_gpu_device: "/dev/dri/card1"
intel_gpu_device: "drm:/dev/dri/card0"
```
When passing a device path, make sure the device is also passed through to the container.
If you are passing in a device path, make sure you've passed the device through to the container.
## AMD-based CPUs
+4 -4
View File
@@ -110,7 +110,7 @@ Here are some common starter configuration examples. These can be configured thr
1. Navigate to <NavPath path="Settings > System > MQTT" /> and configure the MQTT connection to your Home Assistant Mosquitto broker
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `Raspberry Pi (H.264)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
3. Navigate to <NavPath path="Settings > System > Detector hardware" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
4. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
5. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
6. Navigate to <NavPath path="Settings > Camera configuration > Management" /> and add your camera with the appropriate RTSP stream URL
@@ -189,7 +189,7 @@ cameras:
1. Navigate to <NavPath path="Settings > System > MQTT" /> and set **Enable MQTT** to off
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `VAAPI (Intel/AMD GPU)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
3. Navigate to <NavPath path="Settings > System > Detector hardware" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`
4. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
5. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
6. Navigate to <NavPath path="Settings > Camera configuration > Management" /> and add your camera with the appropriate RTSP stream URL
@@ -266,8 +266,8 @@ cameras:
1. Navigate to <NavPath path="Settings > System > MQTT" /> and configure the connection to your MQTT broker
2. Navigate to <NavPath path="Settings > Global configuration > FFmpeg" /> and set **Hardware acceleration arguments** to `VAAPI (Intel/AMD GPU)`
3. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `openvino` and **Device** `AUTO`
4. On the same page, in the **Custom Model** tab, configure the OpenVINO model path and settings
3. Navigate to <NavPath path="Settings > System > Detector hardware" /> and add a detector with **Type** `openvino` and **Device** `AUTO`
4. Navigate to <NavPath path="Settings > System > Detection model" /> and configure the OpenVINO model path and settings
5. Navigate to <NavPath path="Settings > Global configuration > Recording" /> and set **Enable recording** to on, **Motion retention > Retention days** to `7`, **Alert retention > Event retention > Retention days** to `30`, **Alert retention > Event retention > Retention mode** to `motion`, **Detection retention > Event retention > Retention days** to `30`, **Detection retention > Event retention > Retention mode** to `motion`
6. Navigate to <NavPath path="Settings > Global configuration > Snapshots" /> and set **Enable snapshots** to on, **Snapshot retention > Default retention** to `30`
7. Navigate to <NavPath path="Settings > Camera configuration > Management" /> and add your camera with the appropriate RTSP stream URL
+66 -176
View File
@@ -91,7 +91,7 @@ See [common Edge TPU troubleshooting steps](/troubleshooting/edgetpu) if the Edg
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`.
</TabItem>
<TabItem value="yaml">
@@ -111,7 +111,7 @@ detectors:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `usb:0` and `usb:1` as the device for each.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `usb:0` and `usb:1` as the device for each.
</TabItem>
<TabItem value="yaml">
@@ -136,7 +136,7 @@ _warning: may have [compatibility issues](https://github.com/blakeblackshear/fri
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then leave the device field empty.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then leave the device field empty.
</TabItem>
<TabItem value="yaml">
@@ -156,7 +156,7 @@ detectors:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `pci`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `pci`.
</TabItem>
<TabItem value="yaml">
@@ -176,7 +176,7 @@ detectors:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `pci:0` and `pci:1` as the device for each.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors, specifying `pci:0` and `pci:1` as the device for each.
</TabItem>
<TabItem value="yaml">
@@ -199,7 +199,7 @@ detectors:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors with different device types (e.g., `usb` and `pci`).
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add** to add multiple detectors with different device types (e.g., `usb` and `pci`).
</TabItem>
<TabItem value="yaml">
@@ -246,7 +246,7 @@ After placing the downloaded files for the tflite model and labels in your confi
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **EdgeTPU** from the detector type dropdown and click **Add**, then set device to `usb`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure the model settings:
| Field | Value |
| ---------------------------------------- | ----------------------------------------------------------------- |
@@ -309,7 +309,7 @@ Use this configuration for YOLO-based models. When no custom model path or URL i
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure the model settings:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -365,7 +365,7 @@ For SSD-based models, provide either a model path or URL to your compiled SSD mo
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then on the same page, in the **Custom Model** tab, configure the model settings:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure the model settings:
| Field | Value |
| --------------------------------------- | ------ |
@@ -410,7 +410,7 @@ The Hailo detector supports all YOLO models compiled for Hailo hardware that inc
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then on the same page, in the **Custom Model** tab, configure the model settings to match your custom model dimensions and format.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **Hailo-8/Hailo-8L** from the detector type dropdown and click **Add**, then set device to `PCIe`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure the model settings to match your custom model dimensions and format.
</TabItem>
<TabItem value="yaml">
@@ -465,7 +465,7 @@ When using many cameras one detector may not be enough to keep up. Multiple dete
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add** to add multiple detectors, each targeting `GPU` or `NPU`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add** to add multiple detectors, each targeting `GPU` or `NPU`.
</TabItem>
<TabItem value="yaml">
@@ -494,7 +494,7 @@ detectors:
| [YOLO-NAS](#yolo-nas) | ✅ | ✅ | |
| [MobileNet v2](#ssdlite-mobilenet-v2) | ✅ | ✅ | Fast and lightweight model, less accurate than larger models |
| [YOLOX](#yolox) | ✅ | ? | |
| [D-FINE / DEIMv2](#d-fine--deimv2) | ❌ | ❌ | |
| [D-FINE](#d-fine) | ❌ | ❌ | |
#### SSDLite MobileNet v2
@@ -508,7 +508,7 @@ Use the model configuration shown below when using the OpenVINO detector with th
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------ |
@@ -558,7 +558,7 @@ After placing the downloaded onnx model in your config folder, use the following
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -620,7 +620,7 @@ After placing the downloaded onnx model in your config folder, use the following
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU` (or `NPU`). Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -676,7 +676,7 @@ After placing the downloaded onnx model in your `config/model_cache` folder, use
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `GPU`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| --------------------------------------- | --------------------------------- |
@@ -710,13 +710,13 @@ model:
</details>
#### D-FINE / DEIMv2
#### D-FINE
[D-FINE](https://github.com/Peterande/D-FINE) and [DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) are DETR based models that share the same ONNX input/output format. The ONNX exported models are supported, but not included by default. See the models section for downloading [D-FINE](#downloading-d-fine-model) or [DEIMv2](#downloading-deimv2-model) for use in Frigate.
[D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.
:::warning
Currently D-FINE / DEIMv2 models only run on OpenVINO in CPU mode, GPUs currently fail to compile the model
Currently D-FINE models only run on OpenVINO in CPU mode, GPUs currently fail to compile the model
:::
@@ -728,7 +728,7 @@ After placing the downloaded onnx model in your config/model_cache folder, use t
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `CPU`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **OpenVINO** from the detector type dropdown and click **Add**, then set device to `CPU`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------- |
@@ -766,31 +766,6 @@ Note that the labelmap uses a subset of the complete COCO label set that has onl
</details>
<details>
<summary>DEIMv2 Setup & Config</summary>
After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:
```yaml
detectors:
ov:
type: openvino
device: CPU
model:
model_type: dfine
width: 640
height: 640
input_tensor: nchw
input_dtype: float
path: /config/model_cache/deimv2_hgnetv2_n.onnx
labelmap_path: /labelmap/coco-80.txt
```
Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.
</details>
## Apple Silicon detector
The NPU in Apple Silicon can't be accessed from within a container, so the [Apple Silicon detector client](https://github.com/frigate-nvr/apple-silicon-detector) must first be setup. It is recommended to use the Frigate docker image with `-standard-arm64` suffix, for example `ghcr.io/blakeblackshear/frigate:stable-standard-arm64`.
@@ -807,7 +782,7 @@ Using the detector config below will connect to the client:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`.
</TabItem>
<TabItem value="yaml">
@@ -841,7 +816,7 @@ When Frigate is started with the following config it will connect to the detecto
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ZMQ IPC** from the detector type dropdown and click **Add**, then set the endpoint to `tcp://host.docker.internal:5555`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -972,7 +947,7 @@ The AMD GPU kernel is known problematic especially when converting models to mxr
See [ONNX supported models](#supported-models) for supported models, there are some caveats:
- D-FINE / DEIMv2 models are not supported
- D-FINE models are not supported
- YOLO-NAS models are known to not run well on integrated GPUs
## ONNX
@@ -1002,7 +977,7 @@ When using many cameras one detector may not be enough to keep up. Multiple dete
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add** to add multiple detectors.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add** to add multiple detectors.
</TabItem>
<TabItem value="yaml">
@@ -1022,13 +997,13 @@ detectors:
### ONNX Supported Models
| Model | Nvidia GPU | AMD GPU | Notes |
| ------------------------------------ | ---------- | ------- | --------------------------------------------------- |
| [YOLOv9](#yolo-v3-v4-v7-v9-2) | ✅ | ✅ | Supports CUDA Graphs for optimal Nvidia performance |
| [RF-DETR](#rf-detr) | ✅ | ⚠️ | Supports CUDA Graphs for optimal Nvidia performance |
| [YOLO-NAS](#yolo-nas-1) | ⚠️ | ⚠️ | Not supported by CUDA Graphs |
| [YOLOX](#yolox-1) | ✅ | ✅ | Supports CUDA Graphs for optimal Nvidia performance |
| [D-FINE / DEIMv2](#d-fine--deimv2-1) | ⚠️ | ❌ | Not supported by CUDA Graphs |
| Model | Nvidia GPU | AMD GPU | Notes |
| ----------------------------- | ---------- | ------- | --------------------------------------------------- |
| [YOLOv9](#yolo-v3-v4-v7-v9-2) | ✅ | ✅ | Supports CUDA Graphs for optimal Nvidia performance |
| [RF-DETR](#rf-detr) | ✅ | | Supports CUDA Graphs for optimal Nvidia performance |
| [YOLO-NAS](#yolo-nas-1) | ⚠️ | ⚠️ | Not supported by CUDA Graphs |
| [YOLOX](#yolox-1) | ✅ | ✅ | Supports CUDA Graphs for optimal Nvidia performance |
| [D-FINE](#d-fine) | ⚠️ | ❌ | Not supported by CUDA Graphs |
There is no default model provided, the following formats are supported:
@@ -1050,7 +1025,7 @@ After placing the downloaded onnx model in your config folder, use the following
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -1109,7 +1084,7 @@ After placing the downloaded onnx model in your config folder, use the following
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -1158,7 +1133,7 @@ After placing the downloaded onnx model in your config folder, use the following
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------------- |
@@ -1207,7 +1182,7 @@ After placing the downloaded onnx model in your `config/model_cache` folder, use
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| --------------------------------------- | --------------------------------- |
@@ -1240,9 +1215,9 @@ model:
</details>
#### D-FINE / DEIMv2
#### D-FINE
[D-FINE](https://github.com/Peterande/D-FINE) and [DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) are DETR based models that share the same ONNX input/output format. The ONNX exported models are supported, but not included by default. See the models section for downloading [D-FINE](#downloading-d-fine-model) or [DEIMv2](#downloading-deimv2-model) for use in Frigate.
[D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.
<details>
<summary>D-FINE Setup & Config</summary>
@@ -1252,7 +1227,7 @@ After placing the downloaded onnx model in your `config/model_cache` folder, use
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **ONNX** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **ONNX** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------- |
@@ -1287,28 +1262,6 @@ model:
</details>
<details>
<summary>DEIMv2 Setup & Config</summary>
After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:
```yaml
detectors:
onnx:
type: onnx
model:
model_type: dfine
width: 640
height: 640
input_tensor: nchw
input_dtype: float
path: /config/model_cache/deimv2_hgnetv2_n.onnx
labelmap_path: /labelmap/coco-80.txt
```
</details>
Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.
## CPU Detector (not recommended)
@@ -1328,7 +1281,7 @@ A TensorFlow Lite model is provided in the container at `/cpu_model.tflite` and
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **CPU** from the detector type dropdown and click **Add**. Configure the number of threads and click **Add** again to add additional CPU detectors as needed (one per camera is recommended).
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **CPU** from the detector type dropdown and click **Add**. Configure the number of threads and click **Add** again to add additional CPU detectors as needed (one per camera is recommended).
</TabItem>
<TabItem value="yaml">
@@ -1364,7 +1317,7 @@ To integrate CodeProject.AI into Frigate, configure the detector as follows:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **DeepStack** from the detector type dropdown and click **Add**. Set the API URL to point to your CodeProject.AI server (e.g., `http://<your_codeproject_ai_server_ip>:<port>/v1/vision/detection`).
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **DeepStack** from the detector type dropdown and click **Add**. Set the API URL to point to your CodeProject.AI server (e.g., `http://<your_codeproject_ai_server_ip>:<port>/v1/vision/detection`).
</TabItem>
<TabItem value="yaml">
@@ -1403,7 +1356,7 @@ To configure the MemryX detector, use the following example configuration:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`.
</TabItem>
<TabItem value="yaml">
@@ -1423,7 +1376,7 @@ detectors:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add** to add multiple detectors, specifying `PCIe:0`, `PCIe:1`, `PCIe:2`, etc. as the device for each.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add** to add multiple detectors, specifying `PCIe:0`, `PCIe:1`, `PCIe:2`, etc. as the device for each.
</TabItem>
<TabItem value="yaml">
@@ -1452,7 +1405,7 @@ MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the
#### YOLO-NAS
The [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model included in this detector is downloaded from the [Models Section](#downloading-yolo-nas-model) and compiled to DFP with [mx_nc](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage).
The [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model included in this detector is downloaded from the [Models Section](#downloading-yolo-nas-model) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
**Note:** The default model for the MemryX detector is YOLO-NAS 320x320.
@@ -1467,7 +1420,7 @@ Below is the recommended configuration for using the **YOLO-NAS** (small) model
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -1506,7 +1459,7 @@ model:
#### YOLOv9
The YOLOv9s model included in this detector is downloaded from [the original GitHub](https://github.com/WongKinYiu/yolov9) like in the [Models Section](#yolov9-1) and compiled to DFP with [mx_nc](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage).
The YOLOv9s model included in this detector is downloaded from [the original GitHub](https://github.com/WongKinYiu/yolov9) like in the [Models Section](#yolov9-1) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
##### Configuration
@@ -1515,7 +1468,7 @@ Below is the recommended configuration for using the **YOLOv9** (small) model wi
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------- |
@@ -1562,7 +1515,7 @@ Below is the recommended configuration for using the **YOLOX** (small) model wit
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -1609,7 +1562,7 @@ Below is the recommended configuration for using the **SSDLite MobileNet v2** mo
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **MemryX** from the detector type dropdown and click **Add**, then set device to `PCIe:0`. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -1648,39 +1601,19 @@ model:
#### Using a Custom Model
To use your own custom model, first compile it into a [.dfp](https://developer.memryx.com/2p1/specs/files.html#dataflow-program) file, which is the format used by MemryX.
To use your own model:
#### Compile the Model
1. Package your compiled model into a `.zip` file.
Custom models must be compiled using **MemryX SDK 2.1**.
2. The `.zip` must contain the compiled `.dfp` file.
Before compiling your model, install the MemryX Neural Compiler tools from the
[Install Tools](https://developer.memryx.com/2p1/get_started/install_tools.html) page on the **host**.
3. Depending on the model, the compiler may also generate a cropped post-processing network. If present, it will be named with the suffix `_post.onnx`.
> **Note:** It is recommended to compile the model on the host machine, or on another separate machine, rather than inside the Frigate Docker container. Installing the compiler inside Docker may conflict with container packages. It is recommended to create a Python virtual environment and install the compiler there.
4. Bind-mount the `.zip` file into the container and specify its path using `model.path` in your config.
Once the SDK 2.1 environment is set up, follow the
[MemryX Compiler](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage) documentation to compile your model.
5. Update the `labelmap_path` to match your custom model's labels.
Example:
```bash
mx_nc -m yolonas.onnx -c 4 --autocrop -v --dfp_fname yolonas.dfp
```
For detailed instructions on compiling models, refer to the [MemryX Compiler](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage) docs and [Tutorials](https://developer.memryx.com/2p1/tutorials/tutorials.html).
#### Package the Compiled Model
1. Package your compiled model into a `.zip` file.
2. The `.zip` file must contain the compiled `.dfp` file.
3. Depending on the model, the compiler may also generate a cropped post-processing network. If present, it will be named with the suffix `_post.onnx`.
4. Bind-mount the `.zip` file into the container and specify its path using `model.path` in your config.
5. Update `labelmap_path` to match your custom model's labels.
For detailed instructions on compiling models, refer to the [MemryX Compiler](https://developer.memryx.com/tools/neural_compiler.html#usage) docs and [Tutorials](https://developer.memryx.com/tutorials/tutorials.html).
```yaml
# The detector automatically selects the default model if nothing is provided in the config.
@@ -1768,7 +1701,7 @@ Use the config below to work with generated TRT models:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **TensorRT** from the detector type dropdown and click **Add**, then set the device to `0` (the default GPU index). Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **TensorRT** from the detector type dropdown and click **Add**, then set the device to `0` (the default GPU index). Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ------------------------------------------------------------ |
@@ -1825,7 +1758,7 @@ Use the model configuration shown below when using the synaptics detector with t
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **Synaptics** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **Synaptics** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------- |
@@ -1879,7 +1812,7 @@ When using many cameras one detector may not be enough to keep up. Multiple dete
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **RKNN** from the detector type dropdown and click **Add** to add multiple detectors, each with `num_cores` set to `0` for automatic selection.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **RKNN** from the detector type dropdown and click **Add** to add multiple detectors, each with `num_cores` set to `0` for automatic selection.
</TabItem>
<TabItem value="yaml">
@@ -1921,7 +1854,7 @@ This `config.yml` shows all relevant options to configure the detector and expla
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **RKNN** from the detector type dropdown and click **Add**. Set `num_cores` to `0` for automatic selection (increase for better performance on multicore NPUs, e.g., set to `3` on rk3588).
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **RKNN** from the detector type dropdown and click **Add**. Set `num_cores` to `0` for automatic selection (increase for better performance on multicore NPUs, e.g., set to `3` on rk3588).
</TabItem>
<TabItem value="yaml">
@@ -1958,7 +1891,7 @@ The inference time was determined on a rk3588 with 3 NPU cores.
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ----------------------------------------------------------------------- |
@@ -2004,7 +1937,7 @@ The pre-trained YOLO-NAS weights from DeciAI are subject to their license and ca
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | -------------------------------------------------- |
@@ -2044,7 +1977,7 @@ model: # required
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ---------------------------------------------- |
@@ -2138,7 +2071,7 @@ Once completed, configure the detector as follows:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to your AI server (e.g., service name, container name, or `host:port`), the zoo to `degirum/public`, and provide your authentication token if needed.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to your AI server (e.g., service name, container name, or `host:port`), the zoo to `degirum/public`, and provide your authentication token if needed.
</TabItem>
<TabItem value="yaml">
@@ -2181,7 +2114,7 @@ It is also possible to eliminate the need for an AI server and run the hardware
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to `@local`, the zoo to `degirum/public`, and provide your authentication token.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to `@local`, the zoo to `degirum/public`, and provide your authentication token.
</TabItem>
<TabItem value="yaml">
@@ -2218,7 +2151,7 @@ If you do not possess whatever hardware you want to run, there's also the option
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to `@cloud`, the zoo to `degirum/public`, and provide your authentication token.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **DeGirum** from the detector type dropdown and click **Add**. Set the location to `@cloud`, the zoo to `degirum/public`, and provide your authentication token.
</TabItem>
<TabItem value="yaml">
@@ -2274,7 +2207,7 @@ Use the model configuration shown below when using the axengine detector with th
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and select **AXEngine NPU** from the detector type dropdown and click **Add**. Then on the same page, in the **Custom Model** tab, configure:
Navigate to <NavPath path="Settings > System > Detector hardware" /> and select **AXEngine NPU** from the detector type dropdown and click **Add**. Then navigate to <NavPath path="Settings > System > Detection model" /> and configure:
| Field | Value |
| ---------------------------------------- | ----------------------- |
@@ -2341,49 +2274,6 @@ COPY --from=build /dfine/output/dfine_${MODEL_SIZE}_obj2coco.onnx /dfine-${MODEL
EOF
```
### Downloading DEIMv2 Model
[DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) can be exported as ONNX by running the command below. Pretrained weights are available on Hugging Face for two backbone families:
- **HGNetv2** (smaller/faster): `atto`, `femto`, `pico`, `n`
- **DINOv3** (larger/more accurate): `s`, `m`, `l`, `x`
Set `BACKBONE` and `MODEL_SIZE` in the first line to match your desired variant. Hugging Face model names use uppercase (e.g. `HGNetv2_N`, `DINOv3_S`), while config files use lowercase (e.g. `hgnetv2_n`, `dinov3_s`).
```sh
docker build . --rm --build-arg BACKBONE=hgnetv2 --build-arg MODEL_SIZE=n --output . -f- <<'EOF'
FROM python:3.11-slim AS build
RUN apt-get update && apt-get install --no-install-recommends -y git libgl1 libglib2.0-0 && rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:0.8.0 /uv /bin/
WORKDIR /deimv2
RUN git clone https://github.com/Intellindust-AI-Lab/DEIMv2.git .
# Install CPU-only PyTorch first to avoid pulling CUDA variant
RUN uv pip install --no-cache --system torch torchvision --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache --system -r requirements.txt
RUN uv pip install --no-cache --system onnx safetensors huggingface_hub
RUN mkdir -p output
ARG BACKBONE
ARG MODEL_SIZE
# Download from Hugging Face and convert safetensors to pth
RUN python3 -c "\
from huggingface_hub import hf_hub_download; \
from safetensors.torch import load_file; \
import torch; \
backbone = '${BACKBONE}'.replace('hgnetv2','HGNetv2').replace('dinov3','DINOv3'); \
size = '${MODEL_SIZE}'.upper(); \
st = load_file(hf_hub_download('Intellindust/DEIMv2_' + backbone + '_' + size + '_COCO', 'model.safetensors')); \
torch.save({'model': st}, 'output/deimv2.pth')"
RUN sed -i "s/data = torch.rand(2/data = torch.rand(1/" tools/deployment/export_onnx.py
# HuggingFace safetensors omits frozen constants that the model constructor initializes
RUN sed -i "s/cfg.model.load_state_dict(state)/cfg.model.load_state_dict(state, strict=False)/" tools/deployment/export_onnx.py
RUN python3 tools/deployment/export_onnx.py -c configs/deimv2/deimv2_${BACKBONE}_${MODEL_SIZE}_coco.yml -r output/deimv2.pth
FROM scratch
ARG BACKBONE
ARG MODEL_SIZE
COPY --from=build /deimv2/output/deimv2.onnx /deimv2_${BACKBONE}_${MODEL_SIZE}.onnx
EOF
```
### Downloading RF-DETR Model
RF-DETR can be exported as ONNX by running the command below. You can copy and paste the whole thing to your terminal and execute, altering `MODEL_SIZE=Nano` in the first line to `Nano`, `Small`, or `Medium` size.
+1 -1
View File
@@ -195,7 +195,7 @@ Pre and post capture footage is included in the **recording timeline**, visible
## Will Frigate delete old recordings if my storage runs out?
If there is less than an hour left of storage, the oldest hour of recordings will be deleted and a message will be printed in the Frigate logs. This emergency cleanup deletes the oldest recordings first regardless of retention settings to reclaim space as quickly as possible.
As of Frigate 0.12 if there is less than an hour left of storage, the oldest 2 hours of recordings will be deleted.
## Configuring Recording Retention
+8 -3
View File
@@ -236,7 +236,7 @@ Enabling arbitrary exec sources allows execution of arbitrary commands through g
## Advanced Restream Configurations
The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#source-exec) source in go2rtc can be used for custom ffmpeg commands and other applications. An example is below:
The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below:
:::warning
@@ -244,11 +244,16 @@ The `exec:`, `echo:`, and `expr:` sources are disabled by default for security.
:::
NOTE: RTSP output will need to be passed with two curly braces `{{output}}`, whereas pipe output must be passed without curly braces.
:::warning
The `exec:`, `echo:`, and `expr:` sources are disabled by default for security. You must set `GO2RTC_ALLOW_ARBITRARY_EXEC=true` to use them. See [Security: Restricted Stream Sources](#security-restricted-stream-sources) for more information.
:::
NOTE: The output will need to be passed with two curly braces `{{output}}`
```yaml
go2rtc:
streams:
stream1: exec:ffmpeg -hide_banner -re -stream_loop -1 -i /media/BigBuckBunny.mp4 -c copy -rtsp_transport tcp -f rtsp {{output}}
stream2: exec:rpicam-vid -t 0 --libav-format h264 -o -
```
+4 -5
View File
@@ -223,11 +223,10 @@ Apple Silicon can not run within a container, so a ZMQ proxy is utilized to comm
With the [ROCm](../configuration/object_detectors.md#amdrocm-gpu-detector) detector Frigate can take advantage of many discrete AMD GPUs.
| Name | YOLOv9 Inference Time | YOLO-NAS Inference Time | RF-DETR Inference Time |
| -------------- | --------------------------- | ------------------------- | ---------------------- |
| AMD 780M | t-320: ~ 14 ms s-320: 20 ms | 320: ~ 25 ms 640: ~ 50 ms | |
| AMD 8700G | | 320: ~ 20 ms 640: ~ 40 ms | |
| AMD 9060XT 16G | t-320: ~ 4 ms s-320: 5 ms | 320: ~ 6 ms | Nano-320: ~ 90 ms |
| Name | YOLOv9 Inference Time | YOLO-NAS Inference Time |
| --------- | --------------------------- | ------------------------- |
| AMD 780M | t-320: ~ 14 ms s-320: 20 ms | 320: ~ 25 ms 640: ~ 50 ms |
| AMD 8700G | | 320: ~ 20 ms 640: ~ 40 ms |
## Community Supported Detectors
+2 -25
View File
@@ -4,15 +4,12 @@ title: Installation
---
import ShmCalculator from '@site/src/components/ShmCalculator'
import DockerComposeGenerator from '@site/src/components/DockerComposeGenerator'
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Frigate is a Docker container that can be run on any Docker host including as a [Home Assistant App](https://www.home-assistant.io/apps/). Note that the Home Assistant App is **not** the same thing as the integration. The [integration](/integrations/home-assistant) is required to integrate Frigate into Home Assistant, whether you are running Frigate as a standalone Docker container or as a Home Assistant App.
:::tip
If you already have Frigate installed as a Home Assistant App, check out the [getting started guide](../guides/getting_started.md#configuring-frigate) to configure Frigate.
If you already have Frigate installed as a Home Assistant App, check out the [getting started guide](../guides/getting_started#configuring-frigate) to configure Frigate.
:::
@@ -289,7 +286,7 @@ The MemryX MX3 Accelerator is available in the M.2 2280 form factor (like an NVM
#### Installation
To get started with MX3 hardware setup for your system, refer to the [Hardware Setup Guide](https://developer.memryx.com/2p1/get_started/install_hardware.html).
To get started with MX3 hardware setup for your system, refer to the [Hardware Setup Guide](https://developer.memryx.com/get_started/hardware_setup.html).
Then follow these steps for installing the correct driver/runtime configuration:
@@ -298,12 +295,6 @@ Then follow these steps for installing the correct driver/runtime configuration:
3. Run the script with `./user_installation.sh`
4. **Restart your computer** to complete driver installation.
:::warning
For manual setup, use **MemryX SDK 2.1** only. Other SDK versions are not supported for this setup. See the [SDK 2.1 documentation](https://developer.memryx.com/2p1/index.html)
:::
#### Setup
To set up Frigate, follow the default installation instructions, for example: `ghcr.io/blakeblackshear/frigate:stable`
@@ -477,16 +468,6 @@ Finally, configure [hardware object detection](/configuration/object_detectors#a
Running through Docker with Docker Compose is the recommended install method.
<Tabs>
<TabItem value="domestic" label="Docker Compose Generator" default>
Generate a Frigate Docker Compose configuration based on your hardware and requirements.
<DockerComposeGenerator/>
</TabItem>
<TabItem value="original" label="Example Docker Compose File">
```yaml
services:
frigate:
@@ -520,10 +501,6 @@ services:
environment:
FRIGATE_RTSP_PASSWORD: "password"
```
</TabItem>
</Tabs>
**Docker CLI**
If you can't use Docker Compose, you can run the container with something similar to this:
+4 -4
View File
@@ -192,7 +192,7 @@ cameras:
### Step 4: Configure detectors
By default, Frigate will use a single OpenVINO detector running on the CPU.
By default, Frigate will use a single CPU detector.
In many cases, the integrated graphics on Intel CPUs provides sufficient performance for typical Frigate setups. If you have an Intel processor, you can follow the configuration below.
@@ -204,8 +204,8 @@ You need to refer to **Configure hardware acceleration** above to enable the con
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `OpenVINO` and **Device** `GPU`
2. On the same page, in the **Custom Model** tab, configure the model settings for OpenVINO:
1. Navigate to <NavPath path="Settings > System > Detector hardware" /> and add a detector with **Type** `OpenVINO` and **Device** `GPU`
2. Navigate to <NavPath path="Settings > System > Detection model" /> and configure the model settings for OpenVINO:
| Field | Value |
| ---------------------------------------- | ------------------------------------------ |
@@ -273,7 +273,7 @@ services:
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > System > Detectors and model" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`.
Navigate to <NavPath path="Settings > System > Detector hardware" /> and add a detector with **Type** `EdgeTPU` and **Device** `usb`.
</TabItem>
<TabItem value="yaml">
+1 -3
View File
@@ -3,8 +3,6 @@ id: plus
title: Frigate+
---
import NavPath from "@site/src/components/NavPath";
For more information about how to use Frigate+ to improve your model, see the [Frigate+ docs](/plus/).
:::info
@@ -59,7 +57,7 @@ You can view all of your submitted images at [https://plus.frigate.video](https:
Once you have [requested your first model](../plus/first_model.md) and gotten your own model ID, it can be used with a special model path. No other information needs to be configured for Frigate+ models because it fetches the remaining config from Frigate+ automatically.
You can either choose the new model from the <NavPath path="Settings > System > Detectors and model" /> pane in the Frigate UI (the **Frigate+ Model** tab), or manually set the model at the root level in your config:
You can either choose the new model from the Frigate+ pane in the Settings page of the Frigate UI, or manually set the model at the root level in your config:
```yaml
detectors: ...
@@ -37,8 +37,6 @@ The per-clip variation is typically quite low and is mostly an artifact of keyfr
Debug Replay lets you re-run Frigate's detection pipeline against a section of recorded video without manually configuring a dummy camera. It automatically extracts the recording, creates a temporary camera with the same detection settings as the original, and loops the clip through the pipeline so you can observe detections in real time.
Debug Replay isn't intended to be a one-stop pane for all Frigate diagnostics or a comprehensive debugging environment for every Frigate feature. It merely makes it easier to spin up a "dummy camera" and perform some common adjustments in real-time. You'll still need to use the normal tools (logs, an MQTT client, etc) to debug your feature.
### When to use
- Reproducing a detection or tracking issue from a specific time range
+4 -11
View File
@@ -14,11 +14,9 @@
"@docusaurus/theme-mermaid": "^3.7.0",
"@inkeep/docusaurus": "^2.0.16",
"@mdx-js/react": "^3.1.0",
"@types/js-yaml": "^4.0.9",
"clsx": "^2.1.1",
"docusaurus-plugin-openapi-docs": "^4.5.1",
"docusaurus-theme-openapi-docs": "^4.5.1",
"js-yaml": "^4.1.1",
"prism-react-renderer": "^2.4.1",
"raw-loader": "^4.0.2",
"react": "^18.3.1",
@@ -5749,11 +5747,6 @@
"@types/istanbul-lib-report": "*"
}
},
"node_modules/@types/js-yaml": {
"version": "4.0.9",
"resolved": "https://mirrors.tencent.com/npm/@types/js-yaml/-/js-yaml-4.0.9.tgz",
"integrity": "sha512-k4MGaQl5TGo/iipqb2UDG2UwjXziSWkh0uysQelTlJpX1qGlpUZYm8PnO4DxG1qBomtJUdYJ6qR6xdIah10JLg=="
},
"node_modules/@types/json-schema": {
"version": "7.0.15",
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.15.tgz",
@@ -10971,9 +10964,9 @@
"license": "MIT"
},
"node_modules/fast-uri": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.2.tgz",
"integrity": "sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==",
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.0.tgz",
"integrity": "sha512-iPeeDKJSWf4IEOasVVrknXpaBV0IApz/gp7S2bb7Z4Lljbl2MGJRqInZiUrQwV16cpzw/D3S5j5Julj/gT52AA==",
"funding": [
{
"type": "github",
@@ -12890,7 +12883,7 @@
},
"node_modules/js-yaml": {
"version": "4.1.1",
"resolved": "https://mirrors.tencent.com/npm/js-yaml/-/js-yaml-4.1.1.tgz",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"license": "MIT",
"dependencies": {
+2 -5
View File
@@ -3,10 +3,9 @@
"version": "0.0.0",
"private": true,
"scripts": {
"build:config": "node scripts/build-config.mjs",
"docusaurus": "docusaurus",
"start": "npm run build:config && npm run regen-docs && docusaurus start --host 0.0.0.0",
"build": "npm run build:config && npm run regen-docs && docusaurus build",
"start": "npm run regen-docs && docusaurus start --host 0.0.0.0",
"build": "npm run regen-docs && docusaurus build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
@@ -24,11 +23,9 @@
"@docusaurus/theme-mermaid": "^3.7.0",
"@inkeep/docusaurus": "^2.0.16",
"@mdx-js/react": "^3.1.0",
"@types/js-yaml": "^4.0.9",
"clsx": "^2.1.1",
"docusaurus-plugin-openapi-docs": "^4.5.1",
"docusaurus-theme-openapi-docs": "^4.5.1",
"js-yaml": "^4.1.1",
"prism-react-renderer": "^2.4.1",
"raw-loader": "^4.0.2",
"react": "^18.3.1",
-64
View File
@@ -1,64 +0,0 @@
#!/usr/bin/env node
/**
* Build script: reads config.yaml and generates TypeScript files
* for the Docker Compose Generator.
*
* Usage: node scripts/build-config.mjs
*/
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import yaml from "js-yaml";
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const CONFIG_DIR = path.resolve(__dirname, "../src/components/DockerComposeGenerator/config");
const YAML_PATH = path.join(CONFIG_DIR, "config.yaml");
// Read & parse YAML
const raw = fs.readFileSync(YAML_PATH, "utf8");
const config = yaml.load(raw);
if (!config.devices || !config.hardware || !config.ports) {
console.error("config.yaml must contain 'devices', 'hardware', and 'ports' sections.");
process.exit(1);
}
/**
* Generate a .ts file from a section of the YAML config.
*/
function generateTsFile(sectionName, items, typeName, varName, mapVarName, yamlFilename) {
const jsonItems = JSON.stringify(items, null, 2);
// Indent JSON to fit inside the array literal
const indented = jsonItems
.split("\n")
.map((line, i) => (i === 0 ? line : " " + line))
.join("\n");
const content = `/**
* AUTO-GENERATED FILE — do not edit directly.
* Source: ${yamlFilename}
* To update, edit the YAML file and run: npm run build:config
*/
import type { ${typeName} } from "./types";
export const ${varName}: ${typeName}[] = ${indented};
/** Lookup map for quick access by ID */
export const ${mapVarName}: Map<string, ${typeName}> = new Map(${varName}.map((item) => [item.id, item]));
`;
const outPath = path.join(CONFIG_DIR, `${sectionName}.ts`);
fs.writeFileSync(outPath, content, "utf8");
console.log(` ✓ Generated ${sectionName}.ts (${items.length} items)`);
}
console.log("Building config from config.yaml...");
generateTsFile("devices", config.devices, "DeviceConfig", "devices", "deviceMap", "config.yaml");
generateTsFile("hardware", config.hardware, "HardwareOption", "hardwareOptions", "hardwareMap", "config.yaml");
generateTsFile("ports", config.ports, "PortConfig", "ports", "portMap", "config.yaml");
console.log("Done!");
+2 -2
View File
@@ -63,8 +63,8 @@ SYSTEM_NAV: dict[str, tuple[str, str]] = {
"environment_vars": ("System", "Environment variables"),
"telemetry": ("System", "Telemetry"),
"birdseye": ("System", "Birdseye"),
"detectors": ("System", "Detectors and model"),
"model": ("System", "Detectors and model"),
"detectors": ("System", "Detector hardware"),
"model": ("System", "Detection model"),
}
# All known top-level config section keys
@@ -1,108 +0,0 @@
import React from "react";
import Admonition from "@theme/Admonition";
import DeviceSelector from "./components/DeviceSelector";
import HardwareOptions from "./components/HardwareOptions";
import PortConfigSection from "./components/PortConfig";
import StoragePaths from "./components/StoragePaths";
import NvidiaGpuConfig from "./components/NvidiaGpuConfig";
import OtherOptions from "./components/OtherOptions";
import GeneratedOutput from "./components/GeneratedOutput";
import { useConfigGenerator } from "./hooks/useConfigGenerator";
import styles from "./styles.module.css";
/**
* Simple markdown-link-to-React renderer for help text.
* Only supports [text](url) syntax — no nested brackets.
*/
function renderHelpText(text: string): React.ReactNode {
const parts = text.split(/(\[[^\]]+\]\([^)]+\))/g);
return parts.map((part, i) => {
const match = part.match(/^\[([^\]]+)\]\(([^)]+)\)$/);
if (match) {
return (
<a key={i} href={match[2]}>
{match[1]}
</a>
);
}
return <React.Fragment key={i}>{part}</React.Fragment>;
});
}
export default function DockerComposeGenerator() {
const {
deviceId, device, hardwareEnabled,
portEnabled,
nvidiaGpuCount, nvidiaGpuDeviceId,
configPath, mediaPath, rtspPassword, timezone, shmSize,
shmSizeError, gpuDeviceIdError, configPathError, mediaPathError,
hasAnyHardware, generatedYaml,
selectDevice, toggleHardware, togglePort,
handleShmSizeChange, handleConfigPathChange, handleMediaPathChange,
handleNvidiaGpuCountChange, handleNvidiaGpuDeviceIdChange,
setRtspPassword, setTimezone, isHardwareDisabled,
} = useConfigGenerator();
return (
<div className={styles.generator}>
<div className={styles.card}>
<DeviceSelector selectedId={deviceId} onSelect={selectDevice} />
{device.helpText && (
<Admonition type={device.helpType || "info"}>
{renderHelpText(device.helpText)}
</Admonition>
)}
{device.needsNvidiaConfig && (
<NvidiaGpuConfig
gpuCount={nvidiaGpuCount}
gpuDeviceId={nvidiaGpuDeviceId}
gpuDeviceIdError={gpuDeviceIdError}
onGpuCountChange={handleNvidiaGpuCountChange}
onGpuDeviceIdChange={handleNvidiaGpuDeviceIdChange}
/>
)}
<HardwareOptions
deviceId={deviceId}
hardwareEnabled={hardwareEnabled}
onToggle={toggleHardware}
isDisabled={isHardwareDisabled}
/>
<StoragePaths
configPath={configPath}
mediaPath={mediaPath}
configPathError={configPathError}
mediaPathError={mediaPathError}
onConfigPathChange={handleConfigPathChange}
onMediaPathChange={handleMediaPathChange}
/>
<PortConfigSection
portEnabled={portEnabled}
onTogglePort={togglePort}
/>
<OtherOptions
rtspPassword={rtspPassword}
timezone={timezone}
shmSize={shmSize}
shmSizeError={shmSizeError}
onRtspPasswordChange={setRtspPassword}
onTimezoneChange={setTimezone}
onShmSizeChange={handleShmSizeChange}
/>
<GeneratedOutput
yaml={generatedYaml}
configPath={configPath}
mediaPath={mediaPath}
hasAnyHardware={hasAnyHardware}
deviceId={deviceId}
/>
</div>
</div>
);
}
@@ -1,147 +0,0 @@
import React from "react";
import { useColorMode } from "@docusaurus/theme-common";
import { devices } from "../config";
import type { DeviceConfig } from "../config";
import styles from "../styles.module.css";
interface Props {
selectedId: string;
onSelect: (id: string) => void;
}
/**
* Determine the icon type from the icon string:
* - Starts with "<svg" → inline SVG
* - Starts with "/" or "http" → image URL/path
* - Otherwise → emoji text
*/
function getIconType(icon: string): "svg" | "image" | "emoji" {
const trimmed = icon.trim();
if (trimmed.startsWith("<svg")) return "svg";
if (trimmed.startsWith("/") || trimmed.startsWith("http://") || trimmed.startsWith("https://")) return "image";
return "emoji";
}
/**
* Check if the style object contains background-* properties,
* indicating the image should be rendered as a CSS background-image
* rather than an <img> tag.
*/
function hasBackgroundProps(style: React.CSSProperties | undefined): boolean {
if (!style) return false;
return Object.keys(style).some((key) => {
const k = key.toLowerCase().replace(/-/g, "");
return k === "backgroundsize" || k === "backgroundposition" || k === "backgroundrepeat" || k === "backgroundimage";
});
}
/**
* Convert a style object to CSS custom properties (e.g. { width: "24px" } → { "--svg-width": "24px" })
* so they can be consumed by CSS rules targeting child elements like <svg>.
*/
function toCssVars(style: React.CSSProperties | undefined, prefix: string): React.CSSProperties {
if (!style) return {};
const vars: Record<string, string> = {};
for (const [key, value] of Object.entries(style)) {
const cssKey = key.replace(/([A-Z])/g, "-$1").toLowerCase();
vars[`--${prefix}-${cssKey}`] = value;
}
return vars as React.CSSProperties;
}
function DeviceIcon({ device }: { device: DeviceConfig }) {
const { isDarkTheme } = useColorMode();
const iconStr = isDarkTheme && device.iconDark ? device.iconDark : device.icon;
const iconStyle = (isDarkTheme && device.iconDarkStyle
? device.iconDarkStyle
: device.iconStyle) as React.CSSProperties | undefined;
const svgStyle = (isDarkTheme && device.svgDarkStyle
? device.svgDarkStyle
: device.svgStyle) as React.CSSProperties | undefined;
const iconType = getIconType(iconStr);
if (iconType === "svg") {
return (
<div
className={styles.deviceIconSvg}
style={{ ...iconStyle, ...toCssVars(svgStyle, "svg") }}
dangerouslySetInnerHTML={{ __html: iconStr }}
/>
);
}
if (iconType === "image") {
// When iconStyle contains background-* properties, render as background-image
// on the container div instead of an <img> tag, enabling background-size/position control.
if (hasBackgroundProps(iconStyle)) {
return (
<div
className={styles.deviceIconImage}
style={{
backgroundImage: `url(${iconStr})`,
backgroundRepeat: "no-repeat",
backgroundPosition: "center",
backgroundSize: "contain",
...iconStyle,
}}
/>
);
}
return (
<div className={styles.deviceIconImage}>
<img src={iconStr} alt={device.name} style={iconStyle} />
</div>
);
}
return (
<div className={styles.deviceIcon} style={iconStyle}>
{iconStr}
</div>
);
}
function DeviceCard({
device,
active,
onClick,
}: {
device: DeviceConfig;
active: boolean;
onClick: () => void;
}) {
return (
<div
className={`${styles.deviceCard} ${active ? styles.deviceCardActive : ""}`}
onClick={onClick}
role="button"
tabIndex={0}
onKeyDown={(e) => {
if (e.key === "Enter" || e.key === " ") onClick();
}}
>
<DeviceIcon device={device} />
<div className={styles.deviceName}>{device.name}</div>
<div className={styles.deviceDesc}>{device.description}</div>
</div>
);
}
export default function DeviceSelector({ selectedId, onSelect }: Props) {
return (
<div className={styles.formSection}>
<h4>Device Type</h4>
<div className={styles.deviceGrid}>
{devices.map((d) => (
<DeviceCard
key={d.id}
device={d}
active={selectedId === d.id}
onClick={() => onSelect(d.id)}
/>
))}
</div>
</div>
);
}
@@ -1,60 +0,0 @@
import React, { useState, useCallback } from "react";
import CodeBlock from "@theme/CodeBlock";
import Admonition from "@theme/Admonition";
import styles from "../styles.module.css";
interface Props {
yaml: string;
configPath: string;
mediaPath: string;
hasAnyHardware: boolean;
deviceId: string;
}
export default function GeneratedOutput({
yaml,
configPath,
mediaPath,
hasAnyHardware,
deviceId,
}: Props) {
const [copied, setCopied] = useState(false);
const handleCopy = useCallback(() => {
navigator.clipboard.writeText(yaml).then(() => {
setCopied(true);
setTimeout(() => setCopied(false), 2000);
});
}, [yaml]);
return (
<div className={styles.resultSection}>
<div className={styles.resultHeader}>
<h4>Generated Configuration</h4>
<button className="button button--primary button--sm" onClick={handleCopy}>
{copied ? "Copied!" : "Copy"}
</button>
</div>
{!configPath && (
<Admonition type="tip">
<p>You haven&apos;t specified a config file directory. You may want to modify the default path.</p>
</Admonition>
)}
{!mediaPath && (
<Admonition type="tip">
<p>You haven&apos;t specified a recording storage directory. You may want to modify the default path.</p>
</Admonition>
)}
{deviceId === "stable" && !hasAnyHardware && (
<Admonition type="warning">
<p>You haven&apos;t selected any hardware acceleration. Please check if you have supported hardware available.</p>
</Admonition>
)}
<CodeBlock language="yaml" title="docker-compose.yml">
{yaml}
</CodeBlock>
</div>
);
}
@@ -1,62 +0,0 @@
import React from "react";
import { hardwareOptions } from "../config";
import type { HardwareOption } from "../config";
import styles from "../styles.module.css";
interface Props {
deviceId: string;
hardwareEnabled: Record<string, boolean>;
onToggle: (hwId: string) => void;
isDisabled: (hwId: string) => boolean;
}
function renderDescription(text: string): React.ReactNode {
const parts = text.split(/(\[[^\]]+\]\([^)]+\))/g);
return parts.map((part, i) => {
const match = part.match(/^\[([^\]]+)\]\(([^)]+)\)$/);
if (match) {
return <a key={i} href={match[2]}>{match[1]}</a>;
}
return <React.Fragment key={i}>{part}</React.Fragment>;
});
}
function HardwareCheckbox({
hw, disabled, checked, onToggle,
}: {
hw: HardwareOption; disabled: boolean; checked: boolean; onToggle: () => void;
}) {
return (
<div className={styles.hardwareItem}>
<label className={`${styles.checkboxLabel} ${disabled ? styles.checkboxDisabled : ""}`}>
<input type="checkbox" checked={checked} onChange={onToggle} disabled={disabled} />
<span>{hw.label}</span>
</label>
{checked && hw.description && (
<div className={styles.hardwareDescription}>{renderDescription(hw.description)}</div>
)}
</div>
);
}
export default function HardwareOptions({ deviceId, hardwareEnabled, onToggle, isDisabled }: Props) {
return (
<div className={styles.formSection}>
<h4>Generic Hardware Devices</h4>
{deviceId !== "stable" && (
<p className={styles.helpText}>
Some options have been auto-configured based on your device type.
</p>
)}
<div className={styles.checkboxGrid}>
{hardwareOptions.map((hw) => {
const disabled = isDisabled(hw.id);
const checked = disabled ? false : !!hardwareEnabled[hw.id];
return (
<HardwareCheckbox key={hw.id} hw={hw} disabled={disabled} checked={checked} onToggle={() => onToggle(hw.id)} />
);
})}
</div>
</div>
);
}
@@ -1,64 +0,0 @@
import React from "react";
import styles from "../styles.module.css";
interface Props {
gpuCount: string;
gpuDeviceId: string;
gpuDeviceIdError: boolean;
onGpuCountChange: (value: string) => void;
onGpuDeviceIdChange: (value: string) => void;
}
export default function NvidiaGpuConfig({
gpuCount,
gpuDeviceId,
gpuDeviceIdError,
onGpuCountChange,
onGpuDeviceIdChange,
}: Props) {
const showDeviceId = gpuCount !== "";
return (
<div className={styles.nvidiaConfig}>
<div className={styles.formGroup}>
<label htmlFor="dcg-gpu-count" className={styles.label}>
GPU count:
</label>
<input
id="dcg-gpu-count"
type="text"
inputMode="numeric"
pattern="[0-9]*"
className={styles.input}
value={gpuCount}
placeholder="all"
onChange={(e) => onGpuCountChange(e.target.value.replace(/\D/g, ""))}
/>
</div>
{showDeviceId && (
<div className={styles.formGroup}>
<label htmlFor="dcg-gpu-device-id" className={styles.label}>
GPU device IDs (required, comma-separated):
</label>
<input
id="dcg-gpu-device-id"
type="text"
className={`${styles.input} ${gpuDeviceIdError ? styles.inputError : ""}`}
value={gpuDeviceId}
placeholder="0"
onChange={(e) => onGpuDeviceIdChange(e.target.value)}
/>
{gpuDeviceIdError ? (
<p className={styles.helpText}>
GPU device IDs are required when GPU count is a number
</p>
) : (
<p className={styles.helpText}>
Single GPU: 0 &nbsp;|&nbsp; Multiple GPUs: 0,1,2
</p>
)}
</div>
)}
</div>
);
}
@@ -1,122 +0,0 @@
import React, { useMemo } from "react";
import CodeInline from "@theme/CodeInline";
import styles from "../styles.module.css";
const AUTO_TIMEZONE_VALUE = "__auto__";
function getTimezoneList(): string[] {
if (typeof Intl !== "undefined") {
const intl = Intl as typeof Intl & {
supportedValuesOf?: (key: string) => string[];
};
const supported = intl.supportedValuesOf?.("timeZone");
if (supported && supported.length > 0) {
return [...supported].sort();
}
}
const fallback = Intl.DateTimeFormat().resolvedOptions().timeZone;
return fallback ? [fallback] : ["UTC"];
}
interface Props {
rtspPassword: string;
timezone: string;
shmSize: string;
shmSizeError: boolean;
onRtspPasswordChange: (value: string) => void;
onTimezoneChange: (value: string) => void;
onShmSizeChange: (value: string) => void;
}
export default function OtherOptions({
rtspPassword,
timezone,
shmSize,
shmSizeError,
onRtspPasswordChange,
onTimezoneChange,
onShmSizeChange,
}: Props) {
const timezones = useMemo(() => getTimezoneList(), []);
const systemTimezone =
Intl.DateTimeFormat().resolvedOptions().timeZone || "Etc/UTC";
const selectedValue = timezone || AUTO_TIMEZONE_VALUE;
return (
<div className={styles.formSection}>
<h4>Other Options</h4>
<div className={styles.formGrid}>
<div className={styles.formGroup}>
<label htmlFor="dcg-timezone" className={styles.label}>
Timezone:
</label>
<select
id="dcg-timezone"
className={`${styles.input} ${styles.select}`}
value={selectedValue}
onChange={(e) =>
onTimezoneChange(
e.target.value === AUTO_TIMEZONE_VALUE ? "" : e.target.value
)
}
>
<option value={AUTO_TIMEZONE_VALUE}>
Use browser timezone ({systemTimezone})
</option>
{timezones.map((tz) => (
<option key={tz} value={tz}>
{tz}
</option>
))}
</select>
</div>
<div className={styles.formGroup}>
<label htmlFor="dcg-shm-size" className={styles.label}>
Shared memory (SHM):
</label>
<input
id="dcg-shm-size"
type="text"
className={`${styles.input} ${shmSizeError ? styles.inputError : ""}`}
value={shmSize}
placeholder="512mb"
onChange={(e) => onShmSizeChange(e.target.value)}
/>
{shmSizeError ? (
<p className={styles.helpText}>
Invalid format. Use a number followed by a unit (e.g. 512mb, 1gb)
</p>
) : (
<p className={styles.helpText}>
See{" "}
<a href="/frigate/installation#calculating-required-shm-size">
calculating required SHM size
</a>{" "}
for the correct value.
</p>
)}
</div>
<div className={styles.formGroup}>
<label htmlFor="dcg-rtsp-password" className={styles.label}>
RTSP password:
</label>
<input
id="dcg-rtsp-password"
type="text"
className={styles.input}
value={rtspPassword}
placeholder="password"
onChange={(e) => onRtspPasswordChange(e.target.value)}
/>
<p className={styles.helpText}>
Optional. You can specify{" "}
<CodeInline>{"{FRIGATE_RTSP_PASSWORD}"}</CodeInline>{" "}
in the config file to reference camera stream passwords. This is NOT
the Frigate login password.
</p>
</div>
</div>
</div>
);
}
@@ -1,71 +0,0 @@
import React from "react";
import Admonition from "@theme/Admonition";
import { ports } from "../config";
import styles from "../styles.module.css";
interface Props {
portEnabled: Record<string, boolean>;
onTogglePort: (portId: string) => void;
}
function PortItem({
port,
enabled,
onToggle,
}: {
port: typeof ports[number];
enabled: boolean;
onToggle: () => void;
}) {
const showWarning = port.warningContent && (
port.warningWhen === "checked" ? enabled :
port.warningWhen === "unchecked" ? !enabled : enabled
);
return (
<div className={styles.hardwareItem}>
<label className={`${styles.checkboxLabel} ${port.locked ? styles.checkboxDisabled : ""}`}>
<input
type="checkbox"
checked={enabled}
onChange={onToggle}
disabled={port.locked}
/>
<span>
{port.locked && "🔒 "}
Port {port.host}
{port.protocol !== "tcp" && `/${port.protocol}`}
</span>
</label>
{port.description && (
<div className={styles.hardwareDescription}>{port.description}</div>
)}
{showWarning && (
<Admonition type={port.warningType || "warning"}>
{port.warningContent}
</Admonition>
)}
</div>
);
}
export default function PortConfigSection({
portEnabled,
onTogglePort,
}: Props) {
return (
<div className={styles.formSection}>
<h4>Port Configuration</h4>
<div className={styles.checkboxGrid}>
{ports.map((port) => (
<PortItem
key={port.id}
port={port}
enabled={!!portEnabled[port.id]}
onToggle={() => onTogglePort(port.id)}
/>
))}
</div>
</div>
);
}
@@ -1,66 +0,0 @@
import React from "react";
import styles from "../styles.module.css";
interface Props {
configPath: string;
mediaPath: string;
configPathError: boolean;
mediaPathError: boolean;
onConfigPathChange: (value: string) => void;
onMediaPathChange: (value: string) => void;
}
export default function StoragePaths({
configPath,
mediaPath,
configPathError,
mediaPathError,
onConfigPathChange,
onMediaPathChange,
}: Props) {
return (
<div className={styles.formSection}>
<h4>Storage Paths</h4>
<div className={styles.formGrid}>
<div className={styles.formGroup}>
<label htmlFor="dcg-config-path" className={styles.label}>
Config / DB / model cache directory (on your host):
</label>
<input
id="dcg-config-path"
type="text"
className={`${styles.input} ${configPathError ? styles.inputError : ""}`}
value={configPath}
placeholder="/path/to/your/config"
onChange={(e) => onConfigPathChange(e.target.value)}
/>
{configPathError && (
<p className={styles.helpText}>
Path contains invalid characters. Only letters, numbers,
underscores, hyphens, slashes, and dots are allowed.
</p>
)}
</div>
<div className={styles.formGroup}>
<label htmlFor="dcg-media-path" className={styles.label}>
Recording storage directory (on your host):
</label>
<input
id="dcg-media-path"
type="text"
className={`${styles.input} ${mediaPathError ? styles.inputError : ""}`}
value={mediaPath}
placeholder="/path/to/your/storage"
onChange={(e) => onMediaPathChange(e.target.value)}
/>
{mediaPathError && (
<p className={styles.helpText}>
Path contains invalid characters. Only letters, numbers,
underscores, hyphens, slashes, and dots are allowed.
</p>
)}
</div>
</div>
</div>
);
}
File diff suppressed because one or more lines are too long
@@ -1,12 +0,0 @@
export { devices, deviceMap } from "./devices";
export { hardwareOptions, hardwareMap } from "./hardware";
export { ports, portMap } from "./ports";
export type {
DeviceConfig,
DeviceMapping,
VolumeMapping,
HardwareOption,
PortConfig,
NvidiaDeployConfig,
} from "./types";
@@ -1,154 +0,0 @@
/**
* Type definitions for the Docker Compose Generator configuration.
* All device, hardware, and port options are declaratively defined
* so that adding a new device only requires editing config files.
*/
/** A single device mapping entry (e.g. /dev/dri:/dev/dri) */
export interface DeviceMapping {
/** Host device path */
host: string;
/** Container device path (defaults to host if omitted) */
container?: string;
/** Inline comment for this device line */
comment?: string;
}
/** A single volume mapping entry */
export interface VolumeMapping {
/** Host path */
host: string;
/** Container path */
container: string;
/** Whether the mount is read-only */
readOnly?: boolean;
/** Inline comment */
comment?: string;
}
/** NVIDIA deploy configuration for docker-compose */
export interface NvidiaDeployConfig {
/** "all" or a specific number */
count: string;
/** Specific GPU device IDs (when count is a number) */
deviceIds?: string[];
}
/** Full device type definition */
export interface DeviceConfig {
/** Unique identifier, e.g. "intel" */
id: string;
/** Display name, e.g. "Intel GPU" */
name: string;
/** Short description */
description: string;
/**
* Icon for the device card. Supports:
* - Emoji string (e.g. "🖥️")
* - Image URL or static path (e.g. "/img/intel.svg", "https://example.com/icon.png")
* - Inline SVG markup (e.g. "<svg>...</svg>")
*/
icon: string;
/**
* Additional CSS properties applied to the icon element.
* - For image-type icons: if any `background-*` property (e.g. `background-size`,
* `background-position`) is present, the image is rendered as a CSS `background-image`
* on the container div, enabling full background positioning control.
* Otherwise the image is rendered as an `<img>` tag and styles apply to it.
* - For emoji/SVG icons: styles apply to the container div.
*/
iconStyle?: Record<string, string>;
/**
* Additional CSS properties applied directly to the inner `<svg>` element
* when the icon is an inline SVG. Use this to override the default
* `width: 100%; height: 100%` or set `fill`, `transform`, etc.
* Ignored for emoji and image-type icons.
*/
svgStyle?: Record<string, string>;
/**
* Icon for dark mode. Same format as `icon`. When provided, this icon
* replaces `icon` when the user is in dark mode.
*/
iconDark?: string;
/** Additional CSS properties for the dark mode icon container */
iconDarkStyle?: Record<string, string>;
/**
* SVG-specific styles for dark mode. Same as `svgStyle` but applied
* when dark mode is active. Merged over `svgStyle` in dark mode.
*/
svgDarkStyle?: Record<string, string>;
/** Docker image tag, e.g. "stable" */
imageTag: string;
/**
* Image tag suffix appended to the base tag.
* e.g. "-standard-arm64" produces "stable-standard-arm64"
*/
imageTagSuffix?: string;
/** Hardware option IDs to auto-enable when this device is selected */
autoHardware: string[];
/** Help text shown as an admonition when this device is selected */
helpText?: string;
/** Admonition type for help text */
helpType?: "info" | "warning" | "danger";
/** Device mappings always added for this device type */
devices?: DeviceMapping[];
/** Volume mappings always added for this device type */
volumes?: VolumeMapping[];
/** Extra environment variables for this device type */
env?: Record<string, string>;
/** NVIDIA deploy config (only for tensorrt) */
nvidiaDeploy?: NvidiaDeployConfig;
/** Runtime setting, e.g. "nvidia" for Jetson */
runtime?: string;
/** Extra hosts entries, e.g. "host.docker.internal:host-gateway" */
extraHosts?: string[];
/** Security options, e.g. ["apparmor=unconfined"] */
securityOpt?: string[];
/** Whether this device type needs the NVIDIA GPU config UI */
needsNvidiaConfig?: boolean;
}
/** Generic hardware acceleration option definition */
export interface HardwareOption {
/** Unique identifier, e.g. "usbCoral" */
id: string;
/** Display label */
label: string;
/**
* Description shown below the checkbox when this option is enabled.
* Supports markdown link syntax: [text](url)
*/
description?: string;
/** Device IDs that disable this option */
disabledWhen?: string[];
/** Device mappings added when this option is enabled */
devices?: DeviceMapping[];
/** Volume mappings added when this option is enabled */
volumes?: VolumeMapping[];
/** Extra environment variables */
env?: Record<string, string>;
}
/** Port definition */
export interface PortConfig {
/** Unique identifier (also the default host port as string) */
id: string;
/** Host port number */
host: number;
/** Container port number */
container: number;
/** Protocol */
protocol?: "tcp" | "udp";
/** Description of the port's purpose */
description: string;
/** Whether enabled by default */
defaultEnabled: boolean;
/** Whether this port is locked (always enabled, cannot be toggled off) */
locked?: boolean;
/** Admonition type for the warning */
warningType?: "warning" | "danger";
/** Warning content (markdown) */
warningContent?: string;
/** When to show the warning: when the port is checked or unchecked */
warningWhen?: "checked" | "unchecked";
}
@@ -1,250 +0,0 @@
import type {
DeviceConfig,
DeviceMapping,
VolumeMapping,
} from "../config/types";
import { hardwareMap } from "../config";
// ---------------------------------------------------------------------------
// Input type
// ---------------------------------------------------------------------------
export interface GeneratorInput {
device: DeviceConfig;
selectedHardware: string[];
enabledPorts: string[];
configPath: string;
mediaPath: string;
rtspPassword?: string;
timezone: string;
shmSize: string;
nvidiaGpuCount?: string;
nvidiaGpuDeviceId?: string;
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function deviceLine(dm: DeviceMapping): string {
const host = dm.host;
const container = dm.container ?? dm.host;
const mapping = host === container ? host : `${host}:${container}`;
const comment = dm.comment ? ` # ${dm.comment}` : "";
return ` - ${mapping}${comment}`;
}
function volumeLine(vm: VolumeMapping): string {
const ro = vm.readOnly ? ":ro" : "";
const comment = vm.comment ? ` # ${vm.comment}` : "";
return ` - ${vm.host}:${vm.container}${ro}${comment}`;
}
// ---------------------------------------------------------------------------
// YAML builder — each section returns an array of lines
// ---------------------------------------------------------------------------
function buildImage(device: DeviceConfig): string[] {
const tag = device.imageTagSuffix
? `${device.imageTag}${device.imageTagSuffix}`
: device.imageTag;
return [` image: ghcr.io/blakeblackshear/frigate:${tag}`];
}
function buildDevices(
device: DeviceConfig,
hwDevices: DeviceMapping[]
): string[] {
const all: DeviceMapping[] = [
...(device.devices ?? []),
...hwDevices,
];
if (all.length === 0) return [];
return [
" devices:",
...all.map(deviceLine),
];
}
function buildVolumes(
device: DeviceConfig,
hwVolumes: VolumeMapping[],
configPath: string,
mediaPath: string
): string[] {
const all: VolumeMapping[] = [
...(device.volumes ?? []),
...hwVolumes,
];
return [
" volumes:",
" - /etc/localtime:/etc/localtime:ro # Sync host time",
` - ${configPath}:/config # Config file directory`,
` - ${mediaPath}:/media/frigate # Recording storage directory`,
" - type: tmpfs # 1GB in-memory filesystem for recording segment storage",
" target: /tmp/cache",
" tmpfs:",
" size: 1000000000",
...all.map(volumeLine),
];
}
function buildPorts(enabledPorts: string[]): string[] {
return [
" ports:",
...enabledPorts,
];
}
function buildEnvironment(
device: DeviceConfig,
hwEnv: Record<string, string>,
rtspPassword: string | undefined,
timezone: string
): string[] {
const allEnv: Record<string, string> = {
...hwEnv,
...(device.env ?? {}),
};
const lines: string[] = [" environment:"];
if (rtspPassword) {
lines.push(
` FRIGATE_RTSP_PASSWORD: "${rtspPassword}" # RTSP password — change to your own`
);
}
lines.push(` TZ: "${timezone}" # Timezone`);
for (const [key, value] of Object.entries(allEnv)) {
lines.push(` ${key}: "${value}"`);
}
return lines;
}
function buildDeploy(device: DeviceConfig, input: GeneratorInput): string[] {
if (device.id === "stable-tensorrt") {
const count = input.nvidiaGpuCount || "all";
const isAll = count === "all";
const deviceId = input.nvidiaGpuDeviceId?.trim();
if (isAll) {
return [
" deploy:",
" resources:",
" reservations:",
" devices:",
" - driver: nvidia",
" count: all # Use all GPUs",
" capabilities: [gpu]",
];
}
if (deviceId) {
const ids = deviceId
.split(",")
.map((s) => s.trim())
.filter(Boolean)
.map((s) => `'${s}'`)
.join(", ");
return [
" deploy:",
" resources:",
" reservations:",
" devices:",
" - driver: nvidia",
` device_ids: [${ids}] # GPU device IDs`,
` count: ${count} # GPU count`,
" capabilities: [gpu]",
];
}
return [
" deploy:",
" resources:",
" reservations:",
" devices:",
" - driver: nvidia",
` count: ${count} # GPU count`,
" capabilities: [gpu]",
];
}
return [];
}
function buildRuntime(device: DeviceConfig): string[] {
if (device.runtime) {
return [` runtime: ${device.runtime}`];
}
return [];
}
function buildExtraHosts(device: DeviceConfig): string[] {
if (!device.extraHosts?.length) return [];
return [
" extra_hosts:",
...device.extraHosts.map(
(h, i) =>
` - "${h}"${i === 0 ? " # Required to talk to the NPU detector" : ""}`
),
];
}
function buildSecurityOpt(device: DeviceConfig): string[] {
if (!device.securityOpt?.length) return [];
return [
" security_opt:",
...device.securityOpt.map((s) => ` - ${s}`),
];
}
// ---------------------------------------------------------------------------
// Public API
// ---------------------------------------------------------------------------
/**
* Generate a docker-compose YAML string from the given input.
* The output is pure YAML with inline comments (no Shiki annotations).
*/
export function generateDockerCompose(input: GeneratorInput): string {
const { device } = input;
// Collect hardware-level devices, volumes, and env
const hwDevices: DeviceMapping[] = [];
const hwVolumes: VolumeMapping[] = [];
const hwEnv: Record<string, string> = {};
for (const hwId of input.selectedHardware) {
const hw = hardwareMap.get(hwId);
if (!hw) continue;
// Skip GPU device mapping for tensorrt images (it uses deploy instead)
if (hw.id === "gpu" && device.imageTag === "stable-tensorrt") continue;
hwDevices.push(...(hw.devices ?? []));
hwVolumes.push(...(hw.volumes ?? []));
Object.assign(hwEnv, hw.env ?? {});
}
const lines: string[] = [
"services:",
" frigate:",
" container_name: frigate",
" privileged: true # This may not be necessary for all setups",
" restart: unless-stopped",
" stop_grace_period: 30s # Allow enough time to shut down the various services",
...buildImage(device),
` shm_size: "${input.shmSize || "512mb"}" # Update for your cameras based on SHM calculation`,
...buildRuntime(device),
...buildDeploy(device, input),
...buildExtraHosts(device),
...buildSecurityOpt(device),
...buildDevices(device, hwDevices),
...buildVolumes(device, hwVolumes, input.configPath, input.mediaPath),
...buildPorts(input.enabledPorts),
...buildEnvironment(device, hwEnv, input.rtspPassword, input.timezone),
];
return lines.join("\n");
}
@@ -1,195 +0,0 @@
import { useState, useCallback, useMemo } from "react";
import { deviceMap, hardwareMap, portMap } from "../config";
import { generateDockerCompose } from "../generator";
import type { GeneratorInput } from "../generator";
/**
* Main hook that holds all form state and generates the Docker Compose output.
* Configuration is loaded synchronously from build-time generated .ts files.
*/
export function useConfigGenerator() {
const [deviceId, setDeviceId] = useState("stable");
const [hardwareEnabled, setHardwareEnabled] = useState<Record<string, boolean>>(() => {
const defaultDevice = deviceMap.get("stable");
const initial: Record<string, boolean> = {};
if (defaultDevice) {
for (const hwId of defaultDevice.autoHardware) {
initial[hwId] = true;
}
}
return initial;
});
const [portEnabled, setPortEnabled] = useState<Record<string, boolean>>(() => {
const initial: Record<string, boolean> = {};
for (const p of portMap.values()) {
initial[p.id] = p.defaultEnabled;
}
return initial;
});
const [nvidiaGpuCount, setNvidiaGpuCount] = useState("");
const [nvidiaGpuDeviceId, setNvidiaGpuDeviceId] = useState("");
const [configPath, setConfigPath] = useState("");
const [mediaPath, setMediaPath] = useState("");
const [rtspPassword, setRtspPassword] = useState("");
const [timezone, setTimezone] = useState("");
const [shmSize, setShmSize] = useState("512mb");
const [shmSizeError, setShmSizeError] = useState(false);
const [gpuDeviceIdError, setGpuDeviceIdError] = useState(false);
const [configPathError, setConfigPathError] = useState(false);
const [mediaPathError, setMediaPathError] = useState(false);
const device = useMemo(() => deviceMap.get(deviceId)!, [deviceId]);
const selectDevice = useCallback((id: string) => {
const newDevice = deviceMap.get(id);
if (!newDevice) return;
setDeviceId(id);
setHardwareEnabled(() => {
const next: Record<string, boolean> = {};
for (const hwId of newDevice.autoHardware) {
next[hwId] = true;
}
return next;
});
setNvidiaGpuCount("");
setNvidiaGpuDeviceId("");
setGpuDeviceIdError(false);
}, []);
const toggleHardware = useCallback((hwId: string) => {
setHardwareEnabled((prev) => ({ ...prev, [hwId]: !prev[hwId] }));
}, []);
const togglePort = useCallback((portId: string) => {
const port = portMap.get(portId);
if (port?.locked) return;
setPortEnabled((prev) => ({ ...prev, [portId]: !prev[portId] }));
}, []);
const isHardwareDisabled = useCallback(
(hwId: string): boolean => {
const hw = hardwareMap.get(hwId);
if (!hw) return false;
return hw.disabledWhen?.includes(deviceId) ?? false;
},
[deviceId]
);
const validateShmSize = useCallback((value: string): boolean => {
if (!value) return true;
return /^\d+(\.\d+)?[bkmgBKMG]{1,2}$/.test(value);
}, []);
const validatePath = useCallback((value: string): boolean => {
if (!value) return true;
return /^[a-zA-Z0-9_\-/./]+$/.test(value);
}, []);
const handleShmSizeChange = useCallback(
(value: string) => {
const filtered = value.replace(/[^0-9.bkmgBKMG]/g, "");
const valid = validateShmSize(filtered);
setShmSize(filtered);
setShmSizeError(!valid && filtered !== "");
},
[validateShmSize]
);
const handleConfigPathChange = useCallback(
(value: string) => {
const filtered = value.replace(/[^a-zA-Z0-9_\-/./]/g, "");
const valid = validatePath(filtered);
setConfigPath(filtered);
setConfigPathError(!valid && filtered !== "");
},
[validatePath]
);
const handleMediaPathChange = useCallback(
(value: string) => {
const filtered = value.replace(/[^a-zA-Z0-9_\-/./]/g, "");
const valid = validatePath(filtered);
setMediaPath(filtered);
setMediaPathError(!valid && filtered !== "");
},
[validatePath]
);
const handleNvidiaGpuCountChange = useCallback((value: string) => {
// Only allow digits
setNvidiaGpuCount(value);
if (value === "") {
setNvidiaGpuDeviceId("");
setGpuDeviceIdError(false);
} else {
setGpuDeviceIdError(false);
}
}, []);
const handleNvidiaGpuDeviceIdChange = useCallback((value: string) => {
setNvidiaGpuDeviceId(value.trim());
setGpuDeviceIdError(false);
}, []);
const enabledPortLines = useMemo(() => {
const lines: string[] = [];
for (const [id, enabled] of Object.entries(portEnabled)) {
if (!enabled) continue;
const p = portMap.get(id);
if (!p) continue;
const proto = p.protocol && p.protocol !== "tcp" ? `/${p.protocol}` : "";
const comment = p.description ? ` # ${p.description}` : "";
lines.push(` - "${p.host}:${p.container}${proto}"${comment}`);
}
return lines;
}, [portEnabled]);
const selectedHardwareIds = useMemo(() => {
return Object.entries(hardwareEnabled)
.filter(([id, enabled]) => {
if (!enabled) return false;
const hw = hardwareMap.get(id);
if (!hw) return false;
if (hw.disabledWhen?.includes(deviceId)) return false;
return true;
})
.map(([id]) => id);
}, [hardwareEnabled, deviceId]);
const generatedYaml = useMemo(() => {
const input: GeneratorInput = {
device,
selectedHardware: selectedHardwareIds,
enabledPorts: enabledPortLines,
configPath: configPath || "/path/to/your/config",
mediaPath: mediaPath || "/path/to/your/storage",
rtspPassword,
timezone: timezone || Intl.DateTimeFormat().resolvedOptions().timeZone || "Etc/UTC",
shmSize: shmSize || "512mb",
nvidiaGpuCount,
nvidiaGpuDeviceId,
};
return generateDockerCompose(input);
}, [
device, selectedHardwareIds, enabledPortLines,
configPath, mediaPath, rtspPassword, timezone, shmSize,
nvidiaGpuCount, nvidiaGpuDeviceId,
]);
const hasAnyHardware = selectedHardwareIds.length > 0 || !!device?.devices?.length;
return {
deviceId, device, hardwareEnabled, portEnabled,
nvidiaGpuCount, nvidiaGpuDeviceId,
configPath, mediaPath, rtspPassword, timezone, shmSize,
shmSizeError, gpuDeviceIdError, configPathError, mediaPathError,
hasAnyHardware, generatedYaml,
selectDevice, toggleHardware, togglePort,
handleShmSizeChange, handleConfigPathChange, handleMediaPathChange,
handleNvidiaGpuCountChange, handleNvidiaGpuDeviceIdChange,
setRtspPassword, setTimezone, isHardwareDisabled,
};
}
@@ -1 +0,0 @@
export { default } from "./DockerComposeGenerator";
@@ -1,381 +0,0 @@
/* ===================================================================
Docker Compose Generator — styles
Uses Docusaurus / Infima CSS variables for theme compatibility.
=================================================================== */
.generator {
margin: 2rem 0;
}
.card {
background: var(--ifm-background-surface-color);
border: 1px solid var(--ifm-color-emphasis-400);
border-radius: 12px;
padding: 2rem;
box-shadow: var(--ifm-global-shadow-lw);
}
[data-theme="light"] .card {
background: var(--ifm-color-emphasis-100);
border: 1px solid var(--ifm-color-emphasis-300);
}
/* --- Form sections --- */
.formSection {
margin-bottom: 1.5rem;
padding-bottom: 1.5rem;
border-bottom: 1px solid var(--ifm-color-emphasis-400);
}
.formSection:last-child {
border-bottom: none;
margin-bottom: 0;
padding-bottom: 0;
}
.formSection h4 {
margin: 0 0 1rem 0;
color: var(--ifm-font-color-base);
font-size: 1.1rem;
font-weight: var(--ifm-font-weight-semibold);
}
/* --- Form controls --- */
.formGroup {
margin-bottom: 1rem;
}
.formGroup:last-child {
margin-bottom: 0;
}
.label {
display: block;
margin-bottom: 0.25rem;
color: var(--ifm-font-color-base);
font-weight: var(--ifm-font-weight-semibold);
font-size: 0.9rem;
}
.input {
width: 100%;
padding: 0.5rem 0.75rem;
border: 1px solid var(--ifm-color-emphasis-400);
border-radius: 6px;
background: var(--ifm-background-color);
color: var(--ifm-font-color-base);
font-size: 0.95rem;
transition: border-color 0.2s, box-shadow 0.2s;
}
[data-theme="light"] .input {
background: #fff;
border: 1px solid #d0d7de;
}
.input:focus {
outline: none;
border-color: var(--ifm-color-primary);
box-shadow: 0 0 0 3px var(--ifm-color-primary-lightest);
}
[data-theme="dark"] .input {
border-color: var(--ifm-color-emphasis-300);
}
.inputError {
border-color: #e74c3c;
animation: shake 0.3s ease-in-out;
}
@keyframes shake {
0%,
100% {
transform: translateX(0);
}
25% {
transform: translateX(-5px);
}
75% {
transform: translateX(5px);
}
}
/* --- Select dropdown --- */
.select {
cursor: pointer;
appearance: none;
-moz-appearance: none;
-webkit-appearance: none;
background: var(--ifm-background-color)
url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='12' viewBox='0 0 12 12'%3E%3Cpath fill='%23666' d='M6 8L1 3h10z'/%3E%3C/svg%3E")
no-repeat right 0.75rem center / 12px 12px;
padding-right: 2rem;
}
[data-theme="light"] .select {
background: #fff
url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='12' viewBox='0 0 12 12'%3E%3Cpath fill='%23555' d='M6 8L1 3h10z'/%3E%3C/svg%3E")
no-repeat right 0.75rem center / 12px 12px;
}
.helpText {
margin: 0.5rem 0 0 0;
font-size: 0.85rem;
color: var(--ifm-font-color-secondary);
line-height: 1.5;
}
.helpText a {
color: var(--ifm-color-primary);
}
/* --- Device grid --- */
.deviceGrid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(130px, 1fr));
gap: 0.75rem;
margin-top: 0.5rem;
}
.deviceCard {
padding: 0.75rem;
border: 2px solid var(--ifm-color-emphasis-400);
border-radius: 12px;
cursor: pointer;
transition: all 0.2s;
text-align: center;
background: var(--ifm-background-color);
display: flex;
flex-direction: column;
align-items: center;
}
[data-theme="light"] .deviceCard {
border: 2px solid #d0d7de;
background: #fff;
}
.deviceCard:hover {
border-color: var(--ifm-color-primary);
background: var(--ifm-color-emphasis-100);
transform: translateY(-2px);
}
.deviceCardActive {
border-color: var(--ifm-color-primary);
background: var(--ifm-color-primary-lightest);
box-shadow: 0 0 0 1px var(--ifm-color-primary);
}
[data-theme="light"] .deviceCardActive {
background: color-mix(in srgb, var(--ifm-color-primary) 12%, #fff);
}
[data-theme="dark"] .deviceCardActive {
background: color-mix(in srgb, var(--ifm-color-primary) 25%, #1b1b1b);
}
[data-theme="dark"] .deviceCardActive .deviceName {
color: var(--ifm-color-primary-light);
}
[data-theme="dark"] .deviceCardActive .deviceDesc {
color: var(--ifm-color-primary-light);
opacity: 0.85;
}
.deviceIcon {
font-size: 2rem;
margin-bottom: 0.25rem;
height: 40px;
width: 50px;
display: flex;
align-items: center;
justify-content: center;
}
.deviceIconSvg {
margin-bottom: 0.25rem;
height: 40px;
width: 50px;
display: flex;
align-items: center;
justify-content: center;
overflow: visible;
/* Allow iconStyle width/height to override */
flex-shrink: 0;
}
.deviceIconSvg svg {
width: var(--svg-width, 100%);
height: var(--svg-height, 100%);
fill: var(--svg-fill, currentColor);
transform: var(--svg-transform, none);
}
.deviceIconImage {
margin-bottom: 0.25rem;
height: 40px;
width: 50px;
display: flex;
align-items: center;
justify-content: center;
}
.deviceIconImage img {
max-width: 100%;
max-height: 100%;
object-fit: contain;
}
.deviceName {
font-weight: var(--ifm-font-weight-semibold);
color: var(--ifm-font-color-base);
margin-bottom: 0.15rem;
font-size: 0.9rem;
}
.deviceDesc {
font-size: 0.75rem;
color: var(--ifm-font-color-secondary);
line-height: 1.3;
}
/* --- Checkbox grid --- */
.checkboxGrid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 0.5rem;
}
@media (max-width: 576px) {
.checkboxGrid {
grid-template-columns: 1fr;
}
}
.hardwareItem {
margin-bottom: 0;
}
.hardwareDescription {
margin: 0.15rem 0 0.4rem 1.6rem;
font-size: 0.8rem;
color: var(--ifm-font-color-secondary);
line-height: 1.5;
}
.hardwareDescription a {
color: var(--ifm-color-primary);
text-decoration: underline;
text-underline-offset: 2px;
}
.checkboxLabel {
display: flex;
align-items: center;
gap: 0.5rem;
cursor: pointer;
padding: 0.4rem 0.5rem;
border-radius: 6px;
transition: background-color 0.2s;
font-size: 0.9rem;
}
.checkboxLabel:hover {
background: var(--ifm-color-emphasis-100);
}
.checkboxLabel input[type="checkbox"] {
width: 1.1rem;
height: 1.1rem;
cursor: pointer;
flex-shrink: 0;
}
.checkboxLabel span {
color: var(--ifm-font-color-base);
}
.checkboxDisabled {
cursor: not-allowed;
}
.checkboxDisabled:hover {
background: transparent;
}
.checkboxDisabled input[type="checkbox"] {
cursor: not-allowed;
opacity: 0.5;
}
/* --- Form grid (side-by-side) --- */
.formGrid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 1rem;
}
@media (max-width: 576px) {
.formGrid {
grid-template-columns: 1fr;
}
}
.formGrid .formGroup {
margin-bottom: 0;
}
/* --- Port section --- */
.portSection {
margin-bottom: 0.75rem;
}
.warningBadge {
margin-left: auto;
color: #e67e22;
font-size: 0.85rem;
}
/* --- NVIDIA config --- */
.nvidiaConfig {
margin-top: 1rem;
margin-bottom: 1.5rem;
padding: 1rem;
background: var(--ifm-background-color);
border-radius: 8px;
border-left: 3px solid var(--ifm-color-primary);
}
[data-theme="light"] .nvidiaConfig {
background: #f6f8fa;
border-left: 3px solid var(--ifm-color-primary);
}
/* --- Result section --- */
.resultSection {
margin-top: 2rem;
}
.resultHeader {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 1rem;
}
.resultHeader h4 {
margin: 0;
color: var(--ifm-font-color-base);
}
+2 -13
View File
@@ -5997,10 +5997,7 @@ paths:
tags:
- App
summary: Start debug replay
description:
Start a debug replay session from camera recordings. Returns
immediately while clip generation runs as a background job; subscribe
to the 'debug_replay' job_state WS topic to track progress.
description: Start a debug replay session from camera recordings.
operationId: start_debug_replay_debug_replay_start_post
requestBody:
required: true
@@ -6009,16 +6006,12 @@ paths:
schema:
$ref: "#/components/schemas/DebugReplayStartBody"
responses:
"202":
"200":
description: Successful Response
content:
application/json:
schema:
$ref: "#/components/schemas/DebugReplayStartResponse"
"400":
description: Invalid camera, time range, or no recordings
"409":
description: A replay session is already active
"422":
description: Validation Error
content:
@@ -6279,14 +6272,10 @@ components:
replay_camera:
type: string
title: Replay Camera
job_id:
type: string
title: Job Id
type: object
required:
- success
- replay_camera
- job_id
title: DebugReplayStartResponse
description: Response for starting a debug replay session.
DebugReplayStatusResponse:
+9 -111
View File
@@ -96,46 +96,11 @@ def version():
@router.get("/stats", dependencies=[Depends(allow_any_authenticated())])
def stats(
request: Request,
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
stats_data = request.app.stats_emitter.get_latest_stats()
# Admins see the full snapshot
if request.headers.get("remote-role") == "admin":
return JSONResponse(content=stats_data)
allowed_set = set(allowed_cameras)
# Shallow-copy so we don't mutate the cached stats history entry.
filtered = {**stats_data}
cameras = stats_data.get("cameras")
if cameras is not None:
filtered["cameras"] = {
name: data for name, data in cameras.items() if name in allowed_set
}
bandwidth = stats_data.get("bandwidth_usages")
if bandwidth is not None:
filtered["bandwidth_usages"] = {
name: data for name, data in bandwidth.items() if name in allowed_set
}
# cmdline can leak camera URLs/paths; strip but keep cpu/mem so
# client-side problem heuristics still work.
cpu_usages = stats_data.get("cpu_usages")
if cpu_usages is not None:
filtered["cpu_usages"] = {
pid: {k: v for k, v in usage.items() if k != "cmdline"}
for pid, usage in cpu_usages.items()
}
return JSONResponse(content=filtered)
def stats(request: Request):
return JSONResponse(content=request.app.stats_emitter.get_latest_stats())
@router.get("/stats/history", dependencies=[Depends(require_role(["admin"]))])
@router.get("/stats/history", dependencies=[Depends(allow_any_authenticated())])
def stats_history(request: Request, keys: str = None):
if keys:
keys = keys.split(",")
@@ -181,13 +146,8 @@ def config(request: Request):
for name, detector in config_obj.detectors.items()
}
# remove environment_vars for non-admin users
if request.headers.get("remote-role") != "admin":
config.pop("environment_vars", None)
# remove mqtt credentials
# remove the mqtt password
config["mqtt"].pop("password", None)
config["mqtt"].pop("user", None)
# remove the proxy secret
config["proxy"].pop("auth_secret", None)
@@ -534,40 +494,6 @@ def config_save(save_option: str, body: Any = Body(media_type="text/plain")):
)
def _restore_masked_camera_paths(config_data: dict, config: FrigateConfig) -> None:
"""Substitute incoming `*:*` masked credentials with the in-memory ones.
The /config response masks ffmpeg input credentials, so the settings UI
sends the masked path back when sibling fields (e.g. hwaccel_args) are
edited. Without this we'd write `rtsp://*:*@host` into YAML and lose
the real credentials. Mutates `config_data` in place.
"""
cameras = config_data.get("cameras")
if not isinstance(cameras, dict):
return
for camera_name, camera_data in cameras.items():
if not isinstance(camera_data, dict):
continue
inputs = camera_data.get("ffmpeg", {}).get("inputs")
if not isinstance(inputs, list):
continue
existing = config.cameras.get(camera_name)
if existing is None:
continue
existing_paths = [inp.path for inp in existing.ffmpeg.inputs]
for index, input_obj in enumerate(inputs):
if not isinstance(input_obj, dict):
continue
path = input_obj.get("path")
if not isinstance(path, str):
continue
if ("://*:*@" in path or "user=*&password=*" in path) and index < len(
existing_paths
):
input_obj["path"] = existing_paths[index]
def _config_set_in_memory(request: Request, body: AppConfigSetBody) -> JSONResponse:
"""Apply config changes in-memory only, without writing to YAML.
@@ -578,7 +504,6 @@ def _config_set_in_memory(request: Request, body: AppConfigSetBody) -> JSONRespo
try:
updates = {}
if body.config_data:
_restore_masked_camera_paths(body.config_data, request.app.frigate_config)
updates = flatten_config_data(body.config_data)
updates = {k: ("" if v is None else v) for k, v in updates.items()}
@@ -685,9 +610,6 @@ def config_set(request: Request, body: AppConfigSetBody):
if query_string:
updates = process_config_query_string(query_string)
elif body.config_data:
_restore_masked_camera_paths(
body.config_data, request.app.frigate_config
)
updates = flatten_config_data(body.config_data)
# Convert None values to empty strings for deletion (e.g., when deleting masks)
updates = {k: ("" if v is None else v) for k, v in updates.items()}
@@ -774,8 +696,6 @@ def config_set(request: Request, body: AppConfigSetBody):
if request.app.dispatcher is not None:
request.app.dispatcher.config = config
for comm in request.app.dispatcher.comms:
comm.config = config
if body.update_topic:
if body.update_topic.startswith("config/cameras/"):
@@ -872,7 +792,7 @@ def nvinfo():
@router.get(
"/logs/{service}",
tags=[Tags.logs],
dependencies=[Depends(require_role(["admin"]))],
dependencies=[Depends(allow_any_authenticated())],
)
async def logs(
service: str = Path(enum=["frigate", "nginx", "go2rtc"]),
@@ -1077,27 +997,12 @@ def get_media_sync_status(job_id: str):
@router.get("/labels", dependencies=[Depends(allow_any_authenticated())])
def get_labels(
camera: str = "",
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
def get_labels(camera: str = ""):
try:
if camera:
if camera not in allowed_cameras:
return JSONResponse(
content={
"success": False,
"message": f"Access denied to camera '{camera}'",
},
status_code=403,
)
events = Event.select(Event.label).where(Event.camera == camera).distinct()
else:
events = (
Event.select(Event.label)
.where(Event.camera << allowed_cameras)
.distinct()
)
events = Event.select(Event.label).distinct()
except Exception as e:
logger.error(e)
return JSONResponse(
@@ -1110,16 +1015,9 @@ def get_labels(
@router.get("/sub_labels", dependencies=[Depends(allow_any_authenticated())])
def get_sub_labels(
split_joined: Optional[int] = None,
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
def get_sub_labels(split_joined: Optional[int] = None):
try:
events = (
Event.select(Event.sub_label)
.where(Event.camera << allowed_cameras)
.distinct()
)
events = Event.select(Event.sub_label).distinct()
except Exception:
return JSONResponse(
content=({"success": False, "message": "Failed to get sub_labels"}),
+10 -29
View File
@@ -26,7 +26,6 @@ from frigate.api.defs.request.app_body import (
AppPutRoleBody,
)
from frigate.api.defs.tags import Tags
from frigate.api.media_auth import check_camera_access, deny_response_for_media_uri
from frigate.config import AuthConfig, NetworkingConfig, ProxyConfig
from frigate.const import CONFIG_DIR, JWT_SECRET_ENV_VAR, PASSWORD_HASH_ALGORITHM
from frigate.models import User
@@ -634,9 +633,6 @@ def auth(request: Request):
logger.debug("X-Proxy-Secret header does not match configured secret value")
return fail_response
original_url = request.headers.get("x-original-url")
frigate_config = request.app.frigate_config
# if auth is disabled, just apply the proxy header map and return success
if not auth_config.enabled:
# pass the user header value from the upstream proxy if a mapping is specified
@@ -653,11 +649,6 @@ def auth(request: Request):
role = resolve_role(request.headers, proxy_config, config_roles_set)
success_response.headers["remote-role"] = role
deny_status = deny_response_for_media_uri(original_url, role, frigate_config)
if deny_status is not None:
return Response("", status_code=deny_status)
return success_response
# now apply authentication
@@ -752,11 +743,6 @@ def auth(request: Request):
success_response.headers["remote-user"] = user
success_response.headers["remote-role"] = role
deny_status = deny_response_for_media_uri(original_url, role, frigate_config)
if deny_status is not None:
return Response("", status_code=deny_status)
return success_response
except Exception as e:
logger.error(f"Error parsing jwt: {e}")
@@ -826,11 +812,6 @@ limiter = Limiter(key_func=get_remote_addr)
)
@limiter.limit(limit_value=rateLimiter.get_limit)
def login(request: Request, body: AppPostLoginBody):
if not request.app.frigate_config.auth.enabled:
return JSONResponse(
content={"message": "Authentication is disabled"}, status_code=404
)
JWT_COOKIE_NAME = request.app.frigate_config.auth.cookie_name
JWT_COOKIE_SECURE = request.app.frigate_config.auth.cookie_secure
JWT_SESSION_LENGTH = request.app.frigate_config.auth.session_length
@@ -1083,19 +1064,19 @@ async def require_camera_access(
raise HTTPException(status_code=current_user.status_code, detail=detail)
role = current_user["role"]
frigate_config = request.app.frigate_config
all_camera_names = set(request.app.frigate_config.cameras.keys())
roles_dict = request.app.frigate_config.auth.roles
allowed_cameras = User.get_allowed_cameras(role, roles_dict, all_camera_names)
if check_camera_access(role, camera_name, frigate_config):
# Admin or full access bypasses
if role == "admin" or not roles_dict.get(role):
return
all_camera_names = set(frigate_config.cameras.keys())
allowed_cameras = User.get_allowed_cameras(
role, frigate_config.auth.roles, all_camera_names
)
raise HTTPException(
status_code=403,
detail=f"Access denied to camera '{camera_name}'. Allowed: {allowed_cameras}",
)
if camera_name not in allowed_cameras:
raise HTTPException(
status_code=403,
detail=f"Access denied to camera '{camera_name}'. Allowed: {allowed_cameras}",
)
def _get_stream_owner_cameras(request: Request, stream_name: str) -> set[str]:
+5 -40
View File
@@ -19,9 +19,7 @@ from zeep.exceptions import Fault, TransportError
from zeep.transports import AsyncTransport
from frigate.api.auth import (
_get_stream_owner_cameras,
allow_any_authenticated,
get_current_user,
require_go2rtc_stream_access,
require_role,
)
@@ -33,12 +31,11 @@ from frigate.config.camera.updater import (
CameraConfigUpdateTopic,
)
from frigate.config.env import substitute_frigate_vars
from frigate.models import User
from frigate.util.builtin import clean_camera_user_pass
from frigate.util.camera_cleanup import cleanup_camera_db, cleanup_camera_files
from frigate.util.config import find_config_file
from frigate.util.image import run_ffmpeg_snapshot
from frigate.util.services import ffprobe_stream, is_restricted_go2rtc_source
from frigate.util.services import ffprobe_stream
logger = logging.getLogger(__name__)
@@ -69,7 +66,7 @@ def _is_valid_host(host: str) -> bool:
@router.get("/go2rtc/streams", dependencies=[Depends(allow_any_authenticated())])
async def go2rtc_streams(request: Request):
def go2rtc_streams():
r = requests.get("http://127.0.0.1:1984/api/streams")
if not r.ok:
logger.error("Failed to fetch streams from go2rtc")
@@ -78,24 +75,6 @@ async def go2rtc_streams(request: Request):
status_code=500,
)
stream_data = r.json()
# Roles with an explicit camera list see only streams owned by an allowed
# camera. Admin and full-access roles (no list / empty list) see all streams.
current_user = await get_current_user(request)
if not isinstance(current_user, JSONResponse):
role = current_user["role"]
roles_dict = request.app.frigate_config.auth.roles
if role != "admin" and roles_dict.get(role):
all_camera_names = set(request.app.frigate_config.cameras.keys())
allowed_cameras = set(
User.get_allowed_cameras(role, roles_dict, all_camera_names)
)
stream_data = {
name: data
for name, data in stream_data.items()
if _get_stream_owner_cameras(request, name) & allowed_cameras
}
for data in stream_data.values():
for producer in data.get("producers") or []:
producer["url"] = clean_camera_user_pass(producer.get("url", ""))
@@ -147,24 +126,9 @@ def go2rtc_add_stream(request: Request, stream_name: str, src: str = ""):
params = {"name": stream_name}
if src:
try:
resolved_src = substitute_frigate_vars(src)
params["src"] = substitute_frigate_vars(src)
except KeyError:
resolved_src = src
if is_restricted_go2rtc_source(resolved_src):
logger.warning(
"Rejected go2rtc stream '%s' with restricted source type (echo/expr/exec)",
stream_name,
)
return JSONResponse(
content={
"success": False,
"message": "Restricted stream source type",
},
status_code=400,
)
params["src"] = resolved_src
params["src"] = src
r = requests.put(
"http://127.0.0.1:1984/api/streams",
@@ -1002,6 +966,7 @@ async def onvif_probe(
probe = ffprobe_stream(
request.app.frigate_config.ffmpeg, test_uri, detailed=False
)
print(probe)
ok = probe is not None and getattr(probe, "returncode", 1) == 0
tested_candidates.append(
{
+382 -225
View File
@@ -10,7 +10,7 @@ from functools import reduce
from typing import Any, Dict, List, Optional
import cv2
from fastapi import APIRouter, Body, Depends, HTTPException, Request
from fastapi import APIRouter, Body, Depends, Request
from fastapi.responses import JSONResponse, StreamingResponse
from pydantic import BaseModel
@@ -35,13 +35,8 @@ from frigate.api.defs.response.chat_response import (
ToolCall,
)
from frigate.api.defs.tags import Tags
from frigate.api.event import _build_attribute_filter_clause, events
from frigate.api.event import events
from frigate.config import FrigateConfig
from frigate.genai.prompts import (
build_chat_system_prompt,
get_attribute_classifications,
get_tool_definitions,
)
from frigate.genai.utils import build_assistant_message_for_conversation
from frigate.jobs.vlm_watch import (
get_vlm_watch_job,
@@ -72,21 +67,338 @@ class VLMMonitorRequest(BaseModel):
zones: List[str] = []
def get_tool_definitions() -> List[Dict[str, Any]]:
"""
Get OpenAI-compatible tool definitions for Frigate.
Returns a list of tool definitions that can be used with OpenAI-compatible
function calling APIs.
"""
return [
{
"type": "function",
"function": {
"name": "search_objects",
"description": (
"Search the historical record of detected objects in Frigate. "
"Use this ONLY for questions about the PAST — e.g. 'did anyone come by today?', "
"'when was the last car?', 'show me detections from yesterday'. "
"Do NOT use this for monitoring or alerting requests about future events — "
"use start_camera_watch instead for those. "
"An 'object' in Frigate represents a tracked detection (e.g., a person, package, car). "
"When the user asks about a specific name (person, delivery company, animal, etc.), "
"filter by sub_label only and do not set label."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to filter by (optional).",
},
"label": {
"type": "string",
"description": "Object label to filter by (e.g., 'person', 'package', 'car').",
},
"sub_label": {
"type": "string",
"description": "Name of a person, delivery company, animal, etc. When filtering by a specific name, use only sub_label; do not set label.",
},
"after": {
"type": "string",
"description": "Start time in ISO 8601 format (e.g., '2024-01-01T00:00:00Z').",
},
"before": {
"type": "string",
"description": "End time in ISO 8601 format (e.g., '2024-01-01T23:59:59Z').",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "List of zone names to filter by.",
},
"limit": {
"type": "integer",
"description": "Maximum number of objects to return (default: 25).",
"default": 25,
},
},
},
"required": [],
},
},
{
"type": "function",
"function": {
"name": "find_similar_objects",
"description": (
"Find tracked objects that are visually and semantically similar "
"to a specific past event. Use this when the user references a "
"particular object they have seen and wants to find other "
"sightings of the same or similar one ('that green car', 'the "
"person in the red jacket', 'the package that was delivered'). "
"Prefer this over search_objects whenever the user's intent is "
"'find more like this specific one.' Use search_objects first "
"only if you need to locate the anchor event. Requires semantic "
"search to be enabled."
),
"parameters": {
"type": "object",
"properties": {
"event_id": {
"type": "string",
"description": "The id of the anchor event to find similar objects to.",
},
"after": {
"type": "string",
"description": "Start time in ISO 8601 format (e.g., '2024-01-01T00:00:00Z').",
},
"before": {
"type": "string",
"description": "End time in ISO 8601 format (e.g., '2024-01-01T23:59:59Z').",
},
"cameras": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of cameras to restrict to. Defaults to all.",
},
"labels": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of labels to restrict to. Defaults to the anchor event's label.",
},
"sub_labels": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of sub_labels (names) to restrict to.",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of zones. An event matches if any of its zones overlap.",
},
"similarity_mode": {
"type": "string",
"enum": ["visual", "semantic", "fused"],
"description": "Which similarity signal(s) to use. 'fused' (default) combines visual and semantic.",
"default": "fused",
},
"min_score": {
"type": "number",
"description": "Drop matches with a similarity score below this threshold (0.0-1.0).",
},
"limit": {
"type": "integer",
"description": "Maximum number of matches to return (default: 10).",
"default": 10,
},
},
"required": ["event_id"],
},
},
},
{
"type": "function",
"function": {
"name": "set_camera_state",
"description": (
"Change a camera's feature state (e.g., turn detection on/off, enable/disable recordings). "
"Use camera='*' to apply to all cameras at once. "
"Only call this tool when the user explicitly asks to change a camera setting. "
"Requires admin privileges."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to target, or '*' to target all cameras.",
},
"feature": {
"type": "string",
"enum": [
"detect",
"record",
"snapshots",
"audio",
"motion",
"enabled",
"birdseye",
"birdseye_mode",
"improve_contrast",
"ptz_autotracker",
"motion_contour_area",
"motion_threshold",
"notifications",
"audio_transcription",
"review_alerts",
"review_detections",
"object_descriptions",
"review_descriptions",
"profile",
],
"description": (
"The feature to change. Most features accept ON or OFF. "
"birdseye_mode accepts CONTINUOUS, MOTION, or OBJECTS. "
"motion_contour_area and motion_threshold accept a number. "
"profile accepts a profile name or 'none' to deactivate (requires camera='*')."
),
},
"value": {
"type": "string",
"description": "The value to set. ON or OFF for toggles, a number for thresholds, a profile name or 'none' for profile.",
},
},
"required": ["camera", "feature", "value"],
},
},
},
{
"type": "function",
"function": {
"name": "get_live_context",
"description": (
"Get the current live image and detection information for a camera: objects being tracked, "
"zones, timestamps. Use this to understand what is visible in the live view. "
"Call this when answering questions about what is happening right now on a specific camera."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to get live context for.",
},
},
"required": ["camera"],
},
},
},
{
"type": "function",
"function": {
"name": "start_camera_watch",
"description": (
"Start a continuous VLM watch job that monitors a camera and sends a notification "
"when a specified condition is met. Use this when the user wants to be alerted about "
"a future event, e.g. 'tell me when guests arrive' or 'notify me when the package is picked up'. "
"Only one watch job can run at a time. Returns a job ID."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera ID to monitor.",
},
"condition": {
"type": "string",
"description": (
"Natural-language description of the condition to watch for, "
"e.g. 'a person arrives at the front door'."
),
},
"max_duration_minutes": {
"type": "integer",
"description": "Maximum time to watch before giving up (minutes, default 60).",
"default": 60,
},
"labels": {
"type": "array",
"items": {"type": "string"},
"description": "Object labels that should trigger a VLM check (e.g. ['person', 'car']). If omitted, any detection on the camera triggers a check.",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "Zone names to filter by. If specified, only detections in these zones trigger a VLM check.",
},
},
"required": ["camera", "condition"],
},
},
},
{
"type": "function",
"function": {
"name": "stop_camera_watch",
"description": (
"Cancel the currently running VLM watch job. Use this when the user wants to "
"stop a previously started watch, e.g. 'stop watching the front door'."
),
"parameters": {
"type": "object",
"properties": {},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "get_profile_status",
"description": (
"Get the current profile status including the active profile and "
"timestamps of when each profile was last activated. Use this to "
"determine time periods for recap requests — e.g. when the user asks "
"'what happened while I was away?', call this first to find the relevant "
"time window based on profile activation history."
),
"parameters": {
"type": "object",
"properties": {},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "get_recap",
"description": (
"Get a recap of all activity (alerts and detections) for a given time period. "
"Use this after calling get_profile_status to retrieve what happened during "
"a specific window — e.g. 'what happened while I was away?'. Returns a "
"chronological list of activity with camera, objects, zones, and GenAI-generated "
"descriptions when available. Summarize the results for the user."
),
"parameters": {
"type": "object",
"properties": {
"after": {
"type": "string",
"description": "Start of the time period in ISO 8601 format (e.g. '2025-03-15T08:00:00').",
},
"before": {
"type": "string",
"description": "End of the time period in ISO 8601 format (e.g. '2025-03-15T17:00:00').",
},
"cameras": {
"type": "string",
"description": "Comma-separated camera IDs to include, or 'all' for all cameras. Default is 'all'.",
},
"severity": {
"type": "string",
"enum": ["alert", "detection"],
"description": "Filter by severity level. Omit to include both alerts and detections.",
},
},
"required": ["after", "before"],
},
},
},
]
@router.get(
"/chat/tools",
dependencies=[Depends(allow_any_authenticated())],
summary="Get available tools",
description="Returns OpenAI-compatible tool definitions for function calling.",
)
def get_tools(request: Request) -> JSONResponse:
def get_tools() -> JSONResponse:
"""Get list of available tools for LLM function calling."""
config = request.app.frigate_config
semantic_search_enabled = bool(getattr(config.semantic_search, "enabled", False))
attribute_classifications = get_attribute_classifications(config)
tools = get_tool_definitions(
semantic_search_enabled=semantic_search_enabled,
attribute_classifications=attribute_classifications,
)
tools = get_tool_definitions()
return JSONResponse(content={"tools": tools})
@@ -119,29 +431,16 @@ def _resolve_zones(
async def _execute_search_objects(
request: Request,
arguments: Dict[str, Any],
allowed_cameras: List[str],
config: FrigateConfig,
) -> JSONResponse:
"""
Execute the search_objects tool.
Routes to the semantic path when the LLM supplied a `semantic_query`
and semantic search is enabled; otherwise delegates to the standard
events API logic.
This searches for detected objects (events) in Frigate using the same
logic as the events API endpoint.
"""
config = request.app.frigate_config
semantic_query = arguments.get("semantic_query")
if isinstance(semantic_query, str):
semantic_query = semantic_query.strip() or None
else:
semantic_query = None
if semantic_query and getattr(config.semantic_search, "enabled", False):
return await _execute_search_objects_semantic(
request, arguments, allowed_cameras, semantic_query
)
# Parse after/before as server local time; convert to Unix timestamp
after = arguments.get("after")
before = arguments.get("before")
@@ -177,14 +476,11 @@ async def _execute_search_objects(
elif zones is None:
zones = "all"
attribute = arguments.get("attribute")
# Build query parameters compatible with EventsQueryParams
query_params = EventsQueryParams(
cameras=arguments.get("camera", "all"),
labels=arguments.get("label", "all"),
sub_labels=arguments.get("sub_label", "all"), # case-insensitive on the backend
attributes=attribute if attribute else "all",
zones=zones,
zone=zones,
after=after,
@@ -211,124 +507,6 @@ async def _execute_search_objects(
)
async def _execute_search_objects_semantic(
request: Request,
arguments: Dict[str, Any],
allowed_cameras: List[str],
semantic_query: str,
) -> JSONResponse:
"""Search objects via fused thumbnail + description embeddings.
Runs both visual and description vec searches against `semantic_query`,
intersects the candidates with the structured filters (camera, label,
sub_label, zones, time window) the LLM supplied, and ranks the survivors
by fused similarity. Mirrors the candidate-then-filter pattern used by
find_similar_objects since sqlite-vec's IN filter is unreliable.
"""
from peewee import fn
config = request.app.frigate_config
context = request.app.embeddings
if context is None:
logger.warning(
"semantic_query supplied but embeddings context is unavailable; "
"returning empty results."
)
return JSONResponse(content=[])
after = parse_iso_to_timestamp(arguments.get("after"))
before = parse_iso_to_timestamp(arguments.get("before"))
camera_arg = arguments.get("camera")
if camera_arg and camera_arg != "all":
if camera_arg not in allowed_cameras:
return JSONResponse(content=[])
cameras = [camera_arg]
else:
cameras = list(allowed_cameras) if allowed_cameras else []
if not cameras:
return JSONResponse(content=[])
label = arguments.get("label")
sub_label = arguments.get("sub_label")
attribute = arguments.get("attribute")
zones = arguments.get("zones")
if isinstance(zones, list) and zones:
zones = _resolve_zones(zones, config, cameras)
else:
zones = None
limit = int(arguments.get("limit", 25))
limit = max(1, min(limit, 100))
visual_distances: Dict[str, float] = {}
description_distances: Dict[str, float] = {}
try:
rows = context.search_thumbnail(semantic_query)
visual_distances = {row[0]: row[1] for row in rows}
except Exception:
logger.exception(
"search_thumbnail failed for semantic_query: %s", semantic_query
)
try:
rows = context.search_description(semantic_query)
description_distances = {row[0]: row[1] for row in rows}
except Exception:
logger.exception(
"search_description failed for semantic_query: %s", semantic_query
)
vec_ids = set(visual_distances) | set(description_distances)
if not vec_ids:
return JSONResponse(content=[])
clauses = [Event.id.in_(list(vec_ids)), Event.camera.in_(cameras)]
if after is not None:
clauses.append(Event.start_time >= after)
if before is not None:
clauses.append(Event.start_time <= before)
if label:
clauses.append(Event.label == label)
if sub_label:
# case-insensitive match to mirror events() behavior
clauses.append(fn.LOWER(Event.sub_label.cast("text")) == sub_label.lower())
if attribute:
attribute_clause = _build_attribute_filter_clause(attribute)
if attribute_clause is not None:
clauses.append(attribute_clause)
if zones:
zone_clauses = [Event.zones.cast("text") % f'*"{zone}"*' for zone in zones]
clauses.append(reduce(operator.or_, zone_clauses))
eligible = {e.id: e for e in Event.select().where(reduce(operator.and_, clauses))}
scored: List[tuple[str, float]] = []
for eid in eligible:
v_score = (
distance_to_score(visual_distances[eid], context.thumb_stats)
if eid in visual_distances
else None
)
d_score = (
distance_to_score(description_distances[eid], context.desc_stats)
if eid in description_distances
else None
)
fused = fuse_scores(v_score, d_score)
if fused is None:
continue
scored.append((eid, fused))
scored.sort(key=lambda pair: pair[1], reverse=True)
scored = scored[:limit]
results = [hydrate_event(eligible[eid], score=score) for eid, score in scored]
return JSONResponse(content=results)
async def _execute_find_similar_objects(
request: Request,
arguments: Dict[str, Any],
@@ -517,7 +695,9 @@ async def execute_tool(
logger.debug(f"Executing tool: {tool_name} with arguments: {arguments}")
if tool_name == "search_objects":
return await _execute_search_objects(request, arguments, allowed_cameras)
return await _execute_search_objects(
arguments, allowed_cameras, request.app.frigate_config
)
if tool_name == "find_similar_objects":
result = await _execute_find_similar_objects(
@@ -697,7 +877,9 @@ async def _execute_tool_internal(
This is used by the chat completion endpoint to execute tools.
"""
if tool_name == "search_objects":
response = await _execute_search_objects(request, arguments, allowed_cameras)
response = await _execute_search_objects(
arguments, allowed_cameras, request.app.frigate_config
)
try:
if hasattr(response, "body"):
body_str = response.body.decode("utf-8")
@@ -1110,21 +1292,52 @@ async def chat_completion(
status_code=400,
)
config = request.app.frigate_config
semantic_search_enabled = bool(getattr(config.semantic_search, "enabled", False))
attribute_classifications = get_attribute_classifications(config)
tools = get_tool_definitions(
semantic_search_enabled=semantic_search_enabled,
attribute_classifications=attribute_classifications,
)
tools = get_tool_definitions()
conversation = []
system_prompt = build_chat_system_prompt(
config=config,
allowed_cameras=allowed_cameras,
semantic_search_enabled=semantic_search_enabled,
attribute_classifications=attribute_classifications,
)
current_datetime = datetime.now()
current_date_str = current_datetime.strftime("%Y-%m-%d")
current_time_str = current_datetime.strftime("%I:%M:%S %p")
cameras_info = []
config = request.app.frigate_config
for camera_id in allowed_cameras:
if camera_id not in config.cameras:
continue
camera_config = config.cameras[camera_id]
friendly_name = (
camera_config.friendly_name
if camera_config.friendly_name
else camera_id.replace("_", " ").title()
)
zone_names = list(camera_config.zones.keys())
if zone_names:
cameras_info.append(
f" - {friendly_name} (ID: {camera_id}, zones: {', '.join(zone_names)})"
)
else:
cameras_info.append(f" - {friendly_name} (ID: {camera_id})")
cameras_section = ""
if cameras_info:
cameras_section = (
"\n\nAvailable cameras:\n"
+ "\n".join(cameras_info)
+ "\n\nWhen users refer to cameras by their friendly name (e.g., 'Back Deck Camera'), use the corresponding camera ID (e.g., 'back_deck_cam') in tool calls."
)
system_prompt = f"""You are a helpful assistant for Frigate, a security camera NVR system. You help users answer questions about their cameras, detected objects, and events.
Current server local date and time: {current_date_str} at {current_time_str}
Do not start your response with phrases like "I will check...", "Let me see...", or "Let me look...". Answer directly.
Always present times to the user in the server's local timezone. When tool results include start_time_local and end_time_local, use those exact strings when listing or describing detection times—do not convert or invent timestamps. Do not use UTC or ISO format with Z for the user-facing answer unless the tool result only provides Unix timestamps without local time fields.
When users ask about "today", "yesterday", "this week", etc., use the current date above as reference.
When searching for objects or events, use ISO 8601 format for dates (e.g., {current_date_str}T00:00:00Z for the start of today).
Always be accurate with time calculations based on the current date provided.
When a user refers to a specific object they have seen or describe with identifying details ("that green car", "the person in the red jacket", "a package left today"), prefer the find_similar_objects tool over search_objects. Use search_objects first only to locate the anchor event, then pass its id to find_similar_objects. For generic queries like "show me all cars today", keep using search_objects. If a user message begins with [attached_event:<id>], treat that event id as the anchor for any similarity or "tell me more" request in the same message and call find_similar_objects with that id.{cameras_section}"""
conversation.append(
{
@@ -1185,18 +1398,6 @@ async def chat_completion(
)
+ b"\n"
)
elif kind == "reasoning_delta":
yield (
json.dumps({"type": "reasoning", "delta": value}).encode(
"utf-8"
)
+ b"\n"
)
elif kind == "stats":
yield (
json.dumps({"type": "stats", **value}).encode("utf-8")
+ b"\n"
)
elif kind == "message":
msg = value
if msg.get("finish_reason") == "error":
@@ -1292,7 +1493,6 @@ async def chat_completion(
final_content = response.get("content") or ""
if body.stream:
final_reasoning = response.get("reasoning")
async def stream_body() -> Any:
if tool_calls:
@@ -1307,15 +1507,6 @@ async def chat_completion(
).encode("utf-8")
+ b"\n"
)
# Emit the full reasoning trace up front when the
# underlying client did not stream it
if final_reasoning:
yield (
json.dumps(
{"type": "reasoning", "delta": final_reasoning}
).encode("utf-8")
+ b"\n"
)
# Stream content in word-sized chunks for smooth UX
for part in chunk_content(final_content):
yield (
@@ -1336,7 +1527,6 @@ async def chat_completion(
message=ChatMessageResponse(
role="assistant",
content=final_content,
reasoning=response.get("reasoning"),
tool_calls=None,
),
finish_reason=response.get("finish_reason", "stop"),
@@ -1438,7 +1628,6 @@ async def start_vlm_monitor(
dispatcher=request.app.dispatcher,
labels=body.labels,
zones=body.zones,
username=request.headers.get("remote-user", ""),
)
except RuntimeError as e:
logger.error("Failed to start VLM watch job: %s", e, exc_info=True)
@@ -1459,22 +1648,10 @@ async def start_vlm_monitor(
summary="Get current VLM watch job",
description="Returns the current (or most recently completed) VLM watch job.",
)
async def get_vlm_monitor(request: Request) -> JSONResponse:
async def get_vlm_monitor() -> JSONResponse:
job = get_vlm_watch_job()
if job is None:
return JSONResponse(content={"active": False}, status_code=200)
role = request.headers.get("remote-role", "viewer")
username = request.headers.get("remote-user", "")
# Admin and the job's creator always see the job. Other users only see it
# if they have access to the camera being watched; otherwise hide it.
if role != "admin" and username != job.username:
try:
await require_camera_access(job.camera, request=request)
except HTTPException:
return JSONResponse(content={"active": False}, status_code=200)
return JSONResponse(content={"active": True, **job.to_dict()}, status_code=200)
@@ -1484,27 +1661,7 @@ async def get_vlm_monitor(request: Request) -> JSONResponse:
summary="Cancel the current VLM watch job",
description="Cancels the running watch job if one exists.",
)
async def cancel_vlm_monitor(request: Request) -> JSONResponse:
job = get_vlm_watch_job()
if job is None:
return JSONResponse(
content={"success": False, "message": "No active watch job to cancel."},
status_code=404,
)
role = request.headers.get("remote-role", "viewer")
username = request.headers.get("remote-user", "")
# Admin can cancel any job; other users can only cancel jobs they started.
if role != "admin" and username != job.username:
return JSONResponse(
content={
"success": False,
"message": "Not authorized to cancel this watch job.",
},
status_code=403,
)
async def cancel_vlm_monitor() -> JSONResponse:
cancelled = stop_vlm_watch_job()
if not cancelled:
return JSONResponse(
+25 -134
View File
@@ -6,18 +6,10 @@ from datetime import datetime
from fastapi import APIRouter, Depends, Request
from fastapi.responses import JSONResponse
from peewee import DoesNotExist
from pydantic import BaseModel, Field
from frigate.api.auth import require_role
from frigate.api.defs.tags import Tags
from frigate.jobs.debug_replay import (
ExportDebugReplaySource,
RecordingDebugReplaySource,
start_debug_replay_job,
)
from frigate.models import Export
from frigate.util.services import get_video_properties
logger = logging.getLogger(__name__)
@@ -32,28 +24,15 @@ class DebugReplayStartBody(BaseModel):
end_time: float = Field(title="End timestamp")
class DebugReplayStartFromExportBody(BaseModel):
"""Request body for starting a debug replay session from an export."""
export_id: str = Field(title="Export id")
class DebugReplayStartResponse(BaseModel):
"""Response for starting a debug replay session."""
success: bool
replay_camera: str
job_id: str
class DebugReplayStatusResponse(BaseModel):
"""Response for debug replay status.
Returns only session-presence fields. Startup progress and error
details flow through the job_state WebSocket topic via the
debug_replay job (see frigate.jobs.debug_replay); the
Replay page subscribes there with useJobStatus("debug_replay").
"""
"""Response for debug replay status."""
active: bool
replay_camera: str | None = None
@@ -72,35 +51,15 @@ class DebugReplayStopResponse(BaseModel):
@router.post(
"/debug_replay/start",
response_model=DebugReplayStartResponse,
status_code=202,
responses={
400: {"description": "Invalid camera, time range, or no recordings"},
409: {"description": "A replay session is already active"},
},
dependencies=[Depends(require_role(["admin"]))],
summary="Start debug replay",
description="Start a debug replay session from camera recordings. Returns "
"immediately while clip generation runs as a background job; subscribe "
"to the 'debug_replay' job_state WS topic to track progress.",
description="Start a debug replay session from camera recordings.",
)
async def start_debug_replay(request: Request, body: DebugReplayStartBody):
"""Start a debug replay session asynchronously."""
"""Start a debug replay session."""
replay_manager = request.app.replay_manager
source = RecordingDebugReplaySource(
source_camera=body.camera,
start_ts=body.start_time,
end_ts=body.end_time,
)
try:
job_id = await asyncio.to_thread(
start_debug_replay_job,
source=source,
frigate_config=request.app.frigate_config,
config_publisher=request.app.config_publisher,
replay_manager=replay_manager,
)
except RuntimeError:
if replay_manager.active:
return JSONResponse(
content={
"success": False,
@@ -108,102 +67,38 @@ async def start_debug_replay(request: Request, body: DebugReplayStartBody):
},
status_code=409,
)
except ValueError:
logger.exception("Rejected debug replay start request")
return JSONResponse(
content={
"success": False,
"message": "Invalid debug replay parameters",
},
status_code=400,
)
return JSONResponse(
content={
"success": True,
"replay_camera": replay_manager.replay_camera_name,
"job_id": job_id,
},
status_code=202,
)
@router.post(
"/debug_replay/start_from_export",
response_model=DebugReplayStartResponse,
status_code=202,
responses={
400: {"description": "Invalid export, time range, or no recordings"},
404: {"description": "Export not found"},
409: {"description": "A replay session is already active"},
},
dependencies=[Depends(require_role(["admin"]))],
summary="Start debug replay from an export",
description="Start a debug replay session covering an existing export's "
"time range. The end time is derived from the export's video duration.",
)
async def start_debug_replay_from_export(
request: Request, body: DebugReplayStartFromExportBody
):
"""Start a debug replay session from an existing export."""
try:
export: Export = Export.get(Export.id == body.export_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export not found"},
status_code=404,
)
properties = await get_video_properties(
request.app.frigate_config.ffmpeg, export.video_path, get_duration=True
)
duration = properties.get("duration", -1)
if duration is None or duration <= 0:
return JSONResponse(
content={
"success": False,
"message": "Could not determine export duration",
},
status_code=400,
)
replay_manager = request.app.replay_manager
source = ExportDebugReplaySource(export=export, duration=float(duration))
try:
job_id = await asyncio.to_thread(
start_debug_replay_job,
source=source,
replay_camera = await asyncio.to_thread(
replay_manager.start,
source_camera=body.camera,
start_ts=body.start_time,
end_ts=body.end_time,
frigate_config=request.app.frigate_config,
config_publisher=request.app.config_publisher,
replay_manager=replay_manager,
)
except RuntimeError:
return JSONResponse(
content={
"success": False,
"message": "A replay session is already active",
},
status_code=409,
)
except ValueError:
logger.exception("Rejected debug replay start request")
logger.exception("Invalid parameters for debug replay start request")
return JSONResponse(
content={
"success": False,
"message": "Invalid debug replay parameters",
"message": "Invalid debug replay request parameters",
},
status_code=400,
)
except RuntimeError:
logger.exception("Error while starting debug replay session")
return JSONResponse(
content={
"success": False,
"message": "An internal error occurred while starting debug replay",
},
status_code=500,
)
return JSONResponse(
content={
"success": True,
"replay_camera": replay_manager.replay_camera_name,
"job_id": job_id,
},
status_code=202,
return DebugReplayStartResponse(
success=True,
replay_camera=replay_camera,
)
@@ -223,16 +118,12 @@ def get_debug_replay_status(request: Request):
if replay_manager.active and replay_camera:
frame_processor = request.app.detected_frames_processor
frame = (
frame_processor.get_current_frame(replay_camera)
if frame_processor is not None
else None
)
frame = frame_processor.get_current_frame(replay_camera)
if frame is not None:
frame_time = frame_processor.get_current_frame_time(replay_camera)
camera_config = request.app.frigate_config.cameras.get(replay_camera)
retry_interval = 10.0
retry_interval = 10
if camera_config is not None:
retry_interval = float(camera_config.ffmpeg.retry_interval or 10)
@@ -20,10 +20,6 @@ class ChatMessageResponse(BaseModel):
content: Optional[str] = Field(
default=None, description="Message content (None if tool calls present)"
)
reasoning: Optional[str] = Field(
default=None,
description="Separated reasoning/thinking trace if the model emitted one",
)
tool_calls: Optional[list[ToolCallInvocation]] = Field(
default=None, description="Tool calls if LLM wants to call tools"
)
+1 -1
View File
@@ -398,7 +398,7 @@ class _StreamingZipBuffer:
def _unique_archive_name(export: Export, used: set[str]) -> str:
base = sanitize_filename(export.name) if export.name else None
if not base:
base = f"{export.camera}_{int(export.date)}"
base = f"{export.camera}_{int(datetime.datetime.timestamp(export.date))}"
candidate = f"{base}.mp4"
counter = 1
+5 -3
View File
@@ -174,10 +174,12 @@ async def latest_frame(
}
quality_params = get_image_quality_params(extension.value, params.quality)
camera_config = request.app.frigate_config.cameras.get(camera_name)
if camera_config is not None:
if camera_name in request.app.frigate_config.cameras:
frame = frame_processor.get_current_frame(camera_name, draw_options)
retry_interval = float(camera_config.ffmpeg.retry_interval or 10)
retry_interval = float(
request.app.frigate_config.cameras.get(camera_name).ffmpeg.retry_interval
or 10
)
is_offline = False
if frame is None or datetime.now().timestamp() > (
-291
View File
@@ -1,291 +0,0 @@
"""URI-aware authorization for nginx-served static media.
The `/auth` endpoint (used as nginx `auth_request` target) calls into this
module to classify the requested URI from the `X-Original-URL` header and, for
camera-scoped resources, decide whether the current role may access them.
Without this, `auth_request` only verifies the JWT every authenticated user
could read clips, recordings, and exports for *any* camera, bypassing the
per-camera authorization the regular API enforces via `require_camera_access`.
"""
from __future__ import annotations
import logging
import os
from enum import Enum
from typing import Optional
from urllib.parse import unquote, urlparse
from peewee import DoesNotExist
from frigate.config import FrigateConfig
from frigate.const import EXPORT_DIR
from frigate.models import Export, User
logger = logging.getLogger(__name__)
class MediaAuthResolution(str, Enum):
"""Classification of an `X-Original-URL` path for media-auth purposes."""
CAMERA = "camera"
ADMIN_ONLY = "admin_only"
LISTING_MULTI_CAMERA = "listing_multi_camera"
LISTING_NEUTRAL = "listing_neutral"
# Under a recognized media root (/clips, /recordings, /exports) but
# unclassifiable (unknown subtree, no matching DB row, DB error).
# Restricted users are denied; admins/full-access roles are allowed
# (nginx will likely return 404 if the file genuinely doesn't exist).
UNRESOLVED_MEDIA = "unresolved_media"
# Not a media URI at all (e.g. /api/events, /login).
UNKNOWN = "unknown"
def extract_path(original_url: Optional[str]) -> Optional[str]:
"""Return the decoded path component of nginx's `X-Original-URL` header.
nginx forwards the *raw* request URI (with `..` segments intact) via
`$request_uri`. nginx normalizes the path before serving the file, so a
request like `/recordings/.../allowed_cam/../forbidden_cam/file.mp4`
would (1) parse as the allowed camera in our auth check, (2) be served
as the forbidden camera by nginx. To close the bypass we reject any URI
whose path contains `.` or `..` segments outright.
"""
if not original_url:
return None
parsed = urlparse(original_url)
raw_path = parsed.path or original_url
decoded = unquote(raw_path)
if not decoded:
return None
if not decoded.startswith("/"):
decoded = "/" + decoded
segments = decoded.split("/")
if ".." in segments or "." in segments:
return None
return decoded
def resolve_media_uri(
uri: str, frigate_config: Optional[FrigateConfig] = None
) -> tuple[MediaAuthResolution, Optional[str]]:
"""Classify a URI and return the owning camera if applicable.
`frigate_config` is used to disambiguate clip/review filenames whose
camera name contains hyphens by matching against the longest configured
camera-name prefix.
"""
if not uri:
return MediaAuthResolution.UNKNOWN, None
parts = [p for p in uri.split("/") if p]
if not parts:
return MediaAuthResolution.UNKNOWN, None
root = parts[0]
if root == "recordings":
return _resolve_recording(parts)
if root == "clips":
return _resolve_clip(parts, frigate_config)
if root == "exports":
return _resolve_export(parts)
return MediaAuthResolution.UNKNOWN, None
def _resolve_recording(
parts: list[str],
) -> tuple[MediaAuthResolution, Optional[str]]:
# /recordings → neutral
# /recordings/{date} → neutral
# /recordings/{date}/{hour} → multi-camera listing
# /recordings/{date}/{hour}/{cam}/... → camera
if len(parts) <= 2:
return MediaAuthResolution.LISTING_NEUTRAL, None
if len(parts) == 3:
return MediaAuthResolution.LISTING_MULTI_CAMERA, None
return MediaAuthResolution.CAMERA, parts[3]
def _resolve_clip(
parts: list[str], frigate_config: Optional[FrigateConfig]
) -> tuple[MediaAuthResolution, Optional[str]]:
# /clips → multi-camera listing
# /clips/thumbs/{cam}/... → camera
# /clips/previews/{cam}/... → camera
# /clips/review/thumb-{cam}-{review_id}.webp → camera (parsed)
# /clips/faces/... → admin-only
# /clips/genai-requests/... → admin-only
# /clips/preview_restart_cache/... → admin-only
# /clips/{model}/train|dataset/... → admin-only
# /clips/{cam}-{event_id}[-clean].{ext} → camera (parsed)
# other /clips/{subdir}/... → unresolved (deny restricted)
if len(parts) == 1:
return MediaAuthResolution.LISTING_MULTI_CAMERA, None
second = parts[1]
if second in ("thumbs", "previews"):
if len(parts) == 2:
return MediaAuthResolution.LISTING_MULTI_CAMERA, None
return MediaAuthResolution.CAMERA, parts[2]
if second == "review":
if len(parts) == 2:
return MediaAuthResolution.LISTING_MULTI_CAMERA, None
camera = _camera_from_thumb_filename(parts[2], frigate_config)
if camera:
return MediaAuthResolution.CAMERA, camera
return MediaAuthResolution.UNRESOLVED_MEDIA, None
if second in ("faces", "genai-requests", "preview_restart_cache"):
return MediaAuthResolution.ADMIN_ONLY, None
if len(parts) >= 3 and parts[2] in ("train", "dataset"):
return MediaAuthResolution.ADMIN_ONLY, None
if len(parts) == 2:
camera = _camera_from_clip_filename(second, frigate_config)
if camera:
return MediaAuthResolution.CAMERA, camera
return MediaAuthResolution.UNRESOLVED_MEDIA, None
return MediaAuthResolution.UNRESOLVED_MEDIA, None
def _longest_prefix_camera(
stem: str, frigate_config: Optional[FrigateConfig]
) -> Optional[str]:
if frigate_config is None:
return None
for cam in sorted(frigate_config.cameras.keys(), key=len, reverse=True):
if stem.startswith(cam + "-"):
return cam
return None
def _camera_from_clip_filename(
filename: str, frigate_config: Optional[FrigateConfig]
) -> Optional[str]:
"""Match a flat clip filename `{camera}-{event_id}[-clean].{ext}` against
configured camera names. Longest-prefix wins so camera names containing
hyphens (e.g. `front-door`) resolve correctly.
"""
dot = filename.rfind(".")
stem = filename[:dot] if dot > 0 else filename
return _longest_prefix_camera(stem, frigate_config)
def _camera_from_thumb_filename(
filename: str, frigate_config: Optional[FrigateConfig]
) -> Optional[str]:
"""Match a review thumbnail filename `thumb-{camera}-{review_id}.webp`."""
if not filename.startswith("thumb-"):
return None
dot = filename.rfind(".")
stem = filename[len("thumb-") : dot] if dot > 0 else filename[len("thumb-") :]
return _longest_prefix_camera(stem, frigate_config)
def _resolve_export(
parts: list[str],
) -> tuple[MediaAuthResolution, Optional[str]]:
# /exports → multi-camera listing
# /exports/{filename}.mp4 → camera (DB lookup by exact path)
if len(parts) == 1:
return MediaAuthResolution.LISTING_MULTI_CAMERA, None
if len(parts) != 2:
return MediaAuthResolution.UNRESOLVED_MEDIA, None
filename = parts[1]
full_path = os.path.join(EXPORT_DIR, filename)
try:
export = Export.get(Export.video_path == full_path)
return MediaAuthResolution.CAMERA, export.camera
except DoesNotExist:
return MediaAuthResolution.UNRESOLVED_MEDIA, None
except Exception as e:
logger.warning("Export DB lookup failed for %s: %s", filename, e)
return MediaAuthResolution.UNRESOLVED_MEDIA, None
def check_camera_access(role: str, camera: str, frigate_config: FrigateConfig) -> bool:
"""Return True iff `role` may access `camera`.
Mirrors the gating logic in `require_camera_access`: admin and any role
without a non-empty allow-list bypass the check.
"""
if role == "admin":
return True
roles_dict = frigate_config.auth.roles
if not roles_dict.get(role):
return True
all_camera_names = set(frigate_config.cameras.keys())
allowed = User.get_allowed_cameras(role, roles_dict, all_camera_names)
return camera in allowed
def is_role_restricted(role: str, frigate_config: FrigateConfig) -> bool:
"""True if `role` has a non-empty allow-list (i.e. not full-access)."""
if role == "admin":
return False
return bool(frigate_config.auth.roles.get(role))
def deny_response_for_media_uri(
original_url: Optional[str], role: Optional[str], frigate_config: FrigateConfig
) -> Optional[int]:
"""Decide whether the current role should be blocked from `original_url`.
Returns an HTTP status code (403) when access should be denied, or `None`
when the request is allowed.
"""
if not original_url:
return None
path = extract_path(original_url)
# `extract_path` returns None for URIs containing `.` or `..` segments.
# For media-root URIs that's a traversal attempt — deny outright. For
# non-media URIs, pass through (nginx / the backend handle them).
if path is None:
raw = urlparse(original_url).path or original_url
decoded = unquote(raw)
first = decoded.lstrip("/").split("/", 1)[0] if decoded else ""
if first in ("clips", "recordings", "exports"):
return 403
return None
resolution, camera = resolve_media_uri(path, frigate_config)
if resolution == MediaAuthResolution.UNKNOWN:
return None
if not role or role == "admin":
return None
if not is_role_restricted(role, frigate_config):
return None
if resolution == MediaAuthResolution.LISTING_NEUTRAL:
return None
if resolution in (
MediaAuthResolution.LISTING_MULTI_CAMERA,
MediaAuthResolution.ADMIN_ONLY,
MediaAuthResolution.UNRESOLVED_MEDIA,
):
return 403
if resolution == MediaAuthResolution.CAMERA:
if camera and check_camera_access(role, camera, frigate_config):
return None
return 403
return 403
+13 -6
View File
@@ -144,7 +144,7 @@ class FrigateApp:
for d in dirs:
if not os.path.exists(d) and not os.path.islink(d):
logger.info(f"Creating directory: {d}")
os.makedirs(d, exist_ok=True)
os.makedirs(d)
else:
logger.debug(f"Skipping directory: {d}")
@@ -428,11 +428,18 @@ class FrigateApp:
self.camera_maintainer.start()
def start_audio_processor(self) -> None:
self.audio_process = AudioProcessor(
self.config, self.camera_metrics, self.stop_event
)
self.audio_process.start()
self.processes["audio_detector"] = self.audio_process.pid or 0
audio_cameras = [
c
for c in self.config.cameras.values()
if c.enabled and c.audio.enabled_in_config
]
if audio_cameras:
self.audio_process = AudioProcessor(
self.config, audio_cameras, self.camera_metrics, self.stop_event
)
self.audio_process.start()
self.processes["audio_detector"] = self.audio_process.pid or 0
def start_timeline_processor(self) -> None:
self.timeline_processor = TimelineProcessor(
+1 -4
View File
@@ -429,10 +429,7 @@ class WebPushClient(Communicator):
else:
title = base_title
if payload["after"]["data"]["metadata"].get("shortSummary"):
message = payload["after"]["data"]["metadata"]["shortSummary"]
else:
message = f"Detected on {camera_name}"
message = payload["after"]["data"]["metadata"]["shortSummary"]
else:
zone_names = payload["after"]["data"]["zones"]
formatted_zone_names = []
+9 -455
View File
@@ -17,408 +17,9 @@ from ws4py.websocket import WebSocket as WebSocket_
from frigate.comms.base_communicator import Communicator
from frigate.config import FrigateConfig
from frigate.const import (
CLEAR_ONGOING_REVIEW_SEGMENTS,
EXPIRE_AUDIO_ACTIVITY,
INSERT_MANY_RECORDINGS,
INSERT_PREVIEW,
NOTIFICATION_TEST,
REQUEST_REGION_GRID,
UPDATE_AUDIO_ACTIVITY,
UPDATE_AUDIO_TRANSCRIPTION_STATE,
UPDATE_BIRDSEYE_LAYOUT,
UPDATE_CAMERA_ACTIVITY,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
UPDATE_EVENT_DESCRIPTION,
UPDATE_MODEL_STATE,
UPDATE_REVIEW_DESCRIPTION,
UPSERT_REVIEW_SEGMENT,
)
from frigate.models import User
from frigate.output.ws_auth import ws_has_camera_access
logger = logging.getLogger(__name__)
# Internal IPC topics — NEVER allowed from WebSocket, regardless of role
_WS_BLOCKED_TOPICS = frozenset(
{
INSERT_MANY_RECORDINGS,
INSERT_PREVIEW,
REQUEST_REGION_GRID,
UPSERT_REVIEW_SEGMENT,
CLEAR_ONGOING_REVIEW_SEGMENTS,
UPDATE_CAMERA_ACTIVITY,
UPDATE_AUDIO_ACTIVITY,
EXPIRE_AUDIO_ACTIVITY,
UPDATE_EVENT_DESCRIPTION,
UPDATE_REVIEW_DESCRIPTION,
UPDATE_MODEL_STATE,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
UPDATE_BIRDSEYE_LAYOUT,
UPDATE_AUDIO_TRANSCRIPTION_STATE,
NOTIFICATION_TEST,
}
)
# Read-only topics any authenticated user (including viewer) can send
_WS_VIEWER_TOPICS = frozenset(
{
"onConnect",
"modelState",
"audioTranscriptionState",
"birdseyeLayout",
"embeddingsReindexProgress",
"jobState",
}
)
def _check_ws_authorization(
topic: str,
role_header: str | None,
separator: str,
) -> bool:
"""Check if a WebSocket message is authorized.
Args:
topic: The message topic.
role_header: The HTTP_REMOTE_ROLE header value, or None.
separator: The role separator character from proxy config.
Returns:
True if authorized, False if blocked.
"""
# Block IPC-only topics unconditionally
if topic in _WS_BLOCKED_TOPICS:
return False
# No role header: default to viewer (fail-closed)
if role_header is None:
return topic in _WS_VIEWER_TOPICS
# Check if any role is admin
roles = [r.strip() for r in role_header.split(separator)]
if "admin" in roles:
return True
# Non-admin: only viewer topics allowed
return topic in _WS_VIEWER_TOPICS
# ---- Outbound filtering ---------------------------------------------------
#
# Every WebSocket broadcast is classified into one of a small set of scopes,
# then materialized per recipient. Connections with restricted roles only see
# data for cameras they are authorized to access; admin and full-access roles
# behave as today.
# Topics that are safe to broadcast to every authenticated client.
_WS_GLOBAL_OUTBOUND_TOPICS = frozenset(
{
"model_state",
"embeddings_reindex_progress",
"audio_transcription_state",
"profile/state",
"notifications/state",
"notification_test",
}
)
# Topics that restricted roles must never receive. Birdseye composites span
# all cameras, so the existing JSMPEG policy already restricts birdseye access
# to unrestricted roles; the layout broadcast follows the same rule.
_WS_UNRESTRICTED_ONLY_TOPICS = frozenset(
{
"birdseye_layout",
}
)
# Topics whose payload (parsed as JSON) names a single owning camera at the
# given key path. Used to scope events, reviews, triggers, etc.
_WS_PAYLOAD_CAMERA_TOPICS: dict[str, tuple[str, ...]] = {
"events": ("after", "camera"),
"reviews": ("after", "camera"),
"tracked_object_update": ("camera",),
"triggers": ("camera",),
"camera_monitoring": ("camera",),
}
# Topics whose payload is a dict keyed by camera name; filter keys per
# recipient.
_WS_RESHAPE_BY_CAMERA_KEY_TOPICS = frozenset(
{
"camera_activity",
"audio_detections",
}
)
# Topics whose payload is a dict keyed by job_type, where each entry may
# contain a "camera" or "source_camera" field, or a nested ``results.jobs``
# list of per-camera sub-jobs (export broadcasts).
_WS_RESHAPE_JOB_STATE_TOPICS = frozenset(
{
"job_state",
}
)
# Topics whose payload mixes global aggregates with a ``cameras`` sub-dict
# keyed by camera name. Aggregates and detector data stay; per-camera entries
# are filtered.
_WS_RESHAPE_STATS_TOPICS = frozenset(
{
"stats",
}
)
def _collect_zone_names(config: FrigateConfig) -> set[str]:
"""Return the set of all zone names defined across cameras."""
names: set[str] = set()
for camera in config.cameras.values():
zones = getattr(camera, "zones", None) or {}
names.update(zones.keys())
return names
def _parse_json_payload(payload: Any) -> Any:
"""Return payload parsed as JSON if it is a string, else as-is."""
if isinstance(payload, str):
try:
return json.loads(payload)
except (ValueError, TypeError):
return None
return payload
def _scope_job_entry_to_allowed(entry: Any, allowed: set[str]) -> dict[str, Any] | None:
"""Filter a single job_state entry to the recipient's allowed cameras.
Returns the (possibly reshaped) entry, or None to drop it. Four shapes
are handled:
* Top-level ``camera`` or ``source_camera`` (motion_search, vlm_watch,
export sub-job dicts): drop the entry if not allowed.
* Nested ``results.jobs`` list of per-camera sub-jobs (the aggregated
export broadcast): filter the list; drop the entry if nothing remains.
* Nested ``results.camera`` or ``results.source_camera`` (debug_replay,
which puts replay-specific fields inside ``results``): drop the entry
if not allowed.
* No camera anywhere (e.g. ``media_sync``): treat as global and keep.
"""
if not isinstance(entry, dict):
return None
cam = entry.get("camera") or entry.get("source_camera")
if cam is None:
results = entry.get("results")
if isinstance(results, dict):
sub_jobs = results.get("jobs")
if isinstance(sub_jobs, list):
filtered_jobs = [
j
for j in sub_jobs
if isinstance(j, dict)
and (j.get("camera") or j.get("source_camera")) in allowed
]
if not filtered_jobs:
return None
reshaped = dict(entry)
reshaped["results"] = dict(results)
reshaped["results"]["jobs"] = filtered_jobs
return reshaped
cam = results.get("camera") or results.get("source_camera")
if cam is not None:
return entry if cam in allowed else None
return entry
def _extract_payload_camera(payload: Any, path: tuple[str, ...]) -> str | None:
"""Walk the dotted path through a (possibly JSON-encoded) payload."""
cur = _parse_json_payload(payload)
for key in path:
if not isinstance(cur, dict):
return None
cur = cur.get(key)
return cur if isinstance(cur, str) else None
def _classify_outbound(
topic: str, all_cameras: set[str], all_zones: set[str]
) -> tuple[str, Any]:
"""Classify an outbound topic into (kind, extra).
kind values:
- "global" : send to every authenticated client
- "drop" : send to nobody (fail-closed for unknowns)
- "unrestricted_only" : send only to admin/full-access roles
- "camera" : extra is the owning camera name
- "payload_camera" : extra is the JSON key path to the camera name
- "reshape_by_camera_key"
- "reshape_job_state"
- "reshape_stats"
"""
if topic in _WS_GLOBAL_OUTBOUND_TOPICS:
return ("global", None)
if topic in _WS_UNRESTRICTED_ONLY_TOPICS:
return ("unrestricted_only", None)
if topic in _WS_RESHAPE_BY_CAMERA_KEY_TOPICS:
return ("reshape_by_camera_key", None)
if topic in _WS_RESHAPE_JOB_STATE_TOPICS:
return ("reshape_job_state", None)
if topic in _WS_RESHAPE_STATS_TOPICS:
return ("reshape_stats", None)
if topic in _WS_PAYLOAD_CAMERA_TOPICS:
return ("payload_camera", _WS_PAYLOAD_CAMERA_TOPICS[topic])
# Topic-prefix based: first segment names the owning camera or zone.
first = topic.split("/", 1)[0]
if first in all_cameras:
return ("camera", first)
if first in all_zones:
# Zone aggregates span cameras; restricted users see nothing here.
return ("unrestricted_only", None)
return ("drop", None)
def _ws_role_header(ws: Any) -> str | None:
"""Return the HTTP_REMOTE_ROLE header value, if any."""
environ = getattr(ws, "environ", None)
if not environ:
return None
value = environ.get("HTTP_REMOTE_ROLE")
return value if isinstance(value, str) else None
def _ws_valid_roles(ws: Any, config: FrigateConfig) -> list[str]:
"""Return the list of recognized roles for this connection."""
header = _ws_role_header(ws)
if not header:
return []
roles = [r.strip() for r in header.split(config.proxy.separator) if r.strip()]
return [r for r in roles if r in config.auth.roles]
def _ws_is_unrestricted(ws: Any, config: FrigateConfig) -> bool:
"""True when the connection has unrestricted camera access.
Mirrors the policy in ``frigate.output.ws_auth``: admin or any role with
an empty allow-list grants full access.
"""
roles = _ws_valid_roles(ws, config)
if not roles:
return False
roles_dict = config.auth.roles
return any(r == "admin" or not roles_dict.get(r) for r in roles)
def _ws_allowed_cameras(ws: Any, config: FrigateConfig) -> set[str]:
"""Return the union of cameras this connection may access across its roles."""
roles = _ws_valid_roles(ws, config)
if not roles:
return set()
all_cameras = set(config.cameras.keys())
allowed: set[str] = set()
for role in roles:
if role == "admin" or not config.auth.roles.get(role):
return all_cameras
allowed.update(User.get_allowed_cameras(role, config.auth.roles, all_cameras))
return allowed
def _wrap_envelope(topic: str, inner_payload: Any) -> str:
"""Re-serialize a (topic, payload) message after payload reshaping.
Frigate's wire format keeps payloads as JSON-encoded strings inside the
outer envelope, mirroring what producers send today.
"""
return json.dumps({"topic": topic, "payload": json.dumps(inner_payload)})
def _materialize_for_ws(
ws: Any,
topic: str,
full_message: str,
scope: tuple[str, Any],
parsed_payload: Any,
config: FrigateConfig,
) -> str | None:
"""Return the JSON string to deliver to ``ws``, or None to skip it."""
kind, extra = scope
has_role = _ws_role_header(ws) is not None
if kind == "drop":
return None
if kind == "global":
# Globals still require an authenticated connection. Missing role
# falls back to viewer semantics (matching the inbound rule).
return full_message
# Beyond globals, an authenticated role header is required (fail-closed).
if not has_role:
return None
if kind == "unrestricted_only":
return full_message if _ws_is_unrestricted(ws, config) else None
if kind == "camera":
return full_message if ws_has_camera_access(ws, extra, config) else None
if kind == "payload_camera":
camera = _extract_payload_camera(parsed_payload, extra)
if camera is None:
return None
return full_message if ws_has_camera_access(ws, camera, config) else None
if kind == "reshape_by_camera_key":
if _ws_is_unrestricted(ws, config):
return full_message
if not isinstance(parsed_payload, dict):
return None
allowed = _ws_allowed_cameras(ws, config)
filtered = {cam: data for cam, data in parsed_payload.items() if cam in allowed}
if not filtered:
return None
return _wrap_envelope(topic, filtered)
if kind == "reshape_job_state":
if _ws_is_unrestricted(ws, config):
return full_message
if not isinstance(parsed_payload, dict):
return None
allowed = _ws_allowed_cameras(ws, config)
filtered_jobs: dict[str, Any] = {}
for job_type, job_payload in parsed_payload.items():
scoped = _scope_job_entry_to_allowed(job_payload, allowed)
if scoped is not None:
filtered_jobs[job_type] = scoped
if not filtered_jobs:
return None
return _wrap_envelope(topic, filtered_jobs)
if kind == "reshape_stats":
if _ws_is_unrestricted(ws, config):
return full_message
if not isinstance(parsed_payload, dict):
return None
allowed = _ws_allowed_cameras(ws, config)
cameras_block = parsed_payload.get("cameras")
if isinstance(cameras_block, dict):
filtered_cameras = {
name: data for name, data in cameras_block.items() if name in allowed
}
reshaped = dict(parsed_payload)
reshaped["cameras"] = filtered_cameras
return _wrap_envelope(topic, reshaped)
return full_message
return None
class WebSocket(WebSocket_): # type: ignore[misc]
def unhandled_error(self, error: Any) -> None:
@@ -448,7 +49,6 @@ class WebSocketClient(Communicator):
class _WebSocketHandler(WebSocket):
receiver = self._dispatcher
role_separator = self.config.proxy.separator or ","
def received_message(self, message: WebSocket.received_message) -> None: # type: ignore[name-defined]
try:
@@ -463,25 +63,11 @@ class WebSocketClient(Communicator):
)
return
topic = json_message["topic"]
# Authorization check (skip when environ is None — direct internal connection)
role_header = (
self.environ.get("HTTP_REMOTE_ROLE") if self.environ else None
logger.debug(
f"Publishing mqtt message from websockets at {json_message['topic']}."
)
if self.environ is not None and not _check_ws_authorization(
topic, role_header, self.role_separator
):
logger.warning(
"Blocked unauthorized WebSocket message: topic=%s, role=%s",
topic,
role_header,
)
return
logger.debug(f"Publishing mqtt message from websockets at {topic}.")
self.receiver(
topic,
json_message["topic"],
json_message["payload"],
)
@@ -501,10 +87,6 @@ class WebSocketClient(Communicator):
self.websocket_thread.start()
def publish(self, topic: str, payload: Any, _: bool = False) -> None:
if self.websocket_server is None:
logger.debug("Skipping message, websocket not connected yet")
return
try:
ws_message = json.dumps(
{
@@ -517,42 +99,14 @@ class WebSocketClient(Communicator):
logger.debug(f"payload for {topic} wasn't text. Skipping...")
return
all_cameras = set(self.config.cameras.keys())
all_zones = _collect_zone_names(self.config)
scope = _classify_outbound(topic, all_cameras, all_zones)
if scope[0] == "drop":
if self.websocket_server is None:
logger.debug("Skipping message, websocket not connected yet")
return
# Pre-parse payload once for topics that need to read its contents.
parsed_payload: Any = None
if scope[0] in (
"payload_camera",
"reshape_by_camera_key",
"reshape_job_state",
"reshape_stats",
):
parsed_payload = _parse_json_payload(payload)
if parsed_payload is None:
# malformed payload — fail closed
return
manager = self.websocket_server.manager
with manager.lock:
websockets = list(manager.websockets.values())
for ws in websockets:
if getattr(ws, "terminated", False):
continue
message = _materialize_for_ws(
ws, topic, ws_message, scope, parsed_payload, self.config
)
if message is None:
continue
try:
ws.send(message)
except (ConnectionResetError, BrokenPipeError, ValueError):
pass
try:
self.websocket_server.manager.broadcast(ws_message)
except ConnectionResetError:
pass
def stop(self) -> None:
if self.websocket_server is not None:
+1 -1
View File
@@ -76,7 +76,7 @@ class CameraConfig(FrigateBaseModel):
# Options with global fallback
audio: AudioConfig = Field(
default_factory=AudioConfig,
title="Audio detection",
title="Audio events",
description="Settings for audio-based event detection for this camera.",
)
audio_transcription: CameraAudioTranscriptionConfig = Field(
+2 -1
View File
@@ -41,7 +41,8 @@ class GenAIConfig(FrigateBaseModel):
title="Model",
description="The model to use from the provider for generating descriptions or summaries.",
)
provider: GenAIProviderEnum = Field(
provider: GenAIProviderEnum | None = Field(
default=None,
title="Provider",
description="The GenAI provider to use (for example: ollama, gemini, openai).",
)
-3
View File
@@ -121,10 +121,7 @@ class CameraConfigUpdateSubscriber:
elif update_type == CameraConfigUpdateEnum.objects:
config.objects = updated_config
elif update_type == CameraConfigUpdateEnum.record:
old_enabled_in_config = config.record.enabled_in_config
config.record = updated_config
if old_enabled_in_config != updated_config.enabled_in_config:
config.recreate_ffmpeg_cmds()
elif update_type == CameraConfigUpdateEnum.review:
config.review = updated_config
elif update_type == CameraConfigUpdateEnum.review_genai:
+9 -14
View File
@@ -26,11 +26,6 @@ class EnrichmentsDeviceEnum(str, Enum):
CPU = "CPU"
class ModelSizeEnum(str, Enum):
small = "small"
large = "large"
class TriggerType(str, Enum):
THUMBNAIL = "thumbnail"
DESCRIPTION = "description"
@@ -58,13 +53,13 @@ class AudioTranscriptionConfig(FrigateBaseModel):
title="Transcription language",
description="Language code used for transcription/translation (for example 'en' for English). See https://whisper-api.com/docs/languages/ for supported language codes.",
)
device: EnrichmentsDeviceEnum = Field(
device: Optional[EnrichmentsDeviceEnum] = Field(
default=EnrichmentsDeviceEnum.CPU,
title="Transcription device",
description="Device key (CPU/GPU) to run the transcription model on. Only NVIDIA CUDA GPUs are currently supported for transcription.",
)
model_size: ModelSizeEnum = Field(
default=ModelSizeEnum.small,
model_size: str = Field(
default="small",
title="Model size",
description="Model size to use for offline audio event transcription.",
)
@@ -194,8 +189,8 @@ class SemanticSearchConfig(FrigateBaseModel):
return v
return v
model_size: ModelSizeEnum = Field(
default=ModelSizeEnum.small,
model_size: str = Field(
default="small",
title="Model size",
description="Select model size; 'small' runs on CPU and 'large' typically requires GPU.",
)
@@ -258,8 +253,8 @@ class FaceRecognitionConfig(FrigateBaseModel):
title="Enable face recognition",
description="Enable or disable face recognition for all cameras; can be overridden per-camera.",
)
model_size: ModelSizeEnum = Field(
default=ModelSizeEnum.small,
model_size: str = Field(
default="small",
title="Model size",
description="Model size to use for face embeddings (small/large); larger may require GPU.",
)
@@ -340,8 +335,8 @@ class LicensePlateRecognitionConfig(FrigateBaseModel):
title="Enable LPR",
description="Enable or disable license plate recognition for all cameras; can be overridden per-camera.",
)
model_size: ModelSizeEnum = Field(
default=ModelSizeEnum.small,
model_size: str = Field(
default="small",
title="Model size",
description="Model size used for text detection/recognition. Most users should use 'small'.",
)
+18 -45
View File
@@ -1,6 +1,5 @@
from __future__ import annotations
import io
import json
import logging
import os
@@ -26,6 +25,7 @@ from frigate.plus import PlusApi
from frigate.util.builtin import (
deep_merge,
get_ffmpeg_arg_list,
load_labels,
)
from frigate.util.config import (
CURRENT_CONFIG_VERSION,
@@ -80,41 +80,17 @@ logger = logging.getLogger(__name__)
yaml = YAML()
# Pydantic field default applied when an existing config omits `detectors:`.
# Kept as cpu tflite for backwards compatibility with 0.17 configs.
DEFAULT_DETECTORS = {"cpu": {"type": "cpu"}}
# Used by the openvino branch below and rendered into the new-config YAML
# template so first-time setups default to openvino on CPU.
DEFAULT_MODEL = {
"width": 300,
"height": 300,
"input_tensor": "nhwc",
"input_pixel_format": "bgr",
"path": "/openvino-model/ssdlite_mobilenet_v2.xml",
"labelmap_path": "/openvino-model/coco_91cl_bkgr.txt",
}
NEW_CONFIG_DETECTORS = {"ov": {"type": "openvino", "device": "CPU"}}
DEFAULT_DETECT_DIMENSIONS = {"width": 1280, "height": 720}
def _render_default_yaml(data: dict) -> str:
buf = io.StringIO()
_yaml_writer = YAML()
_yaml_writer.indent(mapping=2, sequence=4, offset=2)
_yaml_writer.dump(data, buf)
return buf.getvalue()
DEFAULT_CONFIG = f"""
mqtt:
enabled: False
{_render_default_yaml({"detectors": NEW_CONFIG_DETECTORS, "model": DEFAULT_MODEL})}
cameras: {{}} # No cameras defined, UI wizard should be used
version: {CURRENT_CONFIG_VERSION}
"""
DEFAULT_DETECTORS = {"cpu": {"type": "cpu"}}
DEFAULT_DETECT_DIMENSIONS = {"width": 1280, "height": 720}
# stream info handler
stream_info_retriever = StreamInfoRetriever()
@@ -477,7 +453,7 @@ class FrigateConfig(FrigateBaseModel):
cameras: Dict[str, CameraConfig] = Field(title="Cameras", description="Cameras")
audio: AudioConfig = Field(
default_factory=AudioConfig,
title="Audio detection",
title="Audio events",
description="Settings for audio-based event detection for all cameras; can be overridden per-camera.",
)
birdseye: BirdseyeConfig = Field(
@@ -629,22 +605,26 @@ class FrigateConfig(FrigateBaseModel):
# set default min_score for object attributes
for attribute in self.model.all_attributes:
existing = self.objects.filters.get(attribute)
if existing is None:
if not self.objects.filters.get(attribute):
self.objects.filters[attribute] = FilterConfig(min_score=0.7)
elif "min_score" not in existing.model_fields_set:
existing.min_score = 0.7
elif self.objects.filters[attribute].min_score == 0.5:
self.objects.filters[attribute].min_score = 0.7
# auto detect hwaccel args
if self.ffmpeg.hwaccel_args == "auto":
self.ffmpeg.hwaccel_args = auto_detect_hwaccel()
# Populate global audio filters from listen. Existing user-defined
# entries for labels not in listen are preserved but unused at runtime.
# Populate global audio filters for all audio labels
all_audio_labels = {
label
for label in load_labels("/audio-labelmap.txt", prefill=521).values()
if label
}
if self.audio.filters is None:
self.audio.filters = {}
for key in sorted(set(self.audio.listen) - self.audio.filters.keys()):
for key in sorted(all_audio_labels - self.audio.filters.keys()):
self.audio.filters[key] = AudioFilterConfig()
self.audio.filters = dict(sorted(self.audio.filters.items()))
@@ -699,9 +679,6 @@ class FrigateConfig(FrigateBaseModel):
model_config["path"] = "/cpu_model.tflite"
elif detector_config.type == "edgetpu":
model_config["path"] = "/edgetpu_model.tflite"
elif detector_config.type == "openvino":
for default_key, default_value in DEFAULT_MODEL.items():
model_config.setdefault(default_key, default_value)
model = ModelConfig.model_validate(model_config)
model.check_and_load_plus_model(self.plus_api, detector_config.type)
@@ -836,9 +813,7 @@ class FrigateConfig(FrigateBaseModel):
if camera_config.audio.filters is None:
camera_config.audio.filters = {}
for key in sorted(
set(camera_config.audio.listen) - camera_config.audio.filters.keys()
):
for key in sorted(all_audio_labels - camera_config.audio.filters.keys()):
camera_config.audio.filters[key] = AudioFilterConfig()
camera_config.audio.filters = dict(
@@ -860,9 +835,7 @@ class FrigateConfig(FrigateBaseModel):
if mask_config:
coords = mask_config.coordinates
relative_coords = get_relative_coordinates(
coords,
camera_config.frame_shape,
camera_name=camera_config.name,
coords, camera_config.frame_shape
)
# Create a new ObjectMaskConfig with raw_coordinates set
processed_global_masks[mask_id] = ObjectMaskConfig(
+2 -2
View File
@@ -25,8 +25,8 @@ class StatsConfig(FrigateBaseModel):
)
intel_gpu_device: Optional[str] = Field(
default=None,
title="Intel GPU device",
description="PCI bus address or DRM device path (e.g. /dev/dri/card1) used to pin Intel GPU stats to a specific device when multiple are present.",
title="SR-IOV device",
description="Device identifier used when treating Intel GPUs as SR-IOV to fix GPU stats.",
)
@@ -1073,6 +1073,10 @@ class LicensePlateProcessingMixin:
top_score = score
top_box = bbox
if score > top_score:
top_score = score
top_box = bbox
# Return the top scoring bounding box if found
if top_box is not None:
# expand box by 5% to help with OCR
@@ -1088,6 +1092,9 @@ class LicensePlateProcessingMixin:
]
).clip(0, [input.shape[1], input.shape[0]] * 2)
logger.debug(
f"{camera}: Found license plate. Bounding box: {expanded_box.astype(int)}"
)
return tuple(int(x) for x in expanded_box) # type: ignore[return-value]
else:
return None # No detection above the threshold
@@ -1353,8 +1360,8 @@ class LicensePlateProcessingMixin:
)
# check that license plate is valid
# quadruple the value because we've doubled both dimensions of the car
if license_plate_area < self.config.cameras[camera].lpr.min_area * 4:
# double the value because we've doubled the size of the car
if license_plate_area < self.config.cameras[camera].lpr.min_area * 2:
logger.debug(f"{camera}: License plate is less than min_area")
return
@@ -1458,7 +1465,6 @@ class LicensePlateProcessingMixin:
license_plate_frame,
)
logger.debug(f"{camera}: Found license plate. Bounding box: {list(plate_box)}")
logger.debug(f"{camera}: Running plate recognition for id: {id}.")
# run detection, returns results sorted by confidence, best first
@@ -269,9 +269,7 @@ class ObjectDescriptionProcessor(PostProcessorApi):
if event.has_snapshot and camera_config.objects.genai.use_snapshot:
snapshot_image = self._read_and_crop_snapshot(event)
if not snapshot_image:
self.cleanup_event(event_id)
return
num_thumbnails = len(self.tracked_events.get(event_id, []))
+20 -9
View File
@@ -2,7 +2,7 @@ from typing import Annotated
from pydantic import BaseModel, ConfigDict, Field, StringConstraints
ObservationItem = Annotated[str, StringConstraints(min_length=20, max_length=200)]
ObservationItem = Annotated[str, StringConstraints(min_length=20, max_length=160)]
class ReviewMetadata(BaseModel):
@@ -11,22 +11,33 @@ class ReviewMetadata(BaseModel):
observations: list[ObservationItem] = Field(
...,
min_length=3,
max_length=8,
description="Enumerate the significant observations across all frames, in chronological order.",
max_length=15,
description=(
"Enumerate the significant observations across all frames, in "
"chronological order, BEFORE composing the scene narrative. "
"Include the very start of the activity — for example, a vehicle "
"entering the frame or pulling into the driveway — even if it "
"lasts only a few frames and the rest of the clip is dominated "
"by a longer activity. Include each arrival, departure, motion "
"event, object handled, and notable change in position or state. "
"Each item is a single concrete fact written as a complete "
"sentence. Do not summarize, interpret, or assign meaning here — "
"that belongs in the scene field."
),
)
title: str = Field(
max_length=80,
description="Under 10 words. Name the apparent purpose or outcome of the activity together with the location involved. Do not narrate or list the sequence of actions step by step.",
)
scene: str = Field(
min_length=150,
max_length=600,
description="A chronological narrative of what happens from start to finish, drawing directly from the items in observations.",
)
title: str = Field(
max_length=80,
description="Title for the activity.",
)
shortSummary: str = Field(
min_length=70,
max_length=140,
description="A brief summary for the activity.",
max_length=120,
description="A brief 2-sentence summary of the scene, suitable for notifications.",
)
confidence: float = Field(
ge=0.0,
+7 -3
View File
@@ -229,10 +229,9 @@ class FaceRealTimeProcessor(RealTimeProcessorApi):
logger.debug(f"No person box available for {id}")
return
# YuNet (cv2.FaceDetectorYN) is trained on BGR
bgr = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_I420)
rgb = cv2.cvtColor(frame, cv2.COLOR_YUV2RGB_I420)
left, top, right, bottom = person_box
person = bgr[top:bottom, left:right]
person = rgb[top:bottom, left:right]
face_box = self.__detect_face(person, self.face_config.detection_threshold)
if not face_box:
@@ -251,6 +250,11 @@ class FaceRealTimeProcessor(RealTimeProcessorApi):
)
return
try:
face_frame = cv2.cvtColor(face_frame, cv2.COLOR_RGB2BGR)
except Exception as e:
logger.debug(f"Failed to convert face frame color for {id}: {e}")
return
else:
# don't run for object without attributes
if not obj_data.get("current_attributes"):
+186 -108
View File
@@ -1,15 +1,10 @@
"""Debug replay camera management for replaying recordings with detection overlays.
The startup work (ffmpeg concat + camera config publish) lives in
frigate.jobs.debug_replay. This module owns only session presence
(active), session metadata, and post-session cleanup.
"""
"""Debug replay camera management for replaying recordings with detection overlays."""
import logging
import os
import shutil
import subprocess as sp
import threading
import time
from ruamel.yaml import YAML
@@ -26,15 +21,7 @@ from frigate.const import (
REPLAY_DIR,
THUMB_DIR,
)
from frigate.jobs.debug_replay import (
JOB_TYPE as DEBUG_REPLAY_JOB_TYPE,
)
from frigate.jobs.debug_replay import (
cancel_debug_replay_job,
wait_for_runner,
)
from frigate.jobs.export import JobStatePublisher
from frigate.types import JobStatusTypesEnum
from frigate.models import Recordings
from frigate.util.camera_cleanup import cleanup_camera_db, cleanup_camera_files
from frigate.util.config import find_config_file
@@ -42,14 +29,7 @@ logger = logging.getLogger(__name__)
class DebugReplayManager:
"""Owns the lifecycle pointers for a single debug replay session.
A session exists from the moment mark_starting is called (synchronously,
inside the API handler) until clear_session runs (on success cleanup,
failure, or stop). The active property is the source of truth that the
status bar consumes broader than the startup job, which only covers the
preparing_clip / starting_camera window.
"""
"""Manages a single debug replay session."""
def __init__(self) -> None:
self._lock = threading.Lock()
@@ -58,70 +38,147 @@ class DebugReplayManager:
self.clip_path: str | None = None
self.start_ts: float | None = None
self.end_ts: float | None = None
self._job_state_publisher = JobStatePublisher()
@property
def active(self) -> bool:
"""True from mark_starting until clear_session."""
"""Whether a replay session is currently active."""
return self.replay_camera_name is not None
def mark_starting(
def start(
self,
source_camera: str,
replay_camera_name: str,
start_ts: float,
end_ts: float,
) -> None:
"""Synchronously claim the session before the job runner starts.
Called inside the API handler so the status bar sees active=True
immediately, before the worker thread does any ffmpeg work.
"""
with self._lock:
self.replay_camera_name = replay_camera_name
self.source_camera = source_camera
self.start_ts = start_ts
self.end_ts = end_ts
self.clip_path = None
def mark_session_ready(self, clip_path: str) -> None:
"""Record the on-disk clip path after the camera has been published."""
with self._lock:
self.clip_path = clip_path
def clear_session(self) -> None:
"""Reset session pointers without publishing camera removal.
Used by the job runner on failure paths. stop() does the camera
teardown plus this clear in one step.
"""
with self._lock:
self._clear_locked()
def _clear_locked(self) -> None:
self.replay_camera_name = None
self.source_camera = None
self.clip_path = None
self.start_ts = None
self.end_ts = None
def publish_camera(
self,
source_camera: str,
replay_name: str,
clip_path: str,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
) -> None:
"""Build the in-memory replay camera config and publish the add event.
) -> str:
"""Start a debug replay session.
Called by the job runner during the starting_camera phase.
Args:
source_camera: Name of the source camera to replay
start_ts: Start timestamp
end_ts: End timestamp
frigate_config: Current Frigate configuration
config_publisher: Publisher for camera config updates
Returns:
The replay camera name
Raises:
ValueError: If a session is already active or parameters are invalid
RuntimeError: If clip generation fails
"""
with self._lock:
return self._start_locked(
source_camera, start_ts, end_ts, frigate_config, config_publisher
)
def _start_locked(
self,
source_camera: str,
start_ts: float,
end_ts: float,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
) -> str:
if self.active:
raise ValueError("A replay session is already active")
if source_camera not in frigate_config.cameras:
raise ValueError(f"Camera '{source_camera}' not found")
if end_ts <= start_ts:
raise ValueError("End time must be after start time")
# Query recordings for the source camera in the time range
recordings = (
Recordings.select(
Recordings.path,
Recordings.start_time,
Recordings.end_time,
)
.where(
Recordings.start_time.between(start_ts, end_ts)
| Recordings.end_time.between(start_ts, end_ts)
| ((start_ts > Recordings.start_time) & (end_ts < Recordings.end_time))
)
.where(Recordings.camera == source_camera)
.order_by(Recordings.start_time.asc())
)
if not recordings.count():
raise ValueError(
f"No recordings found for camera '{source_camera}' in the specified time range"
)
# Create replay directory
os.makedirs(REPLAY_DIR, exist_ok=True)
# Generate replay camera name
replay_name = f"{REPLAY_CAMERA_PREFIX}{source_camera}"
# Build concat file for ffmpeg
concat_file = os.path.join(REPLAY_DIR, f"{replay_name}_concat.txt")
clip_path = os.path.join(REPLAY_DIR, f"{replay_name}.mp4")
with open(concat_file, "w") as f:
for recording in recordings:
f.write(f"file '{recording.path}'\n")
# Concatenate recordings into a single clip with -c copy (fast)
ffmpeg_cmd = [
frigate_config.ffmpeg.ffmpeg_path,
"-hide_banner",
"-y",
"-f",
"concat",
"-safe",
"0",
"-i",
concat_file,
"-c",
"copy",
"-movflags",
"+faststart",
clip_path,
]
logger.info(
"Generating replay clip for %s (%.1f - %.1f)",
source_camera,
start_ts,
end_ts,
)
try:
result = sp.run(
ffmpeg_cmd,
capture_output=True,
text=True,
timeout=120,
)
if result.returncode != 0:
logger.error("FFmpeg error: %s", result.stderr)
raise RuntimeError(
f"Failed to generate replay clip: {result.stderr[-500:]}"
)
except sp.TimeoutExpired:
raise RuntimeError("Clip generation timed out")
finally:
# Clean up concat file
if os.path.exists(concat_file):
os.remove(concat_file)
if not os.path.exists(clip_path):
raise RuntimeError("Clip file was not created")
# Build camera config dict for the replay camera
source_config = frigate_config.cameras[source_camera]
camera_dict = self._build_camera_config_dict(
source_config, replay_name, clip_path
)
# Build an in-memory config with the replay camera added
config_file = find_config_file()
yaml_parser = YAML()
with open(config_file, "r") as f:
@@ -134,64 +191,75 @@ class DebugReplayManager:
try:
new_config = FrigateConfig.parse_object(config_data)
except Exception as e:
raise RuntimeError(f"Failed to validate replay camera config: {e}") from e
raise RuntimeError(f"Failed to validate replay camera config: {e}")
# Update the running config
frigate_config.cameras[replay_name] = new_config.cameras[replay_name]
# Publish the add event
config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.add, replay_name),
new_config.cameras[replay_name],
)
# Store session state
self.replay_camera_name = replay_name
self.source_camera = source_camera
self.clip_path = clip_path
self.start_ts = start_ts
self.end_ts = end_ts
logger.info("Debug replay started: %s -> %s", source_camera, replay_name)
return replay_name
def stop(
self,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
) -> None:
"""Cancel any in-flight startup job and tear down the active session.
"""Stop the active replay session and clean up all artifacts.
Safe to call when no session is active (no-op with a warning).
Args:
frigate_config: Current Frigate configuration
config_publisher: Publisher for camera config updates
"""
cancel_debug_replay_job()
wait_for_runner(timeout=2.0)
with self._lock:
if not self.active:
logger.warning("No active replay session to stop")
return
self._stop_locked(frigate_config, config_publisher)
replay_name = self.replay_camera_name
source_camera = self.source_camera
def _stop_locked(
self,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
) -> None:
if not self.active:
logger.warning("No active replay session to stop")
return
# Only publish remove if the camera was actually added to the live
# config (i.e. the runner reached the starting_camera phase).
if replay_name is not None and replay_name in frigate_config.cameras:
config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.remove, replay_name),
frigate_config.cameras[replay_name],
)
replay_name = self.replay_camera_name
if replay_name is not None:
self._cleanup_db(replay_name)
self._cleanup_files(replay_name)
self._job_state_publisher.publish(
{
"id": "stopped",
"job_type": DEBUG_REPLAY_JOB_TYPE,
"status": JobStatusTypesEnum.cancelled,
"start_time": None,
"end_time": time.time(),
"error_message": None,
"results": {
"source_camera": source_camera,
"replay_camera_name": replay_name,
},
}
# Publish remove event so subscribers stop and remove from their config
if replay_name in frigate_config.cameras:
config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.remove, replay_name),
frigate_config.cameras[replay_name],
)
# Do NOT pop here — let subscribers handle removal from the shared
# config dict when they process the ZMQ message to avoid race conditions
self._clear_locked()
# Defensive DB cleanup
self._cleanup_db(replay_name)
logger.info("Debug replay stopped and cleaned up: %s", replay_name)
# Remove filesystem artifacts
self._cleanup_files(replay_name)
# Reset state
self.replay_camera_name = None
self.source_camera = None
self.clip_path = None
self.start_ts = None
self.end_ts = None
logger.info("Debug replay stopped and cleaned up: %s", replay_name)
def _build_camera_config_dict(
self,
@@ -199,7 +267,16 @@ class DebugReplayManager:
replay_name: str,
clip_path: str,
) -> dict:
"""Build a camera config dictionary for the replay camera."""
"""Build a camera config dictionary for the replay camera.
Args:
source_config: Source camera's CameraConfig
replay_name: Name for the replay camera
clip_path: Path to the replay clip file
Returns:
Camera config as a dictionary
"""
# Extract detect config (exclude computed fields)
detect_dict = source_config.detect.model_dump(
exclude={"min_initialized", "max_disappeared", "enabled_in_config"}
@@ -234,6 +311,7 @@ class DebugReplayManager:
zone_dump = zone_config.model_dump(
exclude={"contour", "color"}, exclude_defaults=True
)
# Always include required fields
zone_dump.setdefault("coordinates", zone_config.coordinates)
zones_dict[zone_name] = zone_dump
+3 -19
View File
@@ -79,11 +79,7 @@ def is_openvino_gpu_npu_available() -> bool:
available_devices = get_openvino_available_devices()
# Check for GPU, NPU, or other acceleration devices (excluding CPU)
acceleration_devices = ["GPU", "MYRIAD", "NPU", "GNA", "HDDL"]
return any(
avail_dev == accel_dev or avail_dev.startswith(accel_dev + ".")
for avail_dev in available_devices
for accel_dev in acceleration_devices
)
return any(device in available_devices for device in acceleration_devices)
class BaseModelRunner(ABC):
@@ -136,6 +132,7 @@ class ONNXModelRunner(BaseModelRunner):
return model_type in [
EnrichmentModelTypeEnum.paddleocr.value,
EnrichmentModelTypeEnum.jina_v2.value,
EnrichmentModelTypeEnum.arcface.value,
ModelTypeEnum.rfdetr.value,
ModelTypeEnum.dfine.value,
]
@@ -282,13 +279,6 @@ class OpenVINOModelRunner(BaseModelRunner):
EnrichmentModelTypeEnum.arcface.value,
]
@staticmethod
def is_detection_model(model_type: str) -> bool:
# Import here to avoid circular imports
from frigate.detectors.detector_config import ModelTypeEnum
return model_type in [m.value for m in ModelTypeEnum]
def __init__(self, model_path: str, device: str, model_type: str, **kwargs):
self.model_path = model_path
self.device = device
@@ -317,15 +307,9 @@ class OpenVINOModelRunner(BaseModelRunner):
# Apply performance optimization
self.ov_core.set_property(device, {"PERF_COUNT": "NO"})
if device in ["GPU", "AUTO", "NPU"]:
if device in ["GPU", "AUTO"]:
self.ov_core.set_property(device, {"PERFORMANCE_HINT": "LATENCY"})
if device == "NPU" and OpenVINOModelRunner.is_detection_model(model_type):
try:
self.ov_core.set_property(device, {"NPU_TURBO": "YES"})
except Exception as e:
logger.debug(f"NPU_TURBO not supported by driver: {e}")
# Compile model
self.compiled_model = self.ov_core.compile_model(
model=model_path, device_name=device
-6
View File
@@ -52,12 +52,6 @@ class OvDetector(DetectionApi):
self.h = detector_config.model.height
self.w = detector_config.model.width
logger.info(
"Loading OpenVINO model %s on device %s",
detector_config.model.path,
detector_config.device,
)
self.runner = OpenVINOModelRunner(
model_path=detector_config.model.path,
device=detector_config.device,
+3 -13
View File
@@ -60,11 +60,7 @@ from frigate.data_processing.real_time.license_plate import (
)
from frigate.data_processing.types import DataProcessorMetrics, PostProcessDataEnum
from frigate.db.sqlitevecq import SqliteVecQueueDatabase
from frigate.events.types import (
EventStateEnum,
EventTypeEnum,
RegenerateDescriptionEnum,
)
from frigate.events.types import EventTypeEnum, RegenerateDescriptionEnum
from frigate.genai import GenAIClientManager
from frigate.models import Event, Recordings, ReviewSegment, Trigger
from frigate.types import TrackedObjectUpdateTypesEnum
@@ -232,7 +228,7 @@ class EmbeddingMaintainer(threading.Thread):
)
)
if any(
if self.config.audio_transcription.enabled and any(
c.enabled_in_config and c.audio_transcription.enabled
for c in self.config.cameras.values()
):
@@ -439,7 +435,7 @@ class EmbeddingMaintainer(threading.Thread):
if update is None:
return
source_type, event_type, camera, frame_name, data = update
source_type, _, camera, frame_name, data = update
logger.debug(
f"Received update - source_type: {source_type}, camera: {camera}, data label: {data.get('label') if data else 'None'}"
@@ -489,12 +485,6 @@ class EmbeddingMaintainer(threading.Thread):
for processor in self.post_processors:
if isinstance(processor, ObjectDescriptionProcessor):
# skip end events — _process_finalized handles them via event_end_subscriber.
# processing them here can re-create tracked_events entries after cleanup
# when the event_subscriber queue is backlogged behind event_end_subscriber.
if event_type == EventStateEnum.end:
continue
processor.process_data(
{
"camera": camera,
+15 -40
View File
@@ -84,6 +84,7 @@ class AudioProcessor(FrigateProcess):
def __init__(
self,
config: FrigateConfig,
cameras: list[CameraConfig],
camera_metrics: DictProxy,
stop_event: MpEvent,
):
@@ -92,18 +93,16 @@ class AudioProcessor(FrigateProcess):
)
self.camera_metrics = camera_metrics
self.cameras = cameras
self.config = config
def run(self) -> None:
self.pre_run_setup(self.config.logger)
audio_threads: dict[str, AudioEventMaintainer] = {}
audio_threads: list[AudioEventMaintainer] = []
threading.current_thread().name = "process:audio_manager"
if any(
c.enabled_in_config and c.audio_transcription.enabled
for c in self.config.cameras.values()
):
if self.config.audio_transcription.enabled:
self.transcription_model_runner: AudioTranscriptionModelRunner | None = (
AudioTranscriptionModelRunner(
self.config.audio_transcription.device or "AUTO",
@@ -113,56 +112,32 @@ class AudioProcessor(FrigateProcess):
else:
self.transcription_model_runner = None
config_subscriber = CameraConfigUpdateSubscriber(
self.config,
self.config.cameras,
[
CameraConfigUpdateEnum.add,
CameraConfigUpdateEnum.audio,
CameraConfigUpdateEnum.ffmpeg,
],
)
if len(self.cameras) == 0:
return
def spawn_if_needed(camera: CameraConfig) -> None:
name = camera.name
if name is None or name in audio_threads:
return
if not camera.enabled or not camera.audio.enabled:
return
# ffmpeg update may not have arrived yet; wait for next poll
if not any("audio" in i.roles for i in camera.ffmpeg.inputs):
return
thread = AudioEventMaintainer(
for camera in self.cameras:
audio_thread = AudioEventMaintainer(
camera,
self.config,
self.camera_metrics,
self.transcription_model_runner,
self.stop_event, # type: ignore[arg-type]
)
audio_threads[name] = thread
thread.start()
self.logger.info(f"Audio maintainer started for {name}")
for camera in self.config.cameras.values():
spawn_if_needed(camera)
audio_threads.append(audio_thread)
audio_thread.start()
self.logger.info(f"Audio processor started (pid: {self.pid})")
# poll for newly added cameras or cameras flipped to audio.enabled at runtime
while not self.stop_event.wait(timeout=1.0):
config_subscriber.check_for_updates()
for camera in self.config.cameras.values():
spawn_if_needed(camera)
while not self.stop_event.wait():
pass
config_subscriber.stop()
for thread in audio_threads.values():
for thread in audio_threads:
thread.join(1)
if thread.is_alive():
self.logger.info(f"Waiting for thread {thread.name:s} to exit")
thread.join(10)
for thread in audio_threads.values():
for thread in audio_threads:
if thread.is_alive():
self.logger.warning(f"Thread {thread.name} is still alive")
@@ -209,7 +184,7 @@ class AudioEventMaintainer(threading.Thread):
self.detection_publisher = DetectionPublisher(DetectionTypeEnum.audio.value)
if (
self.camera_config.audio_transcription.enabled
self.config.audio_transcription.enabled
and self.audio_transcription_model_runner is not None
):
# init the transcription processor for this camera
+150 -34
View File
@@ -1,5 +1,6 @@
"""Generative AI module for Frigate."""
import datetime
import importlib
import json
import logging
@@ -8,18 +9,13 @@ import re
from typing import Any, Callable, Optional
import numpy as np
from playhouse.shortcuts import model_to_dict
from pydantic import ValidationError
from frigate.config import CameraConfig, GenAIConfig, GenAIProviderEnum
from frigate.const import CLIPS_DIR
from frigate.data_processing.post.types import ReviewMetadata
from frigate.genai.manager import GenAIClientManager
from frigate.genai.prompts import (
build_object_description_prompt,
build_review_description_prompt,
build_review_description_response_format,
build_review_summary_prompt,
)
from frigate.models import Event
logger = logging.getLogger(__name__)
@@ -65,14 +61,74 @@ class GenAIClient:
activity_context_prompt: str,
) -> ReviewMetadata | None:
"""Generate a description for the review item activity."""
context_prompt = build_review_description_prompt(
review_data,
thumbnails,
concerns,
preferred_language,
activity_context_prompt,
)
def get_concern_prompt() -> str:
if concerns:
concern_list = "\n - ".join(concerns)
return f"""- `other_concerns` (list of strings): Include a list of any of the following concerns that are occurring:
- {concern_list}"""
else:
return ""
def get_language_prompt() -> str:
if preferred_language:
return f"Provide your answer in {preferred_language}"
else:
return ""
def get_objects_list() -> str:
if review_data["unified_objects"]:
return "\n- " + "\n- ".join(review_data["unified_objects"])
else:
return "\n- (No objects detected)"
context_prompt = f"""
Your task is to analyze a sequence of images taken in chronological order from a security camera.
## Normal Activity Patterns for This Property
{activity_context_prompt}
## Task Instructions
Describe the scene based on observable actions and movements, evaluate the activity against the Activity Indicators above, and assign a potential_threat_level (0, 1, or 2) by applying the threat level indicators consistently.
## Analysis Guidelines
When forming your description:
- **CRITICAL: Only describe objects explicitly listed in "Objects in Scene" below.** Do not infer or mention additional people, vehicles, or objects not present in this list, even if visual patterns suggest them. If only a car is listed, do not describe a person interacting with it unless "person" is also in the objects list.
- **Only describe actions actually visible in the frames.** Do not assume or infer actions that you don't observe happening. If someone walks toward furniture but you never see them sit, do not say they sat. Stick to what you can see across the sequence.
- Describe what you observe: actions, movements, interactions with objects and the environment. Include any observable environmental changes (e.g., lighting changes triggered by activity).
- Note visible details such as clothing, items being carried or placed, tools or equipment present, and how they interact with the property or objects.
- Consider the full sequence chronologically: what happens from start to finish, how duration and actions relate to the location and objects involved.
- **Use the actual timestamp provided in "Activity started at"** below for time of day contextdo not infer time from image brightness or darkness. Unusual hours (late night/early morning) should increase suspicion when the observable behavior itself appears questionable. However, recognize that some legitimate activities can occur at any hour.
- **Consider duration as a primary factor**: Apply the duration thresholds defined in the activity patterns above. Brief sequences during normal hours with apparent purpose typically indicate normal activity unless explicit suspicious actions are visible.
- **Weigh all evidence holistically**: Match the activity against the normal and suspicious patterns defined above, then evaluate based on the complete context (zone, objects, time, actions, duration). Apply the threat level indicators consistently. Use your judgment for edge cases.
## Response Field Guidelines
Respond with a JSON object matching the provided schema. Field-specific guidance:
- `scene`: Describe how the sequence begins, then the progression of events all significant movements and actions in order. For example, if a vehicle arrives and then a person exits, describe both sequentially. For named subjects (those with a `` separator in "Objects in Scene"), always use their name do not replace them with generic terms. For unnamed objects (e.g., "person", "car"), refer to them naturally with articles (e.g., "a person", "the car"). Your description should align with and support the threat level you assign.
- `title`: Characterize **what took place and where** interpret the overall purpose or outcome, do not simply compress the scene description into fewer words. Include the relevant location (zone, area, or entry point). For named subjects, always use their name. For unnamed objects, refer to them naturally with articles. No editorial qualifiers like "routine" or "suspicious."
- `potential_threat_level`: Must be consistent with your scene description and the activity patterns above.
{get_concern_prompt()}
## Sequence Details
- Camera: {review_data["camera"]}
- Total frames: {len(thumbnails)} (Frame 1 = earliest, Frame {len(thumbnails)} = latest)
- Activity started at {review_data["start"]} and lasted {review_data["duration"]} seconds
- Zones involved: {", ".join(review_data["zones"]) if review_data["zones"] else "None"}
## Objects in Scene
Each line represents a detection state, not necessarily unique individuals. The `` symbol separates a recognized subject's name from their object type — use only the name (before the `←`) in your response, not the type after it. The same subject may appear across multiple lines if detected multiple times.
**Note: Unidentified objects (without names) are NOT indicators of suspicious activitythey simply mean the system hasn't identified that object.**
{get_objects_list()}
{get_language_prompt()}
"""
logger.debug(
f"Sending {len(thumbnails)} images to create review description on {review_data['camera']}"
)
@@ -86,7 +142,25 @@ class GenAIClient:
) as f:
f.write(context_prompt)
response_format = build_review_description_response_format(concerns)
# Build JSON schema for structured output from ReviewMetadata model
schema = ReviewMetadata.model_json_schema()
schema.get("properties", {}).pop("time", None)
if "time" in schema.get("required", []):
schema["required"].remove("time")
if not concerns:
schema.get("properties", {}).pop("other_concerns", None)
if "other_concerns" in schema.get("required", []):
schema["required"].remove("other_concerns")
response_format = {
"type": "json_schema",
"json_schema": {
"name": "review_metadata",
"strict": True,
"schema": schema,
},
}
response = self._send(context_prompt, thumbnails, response_format)
@@ -165,9 +239,61 @@ class GenAIClient:
debug_save: bool,
) -> str | None:
"""Generate a summary of review item descriptions over a period of time."""
timeline_summary_prompt = build_review_summary_prompt(
start_ts, end_ts, events, preferred_language
)
time_range = f"{datetime.datetime.fromtimestamp(start_ts).strftime('%B %d, %Y at %I:%M %p')} to {datetime.datetime.fromtimestamp(end_ts).strftime('%B %d, %Y at %I:%M %p')}"
timeline_summary_prompt = f"""
You are a security officer writing a concise security report.
Time range: {time_range}
Input format: Each event is a JSON object with:
- "title", "scene", "confidence", "potential_threat_level" (0-2), "other_concerns", "camera", "time", "start_time", "end_time"
- "context": array of related events from other cameras that occurred during overlapping time periods
**Note: Use the "scene" field for event descriptions in the report. Ignore any "shortSummary" field if present.**
Report Structure - Use this EXACT format:
# Security Summary - {time_range}
## Overview
[Write 1-2 sentences summarizing the overall activity pattern during this period.]
---
## Timeline
[Group events by time periods (e.g., "Morning (6:00 AM - 12:00 PM)", "Afternoon (12:00 PM - 5:00 PM)", "Evening (5:00 PM - 9:00 PM)", "Night (9:00 PM - 6:00 AM)"). Use appropriate time blocks based on when events occurred.]
### [Time Block Name]
**HH:MM AM/PM** | [Camera Name] | [Threat Level Indicator]
- [Event title]: [Clear description incorporating contextual information from the "context" array]
- Context: [If context array has items, mention them here, e.g., "Delivery truck present on Front Driveway Cam (HH:MM AM/PM)"]
- Assessment: [Brief assessment incorporating context - if context explains the event, note it here]
[Repeat for each event in chronological order within the time block]
---
## Summary
[One sentence summarizing the period. If all events are normal/explained: "Routine activity observed." If review needed: "Some activity requires review but no security concerns." If security concerns: "Security concerns requiring immediate attention."]
Guidelines:
- List ALL events in chronological order, grouped by time blocks
- Threat level indicators: Normal, Needs review, 🔴 Security concern
- Integrate contextual information naturally - use the "context" array to enrich each event's description
- If context explains the event (e.g., delivery truck explains person at door), describe it accordingly (e.g., "delivery person" not "unidentified person")
- Be concise but informative - focus on what happened and what it means
- If contextual information makes an event clearly normal, reflect that in your assessment
- Only create time blocks that have events - don't create empty sections
"""
timeline_summary_prompt += "\n\nEvents:\n"
for event in events:
timeline_summary_prompt += f"\n{event}\n"
if preferred_language:
timeline_summary_prompt += f"\nProvide your answer in {preferred_language}"
if debug_save:
with open(
@@ -199,7 +325,10 @@ class GenAIClient:
) -> Optional[str]:
"""Generate a description for the frame."""
try:
prompt = build_object_description_prompt(camera_config, event)
prompt = camera_config.objects.genai.object_prompts.get(
str(event.label),
camera_config.objects.genai.prompt,
).format(**model_to_dict(event))
except KeyError as e:
logger.error(f"Invalid key in GenAI prompt: {e}")
return None
@@ -300,10 +429,6 @@ class GenAIClient:
Returns:
Dictionary with:
- 'content': Optional[str] - The text response from the LLM, None if tool calls
- 'reasoning': Optional[str] - The separated reasoning/thinking trace
if the model emitted one (e.g. via OpenAI-compatible
`reasoning_content`). None when the model does not surface a
trace or the provider does not parse it.
- 'tool_calls': Optional[List[Dict]] - List of tool calls if LLM wants to call tools.
Each tool call dict has:
- 'id': str - Unique identifier for this tool call
@@ -315,14 +440,6 @@ class GenAIClient:
- 'length': Hit token limit
- 'error': An error occurred
Streaming counterpart `chat_with_tools_stream` yields
``(kind, value)`` tuples where ``kind`` is one of:
- 'content_delta': value is a string fragment of the answer
- 'reasoning_delta': value is a string fragment of the reasoning
trace (emitted before content for thinking models)
- 'stats': value is a usage stats dict
- 'message': value is the final dict shape described above
Raises:
NotImplementedError: If the provider doesn't implement this method.
"""
@@ -333,15 +450,14 @@ class GenAIClient:
)
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
def load_providers() -> None:
plugins_dir = os.path.join(os.path.dirname(__file__), "plugins")
for filename in os.listdir(plugins_dir):
package_dir = os.path.dirname(__file__)
for filename in os.listdir(package_dir):
if filename.endswith(".py") and filename != "__init__.py":
module_name = f"frigate.genai.plugins.{filename[:-3]}"
module_name = f"frigate.genai.{filename[:-3]}"
importlib.import_module(module_name)
+305
View File
@@ -0,0 +1,305 @@
"""Azure OpenAI Provider for Frigate AI."""
import base64
import json
import logging
from typing import Any, AsyncGenerator, Optional
from urllib.parse import parse_qs, urlparse
from openai import AzureOpenAI
from frigate.config import GenAIProviderEnum
from frigate.genai import GenAIClient, register_genai_provider
logger = logging.getLogger(__name__)
@register_genai_provider(GenAIProviderEnum.azure_openai)
class OpenAIClient(GenAIClient):
"""Generative AI client for Frigate using Azure OpenAI."""
provider: AzureOpenAI
def _init_provider(self) -> AzureOpenAI | None:
"""Initialize the client."""
try:
parsed_url = urlparse(self.genai_config.base_url or "")
query_params = parse_qs(parsed_url.query)
api_version = query_params.get("api-version", [None])[0]
azure_endpoint = f"{parsed_url.scheme}://{parsed_url.netloc}/"
if not api_version:
logger.warning("Azure OpenAI url is missing API version.")
return None
except Exception as e:
logger.warning("Error parsing Azure OpenAI url: %s", str(e))
return None
return AzureOpenAI(
api_key=self.genai_config.api_key,
api_version=api_version,
azure_endpoint=azure_endpoint,
)
def _send(
self,
prompt: str,
images: list[bytes],
response_format: Optional[dict] = None,
) -> Optional[str]:
"""Submit a request to Azure OpenAI."""
encoded_images = [base64.b64encode(image).decode("utf-8") for image in images]
try:
request_params = {
"model": self.genai_config.model,
"messages": [
{
"role": "user",
"content": [{"type": "text", "text": prompt}]
+ [
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{image}",
"detail": "low",
},
}
for image in encoded_images
],
},
],
"timeout": self.timeout,
**self.genai_config.runtime_options,
}
if response_format:
request_params["response_format"] = response_format
result = self.provider.chat.completions.create(**request_params)
except Exception as e:
logger.warning("Azure OpenAI returned an error: %s", str(e))
return None
if len(result.choices) > 0:
return str(result.choices[0].message.content.strip())
return None
def list_models(self) -> list[str]:
"""Return available model IDs from Azure OpenAI."""
try:
return sorted(m.id for m in self.provider.models.list().data)
except Exception as e:
logger.warning("Failed to list Azure OpenAI models: %s", e)
return []
def get_context_size(self) -> int:
"""Get the context window size for Azure OpenAI."""
return 128000
def chat_with_tools(
self,
messages: list[dict[str, Any]],
tools: Optional[list[dict[str, Any]]] = None,
tool_choice: Optional[str] = "auto",
) -> dict[str, Any]:
try:
openai_tool_choice = None
if tool_choice:
if tool_choice == "none":
openai_tool_choice = "none"
elif tool_choice == "auto":
openai_tool_choice = "auto"
elif tool_choice == "required":
openai_tool_choice = "required"
request_params = {
"model": self.genai_config.model,
"messages": messages,
"timeout": self.timeout,
}
if tools:
request_params["tools"] = tools
if openai_tool_choice is not None:
request_params["tool_choice"] = openai_tool_choice
result = self.provider.chat.completions.create(**request_params) # type: ignore[call-overload]
if (
result is None
or not hasattr(result, "choices")
or len(result.choices) == 0
):
return {
"content": None,
"tool_calls": None,
"finish_reason": "error",
}
choice = result.choices[0]
message = choice.message
content = message.content.strip() if message.content else None
tool_calls = None
if message.tool_calls:
tool_calls = []
for tool_call in message.tool_calls:
try:
arguments = json.loads(tool_call.function.arguments)
except (json.JSONDecodeError, AttributeError) as e:
logger.warning(
f"Failed to parse tool call arguments: {e}, "
f"tool: {tool_call.function.name if hasattr(tool_call.function, 'name') else 'unknown'}"
)
arguments = {}
tool_calls.append(
{
"id": tool_call.id if hasattr(tool_call, "id") else "",
"name": tool_call.function.name
if hasattr(tool_call.function, "name")
else "",
"arguments": arguments,
}
)
finish_reason = "error"
if hasattr(choice, "finish_reason") and choice.finish_reason:
finish_reason = choice.finish_reason
elif tool_calls:
finish_reason = "tool_calls"
elif content:
finish_reason = "stop"
return {
"content": content,
"tool_calls": tool_calls,
"finish_reason": finish_reason,
}
except Exception as e:
logger.warning("Azure OpenAI returned an error: %s", str(e))
return {
"content": None,
"tool_calls": None,
"finish_reason": "error",
}
async def chat_with_tools_stream(
self,
messages: list[dict[str, Any]],
tools: Optional[list[dict[str, Any]]] = None,
tool_choice: Optional[str] = "auto",
) -> AsyncGenerator[tuple[str, Any], None]:
"""
Stream chat with tools; yields content deltas then final message.
Implements streaming function calling/tool usage for Azure OpenAI models.
"""
try:
openai_tool_choice = None
if tool_choice:
if tool_choice == "none":
openai_tool_choice = "none"
elif tool_choice == "auto":
openai_tool_choice = "auto"
elif tool_choice == "required":
openai_tool_choice = "required"
request_params = {
"model": self.genai_config.model,
"messages": messages,
"timeout": self.timeout,
"stream": True,
}
if tools:
request_params["tools"] = tools
if openai_tool_choice is not None:
request_params["tool_choice"] = openai_tool_choice
# Use streaming API
content_parts: list[str] = []
tool_calls_by_index: dict[int, dict[str, Any]] = {}
finish_reason = "stop"
stream = self.provider.chat.completions.create(**request_params) # type: ignore[call-overload]
for chunk in stream:
if not chunk or not chunk.choices:
continue
choice = chunk.choices[0]
delta = choice.delta
# Check for finish reason
if choice.finish_reason:
finish_reason = choice.finish_reason
# Extract content deltas
if delta.content:
content_parts.append(delta.content)
yield ("content_delta", delta.content)
# Extract tool calls
if delta.tool_calls:
for tc in delta.tool_calls:
idx = tc.index
fn = tc.function
if idx not in tool_calls_by_index:
tool_calls_by_index[idx] = {
"id": tc.id or "",
"name": fn.name if fn and fn.name else "",
"arguments": "",
}
t = tool_calls_by_index[idx]
if tc.id:
t["id"] = tc.id
if fn and fn.name:
t["name"] = fn.name
if fn and fn.arguments:
t["arguments"] += fn.arguments
# Build final message
full_content = "".join(content_parts).strip() or None
# Convert tool calls to list format
tool_calls_list = None
if tool_calls_by_index:
tool_calls_list = []
for tc in tool_calls_by_index.values():
try:
# Parse accumulated arguments as JSON
parsed_args = json.loads(tc["arguments"])
except (json.JSONDecodeError, Exception):
parsed_args = tc["arguments"]
tool_calls_list.append(
{
"id": tc["id"],
"name": tc["name"],
"arguments": parsed_args,
}
)
finish_reason = "tool_calls"
yield (
"message",
{
"content": full_content,
"tool_calls": tool_calls_list,
"finish_reason": finish_reason,
},
)
except Exception as e:
logger.warning("Azure OpenAI streaming returned an error: %s", str(e))
yield (
"message",
{
"content": None,
"tool_calls": None,
"finish_reason": "error",
},
)
@@ -14,20 +14,6 @@ from frigate.genai import GenAIClient, register_genai_provider
logger = logging.getLogger(__name__)
def _stats_from_gemini_usage(usage: Any) -> Optional[dict[str, Any]]:
"""Build a stats dict from a Gemini usage_metadata object."""
prompt_tokens = getattr(usage, "prompt_token_count", None)
completion_tokens = getattr(usage, "candidates_token_count", None)
if prompt_tokens is None and completion_tokens is None:
return None
stats: dict[str, Any] = {}
if isinstance(prompt_tokens, int):
stats["prompt_tokens"] = prompt_tokens
if isinstance(completion_tokens, int):
stats["completion_tokens"] = completion_tokens
return stats or None
@register_genai_provider(GenAIProviderEnum.gemini)
class GeminiClient(GenAIClient):
"""Generative AI client for Frigate using Gemini."""
@@ -248,13 +234,6 @@ class GeminiClient(GenAIClient):
if tool_config:
config_params["tool_config"] = tool_config
# Ask thinking-capable models (Gemini 2.5+) to include their
# reasoning trace as separate `thought` parts so we can surface
# it on the reasoning channel. Older models ignore this field.
config_params["thinking_config"] = types.ThinkingConfig(
include_thoughts=True
)
# Merge runtime_options
if isinstance(self.genai_config.runtime_options, dict):
config_params.update(self.genai_config.runtime_options)
@@ -269,24 +248,19 @@ class GeminiClient(GenAIClient):
if not response or not response.candidates:
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
candidate = response.candidates[0]
content = None
reasoning_parts: list[str] = []
tool_calls = None
# Extract content, reasoning, and tool calls from response
# Extract content and tool calls from response
if candidate.content and candidate.content.parts:
for part in candidate.content.parts:
if part.text:
if getattr(part, "thought", False):
reasoning_parts.append(part.text)
else:
content = part.text.strip()
content = part.text.strip()
elif part.function_call:
# Handle function call
if tool_calls is None:
@@ -309,8 +283,6 @@ class GeminiClient(GenAIClient):
}
)
reasoning = "".join(reasoning_parts).strip() or None
# Determine finish reason
finish_reason = "error"
if hasattr(candidate, "finish_reason") and candidate.finish_reason:
@@ -336,7 +308,6 @@ class GeminiClient(GenAIClient):
return {
"content": content,
"reasoning": reasoning,
"tool_calls": tool_calls,
"finish_reason": finish_reason,
}
@@ -345,7 +316,6 @@ class GeminiClient(GenAIClient):
logger.warning("Gemini API error during chat_with_tools: %s", str(e))
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
@@ -355,7 +325,6 @@ class GeminiClient(GenAIClient):
)
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
@@ -494,22 +463,14 @@ class GeminiClient(GenAIClient):
if tool_config:
config_params["tool_config"] = tool_config
# Ask thinking-capable models to include their reasoning trace
# as separate `thought` parts (Gemini 2.5+; ignored elsewhere).
config_params["thinking_config"] = types.ThinkingConfig(
include_thoughts=True
)
# Merge runtime_options
if isinstance(self.genai_config.runtime_options, dict):
config_params.update(self.genai_config.runtime_options)
# Use streaming API
content_parts: list[str] = []
reasoning_parts: list[str] = []
tool_calls_by_index: dict[int, dict[str, Any]] = {}
finish_reason = "stop"
usage_stats: Optional[dict[str, Any]] = None
stream = await self.provider.aio.models.generate_content_stream(
model=self.genai_config.model,
@@ -518,12 +479,6 @@ class GeminiClient(GenAIClient):
)
async for chunk in stream:
chunk_usage = getattr(chunk, "usage_metadata", None)
if chunk_usage is not None:
maybe_stats = _stats_from_gemini_usage(chunk_usage)
if maybe_stats is not None:
usage_stats = maybe_stats
if not chunk or not chunk.candidates:
continue
@@ -543,16 +498,12 @@ class GeminiClient(GenAIClient):
]:
finish_reason = "error"
# Extract content, reasoning, and tool calls from chunk
# Extract content and tool calls from chunk
if candidate.content and candidate.content.parts:
for part in candidate.content.parts:
if part.text:
if getattr(part, "thought", False):
reasoning_parts.append(part.text)
yield ("reasoning_delta", part.text)
else:
content_parts.append(part.text)
yield ("content_delta", part.text)
content_parts.append(part.text)
yield ("content_delta", part.text)
elif part.function_call:
# Handle function call
try:
@@ -593,7 +544,6 @@ class GeminiClient(GenAIClient):
# Build final message
full_content = "".join(content_parts).strip() or None
full_reasoning = "".join(reasoning_parts).strip() or None
# Convert tool calls to list format
tool_calls_list = None
@@ -615,14 +565,10 @@ class GeminiClient(GenAIClient):
)
finish_reason = "tool_calls"
if usage_stats is not None:
yield ("stats", usage_stats)
yield (
"message",
{
"content": full_content,
"reasoning": full_reasoning,
"tool_calls": tool_calls_list,
"finish_reason": finish_reason,
},
@@ -634,7 +580,6 @@ class GeminiClient(GenAIClient):
"message",
{
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
},
@@ -647,7 +592,6 @@ class GeminiClient(GenAIClient):
"message",
{
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
},
@@ -4,7 +4,7 @@ import base64
import io
import json
import logging
from typing import Any, AsyncGenerator, Optional, cast
from typing import Any, AsyncGenerator, Optional
import httpx
import numpy as np
@@ -18,86 +18,6 @@ from frigate.genai.utils import parse_tool_calls_from_message
logger = logging.getLogger(__name__)
def _stats_from_llama_cpp_chunk(data: dict[str, Any]) -> Optional[dict[str, Any]]:
"""Build a stats dict from a llama.cpp streaming chunk.
Final-chunk `usage` carries authoritative token counts. Per-chunk
`timings` (enabled via timings_per_token) carries the running token
counts (prompt_n, predicted_n) and generation rate, so live updates
work mid-stream.
"""
usage = data.get("usage") or {}
timings = data.get("timings") or {}
prompt_tokens = usage.get("prompt_tokens")
completion_tokens = usage.get("completion_tokens")
predicted_ms = timings.get("predicted_ms")
tps = timings.get("predicted_per_second")
stats: dict[str, Any] = {}
if not isinstance(prompt_tokens, int):
prompt_n = timings.get("prompt_n")
if isinstance(prompt_n, int):
prompt_tokens = prompt_n
if not isinstance(completion_tokens, int):
predicted_n = timings.get("predicted_n")
if isinstance(predicted_n, int):
completion_tokens = predicted_n
if not isinstance(prompt_tokens, int) and not isinstance(completion_tokens, int):
return None
if isinstance(prompt_tokens, int):
stats["prompt_tokens"] = prompt_tokens
if isinstance(completion_tokens, int):
stats["completion_tokens"] = completion_tokens
if isinstance(predicted_ms, (int, float)) and predicted_ms > 0:
stats["completion_duration_ms"] = float(predicted_ms)
if isinstance(tps, (int, float)) and tps > 0:
stats["tokens_per_second"] = float(tps)
return stats or None
def _parse_launch_arg(args: list[str], flag: str) -> str | None:
"""Return the value following `flag` in a positional argv list, or None."""
try:
idx = args.index(flag)
except ValueError:
return None
if idx + 1 >= len(args):
return None
return args[idx + 1]
def _fetch_llama_props(base_url: str, model: str) -> dict[str, Any]:
"""Fetch /props from a llama.cpp server, with llama-swap fallback.
Raises the underlying RequestException if both endpoints fail; callers
decide how to surface the failure.
"""
try:
response = requests.get(
f"{base_url}/props",
params={"model": model},
timeout=10,
)
response.raise_for_status()
return cast(dict[str, Any], response.json())
except Exception:
response = requests.get(
f"{base_url}/upstream/{model}/props",
timeout=10,
)
response.raise_for_status()
return cast(dict[str, Any], response.json())
def _to_jpeg(img_bytes: bytes) -> bytes | None:
"""Convert image bytes to JPEG. llama.cpp/STB does not support WebP."""
try:
@@ -147,73 +67,28 @@ class LlamaCppClient(GenAIClient):
if base_url is None:
return None
else:
base_url = base_url.replace("/v1", "") # Strip /v1 if included in base_url
configured_model = self.genai_config.model
info = self._get_model_info(base_url, configured_model)
if info is None:
return None
self._context_size = info["context_size"]
self._supports_vision = info["supports_vision"]
self._supports_audio = info["supports_audio"]
self._supports_tools = info["supports_tools"]
self._media_marker = info["media_marker"]
logger.info(
"llama.cpp model '%s' initialized — context: %s, vision: %s, audio: %s, tools: %s",
configured_model,
self._context_size or "unknown",
self._supports_vision,
self._supports_audio,
self._supports_tools,
)
return base_url
def _get_model_info(
self, base_url: str, configured_model: str
) -> dict[str, Any] | None:
"""Resolve model metadata from /v1/models with /props fallback.
Returns a dict of capability fields, or None if the server's model
registry was reachable and reported the configured model as missing.
A reachable-but-unparseable /v1/models is treated as soft-pass and
falls through to /props, matching prior behavior.
After ggml-org/llama.cpp#22952, /v1/models exposes per-model
`architecture.input_modalities` (text/image/audio) the primary
source. When proxied through llama-swap, the same entry carries
`status.args` (server launch argv) and, for the loaded model,
`meta.n_ctx`. /props remains the only source for `media_marker`,
which the server randomizes per startup unless LLAMA_MEDIA_MARKER
is set.
"""
info: dict[str, Any] = {
"context_size": None,
"supports_vision": False,
"supports_audio": False,
"supports_tools": False,
"media_marker": "<__media__>",
}
model_entry: dict[str, Any] | None = None
# Query /v1/models to validate the configured model exists
try:
response = requests.get(f"{base_url}/v1/models", timeout=10)
response = requests.get(
f"{base_url}/v1/models",
timeout=10,
)
response.raise_for_status()
models_data = response.json()
model_found = False
for model in models_data.get("data", []):
model_ids = {model.get("id")}
for alias in model.get("aliases", []):
model_ids.add(alias)
if configured_model in model_ids:
model_entry = model
model_found = True
break
if model_entry is None:
if not model_found:
available = []
for m in models_data.get("data", []):
available.append(m.get("id", "unknown"))
@@ -232,64 +107,65 @@ class LlamaCppClient(GenAIClient):
e,
)
if model_entry is not None:
architecture = model_entry.get("architecture") or {}
input_modalities = architecture.get("input_modalities") or []
if isinstance(input_modalities, list):
info["supports_vision"] = "image" in input_modalities
info["supports_audio"] = "audio" in input_modalities
status = model_entry.get("status") or {}
launch_args = status.get("args") if isinstance(status, dict) else None
if not isinstance(launch_args, list):
launch_args = []
meta = model_entry.get("meta") if isinstance(model_entry, dict) else None
n_ctx = meta.get("n_ctx") if isinstance(meta, dict) else None
if not n_ctx:
n_ctx = _parse_launch_arg(launch_args, "--ctx-size")
if n_ctx:
try:
info["context_size"] = int(n_ctx)
except (TypeError, ValueError):
pass
# Tool calling on llama-server requires --jinja.
if "--jinja" in launch_args:
info["supports_tools"] = True
# Query /props for context size, modalities, and tool support.
# The standard /props?model=<name> endpoint works with llama-server.
# If it fails, try the llama-swap per-model passthrough endpoint which
# returns props for a specific model without requiring it to be loaded.
try:
props = _fetch_llama_props(base_url, configured_model)
try:
response = requests.get(
f"{base_url}/props",
params={"model": configured_model},
timeout=10,
)
response.raise_for_status()
props = response.json()
except Exception:
response = requests.get(
f"{base_url}/upstream/{configured_model}/props",
timeout=10,
)
response.raise_for_status()
props = response.json()
if info["context_size"] is None:
default_settings = props.get("default_generation_settings", {})
n_ctx = default_settings.get("n_ctx")
if n_ctx:
info["context_size"] = int(n_ctx)
# Context size from server runtime config
default_settings = props.get("default_generation_settings", {})
n_ctx = default_settings.get("n_ctx")
if n_ctx:
self._context_size = int(n_ctx)
if not (info["supports_vision"] or info["supports_audio"]):
modalities = props.get("modalities", {})
info["supports_vision"] = bool(modalities.get("vision", False))
info["supports_audio"] = bool(modalities.get("audio", False))
# Modalities (vision, audio)
modalities = props.get("modalities", {})
self._supports_vision = modalities.get("vision", False)
self._supports_audio = modalities.get("audio", False)
if not info["supports_tools"]:
chat_caps = props.get("chat_template_caps", {})
info["supports_tools"] = bool(chat_caps.get("supports_tools", False))
# Tool support from chat template capabilities
chat_caps = props.get("chat_template_caps", {})
self._supports_tools = chat_caps.get("supports_tools", False)
# Media marker for multimodal embeddings; the server randomizes this
# per startup unless LLAMA_MEDIA_MARKER is set, so we must read it
# from /props rather than hardcoding "<__media__>".
media_marker = props.get("media_marker")
if isinstance(media_marker, str) and media_marker:
info["media_marker"] = media_marker
self._media_marker = media_marker
logger.info(
"llama.cpp model '%s' initialized — context: %s, vision: %s, audio: %s, tools: %s",
configured_model,
self._context_size or "unknown",
self._supports_vision,
self._supports_audio,
self._supports_tools,
)
except Exception as e:
logger.warning(
"Failed to query llama.cpp /props endpoint: %s. "
"Image embeddings may fail if the server randomized its media marker.",
"Using defaults for context size and capabilities.",
e,
)
return info
return base_url
def _send(
self,
@@ -517,8 +393,6 @@ class LlamaCppClient(GenAIClient):
}
if stream:
payload["stream"] = True
payload["stream_options"] = {"include_usage": True}
payload["timings_per_token"] = True
if tools:
payload["tools"] = tools
if openai_tool_choice is not None:
@@ -527,28 +401,19 @@ class LlamaCppClient(GenAIClient):
k: v for k, v in self.provider_options.items() if k != "context_size"
}
payload.update(provider_opts)
payload.update(self.genai_config.runtime_options)
return payload
def _message_from_choice(self, choice: dict[str, Any]) -> dict[str, Any]:
"""Parse OpenAI-style choice into {content, reasoning, tool_calls, finish_reason}.
llama.cpp's `--reasoning-format` puts the trace in
`message.reasoning_content` (preferred) or `message.thinking`; both
keys are accepted so different builds work without configuration.
"""
"""Parse OpenAI-style choice into {content, tool_calls, finish_reason}."""
message = choice.get("message", {})
content = message.get("content")
content = content.strip() if content else None
reasoning = message.get("reasoning_content") or message.get("thinking")
reasoning = reasoning.strip() if reasoning else None
tool_calls = parse_tool_calls_from_message(message)
finish_reason = choice.get("finish_reason") or (
"tool_calls" if tool_calls else "stop" if content else "error"
)
return {
"content": content,
"reasoning": reasoning,
"tool_calls": tool_calls,
"finish_reason": finish_reason,
}
@@ -577,31 +442,6 @@ class LlamaCppClient(GenAIClient):
)
return result if result else None
def _refresh_media_marker(self) -> bool:
"""Re-fetch /props and update the cached media marker if it changed.
The server randomizes the marker per startup (unless LLAMA_MEDIA_MARKER
is set), so a stale marker indicates a restart. Returns True iff the
marker was updated to a new value used to gate a one-shot retry of
a failed embeddings request.
"""
if self.provider is None:
return False
try:
props = _fetch_llama_props(self.provider, self.genai_config.model)
except Exception as e:
logger.warning("Failed to refresh llama.cpp media marker: %s", e)
return False
marker = props.get("media_marker")
if not isinstance(marker, str) or not marker or marker == self._media_marker:
return False
logger.info("llama.cpp media marker changed (server restart); refreshed")
self._media_marker = marker
return True
def embed(
self,
texts: list[str] | None = None,
@@ -626,46 +466,30 @@ class LlamaCppClient(GenAIClient):
EMBEDDING_DIM = 768
encoded_images: list[str] = []
content = []
for text in texts:
content.append({"prompt_string": text})
for img in images:
# llama.cpp uses STB which does not support WebP; convert to JPEG
jpeg_bytes = _to_jpeg(img)
to_encode = jpeg_bytes if jpeg_bytes is not None else img
encoded_images.append(base64.b64encode(to_encode).decode("utf-8"))
def build_content() -> list[dict[str, Any]]:
# prompt_string must contain the server's media marker placeholder
# for each image. The marker is randomized per server startup.
content: list[dict[str, Any]] = []
for text in texts:
content.append({"prompt_string": text})
for encoded in encoded_images:
content.append(
{
"prompt_string": f"{self._media_marker}\n",
"multimodal_data": [encoded],
}
)
return content
def post_embeddings() -> requests.Response:
return requests.post(
f"{self.provider}/embeddings",
json={"model": self.genai_config.model, "content": build_content()},
timeout=self.timeout,
encoded = base64.b64encode(to_encode).decode("utf-8")
# prompt_string must contain the server's media marker placeholder.
# The marker is randomized per server startup (read from /props).
content.append(
{
"prompt_string": f"{self._media_marker}\n",
"multimodal_data": [encoded], # type: ignore[dict-item]
}
)
try:
try:
response = post_embeddings()
response.raise_for_status()
except requests.exceptions.RequestException:
# The server may have restarted with a new media marker.
# Refresh from /props; only retry if the marker actually changed.
if not encoded_images or not self._refresh_media_marker():
raise
response = post_embeddings()
response.raise_for_status()
response = requests.post(
f"{self.provider}/embeddings",
json={"model": self.genai_config.model, "content": content},
timeout=self.timeout,
)
response.raise_for_status()
result = response.json()
items = result.get("data", result) if isinstance(result, dict) else result
@@ -811,7 +635,6 @@ class LlamaCppClient(GenAIClient):
try:
payload = self._build_payload(messages, tools, tool_choice, stream=True)
content_parts: list[str] = []
reasoning_parts: list[str] = []
tool_calls_by_index: dict[int, dict[str, Any]] = {}
finish_reason = "stop"
@@ -832,24 +655,12 @@ class LlamaCppClient(GenAIClient):
data = json.loads(data_str)
except json.JSONDecodeError:
continue
maybe_stats = _stats_from_llama_cpp_chunk(data)
if maybe_stats is not None:
yield ("stats", maybe_stats)
choices = data.get("choices") or []
if not choices:
continue
delta = choices[0].get("delta", {})
if choices[0].get("finish_reason"):
finish_reason = choices[0]["finish_reason"]
# llama.cpp emits separated thinking under
# reasoning_content (preferred) or thinking before any
# content tokens arrive
reasoning_delta = delta.get("reasoning_content") or delta.get(
"thinking"
)
if reasoning_delta:
reasoning_parts.append(reasoning_delta)
yield ("reasoning_delta", reasoning_delta)
if delta.get("content"):
content_parts.append(delta["content"])
yield ("content_delta", delta["content"])
@@ -875,7 +686,6 @@ class LlamaCppClient(GenAIClient):
)
full_content = "".join(content_parts).strip() or None
full_reasoning = "".join(reasoning_parts).strip() or None
tool_calls_list = self._streamed_tool_calls_to_list(tool_calls_by_index)
if tool_calls_list:
finish_reason = "tool_calls"
@@ -883,7 +693,6 @@ class LlamaCppClient(GenAIClient):
"message",
{
"content": full_content,
"reasoning": full_reasoning,
"tool_calls": tool_calls_list,
"finish_reason": finish_reason,
},
@@ -1,7 +1,5 @@
"""Ollama Provider for Frigate AI."""
import base64
import binascii
import json
import logging
from typing import Any, AsyncGenerator, Optional
@@ -18,72 +16,6 @@ from frigate.genai.utils import parse_tool_calls_from_message
logger = logging.getLogger(__name__)
def _extract_ollama_stats(response: Any) -> Optional[dict[str, Any]]:
"""Build a stats dict from Ollama's response metadata.
Ollama reports eval_count/eval_duration (generation) and
prompt_eval_count (context size). Durations are nanoseconds.
"""
if not response:
return None
if hasattr(response, "get"):
getter = response.get
else:
getter = lambda key: getattr(response, key, None) # noqa: E731
eval_count = getter("eval_count")
eval_duration_ns = getter("eval_duration")
prompt_eval_count = getter("prompt_eval_count")
if eval_count is None and prompt_eval_count is None:
return None
stats: dict[str, Any] = {}
if isinstance(prompt_eval_count, int):
stats["prompt_tokens"] = prompt_eval_count
if isinstance(eval_count, int):
stats["completion_tokens"] = eval_count
if isinstance(eval_duration_ns, int) and eval_duration_ns > 0:
stats["completion_duration_ms"] = eval_duration_ns / 1_000_000
if isinstance(eval_count, int) and eval_count > 0:
stats["tokens_per_second"] = eval_count / (eval_duration_ns / 1_000_000_000)
return stats or None
def _normalize_multimodal_content(
content: Any,
) -> tuple[Optional[str], Optional[list[bytes]]]:
"""Convert OpenAI-style multimodal content to Ollama's (text, images) shape.
The chat API constructs user messages with content as a list of
``{"type": "text"}`` and ``{"type": "image_url"}`` parts when a tool
returns a live frame. Ollama's SDK requires content to be a string and
images to be passed in a separate field, so we extract each.
"""
if not isinstance(content, list):
return content, None
text_parts: list[str] = []
images: list[bytes] = []
for part in content:
if not isinstance(part, dict):
continue
part_type = part.get("type")
if part_type == "text":
text = part.get("text")
if text:
text_parts.append(str(text))
elif part_type == "image_url":
url = (part.get("image_url") or {}).get("url", "")
if isinstance(url, str) and url.startswith("data:"):
try:
encoded = url.split(",", 1)[1]
images.append(base64.b64decode(encoded, validate=True))
except (ValueError, IndexError, binascii.Error) as e:
logger.debug("Failed to decode multimodal image url: %s", e)
return ("\n".join(text_parts) if text_parts else None), (images or None)
@register_genai_provider(GenAIProviderEnum.ollama)
class OllamaClient(GenAIClient):
"""Generative AI client for Frigate using Ollama."""
@@ -99,12 +31,6 @@ class OllamaClient(GenAIClient):
provider: ApiClient | None
provider_options: dict[str, Any]
def _auth_headers(self) -> dict | None:
if self.genai_config.api_key:
return {"Authorization": "Bearer " + self.genai_config.api_key}
return None
def _init_provider(self) -> ApiClient | None:
"""Initialize the client."""
self.provider_options = {
@@ -113,11 +39,7 @@ class OllamaClient(GenAIClient):
}
try:
client = ApiClient(
host=self.genai_config.base_url,
timeout=self.timeout,
headers=self._auth_headers(),
)
client = ApiClient(host=self.genai_config.base_url, timeout=self.timeout)
# ensure the model is available locally
response = client.show(self.genai_config.model)
if response.get("error"):
@@ -244,9 +166,7 @@ class OllamaClient(GenAIClient):
return []
try:
client = ApiClient(
host=self.genai_config.base_url,
timeout=self.timeout,
headers=self._auth_headers(),
host=self.genai_config.base_url, timeout=self.timeout
)
except Exception:
return []
@@ -275,13 +195,10 @@ class OllamaClient(GenAIClient):
"""Build request_messages and params for chat (sync or stream)."""
request_messages = []
for msg in messages:
content, images = _normalize_multimodal_content(msg.get("content", ""))
msg_dict: dict[str, Any] = {
msg_dict = {
"role": msg.get("role"),
"content": content if content is not None else "",
"content": msg.get("content", ""),
}
if images:
msg_dict["images"] = images
if msg.get("tool_call_id"):
msg_dict["tool_call_id"] = msg["tool_call_id"]
if msg.get("name"):
@@ -309,7 +226,6 @@ class OllamaClient(GenAIClient):
"model": self.genai_config.model,
"messages": request_messages,
**self.provider_options,
**self.genai_config.runtime_options,
}
if stream:
request_params["stream"] = True
@@ -337,9 +253,6 @@ class OllamaClient(GenAIClient):
response.get("done"),
)
content = message.get("content", "").strip() if message.get("content") else None
reasoning = (
message.get("thinking", "").strip() if message.get("thinking") else None
)
tool_calls = parse_tool_calls_from_message(message)
finish_reason = "error"
if response.get("done"):
@@ -352,7 +265,6 @@ class OllamaClient(GenAIClient):
finish_reason = "stop"
return {
"content": content,
"reasoning": reasoning,
"tool_calls": tool_calls,
"finish_reason": finish_reason,
}
@@ -432,19 +344,12 @@ class OllamaClient(GenAIClient):
async_client = OllamaAsyncClient(
host=self.genai_config.base_url,
timeout=self.timeout,
headers=self._auth_headers(),
)
response = await async_client.chat(**request_params)
result = self._message_from_response(response)
reasoning = result.get("reasoning")
if reasoning:
yield ("reasoning_delta", reasoning)
content = result.get("content")
if content:
yield ("content_delta", content)
stats = _extract_ollama_stats(response)
if stats is not None:
yield ("stats", stats)
yield ("message", result)
return
@@ -454,41 +359,27 @@ class OllamaClient(GenAIClient):
async_client = OllamaAsyncClient(
host=self.genai_config.base_url,
timeout=self.timeout,
headers=self._auth_headers(),
)
content_parts: list[str] = []
reasoning_parts: list[str] = []
final_message: dict[str, Any] | None = None
final_chunk: Any = None
stream = await async_client.chat(**request_params)
async for chunk in stream:
if not chunk or "message" not in chunk:
continue
msg = chunk.get("message", {})
reasoning_delta = msg.get("thinking") or ""
if reasoning_delta:
reasoning_parts.append(reasoning_delta)
yield ("reasoning_delta", reasoning_delta)
delta = msg.get("content") or ""
if delta:
content_parts.append(delta)
yield ("content_delta", delta)
if chunk.get("done"):
final_chunk = chunk
full_content = "".join(content_parts).strip() or None
full_reasoning = "".join(reasoning_parts).strip() or None
final_message = {
"content": full_content,
"reasoning": full_reasoning,
"tool_calls": None,
"finish_reason": "stop",
}
break
stats = _extract_ollama_stats(final_chunk)
if stats is not None:
yield ("stats", stats)
if final_message is not None:
yield ("message", final_message)
else:
@@ -496,7 +387,6 @@ class OllamaClient(GenAIClient):
"message",
{
"content": "".join(content_parts).strip() or None,
"reasoning": "".join(reasoning_parts).strip() or None,
"tool_calls": None,
"finish_reason": "stop",
},
@@ -14,22 +14,6 @@ from frigate.genai import GenAIClient, register_genai_provider
logger = logging.getLogger(__name__)
def _stats_from_openai_usage(usage: Any) -> Optional[dict[str, Any]]:
"""Build a stats dict from an OpenAI-compatible usage object."""
if usage is None:
return None
prompt_tokens = getattr(usage, "prompt_tokens", None)
completion_tokens = getattr(usage, "completion_tokens", None)
if prompt_tokens is None and completion_tokens is None:
return None
stats: dict[str, Any] = {}
if isinstance(prompt_tokens, int):
stats["prompt_tokens"] = prompt_tokens
if isinstance(completion_tokens, int):
stats["completion_tokens"] = completion_tokens
return stats or None
@register_genai_provider(GenAIProviderEnum.openai)
class OpenAIClient(GenAIClient):
"""Generative AI client for Frigate using OpenAI."""
@@ -38,11 +22,7 @@ class OpenAIClient(GenAIClient):
context_size: Optional[int] = None
def _init_provider(self) -> OpenAI:
"""Initialize the client.
Subclasses (e.g. Azure) should raise on configuration errors; the
manager catches construction failures and disables the provider.
"""
"""Initialize the client."""
# Extract context_size from provider_options as it's not a valid OpenAI client parameter
# It will be used in get_context_size() instead
provider_opts = {
@@ -207,7 +187,6 @@ class OpenAIClient(GenAIClient):
"model": self.genai_config.model,
"messages": messages,
"timeout": self.timeout,
**self.genai_config.runtime_options,
}
if tools:
@@ -224,7 +203,7 @@ class OpenAIClient(GenAIClient):
}
request_params.update(provider_opts)
result = self.provider.chat.completions.create(**request_params)
result = self.provider.chat.completions.create(**request_params) # type: ignore[call-overload]
if (
result is None
@@ -240,10 +219,6 @@ class OpenAIClient(GenAIClient):
choice = result.choices[0]
message = choice.message
content = message.content.strip() if message.content else None
raw_reasoning = getattr(message, "reasoning_content", None) or getattr(
message, "reasoning", None
)
reasoning = raw_reasoning.strip() if raw_reasoning else None
tool_calls = None
if message.tool_calls:
@@ -278,7 +253,6 @@ class OpenAIClient(GenAIClient):
return {
"content": content,
"reasoning": reasoning,
"tool_calls": tool_calls,
"finish_reason": finish_reason,
}
@@ -287,7 +261,6 @@ class OpenAIClient(GenAIClient):
logger.warning("OpenAI request timed out: %s", str(e))
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
@@ -295,7 +268,6 @@ class OpenAIClient(GenAIClient):
logger.warning("OpenAI returned an error: %s", str(e))
return {
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
}
@@ -326,8 +298,6 @@ class OpenAIClient(GenAIClient):
"messages": messages,
"timeout": self.timeout,
"stream": True,
"stream_options": {"include_usage": True},
**self.genai_config.runtime_options,
}
if tools:
@@ -346,18 +316,12 @@ class OpenAIClient(GenAIClient):
# Use streaming API
content_parts: list[str] = []
reasoning_parts: list[str] = []
tool_calls_by_index: dict[int, dict[str, Any]] = {}
finish_reason = "stop"
usage_stats: Optional[dict[str, Any]] = None
stream = self.provider.chat.completions.create(**request_params)
stream = self.provider.chat.completions.create(**request_params) # type: ignore[call-overload]
for chunk in stream:
chunk_usage = getattr(chunk, "usage", None)
if chunk_usage is not None:
usage_stats = _stats_from_openai_usage(chunk_usage)
if not chunk or not chunk.choices:
continue
@@ -368,15 +332,6 @@ class OpenAIClient(GenAIClient):
if choice.finish_reason:
finish_reason = choice.finish_reason
# Extract reasoning deltas (reasoning_content or reasoning,
# depending on the server)
reasoning_delta = getattr(delta, "reasoning_content", None) or getattr(
delta, "reasoning", None
)
if reasoning_delta:
reasoning_parts.append(reasoning_delta)
yield ("reasoning_delta", reasoning_delta)
# Extract content deltas
if delta.content:
content_parts.append(delta.content)
@@ -405,7 +360,6 @@ class OpenAIClient(GenAIClient):
# Build final message
full_content = "".join(content_parts).strip() or None
full_reasoning = "".join(reasoning_parts).strip() or None
# Convert tool calls to list format
tool_calls_list = None
@@ -427,14 +381,10 @@ class OpenAIClient(GenAIClient):
)
finish_reason = "tool_calls"
if usage_stats is not None:
yield ("stats", usage_stats)
yield (
"message",
{
"content": full_content,
"reasoning": full_reasoning,
"tool_calls": tool_calls_list,
"finish_reason": finish_reason,
},
@@ -446,7 +396,6 @@ class OpenAIClient(GenAIClient):
"message",
{
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
},
@@ -457,7 +406,6 @@ class OpenAIClient(GenAIClient):
"message",
{
"content": None,
"reasoning": None,
"tool_calls": None,
"finish_reason": "error",
},
-1
View File
@@ -1 +0,0 @@
"""GenAI provider plugins."""
-53
View File
@@ -1,53 +0,0 @@
"""Azure OpenAI Provider for Frigate AI.
Azure OpenAI exposes the same chat completions API as OpenAI once the
client is constructed, so this provider inherits all transport, streaming,
reasoning, and tool-calling logic from :class:`OpenAIClient` and only
overrides what is genuinely Azure-specific:
- Client construction: parses ``api-version`` out of the configured
``base_url`` query string and instantiates :class:`openai.AzureOpenAI`
with ``azure_endpoint`` instead of ``base_url``. Raises if the URL is
malformed; :class:`GenAIClientManager` catches the exception and
disables the provider.
- Context size: Azure does not expose a per-model ``max_model_len`` field
reliably, so we keep the historical 128K default rather than the
model-name heuristic used by OpenAI.
"""
import logging
from urllib.parse import parse_qs, urlparse
from openai import AzureOpenAI
from frigate.config import GenAIProviderEnum
from frigate.genai import register_genai_provider
from frigate.genai.plugins.openai import OpenAIClient
logger = logging.getLogger(__name__)
@register_genai_provider(GenAIProviderEnum.azure_openai)
class AzureOpenAIClient(OpenAIClient):
"""Generative AI client for Frigate using Azure OpenAI."""
def _init_provider(self) -> AzureOpenAI:
"""Initialize the AzureOpenAI client from the configured base_url."""
parsed_url = urlparse(self.genai_config.base_url or "")
query_params = parse_qs(parsed_url.query)
api_version = query_params.get("api-version", [None])[0]
if not api_version:
raise ValueError("Azure OpenAI base_url is missing api-version.")
azure_endpoint = f"{parsed_url.scheme}://{parsed_url.netloc}/"
return AzureOpenAI(
api_key=self.genai_config.api_key,
api_version=api_version,
azure_endpoint=azure_endpoint,
)
def get_context_size(self) -> int:
"""Azure does not reliably surface per-model context size; use 128K."""
return 128000
-739
View File
@@ -1,739 +0,0 @@
"""Prompt and response-format builders for GenAI features.
Centralizes the per-feature prompt framing and structured-output schema
shaping so provider clients in :mod:`frigate.genai.plugins` only handle
transport.
"""
import datetime
from typing import Any, Dict, List, Optional
from playhouse.shortcuts import model_to_dict
from frigate.config import CameraConfig, FrigateConfig
from frigate.config.classification import ObjectClassificationType
from frigate.config.ui import UnitSystemEnum
from frigate.data_processing.post.types import ReviewMetadata
from frigate.models import Event
def build_review_description_prompt(
review_data: dict[str, Any],
thumbnails: list[bytes],
concerns: list[str],
preferred_language: str | None,
activity_context_prompt: str,
) -> str:
"""Build the prompt for review activity description generation."""
def get_concern_prompt() -> str:
if concerns:
concern_list = "\n - ".join(concerns)
return (
"\n- `other_concerns` (list of strings): Include a list of any of "
"the following concerns that are occurring:\n"
f" - {concern_list}"
)
else:
return ""
def get_language_prompt() -> str:
if preferred_language:
return f"Provide your answer in {preferred_language}"
else:
return ""
def get_objects_list() -> str:
if review_data["unified_objects"]:
return "\n- " + "\n- ".join(review_data["unified_objects"])
else:
return "\n- (No objects detected)"
return f"""
Your task is to analyze a sequence of images taken in chronological order from a security camera.
## Normal Activity Patterns for This Property
{activity_context_prompt}
## Task Instructions
Describe the scene based on observable actions and movements, evaluate the activity against the Activity Indicators above, and assign a potential_threat_level (0, 1, or 2) by applying the threat level indicators consistently.
## Analysis Guidelines
When forming your description:
- **CRITICAL: Only describe objects explicitly listed in "Objects in Scene" below.** Do not infer or mention additional people, vehicles, or objects not present in this list, even if visual patterns suggest them. If only a car is listed, do not describe a person interacting with it unless "person" is also in the objects list.
- **Only describe actions actually visible in the frames.** Do not assume or infer actions that you don't observe happening. If someone walks toward furniture but you never see them sit, do not say they sat. Stick to what you can see across the sequence.
- Describe what you observe: actions, movements, interactions with objects and the environment. Include any observable environmental changes (e.g., lighting changes triggered by activity).
- Note visible details such as clothing, items being carried or placed, tools or equipment present, and how they interact with the property or objects.
- Consider the full sequence chronologically: what happens from start to finish, how duration and actions relate to the location and objects involved.
- **Use the actual timestamp provided in "Activity started at"** below for time of day contextdo not infer time from image brightness or darkness. Unusual hours (late night/early morning) should increase suspicion when the observable behavior itself appears questionable. However, recognize that some legitimate activities can occur at any hour.
- **Consider duration as a primary factor**: Apply the duration thresholds defined in the activity patterns above. Brief sequences during normal hours with apparent purpose typically indicate normal activity unless explicit suspicious actions are visible.
- **Weigh all evidence holistically**: Match the activity against the normal and suspicious patterns defined above, then evaluate based on the complete context (zone, objects, time, actions, duration). Apply the threat level indicators consistently. Use your judgment for edge cases.
## Response Field Guidelines
Respond with a JSON object matching the provided schema. Field-specific guidance:
- `observations`: Include the very start of the activity for example, a vehicle entering the frame or pulling into the driveway even if it lasts only a few frames and the rest of the clip is dominated by a longer activity. Include each arrival, departure, object handled, and notable change in position or state. Each item is a single concrete fact written as a complete sentence.
- `scene`: Describe how the sequence begins, then the progression of events all significant movements and actions in order. For example, if a vehicle arrives and then a person exits, describe both sequentially. For named subjects (those with a `` separator in "Objects in Scene"), always use their name do not replace them with generic terms. For unnamed objects (e.g., "person", "car"), refer to them naturally with articles (e.g., "a person", "the car"). Your description should align with and support the threat level you assign.
- `title`: Name the primary activity across the observations, together with the location. An activity is what is being done with objects, tools, or surfaces; locomotion through the scene qualifies as the activity only when no other interaction is observed. For named subjects, always use their name. For unnamed objects, refer to them naturally with articles.
- `shortSummary`: Briefly summarize the primary activity across the observations.
- `potential_threat_level`: Must be consistent with your scene description and the activity patterns above.
{get_concern_prompt()}
## Sequence Details
- Camera: {review_data["camera"]}
- Total frames: {len(thumbnails)} (Frame 1 = earliest, Frame {len(thumbnails)} = latest)
- Activity started at {review_data["start"]} and lasted {review_data["duration"]} seconds
- Zones involved: {", ".join(review_data["zones"]) if review_data["zones"] else "None"}
## Objects in Scene
Each line represents a detection state, not necessarily unique individuals. The `` symbol separates a recognized subject's name from their object type — use only the name (before the `←`) in your response, not the type after it. The same subject may appear across multiple lines if detected multiple times.
**Note: Unidentified objects (without names) are NOT indicators of suspicious activitythey simply mean the system hasn't identified that object.**
{get_objects_list()}
{get_language_prompt()}
"""
def build_review_description_response_format(concerns: list[str]) -> dict[str, Any]:
"""Build the structured-output JSON schema for review descriptions.
Strips the `time` field (populated server-side) and drops
`other_concerns` when no concerns are configured.
"""
schema = ReviewMetadata.model_json_schema()
schema.get("properties", {}).pop("time", None)
if "time" in schema.get("required", []):
schema["required"].remove("time")
if not concerns:
schema.get("properties", {}).pop("other_concerns", None)
if "other_concerns" in schema.get("required", []):
schema["required"].remove("other_concerns")
return {
"type": "json_schema",
"json_schema": {
"name": "review_metadata",
"strict": True,
"schema": schema,
},
}
def build_review_summary_prompt(
start_ts: float,
end_ts: float,
events: list[dict[str, Any]],
preferred_language: str | None,
) -> str:
"""Build the prompt for a multi-event review summary."""
time_range = (
f"{datetime.datetime.fromtimestamp(start_ts).strftime('%B %d, %Y at %I:%M %p')}"
f" to "
f"{datetime.datetime.fromtimestamp(end_ts).strftime('%B %d, %Y at %I:%M %p')}"
)
prompt = f"""
You are a security officer writing a concise security report.
Time range: {time_range}
Input format: Each event is a JSON object with:
- "title", "scene", "confidence", "potential_threat_level" (0-2), "other_concerns", "camera", "time", "start_time", "end_time"
- "context": array of related events from other cameras that occurred during overlapping time periods
**Note: Use the "scene" field for event descriptions in the report. Ignore any "shortSummary" field if present.**
Report Structure - Use this EXACT format:
# Security Summary - {time_range}
## Overview
[Write 1-2 sentences summarizing the overall activity pattern during this period.]
---
## Timeline
[Group events by time periods (e.g., "Morning (6:00 AM - 12:00 PM)", "Afternoon (12:00 PM - 5:00 PM)", "Evening (5:00 PM - 9:00 PM)", "Night (9:00 PM - 6:00 AM)"). Use appropriate time blocks based on when events occurred.]
### [Time Block Name]
**HH:MM AM/PM** | [Camera Name] | [Threat Level Indicator]
- [Event title]: [Clear description incorporating contextual information from the "context" array]
- Context: [If context array has items, mention them here, e.g., "Delivery truck present on Front Driveway Cam (HH:MM AM/PM)"]
- Assessment: [Brief assessment incorporating context - if context explains the event, note it here]
[Repeat for each event in chronological order within the time block]
---
## Summary
[One sentence summarizing the period. If all events are normal/explained: "Routine activity observed." If review needed: "Some activity requires review but no security concerns." If security concerns: "Security concerns requiring immediate attention."]
Guidelines:
- List ALL events in chronological order, grouped by time blocks
- Threat level indicators: Normal, Needs review, 🔴 Security concern
- Integrate contextual information naturally - use the "context" array to enrich each event's description
- If context explains the event (e.g., delivery truck explains person at door), describe it accordingly (e.g., "delivery person" not "unidentified person")
- Be concise but informative - focus on what happened and what it means
- If contextual information makes an event clearly normal, reflect that in your assessment
- Only create time blocks that have events - don't create empty sections
"""
prompt += "\n\nEvents:\n"
for event in events:
prompt += f"\n{event}\n"
if preferred_language:
prompt += f"\nProvide your answer in {preferred_language}"
return prompt
def build_object_description_prompt(
camera_config: CameraConfig,
event: Event,
) -> str:
"""Build the prompt for a per-object description.
Pulls the per-label override from `objects.genai.object_prompts`, falling
back to the camera default, and interpolates event fields.
Raises:
KeyError: if the user-defined prompt template references an unknown
event field.
"""
template = camera_config.objects.genai.object_prompts.get(
str(event.label),
camera_config.objects.genai.prompt,
)
return template.format(**model_to_dict(event))
def get_attribute_classifications(config: FrigateConfig) -> List[Dict[str, Any]]:
"""Return enabled custom classification models of `attribute` type.
Each entry: {"name": <model name>, "objects": [<object label>, ...]}.
These models attach attribute metadata to events on the listed object
types, which can later be filtered via the search_objects `attribute`
field.
"""
result: List[Dict[str, Any]] = []
for model_key, model_config in config.classification.custom.items():
if not model_config.enabled or model_config.object_config is None:
continue
if (
model_config.object_config.classification_type
!= ObjectClassificationType.attribute
):
continue
result.append(
{
"name": model_config.name or model_key,
"objects": list(model_config.object_config.objects or []),
}
)
return result
def get_tool_definitions(
semantic_search_enabled: bool = False,
attribute_classifications: Optional[List[Dict[str, Any]]] = None,
) -> List[Dict[str, Any]]:
"""
Get OpenAI-compatible tool definitions for Frigate.
Returns a list of tool definitions that can be used with OpenAI-compatible
function calling APIs. When semantic search is enabled, the search_objects
tool exposes an additional `semantic_query` parameter for descriptive
queries (e.g. "person riding a lawn mower") and find_similar_objects is
included. When attribute classification models are configured, an
`attribute` parameter is exposed for filtering by their labels.
"""
search_objects_properties: Dict[str, Any] = {
"camera": {
"type": "string",
"description": "Camera name to filter by (optional).",
},
"label": {
"type": "string",
"description": (
"Generic object class to filter by — one of the tracked detector "
"labels such as 'person', 'package', 'car', 'dog', 'bird'. Use "
"this for broad queries like 'show me all cars today'. Combine "
"with semantic_query when the user also describes appearance or "
"behavior (e.g. label='person', semantic_query='riding a lawn "
"mower')."
),
},
"sub_label": {
"type": "string",
"description": (
"Filter by a DISCRETE NAMED entity recognized in the detection. "
"Use this for: a known person's name ('John'), a delivery "
"company ('Amazon', 'UPS'), a recognized animal species or "
"breed ('blue jay', 'cardinal', 'golden retriever'), or a "
"license plate string. When filtering by a specific name, set "
"only sub_label and leave label unset. Do NOT use sub_label "
"for descriptions of appearance, clothing, or actions — those "
"belong in semantic_query."
),
},
"after": {
"type": "string",
"description": "Start time in ISO 8601 format (e.g., '2024-01-01T00:00:00Z').",
},
"before": {
"type": "string",
"description": "End time in ISO 8601 format (e.g., '2024-01-01T23:59:59Z').",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "List of zone names to filter by.",
},
"limit": {
"type": "integer",
"description": "Maximum number of objects to return (default: 25).",
"default": 25,
},
}
if attribute_classifications:
model_outline = "; ".join(
f"{m['name']} (applies to {', '.join(m['objects']) or 'any object'})"
for m in attribute_classifications
)
search_objects_properties["attribute"] = {
"type": "string",
"description": (
"Filter by a classification attribute label produced by a "
"configured attribute classification model. Use this INSTEAD "
"of semantic_query when the user's request matches one of "
"these classifications. Configured models: "
f"{model_outline}. "
"Set the value to the attribute label that matches the user's "
"phrasing (case-sensitive)."
),
}
if semantic_search_enabled:
search_objects_properties["semantic_query"] = {
"type": "string",
"description": (
"Optional natural-language description of a PHYSICAL "
"CHARACTERISTIC, APPEARANCE, or ACTIVITY the user mentioned, "
"used to semantically narrow results. Only set this when the "
"user describes something beyond what label and sub_label can "
"express on their own.\n"
"USE for descriptive phrases like: 'riding a lawn mower', "
"'wearing a red jacket', 'carrying a package', 'walking a "
"dog', 'on a bicycle', 'holding an umbrella'.\n"
"DO NOT USE for:\n"
"- specific named people, pets, or delivery companies → use sub_label\n"
"- animal species or breed names like 'blue jay', 'cardinal', "
"'golden retriever' → use sub_label\n"
"- license plate strings → use sub_label\n"
"- generic object queries like 'all cars today' or 'every "
"person' → use label alone with no semantic_query\n"
"When set, combine with label/time/camera/zone filters as "
"usual (e.g. label='person', semantic_query='riding a lawn "
"mower', after='2024-05-01T00:00:00Z')."
),
}
search_objects_description = (
"Search the historical record of detected objects in Frigate. "
"Use this ONLY for questions about the PAST — e.g. 'did anyone come by today?', "
"'when was the last car?', 'show me detections from yesterday'. "
"Do NOT use this for monitoring or alerting requests about future events — "
"use start_camera_watch instead for those. "
"An 'object' in Frigate represents a tracked detection (e.g., a person, package, car).\n\n"
"Choose filters based on what the user is asking for:\n"
"- Generic class query ('show me all cars today'): set `label` only.\n"
"- Specific NAMED entity (known person, delivery company, animal "
"species/breed like 'blue jay' or 'golden retriever', license "
"plate): set `sub_label` only and leave `label` unset.\n"
)
if semantic_search_enabled:
search_objects_description += (
"- Physical CHARACTERISTIC, APPEARANCE, or ACTIVITY that is not a "
"discrete name ('person riding a lawn mower', 'someone in a red "
"jacket', 'person carrying a package'): set `semantic_query` with "
"the descriptive phrase, optionally alongside `label` for the "
"object class. Do NOT put descriptive phrases in sub_label."
)
return [
{
"type": "function",
"function": {
"name": "search_objects",
"description": search_objects_description,
"parameters": {
"type": "object",
"properties": search_objects_properties,
},
"required": [],
},
},
{
"type": "function",
"function": {
"name": "find_similar_objects",
"description": (
"Find tracked objects that are visually and semantically similar "
"to a specific past event. Use this when the user references a "
"particular object they have seen and wants to find other "
"sightings of the same or similar one ('that green car', 'the "
"person in the red jacket', 'the package that was delivered'). "
"Prefer this over search_objects whenever the user's intent is "
"'find more like this specific one.' Use search_objects first "
"only if you need to locate the anchor event. Requires semantic "
"search to be enabled."
),
"parameters": {
"type": "object",
"properties": {
"event_id": {
"type": "string",
"description": "The id of the anchor event to find similar objects to.",
},
"after": {
"type": "string",
"description": "Start time in ISO 8601 format (e.g., '2024-01-01T00:00:00Z').",
},
"before": {
"type": "string",
"description": "End time in ISO 8601 format (e.g., '2024-01-01T23:59:59Z').",
},
"cameras": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of cameras to restrict to. Defaults to all.",
},
"labels": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of labels to restrict to. Defaults to the anchor event's label.",
},
"sub_labels": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of sub_labels (names) to restrict to.",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "Optional list of zones. An event matches if any of its zones overlap.",
},
"similarity_mode": {
"type": "string",
"enum": ["visual", "semantic", "fused"],
"description": "Which similarity signal(s) to use. 'fused' (default) combines visual and semantic.",
"default": "fused",
},
"min_score": {
"type": "number",
"description": "Drop matches with a similarity score below this threshold (0.0-1.0).",
},
"limit": {
"type": "integer",
"description": "Maximum number of matches to return (default: 10).",
"default": 10,
},
},
"required": ["event_id"],
},
},
},
{
"type": "function",
"function": {
"name": "set_camera_state",
"description": (
"Change a camera's feature state (e.g., turn detection on/off, enable/disable recordings). "
"Use camera='*' to apply to all cameras at once. "
"Only call this tool when the user explicitly asks to change a camera setting. "
"Requires admin privileges."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to target, or '*' to target all cameras.",
},
"feature": {
"type": "string",
"enum": [
"detect",
"record",
"snapshots",
"audio",
"motion",
"enabled",
"birdseye",
"birdseye_mode",
"improve_contrast",
"ptz_autotracker",
"motion_contour_area",
"motion_threshold",
"notifications",
"audio_transcription",
"review_alerts",
"review_detections",
"object_descriptions",
"review_descriptions",
"profile",
],
"description": (
"The feature to change. Most features accept ON or OFF. "
"birdseye_mode accepts CONTINUOUS, MOTION, or OBJECTS. "
"motion_contour_area and motion_threshold accept a number. "
"profile accepts a profile name or 'none' to deactivate (requires camera='*')."
),
},
"value": {
"type": "string",
"description": "The value to set. ON or OFF for toggles, a number for thresholds, a profile name or 'none' for profile.",
},
},
"required": ["camera", "feature", "value"],
},
},
},
{
"type": "function",
"function": {
"name": "get_live_context",
"description": (
"Get the current live image and detection information for a camera: objects being tracked, "
"zones, timestamps. Use this to understand what is visible in the live view. "
"Call this when answering questions about what is happening right now on a specific camera."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to get live context for.",
},
},
"required": ["camera"],
},
},
},
{
"type": "function",
"function": {
"name": "start_camera_watch",
"description": (
"Start a continuous VLM watch job that monitors a camera and sends a notification "
"when a specified condition is met. Use this when the user wants to be alerted about "
"a future event, e.g. 'tell me when guests arrive' or 'notify me when the package is picked up'. "
"Only one watch job can run at a time. Returns a job ID."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera ID to monitor.",
},
"condition": {
"type": "string",
"description": (
"Natural-language description of the condition to watch for, "
"e.g. 'a person arrives at the front door'."
),
},
"max_duration_minutes": {
"type": "integer",
"description": "Maximum time to watch before giving up (minutes, default 60).",
"default": 60,
},
"labels": {
"type": "array",
"items": {"type": "string"},
"description": "Object labels that should trigger a VLM check (e.g. ['person', 'car']). If omitted, any detection on the camera triggers a check.",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "Zone names to filter by. If specified, only detections in these zones trigger a VLM check.",
},
},
"required": ["camera", "condition"],
},
},
},
{
"type": "function",
"function": {
"name": "stop_camera_watch",
"description": (
"Cancel the currently running VLM watch job. Use this when the user wants to "
"stop a previously started watch, e.g. 'stop watching the front door'."
),
"parameters": {
"type": "object",
"properties": {},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "get_profile_status",
"description": (
"Get the current profile status including the active profile and "
"timestamps of when each profile was last activated. Use this to "
"determine time periods for recap requests — e.g. when the user asks "
"'what happened while I was away?', call this first to find the relevant "
"time window based on profile activation history."
),
"parameters": {
"type": "object",
"properties": {},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "get_recap",
"description": (
"Get a recap of all activity (alerts and detections) for a given time period. "
"Use this after calling get_profile_status to retrieve what happened during "
"a specific window — e.g. 'what happened while I was away?'. Returns a "
"chronological list of activity with camera, objects, zones, and GenAI-generated "
"descriptions when available. Summarize the results for the user."
),
"parameters": {
"type": "object",
"properties": {
"after": {
"type": "string",
"description": "Start of the time period in ISO 8601 format (e.g. '2025-03-15T08:00:00').",
},
"before": {
"type": "string",
"description": "End of the time period in ISO 8601 format (e.g. '2025-03-15T17:00:00').",
},
"cameras": {
"type": "string",
"description": "Comma-separated camera IDs to include, or 'all' for all cameras. Default is 'all'.",
},
"severity": {
"type": "string",
"enum": ["alert", "detection"],
"description": "Filter by severity level. Omit to include both alerts and detections.",
},
},
"required": ["after", "before"],
},
},
},
]
def build_chat_system_prompt(
config: FrigateConfig,
allowed_cameras: List[str],
semantic_search_enabled: bool,
attribute_classifications: List[Dict[str, Any]],
) -> str:
"""Build the system prompt for the chat completion endpoint.
Composes the static framing with conditional sections describing the
available cameras, speed units, semantic-search routing guidance, and
configured attribute classifications.
"""
current_datetime = datetime.datetime.now()
current_date_str = current_datetime.strftime("%Y-%m-%d")
current_time_str = current_datetime.strftime("%I:%M:%S %p")
cameras_info: List[str] = []
has_speed_zone = False
for camera_id in allowed_cameras:
if camera_id not in config.cameras:
continue
camera_config = config.cameras[camera_id]
friendly_name = (
camera_config.friendly_name
if camera_config.friendly_name
else camera_id.replace("_", " ").title()
)
zone_names = list(camera_config.zones.keys())
if not has_speed_zone:
has_speed_zone = any(
zone.distances for zone in camera_config.zones.values()
)
if zone_names:
cameras_info.append(
f" - {friendly_name} (ID: {camera_id}, zones: {', '.join(zone_names)})"
)
else:
cameras_info.append(f" - {friendly_name} (ID: {camera_id})")
cameras_section = ""
if cameras_info:
cameras_section = (
"\n\nAvailable cameras:\n"
+ "\n".join(cameras_info)
+ "\n\nWhen users refer to cameras by their friendly name (e.g., 'Back Deck Camera'), use the corresponding camera ID (e.g., 'back_deck_cam') in tool calls."
)
speed_units_section = ""
if has_speed_zone:
speed_unit = (
"mph" if config.ui.unit_system == UnitSystemEnum.imperial else "km/h"
)
speed_units_section = f"\n\nReport object speeds to the user in {speed_unit}."
semantic_search_section = ""
if semantic_search_enabled:
semantic_search_section = (
"\n\nWhen routing a search_objects call, pick filters by the shape of the user's request:\n"
"- Generic class ('show me all cars today'): set `label` only.\n"
"- Specific named entity — a known person ('John'), delivery company ('Amazon'), animal species/breed ('blue jay', 'cardinal', 'golden retriever'), or license plate: set `sub_label` only and leave `label` unset.\n"
"- Physical characteristic, appearance, or activity that is NOT a discrete name ('find me people riding a lawn mower', 'someone in a red jacket', 'a person carrying a package'): set `semantic_query` with the descriptive phrase, optionally combined with `label` for the object class. Never put descriptive phrases in `sub_label`."
)
attribute_classification_section = ""
if attribute_classifications:
model_lines = "\n".join(
f"- {m['name']}: applies to {', '.join(m['objects']) or 'any object'}"
for m in attribute_classifications
)
attribute_classification_section = (
"\n\nAttribute classification models are configured for the following object types:\n"
f"{model_lines}\n"
"When the user's request matches one of these classifications, set the search_objects `attribute` field to the matching label rather than using `semantic_query`. Reserve `semantic_query` for descriptive phrases that fall outside the configured attribute labels."
)
return f"""You are a helpful assistant for Frigate, a security camera NVR system. You help users answer questions about their cameras, detected objects, and events.
Current server local date and time: {current_date_str} at {current_time_str}
Do not start your response with phrases like "I will check...", "Let me see...", or "Let me look...". Answer directly.
Always present times to the user in the server's local timezone. When tool results include start_time_local and end_time_local, use those exact strings when listing or describing detection times—do not convert or invent timestamps. Do not use UTC or ISO format with Z for the user-facing answer unless the tool result only provides Unix timestamps without local time fields.
When users ask about "today", "yesterday", "this week", etc., use the current date above as reference.
When searching for objects or events, use ISO 8601 format for dates (e.g., {current_date_str}T00:00:00Z for the start of today).
Always be accurate with time calculations based on the current date provided.
When a user refers to a specific object they have seen or describe with identifying details ("that green car", "the person in the red jacket", "a package left today"), prefer the find_similar_objects tool over search_objects. Use search_objects first only to locate the anchor event, then pass its id to find_similar_objects. For generic queries like "show me all cars today", keep using search_objects. If a user message begins with [attached_event:<id>], treat that event id as the anchor for any similarity or "tell me more" request in the same message and call find_similar_objects with that id.{semantic_search_section}{attribute_classification_section}{cameras_section}{speed_units_section}"""
-488
View File
@@ -1,488 +0,0 @@
"""Debug replay startup job: ffmpeg concat + camera config publish.
The runner orchestrates the async portion of starting a debug replay
session. The DebugReplayManager (in frigate.debug_replay) owns session
presence so the status bar can keep reading a single `active` flag from
/debug_replay/status for the entire session window which is broader
than this job's lifetime.
"""
import logging
import os
import subprocess as sp
import threading
import time
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Optional, cast
from peewee import ModelSelect
from frigate.config import FrigateConfig
from frigate.config.camera.updater import CameraConfigUpdatePublisher
from frigate.const import REPLAY_CAMERA_PREFIX, REPLAY_DIR
from frigate.jobs.export import JobStatePublisher
from frigate.jobs.job import Job
from frigate.jobs.manager import job_is_running, set_current_job
from frigate.models import Export, Recordings
from frigate.types import JobStatusTypesEnum
from frigate.util.ffmpeg import run_ffmpeg_with_progress
if TYPE_CHECKING:
from frigate.debug_replay import DebugReplayManager
logger = logging.getLogger(__name__)
# Coalesce frequent ffmpeg progress callbacks so the WS isn't flooded.
PROGRESS_BROADCAST_MIN_INTERVAL = 1.0
JOB_TYPE = "debug_replay"
STEP_PREPARING_CLIP = "preparing_clip"
STEP_STARTING_CAMERA = "starting_camera"
_active_runner: Optional["DebugReplayJobRunner"] = None
_runner_lock = threading.Lock()
def _set_active_runner(runner: Optional["DebugReplayJobRunner"]) -> None:
global _active_runner
with _runner_lock:
_active_runner = runner
def get_active_runner() -> Optional["DebugReplayJobRunner"]:
with _runner_lock:
return _active_runner
@dataclass
class DebugReplayJob(Job):
"""Job state for a debug replay startup."""
job_type: str = JOB_TYPE
source_camera: str = ""
replay_camera_name: str = ""
start_ts: float = 0.0
end_ts: float = 0.0
current_step: Optional[str] = None
progress_percent: float = 0.0
def to_dict(self) -> dict[str, Any]:
"""Whitelisted payload for the job_state WS topic.
Replay-specific fields land in results so the frontend's
generic Job<TResults> type can be parameterised cleanly.
"""
return {
"id": self.id,
"job_type": self.job_type,
"status": self.status,
"start_time": self.start_time,
"end_time": self.end_time,
"error_message": self.error_message,
"results": {
"current_step": self.current_step,
"progress_percent": self.progress_percent,
"source_camera": self.source_camera,
"replay_camera_name": self.replay_camera_name,
"start_ts": self.start_ts,
"end_ts": self.end_ts,
},
}
def query_recordings(source_camera: str, start_ts: float, end_ts: float) -> ModelSelect:
"""Return the Recordings query for the time range.
Module-level so tests can patch it without instantiating a runner.
"""
query = (
Recordings.select(
Recordings.path,
Recordings.start_time,
Recordings.end_time,
)
.where(
Recordings.start_time.between(start_ts, end_ts)
| Recordings.end_time.between(start_ts, end_ts)
| ((start_ts > Recordings.start_time) & (end_ts < Recordings.end_time))
)
.where(Recordings.camera == source_camera)
.order_by(Recordings.start_time.asc())
)
return cast(ModelSelect, query)
class DebugReplaySource(ABC):
"""Abstract source for a debug replay session.
Provides the camera identity and time range the replay represents,
validates that usable content exists, and supplies the ffmpeg input
args used to build the replay clip.
"""
@property
@abstractmethod
def source_camera(self) -> str:
"""Camera name the replay is derived from."""
@property
@abstractmethod
def start_ts(self) -> float:
"""Unix timestamp marking the start of the replay range."""
@property
@abstractmethod
def end_ts(self) -> float:
"""Unix timestamp marking the end of the replay range."""
@abstractmethod
def validate(self) -> None:
"""Raise ValueError if the source has no usable content."""
@abstractmethod
def ffmpeg_input_args(self, working_dir: str) -> list[str]:
"""Return ffmpeg input args (including -i). May write temp files in working_dir."""
def cleanup(self, working_dir: str) -> None:
"""Remove any temp files the source created in working_dir. Default no-op."""
class RecordingDebugReplaySource(DebugReplaySource):
"""Replay source backed by the Recordings table.
Builds a concat playlist of recording files covering the time range
and feeds it to ffmpeg's concat demuxer.
"""
def __init__(self, source_camera: str, start_ts: float, end_ts: float) -> None:
self._camera = source_camera
self._start_ts = start_ts
self._end_ts = end_ts
self._concat_file: Optional[str] = None
@property
def source_camera(self) -> str:
return self._camera
@property
def start_ts(self) -> float:
return self._start_ts
@property
def end_ts(self) -> float:
return self._end_ts
def validate(self) -> None:
if self._end_ts <= self._start_ts:
raise ValueError("End time must be after start time")
if not query_recordings(self._camera, self._start_ts, self._end_ts).count():
raise ValueError(
f"No recordings found for camera '{self._camera}' in the specified time range"
)
def ffmpeg_input_args(self, working_dir: str) -> list[str]:
replay_name = f"{REPLAY_CAMERA_PREFIX}{self._camera}"
concat_file = os.path.join(working_dir, f"{replay_name}_concat.txt")
recordings = query_recordings(self._camera, self._start_ts, self._end_ts)
with open(concat_file, "w") as f:
for recording in recordings:
f.write(f"file '{recording.path}'\n")
self._concat_file = concat_file
return ["-f", "concat", "-safe", "0", "-i", concat_file]
def cleanup(self, working_dir: str) -> None:
if self._concat_file:
_remove_silent(self._concat_file)
class ExportDebugReplaySource(DebugReplaySource):
"""Replay source backed by an existing Export.
Uses the export's video file directly as the ffmpeg input — does not
require recordings to still exist for the time range.
"""
def __init__(self, export: Export, duration: float) -> None:
self._camera = cast(str, export.camera)
# Export.date is declared DateTimeField but Frigate writes raw unix
# timestamps to the column.
self._start_ts = float(cast(Any, export.date))
self._video_path = cast(str, export.video_path)
self._duration = duration
@property
def source_camera(self) -> str:
return self._camera
@property
def start_ts(self) -> float:
return self._start_ts
@property
def end_ts(self) -> float:
return self._start_ts + self._duration
def validate(self) -> None:
if not os.path.exists(self._video_path):
raise ValueError(f"Export video file not found: {self._video_path}")
def ffmpeg_input_args(self, working_dir: str) -> list[str]:
return ["-i", self._video_path]
class DebugReplayJobRunner(threading.Thread):
"""Worker thread that drives the startup job to completion.
Owns the live ffmpeg Popen reference for cancellation. Cancellation
is two-step (threading.Event + proc.terminate()) so the runner
both knows it should stop and is unblocked from its blocking subprocess
wait.
"""
def __init__(
self,
job: DebugReplayJob,
source: DebugReplaySource,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
replay_manager: "DebugReplayManager",
publisher: Optional[JobStatePublisher] = None,
) -> None:
super().__init__(daemon=True, name=f"debug_replay_{job.id}")
self.job = job
self.source = source
self.frigate_config = frigate_config
self.config_publisher = config_publisher
self.replay_manager = replay_manager
self.publisher = publisher if publisher is not None else JobStatePublisher()
self._cancel_event = threading.Event()
self._active_process: sp.Popen | None = None
self._proc_lock = threading.Lock()
self._last_broadcast_monotonic: float = 0.0
def cancel(self) -> None:
"""Request cancellation. Idempotent."""
self._cancel_event.set()
with self._proc_lock:
proc = self._active_process
if proc is not None:
try:
proc.terminate()
except Exception as exc:
logger.warning("Failed to terminate ffmpeg subprocess: %s", exc)
def is_cancelled(self) -> bool:
return self._cancel_event.is_set()
def _record_proc(self, proc: sp.Popen) -> None:
with self._proc_lock:
self._active_process = proc
# Race: cancel arrived between Popen and _record_proc.
if self._cancel_event.is_set():
try:
proc.terminate()
except Exception:
pass
def _broadcast(self, force: bool = False) -> None:
now = time.monotonic()
if (
not force
and now - self._last_broadcast_monotonic < PROGRESS_BROADCAST_MIN_INTERVAL
):
return
self._last_broadcast_monotonic = now
try:
self.publisher.publish(self.job.to_dict())
except Exception as err:
logger.warning("Publisher raised during job state broadcast: %s", err)
def run(self) -> None:
replay_name = self.job.replay_camera_name
os.makedirs(REPLAY_DIR, exist_ok=True)
clip_path = os.path.join(REPLAY_DIR, f"{replay_name}.mp4")
self.job.status = JobStatusTypesEnum.running
self.job.start_time = time.time()
self.job.current_step = STEP_PREPARING_CLIP
self._broadcast(force=True)
try:
input_args = self.source.ffmpeg_input_args(REPLAY_DIR)
ffmpeg_cmd = [
self.frigate_config.ffmpeg.ffmpeg_path,
"-hide_banner",
"-y",
*input_args,
"-c",
"copy",
"-movflags",
"+faststart",
clip_path,
]
logger.info(
"Generating replay clip for %s (%.1f - %.1f)",
self.job.source_camera,
self.job.start_ts,
self.job.end_ts,
)
def _on_progress(percent: float) -> None:
self.job.progress_percent = percent
self._broadcast()
try:
returncode, stderr = run_ffmpeg_with_progress(
ffmpeg_cmd,
expected_duration_seconds=max(
0.0, self.job.end_ts - self.job.start_ts
),
on_progress=_on_progress,
process_started=self._record_proc,
use_low_priority=True,
)
finally:
with self._proc_lock:
self._active_process = None
if self._cancel_event.is_set():
self._finalize_cancelled(clip_path)
return
if returncode != 0:
raise RuntimeError(f"FFmpeg failed: {stderr[-500:]}")
if not os.path.exists(clip_path):
raise RuntimeError("Clip file was not created")
self.job.current_step = STEP_STARTING_CAMERA
self.job.progress_percent = 100.0
self._broadcast(force=True)
if self._cancel_event.is_set():
self._finalize_cancelled(clip_path)
return
self.replay_manager.publish_camera(
source_camera=self.job.source_camera,
replay_name=replay_name,
clip_path=clip_path,
frigate_config=self.frigate_config,
config_publisher=self.config_publisher,
)
self.replay_manager.mark_session_ready(clip_path)
self.job.status = JobStatusTypesEnum.success
self.job.end_time = time.time()
self._broadcast(force=True)
logger.info(
"Debug replay started: %s -> %s",
self.job.source_camera,
replay_name,
)
except Exception as exc:
logger.exception("Debug replay startup failed")
self.job.status = JobStatusTypesEnum.failed
self.job.error_message = str(exc)
self.job.end_time = time.time()
self._broadcast(force=True)
self.replay_manager.clear_session()
_remove_silent(clip_path)
finally:
self.source.cleanup(REPLAY_DIR)
_set_active_runner(None)
def _finalize_cancelled(self, clip_path: str) -> None:
logger.info("Debug replay startup cancelled")
self.job.status = JobStatusTypesEnum.cancelled
self.job.end_time = time.time()
self._broadcast(force=True)
# The caller of cancel_debug_replay_job (DebugReplayManager.stop) owns
# session cleanup — db rows, filesystem artifacts, clear_session. We
# only clean up the partial concat output we created.
_remove_silent(clip_path)
def _remove_silent(path: str) -> None:
try:
if os.path.exists(path):
os.remove(path)
except OSError:
pass
def start_debug_replay_job(
*,
source: DebugReplaySource,
frigate_config: FrigateConfig,
config_publisher: CameraConfigUpdatePublisher,
replay_manager: "DebugReplayManager",
) -> str:
"""Validate, create job, start runner. Returns the job id.
Raises ValueError for an invalid source (camera missing, source has
no usable content) and RuntimeError if a session is already active.
"""
if job_is_running(JOB_TYPE) or replay_manager.active:
raise RuntimeError("A replay session is already active")
if source.source_camera not in frigate_config.cameras:
raise ValueError(f"Camera '{source.source_camera}' not found")
source.validate()
replay_name = f"{REPLAY_CAMERA_PREFIX}{source.source_camera}"
replay_manager.mark_starting(
source_camera=source.source_camera,
replay_camera_name=replay_name,
start_ts=source.start_ts,
end_ts=source.end_ts,
)
job = DebugReplayJob(
source_camera=source.source_camera,
replay_camera_name=replay_name,
start_ts=source.start_ts,
end_ts=source.end_ts,
)
set_current_job(job)
runner = DebugReplayJobRunner(
job=job,
source=source,
frigate_config=frigate_config,
config_publisher=config_publisher,
replay_manager=replay_manager,
)
_set_active_runner(runner)
runner.start()
return job.id
def cancel_debug_replay_job() -> bool:
"""Signal the active runner to cancel.
Returns True if a runner was signalled, False if no job was active.
"""
runner = get_active_runner()
if runner is None:
return False
runner.cancel()
return True
def wait_for_runner(timeout: float = 2.0) -> bool:
"""Join the active runner. Returns True if the runner ended in time."""
runner = get_active_runner()
if runner is None:
return True
runner.join(timeout=timeout)
return not runner.is_alive()
-3
View File
@@ -45,7 +45,6 @@ class VLMWatchJob(Job):
last_reasoning: str = ""
notification_message: str = ""
iteration_count: int = 0
username: str = ""
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@@ -375,7 +374,6 @@ def start_vlm_watch_job(
dispatcher: Any,
labels: list[str] | None = None,
zones: list[str] | None = None,
username: str = "",
) -> str:
"""Start a new VLM watch job. Returns the job ID.
@@ -399,7 +397,6 @@ def start_vlm_watch_job(
max_duration_minutes=max_duration_minutes,
labels=labels or [],
zones=zones or [],
username=username,
)
cancel_ev = threading.Event()
_current_job = job
+6 -11
View File
@@ -62,10 +62,8 @@ def get_canvas_shape(width: int, height: int) -> tuple[int, int]:
if round(a_w / a_h, 2) != round(width / height, 2):
canvas_width = int(width // 4 * 4)
canvas_height = int((canvas_width / a_w * a_h) // 4 * 4)
logger.error(
f"Birdseye resolution {width}x{height} is not a supported aspect ratio "
f"and may cause visual distortion; falling back to {canvas_width}x{canvas_height}. "
f"Set width and height to a supported aspect ratio (16:9, 20:10, 16:6, 32:9, 12:9, 22:15, 9:16, 9:12, 16:3, or 1:1)"
logger.warning(
f"The birdseye resolution is a non-standard aspect ratio, forcing birdseye resolution to {canvas_width} x {canvas_height}"
)
return (canvas_width, canvas_height)
@@ -798,18 +796,15 @@ class Birdseye:
websocket_server: Any,
) -> None:
self.config = config
canvas_width, canvas_height = get_canvas_shape(
config.birdseye.width, config.birdseye.height
)
self.input: queue.Queue[bytes] = queue.Queue(maxsize=10)
self.converter = FFMpegConverter(
config.ffmpeg,
self.input,
stop_event,
canvas_width,
canvas_height,
canvas_width,
canvas_height,
config.birdseye.width,
config.birdseye.height,
config.birdseye.width,
config.birdseye.height,
config.birdseye.quality,
config.birdseye.restream,
)
-7
View File
@@ -349,13 +349,6 @@ def move_preview_frames(loc: str) -> None:
if not os.path.exists(preview_holdover):
return
if not os.access(preview_holdover, os.R_OK | os.W_OK):
logger.error(
"Insufficient permissions on preview restart cache at %s",
preview_holdover,
)
return
shutil.move(preview_holdover, preview_cache)
except shutil.Error:
logger.error("Failed to restore preview cache.")
+3 -6
View File
@@ -361,17 +361,14 @@ class PreviewRecorder:
small_frame,
cv2.COLOR_YUV2BGR_I420,
)
cache_path = get_cache_image_name(self.camera_name, frame_time)
if not cv2.imwrite(
cache_path,
cv2.imwrite(
get_cache_image_name(self.camera_name, frame_time),
small_frame,
[
int(cv2.IMWRITE_WEBP_QUALITY),
PREVIEW_QUALITY_WEBP[self.config.record.preview.quality],
],
):
logger.error("Failed to write preview frame to %s", cache_path)
)
def write_data(
self,
+3 -5
View File
@@ -351,11 +351,9 @@ class RecordingCleanup(threading.Thread):
)
.where(
ReviewSegment.camera == camera,
# candidate recordings can extend up to continuous_expire_date
# (the no-motion no-audio branch of the recordings query),
# so reviews must cover that full range to avoid deleting
# segments that overlap recent alerts/detections.
ReviewSegment.start_time < continuous_expire_date,
# need to ensure segments for all reviews starting
# before the expire date are included
ReviewSegment.start_time < motion_expire_date,
)
.order_by(ReviewSegment.start_time)
.namedtuples()
+96 -54
View File
@@ -13,7 +13,6 @@ from enum import Enum
from pathlib import Path
from typing import Callable, Optional
import pytz # type: ignore[import-untyped]
from peewee import DoesNotExist
from frigate.config import FfmpegConfig, FrigateConfig
@@ -23,13 +22,13 @@ from frigate.const import (
EXPORT_DIR,
MAX_PLAYLIST_SECONDS,
PREVIEW_FRAME_TYPE,
PROCESS_PRIORITY_LOW,
)
from frigate.ffmpeg_presets import (
EncodeTypeEnum,
parse_preset_hardware_acceleration_encode,
)
from frigate.models import Export, Previews, Recordings, ReviewSegment
from frigate.util.ffmpeg import run_ffmpeg_with_progress
from frigate.util.time import is_current_hour
logger = logging.getLogger(__name__)
@@ -243,43 +242,109 @@ class RecordingExporter(threading.Thread):
return total
def _inject_progress_flags(self, ffmpeg_cmd: list[str]) -> list[str]:
"""Insert FFmpeg progress reporting flags before the output path.
``-progress pipe:2`` writes structured key=value lines to stderr,
``-nostats`` suppresses the noisy default stats output.
"""
if not ffmpeg_cmd:
return ffmpeg_cmd
return ffmpeg_cmd[:-1] + ["-progress", "pipe:2", "-nostats", ffmpeg_cmd[-1]]
def _run_ffmpeg_with_progress(
self,
ffmpeg_cmd: list[str],
playlist_lines: str | list[str],
step: str = "encoding",
) -> tuple[int, str]:
"""Delegate to the shared helper, mapping percent → (step, percent).
"""Run an FFmpeg export command, parsing progress events from stderr.
Returns ``(returncode, captured_stderr)``.
Returns ``(returncode, captured_stderr)``. Stdout is left attached to
the parent process so we don't have to drain it (and risk a deadlock
if the buffer fills). Progress percent is computed against the
expected output duration; values are clamped to [0, 100] inside
:py:meth:`_emit_progress`.
"""
cmd = ["nice", "-n", str(PROCESS_PRIORITY_LOW)] + self._inject_progress_flags(
ffmpeg_cmd
)
if isinstance(playlist_lines, list):
stdin_payload = "\n".join(playlist_lines)
else:
stdin_payload = playlist_lines
return run_ffmpeg_with_progress(
ffmpeg_cmd,
expected_duration_seconds=self._expected_output_duration_seconds(),
on_progress=lambda percent: self._emit_progress(step, percent),
stdin_payload=stdin_payload,
use_low_priority=True,
expected_duration = self._expected_output_duration_seconds()
self._emit_progress(step, 0.0)
proc = sp.Popen(
cmd,
stdin=sp.PIPE,
stderr=sp.PIPE,
text=True,
encoding="ascii",
errors="replace",
)
def get_datetime_from_timestamp(self, timestamp: int) -> str:
# return in iso format using the configured ui.timezone when set,
# so the auto-generated export name reflects local time rather
# than the container's UTC clock
tz_name = self.config.ui.timezone
if tz_name:
assert proc.stdin is not None
assert proc.stderr is not None
try:
proc.stdin.write(stdin_payload)
except (BrokenPipeError, OSError):
# FFmpeg may have rejected the input early; still wait for it
# to terminate so the returncode is meaningful.
pass
finally:
try:
tz = pytz.timezone(tz_name)
except pytz.UnknownTimeZoneError:
tz = None
if tz is not None:
return datetime.datetime.fromtimestamp(timestamp, tz=tz).strftime(
"%Y-%m-%d %H:%M:%S"
)
proc.stdin.close()
except (BrokenPipeError, OSError):
pass
captured: list[str] = []
try:
for raw_line in proc.stderr:
captured.append(raw_line)
line = raw_line.strip()
if not line:
continue
if line.startswith("out_time_us="):
if expected_duration <= 0:
continue
try:
out_time_us = int(line.split("=", 1)[1])
except (ValueError, IndexError):
continue
if out_time_us < 0:
continue
out_seconds = out_time_us / 1_000_000.0
percent = (out_seconds / expected_duration) * 100.0
self._emit_progress(step, percent)
elif line == "progress=end":
self._emit_progress(step, 100.0)
break
except Exception:
logger.exception("Failed reading FFmpeg progress for %s", self.export_id)
proc.wait()
# Drain any remaining stderr so callers can log it on failure.
try:
remaining = proc.stderr.read()
if remaining:
captured.append(remaining)
except Exception:
pass
return proc.returncode, "".join(captured)
def get_datetime_from_timestamp(self, timestamp: int) -> str:
# return in iso format
return datetime.datetime.fromtimestamp(timestamp).strftime("%Y-%m-%d %H:%M:%S")
def _chapter_metadata_path(self) -> str:
@@ -342,7 +407,6 @@ class RecordingExporter(threading.Thread):
return None
total_output = windows[-1][2] + (windows[-1][1] - windows[-1][0])
last_recorded_end = windows[-1][1]
def wall_to_output(t: float) -> float:
t = max(float(self.start_time), min(float(self.end_time), t))
@@ -355,18 +419,8 @@ class RecordingExporter(threading.Thread):
chapter_blocks: list[str] = []
for review in review_rows:
if review.start_time is None:
continue
# In-progress segments have a NULL end_time until the activity
# closes; clamp to the last recorded second so the chapter never
# extends past the actual video.
review_end = (
float(review.end_time)
if review.end_time is not None
else last_recorded_end
)
start_out = wall_to_output(float(review.start_time))
end_out = wall_to_output(review_end)
end_out = wall_to_output(float(review.end_time))
# Drop chapters that fall entirely in a recording gap, or are
# too short to be navigable in a player.
@@ -380,14 +434,9 @@ class RecordingExporter(threading.Thread):
if label and label not in labels:
labels.append(label)
metadata = data.get("metadata") or {}
title = metadata.get("title")
if not title:
title = str(review.severity).capitalize()
if labels:
title = f"{title}: {', '.join(labels)}"
title = str(review.severity).capitalize()
if labels:
title = f"{title}: {', '.join(labels)}"
chapter_blocks.append(
"[CHAPTER]\n"
@@ -454,14 +503,16 @@ class RecordingExporter(threading.Thread):
except DoesNotExist:
return ""
diff = max(0.0, float(self.start_time) - float(preview.start_time))
diff = self.start_time - preview.start_time
minutes = int(diff / 60)
seconds = int(diff % 60)
ffmpeg_cmd = [
"/usr/lib/ffmpeg/7.0/bin/ffmpeg", # hardcode path for exports thumbnail due to missing libwebp support
"-hide_banner",
"-loglevel",
"warning",
"-ss",
f"{diff:.3f}",
f"00:{minutes}:{seconds}",
"-i",
preview.path,
"-frames",
@@ -487,18 +538,12 @@ class RecordingExporter(threading.Thread):
start_file = f"{file_start}{self.start_time}.{PREVIEW_FRAME_TYPE}"
end_file = f"{file_start}{self.end_time}.{PREVIEW_FRAME_TYPE}"
selected_preview = None
# Preview frames are written at most 1-2 fps during activity
# and as little as one every 30s during quiet periods, so a
# short export window can contain zero frames. Track the most
# recent frame before the window as a fallback.
fallback_preview = None
for file in sorted(os.listdir(preview_dir)):
if not file.startswith(file_start):
continue
if file < start_file:
fallback_preview = os.path.join(preview_dir, file)
continue
if file > end_file:
@@ -507,9 +552,6 @@ class RecordingExporter(threading.Thread):
selected_preview = os.path.join(preview_dir, file)
break
if not selected_preview:
selected_preview = fallback_preview
if not selected_preview:
return ""
+2 -1
View File
@@ -610,7 +610,8 @@ class RecordingMaintainer(threading.Thread):
camera,
)
os.makedirs(directory, exist_ok=True)
if not os.path.exists(directory):
os.makedirs(directory)
# file will be in utc due to start_time being in utc
file_name = f"{start_time.strftime('%M.%S.mp4')}"
-109
View File
@@ -1,109 +0,0 @@
"""Resolve human-readable names for Intel GPUs via OpenVINO."""
import logging
import re
from typing import Optional
logger = logging.getLogger(__name__)
class IntelGpuNameResolver:
"""Build a pdev -> normalized device name map by enumerating OpenVINO GPUs.
The lookup is performed once on first access and cached for the process
lifetime. OpenVINO exposes DEVICE_PCI_INFO (domain/bus/device/function) and
FULL_DEVICE_NAME for each GPU it can see, which is enough to associate the
name with the pdev string used by DRM fdinfo.
"""
_names: Optional[dict[str, str]] = None
def get_names(self) -> dict[str, str]:
if self._names is not None:
return self._names
names: dict[str, str] = {}
try:
from openvino import Core
except ImportError:
logger.debug("OpenVINO unavailable; cannot resolve Intel GPU names")
self._names = names
return names
try:
core = Core()
devices = core.available_devices
except Exception as exc:
logger.debug(f"OpenVINO Core initialization failed: {exc}")
self._names = names
return names
cpu_name: Optional[str] = None
if "CPU" in devices:
try:
cpu_name = self._strip_trademarks(
core.get_property("CPU", "FULL_DEVICE_NAME")
)
except Exception as exc:
logger.debug(f"Failed to read CPU FULL_DEVICE_NAME: {exc}")
for device in devices:
if not device.startswith("GPU"):
continue
try:
pci = core.get_property(device, "DEVICE_PCI_INFO")
raw_name = core.get_property(device, "FULL_DEVICE_NAME")
device_type = core.get_property(device, "DEVICE_TYPE")
except Exception as exc:
logger.debug(f"Failed to read properties for {device}: {exc}")
continue
pdev = self._format_pdev(pci)
if not pdev:
continue
names[pdev] = self._resolve_name(raw_name, device_type, cpu_name)
self._names = names
return names
@staticmethod
def _format_pdev(pci) -> Optional[str]:
try:
return f"{pci.domain:04x}:{pci.bus:02x}:{pci.device:02x}.{pci.function:x}"
except AttributeError:
return None
@classmethod
def _resolve_name(cls, raw_name: str, device_type, cpu_name: Optional[str]) -> str:
"""Build a display name for a GPU.
Modern integrated Intel GPUs are reported by OpenVINO with a generic
FULL_DEVICE_NAME like "Intel(R) Graphics (iGPU)" that gives no model
information. Since the iGPU is part of the CPU on these platforms, fall
back to the CPU name (which OpenVINO does report specifically) and
suffix it with "iGPU" so it's clear what the entry is.
"""
is_integrated = "INTEGRATED" in str(device_type).upper()
if is_integrated and cpu_name:
short_cpu = re.sub(r"^Intel\s+", "", cpu_name)
return f"{short_cpu} iGPU"
return cls._normalize_name(raw_name)
@classmethod
def _normalize_name(cls, name: str) -> str:
cleaned = cls._strip_trademarks(name)
cleaned = re.sub(r"\s*\((?:i|d)GPU\)\s*$", "", cleaned, flags=re.IGNORECASE)
return " ".join(cleaned.split())
@staticmethod
def _strip_trademarks(name: str) -> str:
cleaned = re.sub(r"\(R\)|\(TM\)", "", name)
return " ".join(cleaned.split())
intel_gpu_name_resolver = IntelGpuNameResolver()
+11 -16
View File
@@ -230,7 +230,6 @@ async def set_gpu_stats(
hwaccel_args.append(args)
stats: dict[str, dict] = {}
intel_gpu_collected = False
for args in hwaccel_args:
if args in hwaccel_errors:
@@ -243,7 +242,6 @@ async def set_gpu_stats(
if nvidia_usage:
for i in range(len(nvidia_usage)):
stats[nvidia_usage[i]["name"]] = {
"vendor": "nvidia",
"gpu": str(round(float(nvidia_usage[i]["gpu"]), 2)) + "%",
"mem": str(round(float(nvidia_usage[i]["mem"]), 2)) + "%",
"enc": str(round(float(nvidia_usage[i]["enc"]), 2)) + "%",
@@ -252,34 +250,31 @@ async def set_gpu_stats(
}
else:
stats["nvidia-gpu"] = {"vendor": "nvidia", "gpu": "", "mem": ""}
stats["nvidia-gpu"] = {"gpu": "", "mem": ""}
hwaccel_errors.append(args)
elif "nvmpi" in args or "jetson" in args:
# nvidia Jetson
jetson_usage = get_jetson_stats()
if jetson_usage:
stats["jetson-gpu"] = {"vendor": "nvidia", **jetson_usage}
stats["jetson-gpu"] = jetson_usage
else:
stats["jetson-gpu"] = {"vendor": "nvidia", "gpu": "", "mem": ""}
stats["jetson-gpu"] = {"gpu": "", "mem": ""}
hwaccel_errors.append(args)
elif "qsv" in args or ("vaapi" in args and not is_vaapi_amd_driver()):
if not config.telemetry.stats.intel_gpu_stats:
continue
if not intel_gpu_collected:
if "intel-gpu" not in stats:
# intel GPU (QSV or VAAPI both use the same physical GPU)
intel_gpu_collected = True
intel_usage = get_intel_gpu_stats(
config.telemetry.stats.intel_gpu_device
)
if intel_usage:
for entry in intel_usage.values():
name = entry.pop("name")
stats[name] = entry
if intel_usage is not None:
stats["intel-gpu"] = intel_usage or {"gpu": "", "mem": ""}
else:
stats["intel-gpu"] = {"vendor": "intel", "gpu": "", "mem": ""}
stats["intel-gpu"] = {"gpu": "", "mem": ""}
hwaccel_errors.append(args)
elif "vaapi" in args:
if not config.telemetry.stats.amd_gpu_stats:
@@ -289,18 +284,18 @@ async def set_gpu_stats(
amd_usage = get_amd_gpu_stats()
if amd_usage:
stats["amd-vaapi"] = {"vendor": "amd", **amd_usage}
stats["amd-vaapi"] = amd_usage
else:
stats["amd-vaapi"] = {"vendor": "amd", "gpu": "", "mem": ""}
stats["amd-vaapi"] = {"gpu": "", "mem": ""}
hwaccel_errors.append(args)
elif "preset-rk" in args:
rga_usage = get_rockchip_gpu_stats()
if rga_usage:
stats["rockchip"] = {"vendor": "rockchip", **rga_usage}
stats["rockchip"] = rga_usage
elif "v4l2m2m" in args or "rpi" in args:
# RPi v4l2m2m is currently not able to get usage stats
stats["rpi-v4l2m2m"] = {"vendor": "rpi", "gpu": "", "mem": ""}
stats["rpi-v4l2m2m"] = {"gpu": "", "mem": ""}
if stats:
all_stats["gpu_usages"] = stats
@@ -1,124 +0,0 @@
"""Tests for /debug_replay API endpoints."""
from unittest.mock import patch
from frigate.models import Event, Recordings, ReviewSegment
from frigate.test.http_api.base_http_test import AuthTestClient, BaseTestHttp
class TestDebugReplayAPI(BaseTestHttp):
def setUp(self):
super().setUp([Event, Recordings, ReviewSegment])
self.app = self.create_app()
def test_start_returns_202_with_job_id(self):
# Stub the factory to skip validation/threading and just record the
# name on the manager the way the real factory's mark_starting would.
def fake_start(**kwargs):
source = kwargs["source"]
kwargs["replay_manager"].mark_starting(
source_camera=source.source_camera,
replay_camera_name="_replay_front",
start_ts=source.start_ts,
end_ts=source.end_ts,
)
return "job-1234"
with patch(
"frigate.api.debug_replay.start_debug_replay_job",
side_effect=fake_start,
):
with AuthTestClient(self.app) as client:
resp = client.post(
"/debug_replay/start",
json={
"camera": "front",
"start_time": 100,
"end_time": 200,
},
)
self.assertEqual(resp.status_code, 202)
body = resp.json()
self.assertTrue(body["success"])
self.assertEqual(body["job_id"], "job-1234")
self.assertEqual(body["replay_camera"], "_replay_front")
def test_start_returns_400_on_validation_error(self):
with patch(
"frigate.api.debug_replay.start_debug_replay_job",
side_effect=ValueError("Camera 'missing' not found"),
):
with AuthTestClient(self.app) as client:
resp = client.post(
"/debug_replay/start",
json={
"camera": "missing",
"start_time": 100,
"end_time": 200,
},
)
self.assertEqual(resp.status_code, 400)
body = resp.json()
self.assertFalse(body["success"])
# Message is hard-coded so we don't echo exception text back to clients
# (CodeQL: information exposure through an exception).
self.assertEqual(body["message"], "Invalid debug replay parameters")
def test_start_returns_409_when_session_already_active(self):
with patch(
"frigate.api.debug_replay.start_debug_replay_job",
side_effect=RuntimeError("A replay session is already active"),
):
with AuthTestClient(self.app) as client:
resp = client.post(
"/debug_replay/start",
json={
"camera": "front",
"start_time": 100,
"end_time": 200,
},
)
self.assertEqual(resp.status_code, 409)
body = resp.json()
self.assertFalse(body["success"])
def test_status_inactive_when_no_session(self):
with AuthTestClient(self.app) as client:
resp = client.get("/debug_replay/status")
self.assertEqual(resp.status_code, 200)
body = resp.json()
self.assertFalse(body["active"])
self.assertIsNone(body["replay_camera"])
self.assertIsNone(body["source_camera"])
self.assertIsNone(body["start_time"])
self.assertIsNone(body["end_time"])
self.assertFalse(body["live_ready"])
# Make sure deprecated fields are gone
self.assertNotIn("state", body)
self.assertNotIn("progress_percent", body)
self.assertNotIn("error_message", body)
def test_status_active_after_mark_starting(self):
manager = self.app.replay_manager
manager.mark_starting(
source_camera="front",
replay_camera_name="_replay_front",
start_ts=100.0,
end_ts=200.0,
)
with AuthTestClient(self.app) as client:
resp = client.get("/debug_replay/status")
self.assertEqual(resp.status_code, 200)
body = resp.json()
self.assertTrue(body["active"])
self.assertEqual(body["replay_camera"], "_replay_front")
self.assertEqual(body["source_camera"], "front")
self.assertEqual(body["start_time"], 100.0)
self.assertEqual(body["end_time"], 200.0)
self.assertFalse(body["live_ready"])
@@ -1,4 +1,3 @@
import os
from unittest.mock import patch
from fastapi import HTTPException, Request
@@ -358,51 +357,6 @@ class TestGo2rtcStreamAccess(BaseTestHttp):
f"got {resp.status_code}"
)
def test_add_stream_rejects_restricted_source(self):
"""PUT /go2rtc/streams must reject exec:/echo:/expr: sources even for
admins"""
app = self._make_app(_MULTI_CAMERA_CONFIG)
with AuthTestClient(app) as client:
for src in (
"exec:/tmp/rev.sh",
"echo:foo",
"expr:bar",
" exec:/tmp/rev.sh",
):
resp = client.put(f"/go2rtc/streams/revshell?src={src}")
assert resp.status_code == 400, (
f"Expected 400 for restricted src {src!r}; got {resp.status_code}"
)
assert resp.json().get("success") is False
def test_add_stream_allows_non_restricted_source(self):
"""A normal stream URL should pass the restricted-source check and reach
the (unavailable in tests) go2rtc proxy so we expect 500, not 400."""
app = self._make_app(_MULTI_CAMERA_CONFIG)
with AuthTestClient(app) as client:
resp = client.put("/go2rtc/streams/legit?src=rtsp://10.0.0.1:554/video")
assert resp.status_code != 400, (
f"Non-restricted source should not be rejected with 400; got {resp.status_code}"
)
def test_add_stream_allows_restricted_source_when_override_set(self):
"""When GO2RTC_ALLOW_ARBITRARY_EXEC is set, the API must defer to operator
intent and forward the request to go2rtc instead of short-circuiting with 400."""
app = self._make_app(_MULTI_CAMERA_CONFIG)
mock_response = type("R", (), {"ok": True, "status_code": 200, "text": "ok"})()
with patch.dict(os.environ, {"GO2RTC_ALLOW_ARBITRARY_EXEC": "true"}):
with patch(
"frigate.api.camera.requests.put", return_value=mock_response
) as mock_put:
with AuthTestClient(app) as client:
resp = client.put("/go2rtc/streams/legit?src=exec:/tmp/something")
assert resp.status_code == 200, (
f"Restricted src should be forwarded when override set; got {resp.status_code}"
)
mock_put.assert_called_once()
forwarded_src = mock_put.call_args.kwargs["params"]["src"]
assert forwarded_src == "exec:/tmp/something"
def test_stream_alias_blocked_when_owning_camera_disallowed(self):
"""limited_user cannot access a stream alias that belongs to a camera they
are not allowed to see."""
+10 -62
View File
@@ -10,7 +10,7 @@ from ruamel.yaml.constructor import DuplicateKeyError
from frigate.config import BirdseyeModeEnum, FrigateConfig
from frigate.const import MODEL_CACHE_DIR
from frigate.detectors import DetectorTypeEnum
from frigate.util.builtin import deep_merge
from frigate.util.builtin import deep_merge, load_labels
class TestConfig(unittest.TestCase):
@@ -309,11 +309,16 @@ class TestConfig(unittest.TestCase):
}
frigate_config = FrigateConfig(**config)
assert set(frigate_config.cameras["back"].audio.filters.keys()) == {
"speech",
"yell",
all_audio_labels = {
label
for label in load_labels("/audio-labelmap.txt", prefill=521).values()
if label
}
assert all_audio_labels.issubset(
set(frigate_config.cameras["back"].audio.filters.keys())
)
def test_override_audio_filters(self):
config = {
"mqtt": {"host": "mqtt"},
@@ -340,8 +345,7 @@ class TestConfig(unittest.TestCase):
frigate_config = FrigateConfig(**config)
assert "speech" in frigate_config.cameras["back"].audio.filters
assert frigate_config.cameras["back"].audio.filters["speech"].threshold == 0.9
assert "yell" in frigate_config.cameras["back"].audio.filters
assert "babbling" not in frigate_config.cameras["back"].audio.filters
assert "babbling" in frigate_config.cameras["back"].audio.filters
def test_inherit_object_filters(self):
config = {
@@ -1001,7 +1005,6 @@ class TestConfig(unittest.TestCase):
config = {
"mqtt": {"host": "mqtt"},
"detectors": {"cpu": {"type": "cpu"}},
"model": {"path": "plus://test"},
"cameras": {
"back": {
@@ -1673,60 +1676,5 @@ class TestConfig(unittest.TestCase):
self.assertRaises(ValueError, lambda: FrigateConfig(**config))
class TestAttributeFilterDefaults(unittest.TestCase):
"""Verify attribute filter min_score handling at config load."""
def setUp(self):
self.minimal = {
"mqtt": {"host": "mqtt"},
"cameras": {
"back": {
"ffmpeg": {
"inputs": [
{"path": "rtsp://10.0.0.1:554/video", "roles": ["detect"]}
]
},
"detect": {
"height": 1080,
"width": 1920,
"fps": 5,
},
}
},
}
def _build_config(self, object_filters: dict | None = None) -> FrigateConfig:
config = deep_merge({}, self.minimal)
if object_filters is not None:
config.setdefault("objects", {})["filters"] = object_filters
return FrigateConfig(**config)
def test_attribute_with_no_filter_gets_default_min_score(self):
"""Attribute with no user-provided filter gets created with min_score=0.7."""
config = self._build_config()
face_filter = config.objects.filters.get("face")
self.assertIsNotNone(face_filter)
self.assertEqual(face_filter.min_score, 0.7)
def test_attribute_filter_without_min_score_gets_bumped(self):
"""If user sets some FilterConfig field but not min_score, min_score is bumped to 0.7."""
config = self._build_config({"face": {"min_area": 500}})
face_filter = config.objects.filters["face"]
self.assertEqual(face_filter.min_area, 500)
self.assertEqual(face_filter.min_score, 0.7)
def test_attribute_filter_explicit_min_score_half_is_preserved(self):
"""User-provided min_score=0.5 must NOT be silently rewritten to 0.7."""
config = self._build_config({"face": {"min_score": 0.5}})
face_filter = config.objects.filters["face"]
self.assertEqual(face_filter.min_score, 0.5)
def test_attribute_filter_explicit_min_score_other_value_is_preserved(self):
"""Sanity: explicit non-0.5 values pass through unchanged."""
config = self._build_config({"face": {"min_score": 0.3}})
face_filter = config.objects.filters["face"]
self.assertEqual(face_filter.min_score, 0.3)
if __name__ == "__main__":
unittest.main(verbosity=2)
-250
View File
@@ -1,250 +0,0 @@
"""Tests for the simplified DebugReplayManager.
Startup orchestration lives in ``frigate.jobs.debug_replay`` (covered by
``test_debug_replay_job``). The manager owns only session presence and
cleanup.
"""
import unittest
import unittest.mock
from unittest.mock import MagicMock, patch
class TestDebugReplayManagerSession(unittest.TestCase):
def test_inactive_by_default(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
self.assertFalse(manager.active)
self.assertIsNone(manager.replay_camera_name)
self.assertIsNone(manager.source_camera)
self.assertIsNone(manager.clip_path)
self.assertIsNone(manager.start_ts)
self.assertIsNone(manager.end_ts)
def test_mark_starting_sets_session_pointers_and_active(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting(
source_camera="front",
replay_camera_name="_replay_front",
start_ts=100.0,
end_ts=200.0,
)
self.assertTrue(manager.active)
self.assertEqual(manager.replay_camera_name, "_replay_front")
self.assertEqual(manager.source_camera, "front")
self.assertEqual(manager.start_ts, 100.0)
self.assertEqual(manager.end_ts, 200.0)
self.assertIsNone(manager.clip_path)
def test_mark_session_ready_sets_clip_path(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting("front", "_replay_front", 100.0, 200.0)
manager.mark_session_ready(clip_path="/tmp/replay/_replay_front.mp4")
self.assertEqual(manager.clip_path, "/tmp/replay/_replay_front.mp4")
self.assertTrue(manager.active)
def test_clear_session_resets_all_pointers(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting("front", "_replay_front", 100.0, 200.0)
manager.mark_session_ready("/tmp/replay/clip.mp4")
manager.clear_session()
self.assertFalse(manager.active)
self.assertIsNone(manager.replay_camera_name)
self.assertIsNone(manager.source_camera)
self.assertIsNone(manager.clip_path)
self.assertIsNone(manager.start_ts)
self.assertIsNone(manager.end_ts)
class TestDebugReplayManagerStop(unittest.TestCase):
def setUp(self) -> None:
# stop() publishes a terminal job_state via a real JobStatePublisher,
# which opens a ZMQ REQ socket and blocks on REP. No dispatcher runs
# in unit tests, so substitute a no-op publisher.
patcher = patch("frigate.debug_replay.JobStatePublisher")
patcher.start()
self.addCleanup(patcher.stop)
def test_stop_when_inactive_is_a_noop(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
frigate_config = MagicMock()
frigate_config.cameras = {}
publisher = MagicMock()
# Should not raise; should not publish any events.
manager.stop(frigate_config=frigate_config, config_publisher=publisher)
publisher.publish_update.assert_not_called()
def test_stop_publishes_remove_when_camera_was_published(self) -> None:
from frigate.config.camera.updater import CameraConfigUpdateEnum
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting("front", "_replay_front", 100.0, 200.0)
manager.mark_session_ready("/tmp/replay/_replay_front.mp4")
camera_config = MagicMock()
frigate_config = MagicMock()
frigate_config.cameras = {"_replay_front": camera_config}
publisher = MagicMock()
with (
patch.object(manager, "_cleanup_db"),
patch.object(manager, "_cleanup_files"),
patch("frigate.debug_replay.cancel_debug_replay_job", return_value=False),
):
manager.stop(frigate_config=frigate_config, config_publisher=publisher)
# One publish_update call with a remove topic.
self.assertEqual(publisher.publish_update.call_count, 1)
topic_arg = publisher.publish_update.call_args.args[0]
self.assertEqual(topic_arg.update_type, CameraConfigUpdateEnum.remove)
self.assertFalse(manager.active)
def test_stop_skips_remove_publish_when_camera_not_in_config(self) -> None:
"""Cancellation during preparing_clip: no camera was published yet."""
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting("front", "_replay_front", 100.0, 200.0)
# clip_path stays None because we cancelled before camera publish.
frigate_config = MagicMock()
frigate_config.cameras = {} # _replay_front not present
publisher = MagicMock()
with (
patch.object(manager, "_cleanup_db"),
patch.object(manager, "_cleanup_files"),
patch("frigate.debug_replay.cancel_debug_replay_job", return_value=True),
):
manager.stop(frigate_config=frigate_config, config_publisher=publisher)
publisher.publish_update.assert_not_called()
self.assertFalse(manager.active)
def test_stop_calls_cancel_debug_replay_job(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
manager.mark_starting("front", "_replay_front", 100.0, 200.0)
frigate_config = MagicMock()
frigate_config.cameras = {}
publisher = MagicMock()
with (
patch.object(manager, "_cleanup_db"),
patch.object(manager, "_cleanup_files"),
patch(
"frigate.debug_replay.cancel_debug_replay_job",
return_value=True,
) as mock_cancel,
):
manager.stop(frigate_config=frigate_config, config_publisher=publisher)
mock_cancel.assert_called_once()
class TestDebugReplayManagerPublishCamera(unittest.TestCase):
def test_publish_camera_invokes_publisher_with_add_topic(self) -> None:
from frigate.config.camera.updater import CameraConfigUpdateEnum
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
source_config = MagicMock()
new_camera_config = MagicMock()
frigate_config = MagicMock()
frigate_config.cameras = {"front": source_config}
publisher = MagicMock()
with (
patch.object(
manager,
"_build_camera_config_dict",
return_value={"enabled": True},
),
patch("frigate.debug_replay.find_config_file", return_value="/cfg.yml"),
patch("frigate.debug_replay.YAML") as yaml_cls,
patch("frigate.debug_replay.FrigateConfig.parse_object") as parse_object,
patch("builtins.open", unittest.mock.mock_open(read_data="cameras:\n")),
):
yaml_instance = yaml_cls.return_value
yaml_instance.load.return_value = {"cameras": {}}
parsed = MagicMock()
parsed.cameras = {"_replay_front": new_camera_config}
parse_object.return_value = parsed
manager.publish_camera(
source_camera="front",
replay_name="_replay_front",
clip_path="/tmp/clip.mp4",
frigate_config=frigate_config,
config_publisher=publisher,
)
# Camera registered into the live config dict
self.assertIn("_replay_front", frigate_config.cameras)
# Publisher invoked with an add topic
self.assertEqual(publisher.publish_update.call_count, 1)
topic_arg = publisher.publish_update.call_args.args[0]
self.assertEqual(topic_arg.update_type, CameraConfigUpdateEnum.add)
def test_publish_camera_wraps_parse_failure_in_runtime_error(self) -> None:
from frigate.debug_replay import DebugReplayManager
manager = DebugReplayManager()
frigate_config = MagicMock()
frigate_config.cameras = {"front": MagicMock()}
publisher = MagicMock()
with (
patch.object(
manager,
"_build_camera_config_dict",
return_value={"enabled": True},
),
patch("frigate.debug_replay.find_config_file", return_value="/cfg.yml"),
patch("frigate.debug_replay.YAML") as yaml_cls,
patch(
"frigate.debug_replay.FrigateConfig.parse_object",
side_effect=ValueError("zone foo has invalid coordinates"),
),
patch("builtins.open", unittest.mock.mock_open(read_data="cameras:\n")),
):
yaml_cls.return_value.load.return_value = {"cameras": {}}
with self.assertRaises(RuntimeError) as ctx:
manager.publish_camera(
source_camera="front",
replay_name="_replay_front",
clip_path="/tmp/clip.mp4",
frigate_config=frigate_config,
config_publisher=publisher,
)
self.assertIn("replay camera config", str(ctx.exception))
self.assertIn("invalid coordinates", str(ctx.exception))
publisher.publish_update.assert_not_called()
if __name__ == "__main__":
unittest.main()
-461
View File
@@ -1,461 +0,0 @@
"""Tests for the debug replay job runner and factory."""
import threading
import time
import unittest
import unittest.mock
from unittest.mock import MagicMock, patch
from frigate.debug_replay import DebugReplayManager
from frigate.jobs.debug_replay import (
DebugReplayJob,
RecordingDebugReplaySource,
cancel_debug_replay_job,
get_active_runner,
start_debug_replay_job,
)
from frigate.jobs.export import JobStatePublisher
from frigate.jobs.manager import _completed_jobs, _current_jobs
from frigate.types import JobStatusTypesEnum
def _reset_job_manager() -> None:
"""Clear the global job manager state between tests."""
_current_jobs.clear()
_completed_jobs.clear()
def _patch_publisher(test_case: unittest.TestCase) -> None:
"""Replace JobStatePublisher.publish with a no-op to avoid hanging on IPC."""
publisher_patch = patch.object(
JobStatePublisher, "publish", lambda self, payload: None
)
publisher_patch.start()
test_case.addCleanup(publisher_patch.stop)
class TestDebugReplayJob(unittest.TestCase):
def test_default_fields(self) -> None:
job = DebugReplayJob()
self.assertEqual(job.job_type, "debug_replay")
self.assertEqual(job.status, JobStatusTypesEnum.queued)
self.assertIsNone(job.current_step)
self.assertEqual(job.progress_percent, 0.0)
def test_to_dict_whitelist(self) -> None:
job = DebugReplayJob(
source_camera="front",
replay_camera_name="_replay_front",
start_ts=100.0,
end_ts=200.0,
)
job.current_step = "preparing_clip"
job.progress_percent = 42.5
payload = job.to_dict()
# Top-level matches the standard Job<TResults> shape.
for key in (
"id",
"job_type",
"status",
"start_time",
"end_time",
"error_message",
"results",
):
self.assertIn(key, payload, f"missing top-level field: {key}")
results = payload["results"]
self.assertEqual(results["source_camera"], "front")
self.assertEqual(results["replay_camera_name"], "_replay_front")
self.assertEqual(results["current_step"], "preparing_clip")
self.assertEqual(results["progress_percent"], 42.5)
self.assertEqual(results["start_ts"], 100.0)
self.assertEqual(results["end_ts"], 200.0)
class TestStartDebugReplayJob(unittest.TestCase):
def setUp(self) -> None:
_reset_job_manager()
_patch_publisher(self)
self.manager = DebugReplayManager()
self.frigate_config = MagicMock()
self.frigate_config.cameras = {"front": MagicMock()}
self.frigate_config.ffmpeg.ffmpeg_path = "/bin/true"
self.publisher = MagicMock()
self.recordings_qs = MagicMock()
self.recordings_qs.count.return_value = 1
self.recordings_qs.__iter__.return_value = iter([MagicMock(path="/tmp/r1.mp4")])
def tearDown(self) -> None:
runner = get_active_runner()
if runner is not None:
runner.cancel()
runner.join(timeout=2.0)
_reset_job_manager()
def test_rejects_unknown_camera(self) -> None:
with self.assertRaises(ValueError):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="missing", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
def test_rejects_invalid_time_range(self) -> None:
with self.assertRaises(ValueError):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=200.0, end_ts=100.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
def test_rejects_when_no_recordings(self) -> None:
empty_qs = MagicMock()
empty_qs.count.return_value = 0
with patch("frigate.jobs.debug_replay.query_recordings", return_value=empty_qs):
with self.assertRaises(ValueError):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
def test_returns_job_id_and_marks_session_starting(self) -> None:
block = threading.Event()
def slow_helper(cmd, **kwargs):
block.wait(timeout=5)
return 0, ""
with (
patch(
"frigate.jobs.debug_replay.query_recordings",
return_value=self.recordings_qs,
),
patch(
"frigate.jobs.debug_replay.run_ffmpeg_with_progress",
side_effect=slow_helper,
),
patch.object(self.manager, "publish_camera"),
patch("os.path.exists", return_value=True),
patch("os.makedirs"),
patch("builtins.open", unittest.mock.mock_open()),
):
job_id = start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
self.assertIsInstance(job_id, str)
self.assertTrue(self.manager.active)
self.assertEqual(self.manager.replay_camera_name, "_replay_front")
self.assertEqual(self.manager.source_camera, "front")
block.set()
def test_rejects_concurrent_calls(self) -> None:
block = threading.Event()
def slow_helper(cmd, **kwargs):
block.wait(timeout=5)
return 0, ""
with (
patch(
"frigate.jobs.debug_replay.query_recordings",
return_value=self.recordings_qs,
),
patch(
"frigate.jobs.debug_replay.run_ffmpeg_with_progress",
side_effect=slow_helper,
),
patch.object(self.manager, "publish_camera"),
patch("os.path.exists", return_value=True),
patch("os.makedirs"),
patch("builtins.open", unittest.mock.mock_open()),
):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
with self.assertRaises(RuntimeError):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
block.set()
class TestRunnerHappyPath(unittest.TestCase):
def setUp(self) -> None:
_reset_job_manager()
_patch_publisher(self)
self.manager = DebugReplayManager()
self.frigate_config = MagicMock()
self.frigate_config.cameras = {"front": MagicMock()}
self.frigate_config.ffmpeg.ffmpeg_path = "/bin/true"
self.publisher = MagicMock()
self.recordings_qs = MagicMock()
self.recordings_qs.count.return_value = 1
self.recordings_qs.__iter__.return_value = iter([MagicMock(path="/tmp/r1.mp4")])
def tearDown(self) -> None:
runner = get_active_runner()
if runner is not None:
runner.cancel()
runner.join(timeout=2.0)
_reset_job_manager()
def _wait_for(self, predicate, timeout: float = 5.0) -> bool:
deadline = time.time() + timeout
while time.time() < deadline:
if predicate():
return True
time.sleep(0.02)
return False
def test_progress_callback_updates_job_percent(self) -> None:
captured: list[float] = []
def fake_helper(cmd, *, on_progress=None, **kwargs):
on_progress(0.0)
on_progress(50.0)
on_progress(100.0)
return 0, ""
with (
patch(
"frigate.jobs.debug_replay.query_recordings",
return_value=self.recordings_qs,
),
patch(
"frigate.jobs.debug_replay.run_ffmpeg_with_progress",
side_effect=fake_helper,
),
patch.object(
self.manager,
"publish_camera",
side_effect=lambda *a, **kw: captured.append("published"),
),
patch("os.path.exists", return_value=True),
patch("os.makedirs"),
patch("builtins.open", unittest.mock.mock_open()),
):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
self.assertTrue(
self._wait_for(lambda: get_active_runner() is None),
"runner did not finish",
)
from frigate.jobs.manager import get_current_job
job = get_current_job("debug_replay")
self.assertIsNotNone(job)
self.assertEqual(job.status, JobStatusTypesEnum.success)
self.assertEqual(job.progress_percent, 100.0)
self.assertEqual(captured, ["published"])
# Manager should have been told the session is ready with the clip path.
self.assertIsNotNone(self.manager.clip_path)
class TestRunnerFailurePath(unittest.TestCase):
def setUp(self) -> None:
_reset_job_manager()
_patch_publisher(self)
self.manager = DebugReplayManager()
self.frigate_config = MagicMock()
self.frigate_config.cameras = {"front": MagicMock()}
self.frigate_config.ffmpeg.ffmpeg_path = "/bin/true"
self.publisher = MagicMock()
self.recordings_qs = MagicMock()
self.recordings_qs.count.return_value = 1
self.recordings_qs.__iter__.return_value = iter([MagicMock(path="/tmp/r1.mp4")])
def tearDown(self) -> None:
runner = get_active_runner()
if runner is not None:
runner.cancel()
runner.join(timeout=2.0)
_reset_job_manager()
def _wait_for(self, predicate, timeout: float = 5.0) -> bool:
deadline = time.time() + timeout
while time.time() < deadline:
if predicate():
return True
time.sleep(0.02)
return False
def test_ffmpeg_failure_marks_job_failed_and_clears_session(self) -> None:
def failing_helper(cmd, **kwargs):
return 1, "ffmpeg exploded"
with (
patch(
"frigate.jobs.debug_replay.query_recordings",
return_value=self.recordings_qs,
),
patch(
"frigate.jobs.debug_replay.run_ffmpeg_with_progress",
side_effect=failing_helper,
),
patch("os.path.exists", return_value=True),
patch("os.makedirs"),
patch("os.remove"),
patch("builtins.open", unittest.mock.mock_open()),
):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
self.assertTrue(
self._wait_for(lambda: get_active_runner() is None),
"runner did not finish",
)
from frigate.jobs.manager import get_current_job
job = get_current_job("debug_replay")
self.assertIsNotNone(job)
self.assertEqual(job.status, JobStatusTypesEnum.failed)
self.assertIsNotNone(job.error_message)
self.assertIn("ffmpeg", job.error_message.lower())
# Session cleared so a new /start is allowed
self.assertFalse(self.manager.active)
class TestRunnerCancellation(unittest.TestCase):
def setUp(self) -> None:
_reset_job_manager()
_patch_publisher(self)
self.manager = DebugReplayManager()
self.frigate_config = MagicMock()
self.frigate_config.cameras = {"front": MagicMock()}
self.frigate_config.ffmpeg.ffmpeg_path = "/bin/true"
self.publisher = MagicMock()
self.recordings_qs = MagicMock()
self.recordings_qs.count.return_value = 1
self.recordings_qs.__iter__.return_value = iter([MagicMock(path="/tmp/r1.mp4")])
def tearDown(self) -> None:
runner = get_active_runner()
if runner is not None:
runner.cancel()
runner.join(timeout=2.0)
_reset_job_manager()
def _wait_for(self, predicate, timeout: float = 5.0) -> bool:
deadline = time.time() + timeout
while time.time() < deadline:
if predicate():
return True
time.sleep(0.02)
return False
def test_cancel_terminates_ffmpeg_and_marks_cancelled(self) -> None:
terminated = threading.Event()
fake_proc = MagicMock()
fake_proc.terminate = MagicMock(side_effect=lambda: terminated.set())
def fake_helper(cmd, *, process_started=None, **kwargs):
if process_started is not None:
process_started(fake_proc)
terminated.wait(timeout=5)
return -15, "killed"
with (
patch(
"frigate.jobs.debug_replay.query_recordings",
return_value=self.recordings_qs,
),
patch(
"frigate.jobs.debug_replay.run_ffmpeg_with_progress",
side_effect=fake_helper,
),
patch("os.path.exists", return_value=True),
patch("os.makedirs"),
patch("os.remove"),
patch("builtins.open", unittest.mock.mock_open()),
):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front", start_ts=100.0, end_ts=200.0
),
frigate_config=self.frigate_config,
config_publisher=self.publisher,
replay_manager=self.manager,
)
# Wait for the runner to register the active process.
self.assertTrue(
self._wait_for(
lambda: (
get_active_runner() is not None
and get_active_runner()._active_process is fake_proc
)
)
)
cancelled = cancel_debug_replay_job()
self.assertTrue(cancelled)
self.assertTrue(fake_proc.terminate.called)
self.assertTrue(
self._wait_for(lambda: get_active_runner() is None),
"runner did not finish",
)
from frigate.jobs.manager import get_current_job
job = get_current_job("debug_replay")
self.assertEqual(job.status, JobStatusTypesEnum.cancelled)
# Runner must not clear the manager session on cancellation —
# that belongs to the caller of cancel_debug_replay_job (stop()).
# If the runner cleared it, stop() would log "no active session"
# and skip its cleanup_db / cleanup_files calls.
self.assertTrue(self.manager.active)
if __name__ == "__main__":
unittest.main()
+5 -173
View File
@@ -1,9 +1,6 @@
"""Tests for export progress tracking, broadcast, and FFmpeg parsing."""
import io
import os
import shutil
import tempfile
import unittest
from unittest.mock import MagicMock, patch
@@ -14,7 +11,6 @@ from frigate.jobs.export import (
)
from frigate.record.export import PlaybackSourceEnum, RecordingExporter
from frigate.types import JobStatusTypesEnum
from frigate.util.ffmpeg import inject_progress_flags
def _make_exporter(
@@ -119,9 +115,10 @@ class TestExpectedOutputDuration(unittest.TestCase):
class TestProgressFlagInjection(unittest.TestCase):
def test_inserts_before_output_path(self) -> None:
exporter = _make_exporter()
cmd = ["ffmpeg", "-i", "input.m3u8", "-c", "copy", "/tmp/output.mp4"]
result = inject_progress_flags(cmd)
result = exporter._inject_progress_flags(cmd)
assert result == [
"ffmpeg",
@@ -136,7 +133,8 @@ class TestProgressFlagInjection(unittest.TestCase):
]
def test_handles_empty_cmd(self) -> None:
assert inject_progress_flags([]) == []
exporter = _make_exporter()
assert exporter._inject_progress_flags([]) == []
class TestFfmpegProgressParsing(unittest.TestCase):
@@ -166,7 +164,7 @@ class TestFfmpegProgressParsing(unittest.TestCase):
fake_proc.returncode = 0
fake_proc.wait = MagicMock(return_value=0)
with patch("frigate.util.ffmpeg.sp.Popen", return_value=fake_proc):
with patch("frigate.record.export.sp.Popen", return_value=fake_proc):
returncode, _stderr = exporter._run_ffmpeg_with_progress(
["ffmpeg", "-i", "x.m3u8", "/tmp/out.mp4"], "playlist", step="encoding"
)
@@ -365,121 +363,6 @@ class TestBroadcastAggregation(unittest.TestCase):
assert job.progress_percent == 33.0
class TestGetDatetimeFromTimestamp(unittest.TestCase):
"""Auto-generated export name should honor config.ui.timezone, not
fall back to the container's UTC clock when a timezone is configured.
"""
def test_uses_configured_ui_timezone(self) -> None:
exporter = _make_exporter()
exporter.config.ui.timezone = "America/New_York"
# 2025-01-15 12:00:00 UTC is 07:00:00 EST
assert exporter.get_datetime_from_timestamp(1736942400) == "2025-01-15 07:00:00"
def test_falls_back_to_local_when_timezone_unset(self) -> None:
exporter = _make_exporter()
exporter.config.ui.timezone = None
# No assertion on the exact wall-clock value — just confirm no
# exception and that pytz isn't required when the field is unset.
assert isinstance(exporter.get_datetime_from_timestamp(1736942400), str)
def test_invalid_timezone_falls_back_to_local(self) -> None:
exporter = _make_exporter()
exporter.config.ui.timezone = "Not/A_Real_Zone"
assert isinstance(exporter.get_datetime_from_timestamp(1736942400), str)
class TestSaveThumbnailFromPreviewFrames(unittest.TestCase):
"""Short exports in the current hour can fall between preview frame
writes (1-2 fps during activity, every 30s otherwise). When no frame
falls inside the export window, save_thumbnail should fall back to
the most recent prior frame instead of returning no thumbnail."""
def setUp(self) -> None:
self.tmp_root = tempfile.mkdtemp(prefix="frigate_thumb_test_")
self.preview_dir = os.path.join(self.tmp_root, "cache", "preview_frames")
self.export_clips = os.path.join(self.tmp_root, "clips", "export")
os.makedirs(self.preview_dir, exist_ok=True)
os.makedirs(self.export_clips, exist_ok=True)
def tearDown(self) -> None:
shutil.rmtree(self.tmp_root, ignore_errors=True)
def _write_frame(self, camera: str, frame_time: float) -> str:
path = os.path.join(self.preview_dir, f"preview_{camera}-{frame_time}.webp")
with open(path, "wb") as f:
f.write(b"fake-webp-bytes")
return path
def _make_short_current_hour_exporter(self) -> RecordingExporter:
# Use a "now-ish" timestamp so save_thumbnail's start-of-hour
# comparison takes the current-hour branch (preview frames).
import datetime
now = datetime.datetime.now(datetime.timezone.utc).timestamp()
exporter = _make_exporter()
exporter.export_id = "thumb_short"
exporter.start_time = now
exporter.end_time = now + 3
return exporter
def test_short_export_falls_back_to_prior_preview_frame(self) -> None:
exporter = self._make_short_current_hour_exporter()
# Most recent preview frame is 10s before the export window
prior = self._write_frame(exporter.camera, exporter.start_time - 10.0)
thumb_target = os.path.join(self.export_clips, f"{exporter.export_id}.webp")
with (
patch(
"frigate.record.export.CACHE_DIR", os.path.join(self.tmp_root, "cache")
),
patch(
"frigate.record.export.CLIPS_DIR", os.path.join(self.tmp_root, "clips")
),
):
result = exporter.save_thumbnail(exporter.export_id)
assert result == thumb_target
assert os.path.isfile(thumb_target)
with open(thumb_target, "rb") as f, open(prior, "rb") as src:
assert f.read() == src.read()
def test_returns_empty_when_no_preview_frames_exist(self) -> None:
exporter = self._make_short_current_hour_exporter()
with (
patch(
"frigate.record.export.CACHE_DIR", os.path.join(self.tmp_root, "cache")
),
patch(
"frigate.record.export.CLIPS_DIR", os.path.join(self.tmp_root, "clips")
),
):
result = exporter.save_thumbnail(exporter.export_id)
assert result == ""
def test_prefers_in_window_frame_over_prior_frame(self) -> None:
exporter = self._make_short_current_hour_exporter()
self._write_frame(exporter.camera, exporter.start_time - 10.0)
in_window = self._write_frame(exporter.camera, exporter.start_time + 1.0)
thumb_target = os.path.join(self.export_clips, f"{exporter.export_id}.webp")
with (
patch(
"frigate.record.export.CACHE_DIR", os.path.join(self.tmp_root, "cache")
),
patch(
"frigate.record.export.CLIPS_DIR", os.path.join(self.tmp_root, "clips")
),
):
result = exporter.save_thumbnail(exporter.export_id)
assert result == thumb_target
with open(thumb_target, "rb") as f, open(in_window, "rb") as src:
assert f.read() == src.read()
class TestSchedulesCleanup(unittest.TestCase):
def test_schedule_job_cleanup_removes_after_delay(self) -> None:
config = MagicMock()
@@ -498,56 +381,5 @@ class TestSchedulesCleanup(unittest.TestCase):
assert job.id not in manager.jobs
class TestChapterMetadataInProgressReview(unittest.TestCase):
"""Regression: in-progress review segments have end_time=NULL until the
activity closes. The chapter builder must clamp the chapter end to the
last recorded second instead of crashing on float(None)."""
def _fake_select_returning(self, rows: list) -> MagicMock:
mock_query = MagicMock()
mock_query.where.return_value = mock_query
mock_query.order_by.return_value = mock_query
mock_query.iterator.return_value = iter(rows)
return mock_query
def test_in_progress_review_does_not_crash_and_clamps_to_last_recording(
self,
) -> None:
exporter = _make_exporter(end_minus_start=200)
# Recordings cover [1000, 1150]; export window is [1000, 1200] so
# the last recorded second is 1150 (a 50s gap at the tail).
recordings = [
MagicMock(start_time=1000.0, end_time=1150.0),
]
in_progress = MagicMock(
start_time=1100.0,
end_time=None,
severity="alert",
data={"objects": ["person"]},
)
with tempfile.TemporaryDirectory() as tmpdir:
chapter_path = os.path.join(tmpdir, "chapters.txt")
exporter._chapter_metadata_path = lambda: chapter_path # type: ignore[method-assign]
with patch(
"frigate.record.export.ReviewSegment.select",
return_value=self._fake_select_returning([in_progress]),
):
result = exporter._build_chapter_metadata_file(recordings)
assert result == chapter_path
with open(chapter_path) as f:
content = f.read()
# Output time is windows[-1][1] - windows[-1][0] = 150s.
# Review starts at wall=1100, output offset = 100s -> 100000ms.
# Clamped end = last_recorded_end (1150) -> output offset = 150s -> 150000ms.
assert "[CHAPTER]" in content
assert "START=100000" in content
assert "END=150000" in content
assert "title=Alert: person" in content
if __name__ == "__main__":
unittest.main()
-111
View File
@@ -1,111 +0,0 @@
"""Tests for the shared ffmpeg progress helper."""
import unittest
from unittest.mock import MagicMock, patch
from frigate.util.ffmpeg import inject_progress_flags, run_ffmpeg_with_progress
class TestInjectProgressFlags(unittest.TestCase):
def test_inserts_flags_before_output_path(self):
cmd = ["ffmpeg", "-i", "in.mp4", "-c", "copy", "out.mp4"]
result = inject_progress_flags(cmd)
self.assertEqual(
result,
[
"ffmpeg",
"-i",
"in.mp4",
"-c",
"copy",
"-progress",
"pipe:2",
"-nostats",
"out.mp4",
],
)
def test_empty_cmd_returns_empty(self):
self.assertEqual(inject_progress_flags([]), [])
class TestRunFfmpegWithProgress(unittest.TestCase):
def _make_fake_proc(self, stderr_lines, returncode=0):
proc = MagicMock()
proc.stderr = iter(stderr_lines)
proc.stdin = MagicMock()
proc.returncode = returncode
proc.wait = MagicMock()
return proc
def test_emits_percent_from_out_time_us_lines(self):
captured: list[float] = []
def on_progress(percent: float) -> None:
captured.append(percent)
stderr_lines = [
"out_time_us=1000000\n",
"out_time_us=5000000\n",
"progress=end\n",
]
proc = self._make_fake_proc(stderr_lines)
proc.stderr = MagicMock()
proc.stderr.__iter__ = lambda self: iter(stderr_lines)
proc.stderr.read = MagicMock(return_value="")
with patch("subprocess.Popen", return_value=proc):
returncode, _stderr = run_ffmpeg_with_progress(
["ffmpeg", "-i", "in", "out"],
expected_duration_seconds=10.0,
on_progress=on_progress,
use_low_priority=False,
)
self.assertEqual(returncode, 0)
self.assertEqual(len(captured), 4) # initial 0.0 + two parsed + final 100.0
self.assertAlmostEqual(captured[0], 0.0)
self.assertAlmostEqual(captured[1], 10.0)
self.assertAlmostEqual(captured[2], 50.0)
self.assertAlmostEqual(captured[3], 100.0)
def test_passes_started_process_to_callback(self):
proc = self._make_fake_proc([])
proc.stderr = MagicMock()
proc.stderr.__iter__ = lambda self: iter([])
proc.stderr.read = MagicMock(return_value="")
seen: list = []
with patch("subprocess.Popen", return_value=proc):
run_ffmpeg_with_progress(
["ffmpeg", "out"],
expected_duration_seconds=1.0,
process_started=lambda p: seen.append(p),
use_low_priority=False,
)
self.assertEqual(seen, [proc])
def test_clamps_percent_to_0_100(self):
captured: list[float] = []
def on_progress(percent: float) -> None:
captured.append(percent)
stderr_lines = ["out_time_us=999999999999\n"]
proc = self._make_fake_proc(stderr_lines)
proc.stderr = MagicMock()
proc.stderr.__iter__ = lambda self: iter(stderr_lines)
proc.stderr.read = MagicMock(return_value="")
with patch("subprocess.Popen", return_value=proc):
run_ffmpeg_with_progress(
["ffmpeg", "out"],
expected_duration_seconds=10.0,
on_progress=on_progress,
use_low_priority=False,
)
# initial 0.0 then a clamped reading
self.assertEqual(captured[-1], 100.0)
+28 -76
View File
@@ -7,6 +7,8 @@ from frigate.util.services import get_amd_gpu_stats, get_intel_gpu_stats
class TestGpuStats(unittest.TestCase):
def setUp(self):
self.amd_results = "Unknown Radeon card. <= R500 won't work, new cards might.\nDumping to -, line limit 1.\n1664070990.607556: bus 10, gpu 4.17%, ee 0.00%, vgt 0.00%, ta 0.00%, tc 0.00%, sx 0.00%, sh 0.00%, spi 0.83%, smx 0.00%, cr 0.00%, sc 0.00%, pa 0.00%, db 0.00%, cb 0.00%, vram 60.37% 294.04mb, gtt 0.33% 52.21mb, mclk 100.00% 1.800ghz, sclk 26.65% 0.533ghz\n"
self.intel_results = """{"period":{"duration":1.194033,"unit":"ms"},"frequency":{"requested":0.000000,"actual":0.000000,"unit":"MHz"},"interrupts":{"count":3349.991164,"unit":"irq/s"},"rc6":{"value":47.844741,"unit":"%"},"engines":{"Render/3D/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"Blitter/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"Video/0":{"busy":4.533124,"sema":0.000000,"wait":0.000000,"unit":"%"},"Video/1":{"busy":6.194385,"sema":0.000000,"wait":0.000000,"unit":"%"},"VideoEnhance/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"}}},{"period":{"duration":1.189291,"unit":"ms"},"frequency":{"requested":0.000000,"actual":0.000000,"unit":"MHz"},"interrupts":{"count":0.000000,"unit":"irq/s"},"rc6":{"value":100.000000,"unit":"%"},"engines":{"Render/3D/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"Blitter/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"Video/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"Video/1":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"},"VideoEnhance/0":{"busy":0.000000,"sema":0.000000,"wait":0.000000,"unit":"%"}}}"""
self.nvidia_results = "name, utilization.gpu [%], memory.used [MiB], memory.total [MiB]\nNVIDIA GeForce RTX 3050, 42 %, 5036 MiB, 8192 MiB\n"
@patch("subprocess.run")
def test_amd_gpu_stats(self, sp):
@@ -17,82 +19,32 @@ class TestGpuStats(unittest.TestCase):
amd_stats = get_amd_gpu_stats()
assert amd_stats == {"gpu": "4.17%", "mem": "60.37%"}
@patch("frigate.stats.intel_gpu_info.intel_gpu_name_resolver.get_names")
@patch("frigate.util.services.time.sleep")
@patch("frigate.util.services.time.monotonic")
@patch("frigate.util.services._read_intel_drm_fdinfo")
def test_intel_gpu_stats_fdinfo(self, read_fdinfo, monotonic, sleep, get_names):
# 1 second of wall clock between snapshots
monotonic.side_effect = [0.0, 1.0]
get_names.return_value = {"0000:00:02.0": "Intel Graphics"}
# @patch("subprocess.run")
# def test_nvidia_gpu_stats(self, sp):
# process = MagicMock()
# process.returncode = 0
# process.stdout = self.nvidia_results
# sp.return_value = process
# nvidia_stats = get_nvidia_gpu_stats()
# assert nvidia_stats == {
# "name": "NVIDIA GeForce RTX 3050",
# "gpu": "42 %",
# "mem": "61.5 %",
# }
# Two i915 clients on the same iGPU. Engine values are cumulative ns.
# Deltas over the 1s window:
# client A (pid 100): render +200_000_000 (20%), video +500_000_000 (50%),
# video-enhance +100_000_000 (10%)
# client B (pid 200): compute +100_000_000 (10%)
# Engine totals → render 20, video 50, video-enhance 10, compute 10
# → compute = render + compute = 30
# → dec = video + video-enhance = 60
# → gpu = compute + dec = 90
snapshot_a = {
("0000:00:02.0", "1", "100"): {
"driver": "i915",
"pid": "100",
"engines": {
"render": (1_000_000_000, 0),
"video": (5_000_000_000, 0),
"video-enhance": (200_000_000, 0),
"compute": (0, 0),
},
},
("0000:00:02.0", "2", "200"): {
"driver": "i915",
"pid": "200",
"engines": {
"render": (0, 0),
"compute": (2_000_000_000, 0),
},
},
}
snapshot_b = {
("0000:00:02.0", "1", "100"): {
"driver": "i915",
"pid": "100",
"engines": {
"render": (1_200_000_000, 0),
"video": (5_500_000_000, 0),
"video-enhance": (300_000_000, 0),
"compute": (0, 0),
},
},
("0000:00:02.0", "2", "200"): {
"driver": "i915",
"pid": "200",
"engines": {
"render": (0, 0),
"compute": (2_100_000_000, 0),
},
},
}
read_fdinfo.side_effect = [snapshot_a, snapshot_b]
intel_stats = get_intel_gpu_stats(None)
sleep.assert_called_once()
@patch("subprocess.run")
def test_intel_gpu_stats(self, sp):
process = MagicMock()
process.returncode = 124
process.stdout = self.intel_results
sp.return_value = process
intel_stats = get_intel_gpu_stats(False)
# rc6 values: 47.844741 and 100.0 → avg 73.92 → gpu = 100 - 73.92 = 26.08%
# Render/3D/0: 0.0 and 0.0 → enc = 0.0%
# Video/0: 4.533124 and 0.0 → dec = 2.27%
assert intel_stats == {
"0000:00:02.0": {
"name": "Intel Graphics",
"vendor": "intel",
"gpu": "90.0%",
"mem": "-%",
"compute": "30.0%",
"dec": "60.0%",
"clients": {"100": "80.0%", "200": "10.0%"},
},
"gpu": "26.08%",
"mem": "-%",
"compute": "0.0%",
"dec": "2.27%",
}
@patch("frigate.util.services._read_intel_drm_fdinfo")
def test_intel_gpu_stats_no_clients(self, read_fdinfo):
read_fdinfo.return_value = {}
assert get_intel_gpu_stats(None) is None
-381
View File
@@ -1,381 +0,0 @@
"""Unit tests for `frigate.api.media_auth`.
Covers URI classification, the role-vs-camera decision matrix, and the export
DB-lookup path. These are pure functions/DB lookups no HTTP stack involved.
"""
import datetime
import logging
import os
import unittest
from peewee_migrate import Router
from playhouse.sqlite_ext import SqliteExtDatabase
from playhouse.sqliteq import SqliteQueueDatabase
from frigate.api.media_auth import (
MediaAuthResolution,
deny_response_for_media_uri,
extract_path,
resolve_media_uri,
)
from frigate.config import FrigateConfig
from frigate.models import Event, Export, Recordings, ReviewSegment
from frigate.test.const import TEST_DB, TEST_DB_CLEANUPS
_CONFIG = {
"mqtt": {"host": "mqtt"},
"auth": {"roles": {"limited_user": ["front_door"]}},
"cameras": {
"front_door": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.1:554/video", "roles": ["detect"]}]
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
"back_door": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.2:554/video", "roles": ["detect"]}]
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
# Camera name with a hyphen — exercises longest-prefix match.
"back-yard": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.3:554/video", "roles": ["detect"]}]
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
},
}
class TestExtractPath(unittest.TestCase):
def test_full_url(self):
self.assertEqual(
extract_path("http://host:8971/clips/front_door-1.jpg"),
"/clips/front_door-1.jpg",
)
def test_strips_query_string(self):
self.assertEqual(
extract_path("http://h/recordings/2026-05-11/14/front_door/00.00.mp4?t=1"),
"/recordings/2026-05-11/14/front_door/00.00.mp4",
)
def test_path_only(self):
self.assertEqual(extract_path("/exports/x.mp4"), "/exports/x.mp4")
def test_percent_decoded(self):
self.assertEqual(
extract_path("http://h/clips/front%20door-1.jpg"),
"/clips/front door-1.jpg",
)
def test_empty(self):
self.assertIsNone(extract_path(None))
self.assertIsNone(extract_path(""))
class TestResolveMediaUri(unittest.TestCase):
def setUp(self):
self.config = FrigateConfig(**_CONFIG)
def _assert(self, uri, resolution, camera=None):
got_resolution, got_camera = resolve_media_uri(uri, self.config)
self.assertEqual(got_resolution, resolution, uri)
self.assertEqual(got_camera, camera, uri)
def test_unknown_paths(self):
self._assert("/api/events", MediaAuthResolution.UNKNOWN)
self._assert("/", MediaAuthResolution.UNKNOWN)
self._assert("", MediaAuthResolution.UNKNOWN)
def test_recordings(self):
self._assert("/recordings/", MediaAuthResolution.LISTING_NEUTRAL)
self._assert("/recordings/2026-05-11/", MediaAuthResolution.LISTING_NEUTRAL)
self._assert(
"/recordings/2026-05-11/14/", MediaAuthResolution.LISTING_MULTI_CAMERA
)
self._assert(
"/recordings/2026-05-11/14/front_door/",
MediaAuthResolution.CAMERA,
camera="front_door",
)
self._assert(
"/recordings/2026-05-11/14/back_door/00.00.mp4",
MediaAuthResolution.CAMERA,
camera="back_door",
)
def test_clip_flat_filename_resolves_camera(self):
self._assert(
"/clips/front_door-1234.jpg",
MediaAuthResolution.CAMERA,
camera="front_door",
)
self._assert(
"/clips/back_door-1234-clean.webp",
MediaAuthResolution.CAMERA,
camera="back_door",
)
def test_clip_filename_with_hyphenated_camera_name(self):
# Camera name "back-yard" itself contains a hyphen; longest-prefix
# match must pick `back-yard`, not the bogus `back` prefix.
self._assert(
"/clips/back-yard-1234.jpg",
MediaAuthResolution.CAMERA,
camera="back-yard",
)
def test_clip_filename_no_matching_camera(self):
# Looks like a media path but couldn't classify — fail closed for
# restricted users (UNRESOLVED_MEDIA), not pass-through.
self._assert(
"/clips/nonexistent-1234.jpg", MediaAuthResolution.UNRESOLVED_MEDIA
)
def test_clip_thumbs(self):
self._assert("/clips/thumbs/", MediaAuthResolution.LISTING_MULTI_CAMERA)
self._assert(
"/clips/thumbs/front_door/",
MediaAuthResolution.CAMERA,
camera="front_door",
)
self._assert(
"/clips/thumbs/back_door/abc.webp",
MediaAuthResolution.CAMERA,
camera="back_door",
)
def test_clip_previews(self):
self._assert("/clips/previews/", MediaAuthResolution.LISTING_MULTI_CAMERA)
self._assert(
"/clips/previews/front_door/",
MediaAuthResolution.CAMERA,
camera="front_door",
)
self._assert(
"/clips/previews/back_door/segment.mp4",
MediaAuthResolution.CAMERA,
camera="back_door",
)
def test_clip_review_thumbs(self):
# Format: /clips/review/thumb-{camera}-{review_id}.webp (frigate/review/maintainer.py).
self._assert(
"/clips/review/thumb-front_door-abc123.webp",
MediaAuthResolution.CAMERA,
camera="front_door",
)
# Hyphenated camera name — longest-prefix match.
self._assert(
"/clips/review/thumb-back-yard-abc123.webp",
MediaAuthResolution.CAMERA,
camera="back-yard",
)
# Unknown camera prefix → unresolved, not allowed for restricted users.
self._assert(
"/clips/review/thumb-unknown-cam-abc123.webp",
MediaAuthResolution.UNRESOLVED_MEDIA,
)
def test_clip_admin_only_subtrees(self):
self._assert("/clips/faces/train/foo.webp", MediaAuthResolution.ADMIN_ONLY)
self._assert("/clips/faces/", MediaAuthResolution.ADMIN_ONLY)
self._assert("/clips/genai-requests/x/0.webp", MediaAuthResolution.ADMIN_ONLY)
self._assert(
"/clips/preview_restart_cache/x.mp4", MediaAuthResolution.ADMIN_ONLY
)
self._assert("/clips/some_model/train/x.jpg", MediaAuthResolution.ADMIN_ONLY)
self._assert("/clips/some_model/dataset/x.jpg", MediaAuthResolution.ADMIN_ONLY)
def test_clip_unknown_subtree_is_unresolved(self):
# Unknown /clips/{x}/{y}/... subtree falls through as unresolved (not
# admin-only) so restricted users get 403 without admins being denied
# access to legitimate but unrecognized resources.
self._assert("/clips/random_dir/foo.jpg", MediaAuthResolution.UNRESOLVED_MEDIA)
def test_clip_top_level_listing(self):
self._assert("/clips/", MediaAuthResolution.LISTING_MULTI_CAMERA)
def test_exports_listing(self):
self._assert("/exports/", MediaAuthResolution.LISTING_MULTI_CAMERA)
class TestExportResolution(unittest.TestCase):
"""Export resolution requires a DB lookup."""
def setUp(self):
migrate_db = SqliteExtDatabase("test.db")
del logging.getLogger("peewee_migrate").handlers[:]
Router(migrate_db).run()
migrate_db.close()
self.db = SqliteQueueDatabase(TEST_DB)
self.db.bind([Event, ReviewSegment, Recordings, Export])
self.config = FrigateConfig(**_CONFIG)
def tearDown(self):
if not self.db.is_closed():
self.db.close()
for f in TEST_DB_CLEANUPS:
try:
os.remove(f)
except OSError:
pass
def _insert_export(self, export_id, camera, filename):
Export.insert(
id=export_id,
camera=camera,
name=f"export-{export_id}",
date=int(datetime.datetime.now().timestamp()),
video_path=f"/media/frigate/exports/{filename}",
thumb_path=f"/media/frigate/exports/{filename}.jpg",
in_progress=False,
).execute()
def test_export_resolves_camera(self):
self._insert_export(
"exp1", "back_door", "back_door_20260511_140000-20260511_150000_abc123.mp4"
)
resolution, camera = resolve_media_uri(
"/exports/back_door_20260511_140000-20260511_150000_abc123.mp4",
self.config,
)
self.assertEqual(resolution, MediaAuthResolution.CAMERA)
self.assertEqual(camera, "back_door")
def test_unknown_export_is_unresolved(self):
# No matching row → UNRESOLVED_MEDIA (fail closed for restricted users),
# not UNKNOWN (which would pass-through).
resolution, camera = resolve_media_uri(
"/exports/does_not_exist.mp4", self.config
)
self.assertEqual(resolution, MediaAuthResolution.UNRESOLVED_MEDIA)
self.assertIsNone(camera)
def test_export_anchored_match_not_endswith(self):
# Anchored exact-path equality must NOT match by filename suffix.
# A request like /exports/clip.mp4 must not authorize against a row at
# /media/frigate/exports/back_door_clip.mp4 just because the suffix matches.
self._insert_export("exp_bd", "back_door", "back_door_clip.mp4")
self._insert_export("exp_fd", "front_door", "front_door_clip.mp4")
resolution, _ = resolve_media_uri("/exports/clip.mp4", self.config)
self.assertEqual(resolution, MediaAuthResolution.UNRESOLVED_MEDIA)
class TestDenyResponseForMediaUri(unittest.TestCase):
"""End-to-end decision check used by /auth."""
def setUp(self):
self.config = FrigateConfig(**_CONFIG)
def _deny(self, url, role):
return deny_response_for_media_uri(url, role, self.config)
def test_admin_always_allowed(self):
self.assertIsNone(self._deny("/clips/back_door-1.jpg", "admin"))
self.assertIsNone(self._deny("/clips/", "admin"))
self.assertIsNone(self._deny("/clips/faces/x.webp", "admin"))
self.assertIsNone(
self._deny("/recordings/2026-05-11/14/back_door/00.00.mp4", "admin")
)
def test_unrestricted_role_allowed(self):
# "viewer" role has no entry in roles_dict → full access (matches the
# behavior of require_camera_access).
self.assertIsNone(self._deny("/clips/back_door-1.jpg", "viewer"))
self.assertIsNone(self._deny("/clips/", "viewer"))
def test_restricted_role_allowed_camera(self):
self.assertIsNone(self._deny("/clips/front_door-1.jpg", "limited_user"))
self.assertIsNone(
self._deny("/recordings/2026-05-11/14/front_door/00.00.mp4", "limited_user")
)
self.assertIsNone(
self._deny("/clips/thumbs/front_door/abc.webp", "limited_user")
)
def test_restricted_role_blocked_other_camera(self):
self.assertEqual(self._deny("/clips/back_door-1.jpg", "limited_user"), 403)
self.assertEqual(
self._deny("/recordings/2026-05-11/14/back_door/00.00.mp4", "limited_user"),
403,
)
self.assertEqual(
self._deny("/clips/thumbs/back_door/abc.webp", "limited_user"), 403
)
def test_restricted_role_blocked_admin_only(self):
self.assertEqual(self._deny("/clips/faces/train/foo.webp", "limited_user"), 403)
def test_restricted_role_blocked_multi_camera_listing(self):
self.assertEqual(self._deny("/clips/", "limited_user"), 403)
self.assertEqual(self._deny("/exports/", "limited_user"), 403)
self.assertEqual(self._deny("/recordings/2026-05-11/14/", "limited_user"), 403)
def test_restricted_role_allowed_neutral_listing(self):
self.assertIsNone(self._deny("/recordings/", "limited_user"))
self.assertIsNone(self._deny("/recordings/2026-05-11/", "limited_user"))
def test_non_media_uri_passes_through(self):
self.assertIsNone(self._deny("/api/events", "limited_user"))
self.assertIsNone(self._deny("http://h/login", "limited_user"))
def test_missing_header(self):
self.assertIsNone(self._deny(None, "limited_user"))
self.assertIsNone(self._deny("", "limited_user"))
def test_traversal_in_media_uri_denied_for_all_roles(self):
# Bypass attempt: parts[3] looks like an allowed camera, but the
# normalized path nginx would serve points at a forbidden camera.
# Both restricted and admin should be denied — the URI is malformed
# and we refuse to make an auth decision against it.
traversal_uris = [
"/recordings/2026-05-11/14/front_door/../back_door/00.00.mp4",
"/clips/front_door-1.jpg/../back_door-1.jpg",
"/exports/../recordings/2026-05-11/14/back_door/00.00.mp4",
"/clips/./back_door-1.jpg",
]
for uri in traversal_uris:
self.assertEqual(self._deny(uri, "limited_user"), 403, uri)
self.assertEqual(self._deny(uri, "admin"), 403, uri)
self.assertEqual(self._deny(uri, "viewer"), 403, uri)
def test_traversal_outside_media_passes_through(self):
# `..` in non-media URIs is not our problem; the backend handles it.
self.assertIsNone(self._deny("/api/foo/../bar", "limited_user"))
def test_percent_encoded_traversal_denied(self):
# nginx may decode percent-encoded `%2E%2E` to `..` before serving;
# we must apply the same denial after percent-decoding.
self.assertEqual(
self._deny(
"/recordings/2026-05-11/14/front_door/%2E%2E/back_door/00.mp4",
"limited_user",
),
403,
)
def test_unresolved_media_fails_closed_for_restricted(self):
# Restricted user requesting a media URI we can't classify (no DB row,
# unknown clip prefix, unknown clip subtree) must be denied.
self.assertEqual(self._deny("/clips/nonexistent-1.jpg", "limited_user"), 403)
self.assertEqual(self._deny("/clips/random_dir/foo.jpg", "limited_user"), 403)
self.assertEqual(
self._deny("/clips/review/thumb-unknown_cam-1.webp", "limited_user"),
403,
)
def test_unresolved_media_allowed_for_admin(self):
# Admin and full-access roles are *not* denied on UNRESOLVED_MEDIA —
# nginx returns 404 if the file doesn't exist on disk anyway, and we
# don't want a stale DB to lock out admins.
self.assertIsNone(self._deny("/clips/nonexistent-1.jpg", "admin"))
self.assertIsNone(self._deny("/clips/nonexistent-1.jpg", "viewer"))
if __name__ == "__main__":
unittest.main()
-166
View File
@@ -1,166 +0,0 @@
"""Tests for WebSocket authorization checks."""
import unittest
from frigate.comms.ws import _check_ws_authorization
from frigate.const import INSERT_MANY_RECORDINGS, UPDATE_CAMERA_ACTIVITY
class TestCheckWsAuthorization(unittest.TestCase):
"""Tests for the _check_ws_authorization pure function."""
DEFAULT_SEPARATOR = ","
# --- IPC topic blocking (unconditional, regardless of role) ---
def test_ipc_topic_blocked_for_admin(self):
self.assertFalse(
_check_ws_authorization(
INSERT_MANY_RECORDINGS, "admin", self.DEFAULT_SEPARATOR
)
)
def test_ipc_topic_blocked_for_viewer(self):
self.assertFalse(
_check_ws_authorization(
UPDATE_CAMERA_ACTIVITY, "viewer", self.DEFAULT_SEPARATOR
)
)
def test_ipc_topic_blocked_when_no_role(self):
self.assertFalse(
_check_ws_authorization(
INSERT_MANY_RECORDINGS, None, self.DEFAULT_SEPARATOR
)
)
# --- Viewer allowed topics ---
def test_viewer_can_send_on_connect(self):
self.assertTrue(
_check_ws_authorization("onConnect", "viewer", self.DEFAULT_SEPARATOR)
)
def test_viewer_can_send_model_state(self):
self.assertTrue(
_check_ws_authorization("modelState", "viewer", self.DEFAULT_SEPARATOR)
)
def test_viewer_can_send_audio_transcription_state(self):
self.assertTrue(
_check_ws_authorization(
"audioTranscriptionState", "viewer", self.DEFAULT_SEPARATOR
)
)
def test_viewer_can_send_birdseye_layout(self):
self.assertTrue(
_check_ws_authorization("birdseyeLayout", "viewer", self.DEFAULT_SEPARATOR)
)
def test_viewer_can_send_embeddings_reindex_progress(self):
self.assertTrue(
_check_ws_authorization(
"embeddingsReindexProgress", "viewer", self.DEFAULT_SEPARATOR
)
)
# --- Viewer blocked from admin topics ---
def test_viewer_blocked_from_restart(self):
self.assertFalse(
_check_ws_authorization("restart", "viewer", self.DEFAULT_SEPARATOR)
)
def test_viewer_blocked_from_camera_detect_set(self):
self.assertFalse(
_check_ws_authorization(
"front_door/detect/set", "viewer", self.DEFAULT_SEPARATOR
)
)
def test_viewer_blocked_from_camera_ptz(self):
self.assertFalse(
_check_ws_authorization("front_door/ptz", "viewer", self.DEFAULT_SEPARATOR)
)
def test_viewer_blocked_from_global_notifications_set(self):
self.assertFalse(
_check_ws_authorization(
"notifications/set", "viewer", self.DEFAULT_SEPARATOR
)
)
def test_viewer_blocked_from_camera_notifications_suspend(self):
self.assertFalse(
_check_ws_authorization(
"front_door/notifications/suspend", "viewer", self.DEFAULT_SEPARATOR
)
)
def test_viewer_blocked_from_arbitrary_unknown_topic(self):
self.assertFalse(
_check_ws_authorization(
"some_random_topic", "viewer", self.DEFAULT_SEPARATOR
)
)
# --- Admin access ---
def test_admin_can_send_restart(self):
self.assertTrue(
_check_ws_authorization("restart", "admin", self.DEFAULT_SEPARATOR)
)
def test_admin_can_send_camera_detect_set(self):
self.assertTrue(
_check_ws_authorization(
"front_door/detect/set", "admin", self.DEFAULT_SEPARATOR
)
)
def test_admin_can_send_camera_ptz(self):
self.assertTrue(
_check_ws_authorization("front_door/ptz", "admin", self.DEFAULT_SEPARATOR)
)
# --- Comma-separated roles ---
def test_comma_separated_admin_viewer_grants_admin(self):
self.assertTrue(
_check_ws_authorization("restart", "admin,viewer", self.DEFAULT_SEPARATOR)
)
def test_comma_separated_viewer_admin_grants_admin(self):
self.assertTrue(
_check_ws_authorization("restart", "viewer,admin", self.DEFAULT_SEPARATOR)
)
def test_comma_separated_with_spaces(self):
self.assertTrue(
_check_ws_authorization("restart", "viewer, admin", self.DEFAULT_SEPARATOR)
)
# --- Custom separator ---
def test_pipe_separator(self):
self.assertTrue(_check_ws_authorization("restart", "viewer|admin", "|"))
def test_pipe_separator_no_admin(self):
self.assertFalse(_check_ws_authorization("restart", "viewer|editor", "|"))
# --- No role header (fail-closed) ---
def test_no_role_header_blocks_admin_topics(self):
self.assertFalse(
_check_ws_authorization("restart", None, self.DEFAULT_SEPARATOR)
)
def test_no_role_header_allows_viewer_topics(self):
self.assertTrue(
_check_ws_authorization("onConnect", None, self.DEFAULT_SEPARATOR)
)
if __name__ == "__main__":
unittest.main()
-806
View File
@@ -1,806 +0,0 @@
"""Tests for outbound WebSocket broadcast filtering."""
import json
import threading
import unittest
from types import SimpleNamespace
from typing import Any
from frigate.comms.ws import (
WebSocketClient,
_classify_outbound,
_collect_zone_names,
_extract_payload_camera,
_materialize_for_ws,
_ws_allowed_cameras,
_ws_is_unrestricted,
)
from frigate.config import FrigateConfig
def _build_config(
*,
extra_roles: dict[str, list[str]] | None = None,
extra_cameras: dict[str, dict[str, Any]] | None = None,
extra_zones: dict[str, dict[str, dict[str, Any]]] | None = None,
) -> FrigateConfig:
"""Construct a FrigateConfig used by the outbound filter tests.
The default fixture has three cameras: front_door, back_door, garage.
Restricted role "house_only" sees front_door + back_door but not garage.
"""
cameras: dict[str, dict[str, Any]] = {
"front_door": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.1:554/v", "roles": ["detect"]}],
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
"back_door": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.2:554/v", "roles": ["detect"]}],
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
"garage": {
"ffmpeg": {
"inputs": [{"path": "rtsp://10.0.0.3:554/v", "roles": ["detect"]}],
},
"detect": {"height": 1080, "width": 1920, "fps": 5},
},
}
if extra_cameras:
cameras.update(extra_cameras)
if extra_zones:
for cam_name, zones in extra_zones.items():
cameras[cam_name]["zones"] = zones
roles = {"house_only": ["front_door", "back_door"]}
if extra_roles:
roles.update(extra_roles)
return FrigateConfig(
mqtt={"host": "mqtt"},
auth={"roles": roles},
cameras=cameras,
)
def _ws(role: str | None) -> Any:
"""Build a fake ws4py-style websocket exposing ``environ``."""
environ = {} if role is None else {"HTTP_REMOTE_ROLE": role}
return SimpleNamespace(environ=environ, terminated=False, sent=[])
class TestClassifyOutbound(unittest.TestCase):
"""The pure classifier — bucket every topic into a scope."""
def setUp(self):
self.config = _build_config(
extra_zones={"front_door": {"driveway": {"coordinates": "0,0,1,0,1,1,0,1"}}}
)
self.all_cameras = set(self.config.cameras.keys())
self.all_zones = _collect_zone_names(self.config)
def _classify(self, topic: str) -> tuple[str, Any]:
return _classify_outbound(topic, self.all_cameras, self.all_zones)
# --- Global allowlist ---
def test_model_state_is_global(self):
self.assertEqual(self._classify("model_state"), ("global", None))
def test_profile_state_is_global(self):
self.assertEqual(self._classify("profile/state"), ("global", None))
def test_bare_notifications_state_is_global(self):
"""The 2-segment ``notifications/state`` is global; the 3-segment
``<camera>/notifications/state`` is camera-scoped (see below)."""
self.assertEqual(self._classify("notifications/state"), ("global", None))
def test_notification_test_is_global(self):
self.assertEqual(self._classify("notification_test"), ("global", None))
# --- Unrestricted-only ---
def test_birdseye_layout_is_unrestricted_only(self):
self.assertEqual(self._classify("birdseye_layout"), ("unrestricted_only", None))
# --- Camera-prefixed ---
def test_camera_state_topic_resolves_to_camera(self):
self.assertEqual(
self._classify("front_door/detect/state"), ("camera", "front_door")
)
def test_camera_motion_topic_resolves_to_camera(self):
self.assertEqual(self._classify("back_door/motion"), ("camera", "back_door"))
def test_camera_per_notification_topic_resolves_to_camera(self):
self.assertEqual(
self._classify("front_door/notifications/state"),
("camera", "front_door"),
)
def test_camera_label_counter_resolves_to_camera(self):
self.assertEqual(self._classify("front_door/person"), ("camera", "front_door"))
def test_camera_object_mask_state_resolves_to_camera(self):
self.assertEqual(
self._classify("front_door/object_mask/zone_1/state"),
("camera", "front_door"),
)
# --- Zone-prefixed ---
def test_zone_aggregate_topic_is_unrestricted_only(self):
self.assertEqual(self._classify("driveway/person"), ("unrestricted_only", None))
def test_zone_all_topic_is_unrestricted_only(self):
self.assertEqual(self._classify("driveway/all"), ("unrestricted_only", None))
# --- Payload-camera ---
def test_events_topic_marks_payload_camera_path(self):
self.assertEqual(
self._classify("events"), ("payload_camera", ("after", "camera"))
)
def test_reviews_topic_marks_payload_camera_path(self):
self.assertEqual(
self._classify("reviews"), ("payload_camera", ("after", "camera"))
)
def test_triggers_topic_marks_payload_camera_path(self):
self.assertEqual(self._classify("triggers"), ("payload_camera", ("camera",)))
def test_tracked_object_update_marks_payload_camera_path(self):
self.assertEqual(
self._classify("tracked_object_update"), ("payload_camera", ("camera",))
)
# --- Reshape ---
def test_camera_activity_is_reshape_by_camera_key(self):
self.assertEqual(
self._classify("camera_activity"), ("reshape_by_camera_key", None)
)
def test_audio_detections_is_reshape_by_camera_key(self):
self.assertEqual(
self._classify("audio_detections"), ("reshape_by_camera_key", None)
)
def test_job_state_is_reshape_job_state(self):
self.assertEqual(self._classify("job_state"), ("reshape_job_state", None))
def test_stats_is_reshape_stats(self):
self.assertEqual(self._classify("stats"), ("reshape_stats", None))
# --- Fail-closed ---
def test_unknown_topic_is_dropped(self):
self.assertEqual(self._classify("some_random_topic"), ("drop", None))
def test_unknown_camera_prefix_is_dropped(self):
self.assertEqual(self._classify("ghost_camera/detect/state"), ("drop", None))
class TestCollectZoneNames(unittest.TestCase):
def test_zones_from_all_cameras(self):
config = _build_config(
extra_zones={
"front_door": {"driveway": {"coordinates": "0,0,1,0,1,1,0,1"}},
"back_door": {"yard": {"coordinates": "0,0,1,0,1,1,0,1"}},
}
)
self.assertEqual(_collect_zone_names(config), {"driveway", "yard"})
def test_no_zones_returns_empty(self):
self.assertEqual(_collect_zone_names(_build_config()), set())
class TestExtractPayloadCamera(unittest.TestCase):
def test_extract_from_dict_path(self):
payload = {"after": {"camera": "front_door"}}
self.assertEqual(
_extract_payload_camera(payload, ("after", "camera")), "front_door"
)
def test_extract_from_json_string(self):
payload = json.dumps({"after": {"camera": "front_door"}})
self.assertEqual(
_extract_payload_camera(payload, ("after", "camera")), "front_door"
)
def test_extract_single_segment_path(self):
self.assertEqual(
_extract_payload_camera({"camera": "garage"}, ("camera",)), "garage"
)
def test_missing_key_returns_none(self):
self.assertIsNone(_extract_payload_camera({}, ("after", "camera")))
def test_malformed_json_returns_none(self):
self.assertIsNone(_extract_payload_camera("not-json", ("camera",)))
def test_non_string_camera_returns_none(self):
self.assertIsNone(_extract_payload_camera({"camera": 42}, ("camera",)))
class TestWsRoleHelpers(unittest.TestCase):
def setUp(self):
self.config = _build_config()
def test_admin_is_unrestricted(self):
self.assertTrue(_ws_is_unrestricted(_ws("admin"), self.config))
def test_viewer_is_unrestricted(self):
self.assertTrue(_ws_is_unrestricted(_ws("viewer"), self.config))
def test_restricted_role_is_not_unrestricted(self):
self.assertFalse(_ws_is_unrestricted(_ws("house_only"), self.config))
def test_missing_role_is_not_unrestricted(self):
self.assertFalse(_ws_is_unrestricted(_ws(None), self.config))
def test_unknown_role_is_not_unrestricted(self):
self.assertFalse(_ws_is_unrestricted(_ws("ghost"), self.config))
def test_admin_allowed_cameras_is_all(self):
self.assertEqual(
_ws_allowed_cameras(_ws("admin"), self.config),
{"front_door", "back_door", "garage"},
)
def test_restricted_role_allowed_cameras_is_subset(self):
self.assertEqual(
_ws_allowed_cameras(_ws("house_only"), self.config),
{"front_door", "back_door"},
)
def test_missing_role_allowed_cameras_is_empty(self):
self.assertEqual(_ws_allowed_cameras(_ws(None), self.config), set())
def test_multi_role_union_grants_widest(self):
self.assertEqual(
_ws_allowed_cameras(_ws("house_only,admin"), self.config),
{"front_door", "back_door", "garage"},
)
class TestMaterializeForWs(unittest.TestCase):
def setUp(self):
self.config = _build_config(
extra_zones={"front_door": {"driveway": {"coordinates": "0,0,1,0,1,1,0,1"}}}
)
self.all_cameras = set(self.config.cameras.keys())
self.all_zones = _collect_zone_names(self.config)
def _materialize(self, ws: Any, topic: str, payload: Any) -> str | None:
scope = _classify_outbound(topic, self.all_cameras, self.all_zones)
from frigate.comms.ws import _parse_json_payload
parsed = (
_parse_json_payload(payload)
if scope[0]
in (
"payload_camera",
"reshape_by_camera_key",
"reshape_job_state",
"reshape_stats",
)
else None
)
full = json.dumps({"topic": topic, "payload": payload})
return _materialize_for_ws(ws, topic, full, scope, parsed, self.config)
# --- Globals: every authenticated client sees them ---
def test_globals_reach_admin(self):
self.assertIsNotNone(self._materialize(_ws("admin"), "model_state", "{}"))
def test_globals_reach_restricted(self):
self.assertIsNotNone(self._materialize(_ws("house_only"), "model_state", "{}"))
def test_globals_reach_no_role(self):
"""A missing role header still gets globals (matches viewer-default
for inbound)."""
self.assertIsNotNone(self._materialize(_ws(None), "model_state", "{}"))
# --- Unknown topic dropped for everyone ---
def test_unknown_topic_dropped_for_admin(self):
self.assertIsNone(self._materialize(_ws("admin"), "rogue_topic", "{}"))
# --- Non-global topics require a role (fail-closed) ---
def test_no_role_blocked_from_camera_topic(self):
self.assertIsNone(self._materialize(_ws(None), "front_door/detect/state", "ON"))
def test_no_role_blocked_from_events(self):
payload = json.dumps({"after": {"camera": "front_door"}})
self.assertIsNone(self._materialize(_ws(None), "events", payload))
# --- Camera-prefixed ---
def test_restricted_role_sees_allowed_camera(self):
self.assertIsNotNone(
self._materialize(_ws("house_only"), "front_door/detect/state", "ON")
)
def test_restricted_role_blocked_from_unallowed_camera(self):
self.assertIsNone(
self._materialize(_ws("house_only"), "garage/detect/state", "ON")
)
def test_admin_sees_all_camera_topics(self):
self.assertIsNotNone(
self._materialize(_ws("admin"), "garage/detect/state", "ON")
)
# --- Unrestricted-only (zones, birdseye_layout) ---
def test_zone_aggregate_blocked_for_restricted(self):
self.assertIsNone(self._materialize(_ws("house_only"), "driveway/person", 3))
def test_zone_aggregate_visible_to_admin(self):
self.assertIsNotNone(self._materialize(_ws("admin"), "driveway/person", 3))
def test_birdseye_layout_blocked_for_restricted(self):
payload = json.dumps(
{"front_door": {"x": 0, "y": 0, "width": 100, "height": 100}}
)
self.assertIsNone(
self._materialize(_ws("house_only"), "birdseye_layout", payload)
)
def test_birdseye_layout_visible_to_admin(self):
payload = json.dumps(
{"front_door": {"x": 0, "y": 0, "width": 100, "height": 100}}
)
self.assertIsNotNone(
self._materialize(_ws("admin"), "birdseye_layout", payload)
)
# --- Payload-camera ---
def test_events_filtered_by_payload_camera(self):
payload = json.dumps({"after": {"camera": "garage"}})
self.assertIsNone(self._materialize(_ws("house_only"), "events", payload))
payload = json.dumps({"after": {"camera": "front_door"}})
self.assertIsNotNone(self._materialize(_ws("house_only"), "events", payload))
def test_events_with_missing_camera_dropped(self):
payload = json.dumps({"after": {}})
self.assertIsNone(self._materialize(_ws("house_only"), "events", payload))
def test_triggers_filtered_by_payload_camera(self):
payload = json.dumps({"name": "t1", "camera": "garage"})
self.assertIsNone(self._materialize(_ws("house_only"), "triggers", payload))
# --- Reshape: dict keyed by camera ---
def test_camera_activity_filtered_to_allowed_keys(self):
payload = json.dumps(
{
"front_door": {"objects": 1},
"back_door": {"objects": 0},
"garage": {"objects": 2},
}
)
message = self._materialize(_ws("house_only"), "camera_activity", payload)
self.assertIsNotNone(message)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertEqual(set(inner.keys()), {"front_door", "back_door"})
self.assertNotIn("garage", inner)
def test_camera_activity_unchanged_for_admin(self):
payload = json.dumps({"front_door": {}, "back_door": {}, "garage": {}})
message = self._materialize(_ws("admin"), "camera_activity", payload)
envelope = json.loads(message) # type: ignore[arg-type]
self.assertEqual(envelope["payload"], payload)
def test_camera_activity_with_no_allowed_returns_none(self):
payload = json.dumps({"garage": {"objects": 2}})
self.assertIsNone(
self._materialize(_ws("house_only"), "camera_activity", payload)
)
def test_audio_detections_filtered_to_allowed_keys(self):
payload = json.dumps({"front_door": {"bark": {}}, "garage": {"speech": {}}})
message = self._materialize(_ws("house_only"), "audio_detections", payload)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertEqual(set(inner.keys()), {"front_door"})
# --- Reshape: job_state ---
def test_job_state_admin_sees_full_payload(self):
payload = json.dumps(
{
"motion_search": {"job_type": "motion_search", "camera": "garage"},
"media_sync": {"job_type": "media_sync"},
}
)
message = self._materialize(_ws("admin"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
self.assertEqual(envelope["payload"], payload)
def test_job_state_restricted_keeps_allowed_camera_jobs(self):
"""Top-level camera field on a job entry: drop if not allowed."""
payload = json.dumps(
{
"motion_search": {"job_type": "motion_search", "camera": "front_door"},
"vlm_watch": {"job_type": "vlm_watch", "camera": "garage"},
}
)
message = self._materialize(_ws("house_only"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertIn("motion_search", inner)
self.assertNotIn("vlm_watch", inner)
def test_job_state_export_results_jobs_filtered_per_recipient(self):
"""The aggregated export broadcast nests per-camera sub-jobs under
``results.jobs``. Restricted users must only see allowed entries."""
payload = json.dumps(
{
"export": {
"job_type": "export",
"status": "running",
"results": {
"jobs": [
{"job_type": "export", "camera": "front_door", "id": "a"},
{"job_type": "export", "camera": "garage", "id": "b"},
{"job_type": "export", "camera": "back_door", "id": "c"},
]
},
}
}
)
message = self._materialize(_ws("house_only"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertIn("export", inner)
kept_cameras = [j["camera"] for j in inner["export"]["results"]["jobs"]]
self.assertEqual(kept_cameras, ["front_door", "back_door"])
# Sibling fields like ``status`` must survive reshaping.
self.assertEqual(inner["export"]["status"], "running")
def test_job_state_export_entry_dropped_when_no_jobs_allowed(self):
payload = json.dumps(
{
"export": {
"job_type": "export",
"status": "running",
"results": {
"jobs": [
{"job_type": "export", "camera": "garage", "id": "b"},
]
},
}
}
)
self.assertIsNone(self._materialize(_ws("house_only"), "job_state", payload))
# --- Reshape: stats ---
def _stats_payload(self) -> str:
return json.dumps(
{
"cameras": {
"front_door": {"camera_fps": 5.0, "pid": 1234},
"back_door": {"camera_fps": 5.0, "pid": 1235},
"garage": {"camera_fps": 5.0, "pid": 1236},
},
"detectors": {"cpu": {"detection_start": 0.0, "inference_speed": 10}},
"service": {"uptime": 12345, "version": "0.16.0"},
"camera_fps": 15.0,
"detection_fps": 6.0,
}
)
def test_stats_admin_sees_full_payload(self):
message = self._materialize(_ws("admin"), "stats", self._stats_payload())
envelope = json.loads(message) # type: ignore[arg-type]
self.assertEqual(envelope["payload"], self._stats_payload())
def test_stats_restricted_filters_camera_keys_but_keeps_aggregates(self):
message = self._materialize(_ws("house_only"), "stats", self._stats_payload())
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertEqual(set(inner["cameras"].keys()), {"front_door", "back_door"})
self.assertNotIn("garage", inner["cameras"])
# Aggregates, detectors, and service block must survive.
self.assertEqual(inner["camera_fps"], 15.0)
self.assertEqual(inner["detection_fps"], 6.0)
self.assertIn("detectors", inner)
self.assertIn("service", inner)
def test_stats_restricted_with_no_allowed_cameras_still_sends_aggregates(self):
"""A restricted role whose allow-list contains only nonexistent cameras
still gets the global aggregates and service block."""
config = _build_config(extra_roles={"empty_role": ["nonexistent"]})
from frigate.comms.ws import _parse_json_payload
payload = self._stats_payload()
all_cameras = set(config.cameras.keys())
scope = _classify_outbound("stats", all_cameras, _collect_zone_names(config))
full = json.dumps({"topic": "stats", "payload": payload})
message = _materialize_for_ws(
_ws("empty_role"),
"stats",
full,
scope,
_parse_json_payload(payload),
config,
)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertEqual(inner["cameras"], {})
self.assertEqual(inner["camera_fps"], 15.0)
self.assertIn("service", inner)
def test_stats_without_cameras_key_passes_through(self):
"""A malformed stats payload missing the cameras sub-dict shouldn't
break delivery for restricted users fall back to the full message."""
payload = json.dumps({"detectors": {}, "service": {}, "detection_fps": 0.0})
message = self._materialize(_ws("house_only"), "stats", payload)
envelope = json.loads(message) # type: ignore[arg-type]
self.assertEqual(envelope["payload"], payload)
def test_job_state_export_entry_unchanged_for_admin(self):
payload = json.dumps(
{
"export": {
"job_type": "export",
"status": "running",
"results": {
"jobs": [
{"job_type": "export", "camera": "garage", "id": "b"},
]
},
}
}
)
message = self._materialize(_ws("admin"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
self.assertEqual(envelope["payload"], payload)
def test_job_state_restricted_keeps_global_jobs(self):
"""media_sync has no camera field; restricted users still see it."""
payload = json.dumps(
{"media_sync": {"job_type": "media_sync", "status": "running"}}
)
message = self._materialize(_ws("house_only"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertIn("media_sync", inner)
def test_job_state_debug_replay_nested_source_camera_filtered(self):
"""debug_replay puts ``source_camera`` inside ``results`` (see
jobs/debug_replay.py:to_dict). Restricted users must not receive
entries whose nested source camera is unauthorized."""
payload = json.dumps(
{
"debug_replay": {
"id": "bd6dc99d-a7d",
"job_type": "debug_replay",
"status": "running",
"start_time": 1.0,
"end_time": None,
"error_message": None,
"results": {
"current_step": "preparing_clip",
"progress_percent": 0.0,
"source_camera": "garage",
"replay_camera_name": "_replay_garage",
"start_ts": 0.0,
"end_ts": 1.0,
},
}
}
)
self.assertIsNone(self._materialize(_ws("house_only"), "job_state", payload))
def test_job_state_debug_replay_nested_source_camera_allowed(self):
payload = json.dumps(
{
"debug_replay": {
"id": "bd6dc99d-a7d",
"job_type": "debug_replay",
"status": "running",
"results": {
"source_camera": "front_door",
"replay_camera_name": "_replay_front_door",
},
}
}
)
message = self._materialize(_ws("house_only"), "job_state", payload)
envelope = json.loads(message) # type: ignore[arg-type]
inner = json.loads(envelope["payload"])
self.assertIn("debug_replay", inner)
self.assertEqual(
inner["debug_replay"]["results"]["source_camera"], "front_door"
)
class _FakeManager:
"""Minimal ws4py manager: holds clients and exposes a lock."""
def __init__(self, clients: list[Any]) -> None:
self.lock = threading.Lock()
self.websockets = {id(c): c for c in clients}
class _FakeServer:
def __init__(self, manager: _FakeManager) -> None:
self.manager = manager
class _CapturingWs(SimpleNamespace):
"""Fake ws4py client that records what was sent."""
def __init__(self, role: str | None) -> None:
environ = {} if role is None else {"HTTP_REMOTE_ROLE": role}
super().__init__(environ=environ, terminated=False)
self.sent: list[str] = []
def send(self, message: str) -> None: # noqa: D401 - matches ws4py API
self.sent.append(message)
class TestPublishEndToEnd(unittest.TestCase):
"""Drive WebSocketClient.publish() against fake clients with different roles."""
def setUp(self):
self.config = _build_config(
extra_zones={"front_door": {"driveway": {"coordinates": "0,0,1,0,1,1,0,1"}}}
)
self.admin = _CapturingWs("admin")
self.restricted = _CapturingWs("house_only")
self.anon = _CapturingWs(None)
self.client = WebSocketClient(self.config)
self.client.websocket_server = _FakeServer(
_FakeManager([self.admin, self.restricted, self.anon])
)
def _payloads(self, ws: _CapturingWs) -> list[Any]:
return [json.loads(m)["payload"] for m in ws.sent]
def test_global_topic_reaches_everyone(self):
self.client.publish("model_state", "{}")
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 1)
self.assertEqual(len(self.anon.sent), 1)
def test_camera_topic_filters_restricted_recipient(self):
self.client.publish("garage/detect/state", "ON")
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 0)
self.assertEqual(len(self.anon.sent), 0)
def test_camera_topic_allows_restricted_recipient_for_allowed_camera(self):
self.client.publish("front_door/detect/state", "ON")
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 1)
self.assertEqual(len(self.anon.sent), 0)
def test_events_payload_filtered(self):
self.client.publish("events", json.dumps({"after": {"camera": "garage"}}))
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 0)
def test_camera_activity_reshaped_per_recipient(self):
self.client.publish(
"camera_activity",
json.dumps(
{
"front_door": {"objects": 1},
"back_door": {"objects": 0},
"garage": {"objects": 2},
}
),
)
self.assertEqual(len(self.admin.sent), 1)
admin_inner = json.loads(self._payloads(self.admin)[0])
self.assertEqual(set(admin_inner.keys()), {"front_door", "back_door", "garage"})
self.assertEqual(len(self.restricted.sent), 1)
restricted_inner = json.loads(self._payloads(self.restricted)[0])
self.assertEqual(set(restricted_inner.keys()), {"front_door", "back_door"})
self.assertEqual(len(self.anon.sent), 0)
def test_birdseye_layout_blocked_for_restricted_and_anon(self):
self.client.publish(
"birdseye_layout",
json.dumps({"front_door": {"x": 0, "y": 0, "width": 1, "height": 1}}),
)
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 0)
self.assertEqual(len(self.anon.sent), 0)
def test_zone_aggregate_blocked_for_restricted(self):
self.client.publish("driveway/person", 2)
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 0)
def test_stats_reshaped_per_recipient(self):
self.client.publish(
"stats",
json.dumps(
{
"cameras": {
"front_door": {"camera_fps": 5.0},
"garage": {"camera_fps": 5.0},
},
"service": {"uptime": 1},
"camera_fps": 10.0,
}
),
)
self.assertEqual(len(self.admin.sent), 1)
admin_inner = json.loads(self._payloads(self.admin)[0])
self.assertEqual(set(admin_inner["cameras"].keys()), {"front_door", "garage"})
self.assertEqual(len(self.restricted.sent), 1)
restricted_inner = json.loads(self._payloads(self.restricted)[0])
self.assertEqual(set(restricted_inner["cameras"].keys()), {"front_door"})
self.assertEqual(restricted_inner["camera_fps"], 10.0)
self.assertIn("service", restricted_inner)
# Stats requires a role; anonymous gets nothing.
self.assertEqual(len(self.anon.sent), 0)
def test_export_job_state_filters_results_jobs_per_recipient(self):
self.client.publish(
"job_state",
json.dumps(
{
"export": {
"job_type": "export",
"status": "running",
"results": {
"jobs": [
{"camera": "front_door", "id": "a"},
{"camera": "garage", "id": "b"},
]
},
}
}
),
)
self.assertEqual(len(self.admin.sent), 1)
admin_inner = json.loads(self._payloads(self.admin)[0])
self.assertEqual(
[j["camera"] for j in admin_inner["export"]["results"]["jobs"]],
["front_door", "garage"],
)
self.assertEqual(len(self.restricted.sent), 1)
restricted_inner = json.loads(self._payloads(self.restricted)[0])
self.assertEqual(
[j["camera"] for j in restricted_inner["export"]["results"]["jobs"]],
["front_door"],
)
def test_unknown_topic_dropped_for_everyone(self):
self.client.publish("some_rogue_topic", "data")
self.assertEqual(self.admin.sent, [])
self.assertEqual(self.restricted.sent, [])
self.assertEqual(self.anon.sent, [])
def test_terminated_client_is_skipped(self):
self.restricted.terminated = True
self.client.publish("front_door/detect/state", "ON")
self.assertEqual(len(self.admin.sent), 1)
self.assertEqual(len(self.restricted.sent), 0)
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More