Compare commits

..
32 Commits
Author SHA1 Message Date
Josh Hawkins 3524d53acb i18n tweaks 2026-08-21 19:16:52 -05:00
Josh Hawkins bb84f2d009 use yml as default config file extension when not found 2026-08-21 19:16:52 -05:00
Josh Hawkins 46c23c4064 add light/dark mode icon switcher 2026-08-21 19:16:52 -05:00
Josh Hawkins 9b65fdf13a clean up 2026-08-21 19:16:52 -05:00
Josh Hawkins 0a5d9e5126 resolve hwaccel per camera and clarify recording retention
The hwaccel step listed every preset Frigate ships, so an Intel box was offered Raspberry Pi and Rockchip decoding, and the codec specific presets (`preset-intel-qsv-h264` vs `-h265`) were offered as global values that break as soon as two cameras use different codecs. `/hardware/hwaccel` now returns the decoding families the probed hardware can actually use, each carrying a preset per codec, and the wizard resolves the family against the detect stream codec the camera wizard already probed: one global `ffmpeg.hwaccel_args` when every camera agrees, per-camera `cameras.<name>.ffmpeg.hwaccel_args` when they don't. The global stays on `auto` in that case so cameras added later still resolve at startup. A gen13+ Intel machine keeps its QuickSync recommendation with mixed h264 and h265 cameras instead of dropping to vaapi.

The recording step's "Days to retain recordings" only wrote alert and detection retention, and the storage estimate under it assumed continuous recording. It now asks what to record in plain language, writes `record.continuous.days` to match, shows the estimate only for continuous, and drops the spinner arrows on the number input.
2026-08-21 19:16:52 -05:00
Josh Hawkins cfda6baa06 add onboarding wizard for new users 2026-08-21 19:15:08 -05:00
Josh HawkinsandGitHub df5f943e33 fix clip download deadlock from unread ffmpeg stderr (#24032)
ffmpeg's stderr was piped but never read, so recording segments that generate more than 64 KB of ffmpeg warnings blocked ffmpeg mid-write, stranding the streaming thread and its anyio threadpool token for good. Enough of those and every sync endpoint stops responding until restart. The trigger is how noisy the segments are, not how long the clip is.

Send stderr to a temp file instead, and guarantee ffmpeg teardown and playlist cleanup on every exit path, including client disconnect.

Also fixes two bugs the deadlock hid: the failure branch was unreachable because returncode is None mid-loop, so the playlist file leaked and ffmpeg's logs were never reported. Playlist files now get a unique name so concurrent requests for one range cannot delete each other's input.

Extracts the terminate helper motion search already had into frigate/util/ffmpeg.py, now shared by both streaming call sites.
2026-08-19 07:17:49 -06:00
Josh Hawkins bcf81e7c65 fix the model lookup KeyError for cameras added at runtime (#24026) 2026-08-18 16:25:18 -05:00
Josh Hawkins 90e56b9137 Add import/export for camera group layouts and per-camera streaming settings (#24025)
* add import/export for camera group layouts and streaming settings

Camera group layouts and per-camera streaming settings are stored in the browser's IndexedDB, so they are tied to a single browser on a single device. Users with more than one device have to rebuild every group layout and re-pick every camera's stream settings by hand, and clearing browser data loses the work.

Add a Backup & Restore card to Settings > UI Settings that exports these settings to a JSON file and imports that file on another device. Import shows a confirmation dialog with per-section counts, switches for layouts, streaming settings, and UI preferences, and warnings about camera groups or cameras in the file that are not on this server.

Server-side storage is deliberately avoided. These are per-device presentation settings: a layout arranged for a desktop is wrong on a tablet, and continuous full-resolution streams that are free on a wired LAN are not on a phone. An explicit file moves settings only when the user chooses to move them.

Implementation notes:

- web/src/utils/uiSettingsTransfer.ts owns a registry of transferable IndexedDB keys. Each entry records whether the key is user-namespaced, matching which persistence hook wrote it, plus a zod schema for its value.
- Only registry-known keys are ever written, and only when their value passes that schema. The file format deliberately lets unknown keys survive parsing, so this filter is what prevents a hand-edited file from writing arbitrary storage keys or out-of-range values.
- Export falls back to the legacy un-namespaced key, because the username migration runs lazily on first mount of each owning hook.
- Streaming settings merge per group rather than replacing the whole map, so groups configured only on the receiving device survive.
- Import writes storage and then reloads, because useUserPersistence reads a key only on mount and StreamingSettingsProvider would otherwise write its stale in-memory state back over the import.
- playbackBandwidthEstimate, frigate-search-history, and live-layout are excluded: the first two are measurements and user data rather than preferences, and live-layout's default is derived from the device.

* merge imported streaming settings per camera instead of per group
2026-08-18 16:25:18 -05:00
Nicolas MowenandJosh Hawkins 4fb13b72ef Implement UI for managing multiple models (#24023)
* Implement hardware detection and UI management

* Cleanup Frigate+ detection

* Don't count model as changed

* Fixes for audio map error

* Add descriptions

* Enforce that all model must exist

* Fix hardware picking

* Docs fixes

* WebUI cleanup

* Cleanup handling of scenes

* UI refinement

* Cleanup recommended UI

* test fixews
2026-08-18 16:25:18 -05:00
Josh Hawkins 5aee6861a8 Base emergency cleanup on the streams a camera is currently recording (#24022)
* gate emergency cleanup bandwidth on the streams a camera currently records

* settle bandwidth samples per stream instead of per camera

* fix mypy
2026-08-18 16:25:18 -05:00
Nicolas MowenandJosh Hawkins 591464e82f Refactor detector and model management (#23995)
* Refactor detector and model management

* Fix model resolution field
2026-08-18 16:25:18 -05:00
Ersa Oktavian RamadanandJosh Hawkins 13a7d8ecae Add audio labelmap grouping (#24004)
Allow audio classes to be grouped under a shared configured label.

Keep audio overrides separate from object labels and retain only the highest-scoring grouped detection.

Refs #23967
2026-08-18 16:25:18 -05:00
Josh Hawkins ac73ad210e Show main and sub stream usage separately in Storage Metrics (#24015)
* backend

* frontend

* docs

* test

* report null instead of 0 for a stream with no cached bandwidth sample
2026-08-18 16:25:18 -05:00
Josh Hawkins c6ec533430 Refactor MQTT (#24010)
* refactor mqtt so that Frigate owns the transport lifecycle instead of delegating it to paho

* release the shutdown barrier on worker crash and replay retained publishes the broker never acked

* collapse in-flight retained values by topic and release the shutdown barrier from a finally

* replay the outage buffer before the publish queue so newer values are not reverted
2026-08-18 16:25:18 -05:00
Josh Hawkins dcf711d5a0 Refactor birdseye activity modes as a list and add alerts/detections (#24012)
* backend

* tests

* frontend and i18n

* e2e test schema

* docs
2026-08-18 16:25:18 -05:00
Josh Hawkins 83750a9596 Improve History's seek startup time and recordings query performance (#24011)
* serve a segment startup ladder so seeks begin playing sooner

nginx-vod was handed one 10s segment per recording file, so every playlist start had to download and decode a full segment before the first frame. Declare real keyframe data per clip and let nginx cut short leading segments from it.

- add vod_bootstrap_segment_durations 1000/2000/4000 so each playlist starts with 1s/2s/4s segments before settling at 10s
- emit real clip-relative keyFrameDurations (plus firstKeyFrameOffset when nonzero) from the recording keyframe index; rows without an index keep the whole-clip declaration, the only safe cut without keyframe knowledge
- drop the manifest's segment_duration field, which was always inert: nginx-vod parses only camelCase segmentDuration
- rebuild the player source at the seek target, quantized to a 10s grid, so the ladder applies to every seek and seek URLs stay repeatable for nginx's mapping and response caches
- route the seek model, in-range checks, and the stale-report guard through the source window rather than the chunk range
- bridge repositioning seeks (>2s from the last played timestamp) through the preview player and hold the release anchor one commit, so neither path paints a stale frame
- clear a pending loading timer before replacing it; an orphaned timer escaped onPlaying's clearTimeout and flashed loading mid-playback

* keep recordings queries on their indexes

Several recordings queries degraded into full scans or large sorts on big databases: the planner ignored index order, or the query shape gave it nothing tight to seek on. Reshape them into bounded seeks and add the composite index the per-stream lookups need.

- index recordings on (camera, stream_type, start_time DESC) and drop the (camera, stream_type) index it supersedes
- walk the recordings summary day by day with EXISTS probes and per-camera MIN/MAX seeks, skipping ahead over empty gaps instead of bucketing every row for the requested cameras
- run the summary endpoint on the event loop rather than the threadpool
- bound the unavailable-recordings query by start_time per camera and merge the results in Python
- bound the expire query's start_time so it seeks the retention window instead of scanning a camera's whole history
- enumerate deleted cameras with one index seek each rather than a camera NOT IN (...) scan
- compute bandwidth with segment_size filtered in a CASE projection; as a WHERE predicate it baited the planner into the (camera, segment_size) index plus a full sort of the camera's history
- fall back to a 1000-segment window when the recent 100 are all zero-size, so an ingest glitch doesn't report zero bandwidth
- limit the needs_refresh count instead of counting every segment
- cover sub-only and sparse calendar days, midnight-spanning day attribution, multi-camera gap merging, deleted-camera expiry, and zero-size segment runs

* fix mypy
2026-08-18 16:25:18 -05:00
Josh Hawkins 83fbce1533 Enable PTZ control setup in the Add Camera Wizard (#23444)
* add ptz controls to camera via wizard when onvif has already been probed

* i18n

* add e2e test

* backend add and remove subscriber

* tweaks

* turn on switch by default if pan and/or tilt capability is available

* fix test
2026-08-18 16:25:18 -05:00
Josh Hawkins 19fdc11ab3 Add sub stream recording with adaptive quality playback (#24009)
* add sub stream recording with adaptive quality playback

Optionally record a second, lower bitrate stream alongside the main
recording stream via a `record_sub` input role and `record.sub` config block, with its own retention windows.
Recordings rows now carry the stream type plus the media details needed to serve both streams from one manifest: video codec, audio presence, audio codec and rate, and a record-time keyframe index.

Playback resolves coverage across both streams and merges them into a single VOD sequence, falling back to a discontinuity manifest with per-clip init segments when the media signatures differ. The player exposes a quality selector, and an auto governor picks the stream from stall time, bandwidth, codec support, and the save-data hint.

* fix tests and i18n
2026-08-18 16:25:18 -05:00
Josh Hawkins a85304e26e stop creating a config subscriber per capture thread (#24002) 2026-08-18 16:25:18 -05:00
Josh Hawkins 4bfd89fd90 Guard lookups when adding/deleting cameras at runtime (#23994)
* Guard object processor queue handlers against unknown cameras

* Skip embeddings post processing for removed cameras

* End review segments for removed cameras

* Drop queued autotracker moves for removed cameras

* Release tracked event thumbnails when skipping a removed camera

* Add locked accessors for camera states

* Read camera states through the processor accessors

* Guard output and recording paths against cameras not yet known

* Resolve camera state once in ONVIF, notification, and transcription paths
2026-08-18 16:25:18 -05:00
Ersa Oktavian RamadanandJosh Hawkins 811887c70d Refactor Birdseye activity types as composable booleans (#23940)
* Add combined motion and object Birdseye mode

Add a motion_objects mode that keeps Birdseye active when motion is detected or a confirmed tracked object is present, including stationary objects.

Wire the mode through configuration, runtime commands, API schemas, documentation, and UI labels. Exclude false-positive trackers and add regression coverage for Birdseye activation and MQTT validation.

* Refactor Birdseye activity types as booleans

Replace combination-specific Birdseye modes with composable boolean activity types for motion, active objects, stationary objects, and continuous display.

Preserve legacy single-mode configuration and MQTT inputs, support canonical comma-separated MQTT combinations, and allow scalar YAML values to be replaced by nested settings through the config API.

* Preserve OpenVINO config translations

Regenerate the configuration translations with the OpenVINO detector schema available so the unrelated production detector labels remain intact.

* Preserve partial Birdseye mode overrides

Allow an empty activity selection with a canonical NONE MQTT state so partial camera and profile overrides can disable inherited flags without failing validation.

Add regression coverage for camera and profile inheritance, document the NONE contract, and keep the generated schema fixture scoped to Birdseye.

* Address Birdseye activity review feedback

Move scalar mode compatibility into the 0.18-1 config migration and reject empty activity selections instead of publishing a NONE state.

Pass activity signals through a frozen dataclass, preserve existing active-object tracker behavior, and require confirmed stationary objects. Revert the generic YAML mutation and cover migration, inheritance, MQTT, and activation regressions.

* Move Birdseye migration to 0.19

Use the 0.19-0 configuration revision for converting scalar Birdseye modes to composable activity flags, and update the migration regression coverage accordingly.

* Remove Birdseye migration test

Drop the dedicated config migration test as requested during review while retaining the 0.19-0 migration implementation.
2026-08-18 16:25:18 -05:00
Josh Hawkins e6812f9282 Fix birdseye layout overlap with mixed landscape/portrait cameras (#22917)
* fix birdseye layout calculation

replace the two pass layout with a single pass pixel space algorithm

* add test
2026-08-18 16:25:18 -05:00
Nicolas MowenandJosh Hawkins 923a0c8310 Don't require object type for parameter in categorized names tool 2026-08-18 16:25:18 -05:00
13a1711adf Dynamically resolve Intel NPU (#23761)
* Add support for newer Intel NPU busy time counter

* Resolve Intel NPU device dynamically

---------

Co-authored-by: Filious Louis <1417132+fjlouis@users.noreply.github.com>
2026-08-18 16:25:18 -05:00
DoFabienandJosh Hawkins 6ca5b59281 Improve recording timeline and VOD query performance (#23862)
* Improve recording timeline and VOD query performance

* Add recording query boundary tests
2026-08-18 16:25:18 -05:00
Nicolas MowenandJosh Hawkins 7f929cb0dc GenAI Chat Prompt Refinements (#23864)
* Prompt refactoring and optimization

* Update spec
2026-08-18 16:25:18 -05:00
Nicolas MowenandJosh Hawkins d99ce0a9ed Update to 0.19 2026-08-18 16:25:18 -05:00
Josh HawkinsandGitHub 036bae4ea9 Return a specific 404 when starting a debug replay with no recordings in range (#24024)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
2026-08-18 10:08:52 -05:00
dtigheandGitHub 8384a8c5b3 Fix Gemini tool calling on 3.6+ by using documented function response role (#24013)
Gemini 3.6 and newer reject role="function" on the function response
Content with 400 INVALID_ARGUMENT, breaking any chat query that triggers
a tool call. The tool call itself succeeds; only the hand-back to the
model fails, and because the error surfaces mid-stream the request still
returns HTTP 200, so it is easy to miss.

Google's function calling documentation specifies role="user" for
returning function results:

    contents.append(response.candidates[0].content)
    contents.append(types.Content(role="user", parts=[function_response_part]))

https://ai.google.dev/gemini-api/docs/generate-content/function-calling

Verified with my local setup.
2026-08-18 08:16:47 -06:00
Josh HawkinsandGitHub 77fc2ce174 Miscellaneous fixes (0.18 beta) (#24016)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* fix classification drawer closing instead of scrolling when list is long on mobile

* add qwen3.8 to genai docs

* add titles to more clearly separate model types
2026-08-18 07:01:22 -06:00
Josh HawkinsandGitHub 8425a76558 Miscellaneous fixes (0.18 beta) (#23993)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* subscribe to add in webpush

* add docs for detector cpu usage

* rebuild notification camera access when a camera is added at runtime

* document how frigate shows CPU usage metrics

* add faq about version key in config
2026-08-16 12:39:28 -06:00
17 changed files with 466 additions and 58 deletions
+9 -5
View File
@@ -59,13 +59,17 @@ Running Generative AI models on CPU is not recommended, as high inference times
### Recommended Local Models
#### Vision models
You must use a vision-capable model with Frigate. The following models are recommended for local deployment of the `descriptions` and `chat` roles:
| Model | Notes |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. |
| `qwen3.6` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. |
| Model | Notes |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. |
| `qwen3.6`/`qwen3.8` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. |
#### Embedding models
The `embeddings` role needs a different kind of model. Text queries are matched against the stored image embeddings, so the model must be trained to place images and text into the same vector space. A chat or description model will still return vectors when asked, but those vectors are not trained for retrieval and text searches will return poor matches with no error to indicate why.
+41 -1
View File
@@ -3,7 +3,31 @@ id: cpu
title: High CPU Usage
---
High CPU usage can impact Frigate's performance and responsiveness. This guide outlines the most effective configuration changes to help reduce CPU consumption and optimize resource usage.
High CPU usage can impact Frigate's performance and responsiveness. This guide explains how to interpret the CPU values Frigate reports and outlines the most effective configuration changes to help reduce CPU consumption and optimize resource usage.
## Understanding Frigate's Reported CPU Usage
Frigate's CPU percentages often look much higher than what the host reports. Usually both numbers are correct and are simply measured against different denominators, so confirm you actually have a problem before tuning anything.
### Per-process values are relative to a single core
The values Frigate reports for FFmpeg, capture, detect, detector, and other processes follow the same convention as `top`: 100% means one CPU core is fully saturated, not that the whole system is saturated. A multithreaded process such as FFmpeg can legitimately report well over 100%.
Host and hypervisor tools instead report a percentage of the machine's total capacity across all cores. This includes `docker stats`, the `htop` summary, the Proxmox summary graph, the Unraid dashboard, Synology Resource Monitor, and Home Assistant's system monitor sensors. To reconcile the two:
```
host percentage ≈ (sum of Frigate's process percentages) / (number of cores)
```
On a 4 core system, an FFmpeg process reporting 100% is consuming one quarter of the machine, so the host will show roughly 25 to 30% once the remaining Frigate processes are included. That same 100% on a 16 core system is about 6%. Frigate's own warning thresholds use the per-core convention as well, so an FFmpeg process is flagged at 20% of a single core, not 20% of the system.
### Instantaneous samples and averages measure different things
Frigate collects stats every 15 seconds, and the `cpu` value covers only the interval since the previous collection. The `cpu_average` value in the stats API and MQTT payload is the average across the entire life of the process, and it is what the high CPU usage warnings are based on. Host dashboards generally plot data averaged over a longer window, so a single Frigate sample can show a peak that a host graph never displays. A process that has just started, such as FFmpeg after a camera reconnect, reports 0 until it has been sampled twice.
### The system-wide value depends on what the container can see
The system CPU value is read from `/proc/stat`. Under Docker that file belongs to the host, so the value covers the entire machine including workloads unrelated to Frigate, and it will not match `docker stats` for the Frigate container. Under an LXC container, lxcfs virtualizes `/proc/stat` and the value reflects only the cores assigned to the container. In a virtual machine, the guest sees only its assigned vCPUs while the hypervisor divides by every physical thread on the node, so guest and host percentages will not agree even when both are accurate.
## 1. Hardware Acceleration for Video Decoding
@@ -72,3 +96,19 @@ The model you use significantly impacts detector performance. Frigate provides d
- Larger models (640x640): Slower inference, can sometimes have higher accuracy on very large objects that take up a majority of the frame.
For more detail on picking the right size, see [Choosing a model size](../configuration/object_detectors.md#choosing-a-model-size).
## 3. Reducing Detector CPU Usage
**Priority: High**
The **Detector CPU Usage** metric measures the CPU spent converting frames into the tensor format the model expects and post-processing the model's output. It does not include inference, so this value can be high even when you've configured a GPU, NPU, or Coral for object detection.
This metric scales with how many detections per second Frigate runs and how expensive each one is to prepare. Tuning [motion detection](../configuration/motion_detection) is usually the first recommendation to reduce the number of detections. Additionally, you can:
- **Lower `detect -> fps`.** 5 is the recommended value for nearly all cameras. Running at 10 doubles the frames eligible for detection and is one of the largest contributors to this metric.
- **Use a 320x320 model.** A 640x640 model has 4 times as many pixels to transpose, convert, and copy on every inference.
- **Prefer a model that takes integer input.** Models configured with `input_dtype: float` require each frame to be converted to float32 and normalized on the CPU first. Models taking `int` input, such as the tflite models used by the Edge TPU, skip that step.
- **Do not match the detect resolution to the model resolution.** The detect stream should match your camera's aspect ratio, for example `1280x720`, not the model's input size. Frigate crops and scales regions of motion itself, so an oversized detect stream only adds work.
- **Tune stationary object behavior.** Objects that never settle into a stationary state are re-detected continuously. Raising `detect -> stationary -> interval` reduces how often detection runs on objects that are already parked. See [stationary objects](../configuration/stationary_objects).
Adding [more detector instances](#multiple-detector-instances) spreads this work across more CPU cores, but does not reduce the total CPU used.
+6
View File
@@ -133,6 +133,12 @@ cameras:
height: 720
```
### What is the `version` key in my config file?
`version` records the config format that your config was last migrated to. On startup Frigate compares it against the format the running version expects, and if it is older it copies your config to `/config/backup_config.yaml`, rewrites it to the new format, and updates `version` as the final step. A config with no `version` key is assumed to predate 0.14 and is migrated from there.
Frigate manages this key for you, so do not set or edit it. Raising it makes Frigate skip migrations your config still needs, and lowering it re-runs migrations against config that has already been converted. Either can leave you with a config that no longer validates.
### Why does Frigate keep creating new tracked objects for my parked car?
Stationary tracking is designed to _prevent_ this: a parked car should remain a single tracked object rather than generating new ones. If you're repeatedly getting new tracked objects for the same car, it's likely that Frigate is losing the object and re-detecting it as a new one.
+92 -1
View File
@@ -4053,6 +4053,58 @@ paths:
security:
- frigateAdminAuth: []
x-required-role: admin
/hardware/hwaccel:
get:
tags:
- Hardware
summary: Hwaccel Recommendation
description: |-
**Access:** Admin role required.
Get the hardware decoding this system can do.
Args:
detector: Hardware key of the detection hardware in use, which biases
the recommendation toward that hardware's GPU
codecs: Comma separated codecs of the streams that will be decoded,
used to drop families that cannot decode one of them
Returns:
The recommended family (empty when none fits) and every usable family
operationId: hwaccel_recommendation_hardware_hwaccel_get
parameters:
- name: detector
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
title: Detector
- name: codecs
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
title: Codecs
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/HwaccelRecommendation'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- frigateAdminAuth: []
x-required-role: admin
/events:
get:
tags:
@@ -7306,7 +7358,9 @@ paths:
schema:
$ref: '#/components/schemas/DebugReplayStartResponse'
'400':
description: Invalid camera, time range, or no recordings
description: Invalid camera or time range
'404':
description: No recordings in the requested time range
'409':
description: A replay session is already active
'422':
@@ -8668,6 +8722,43 @@ components:
- label
title: HardwareUnit
description: One physical piece of hardware.
HwaccelFamily:
properties:
key:
type: string
title: Family key
description: Stable identifier for this kind of hardware decoding.
presets:
additionalProperties:
type: string
type: object
title: Presets
description: The ffmpeg preset for each codec this family decodes, or
a single 'any' preset when it decodes every codec.
type: object
required:
- key
- presets
title: HwaccelFamily
description: A kind of hardware decoding, and the presets that drive it.
HwaccelRecommendation:
properties:
recommended:
type: string
title: Recommended family
description: Key of the family that fits this system best, or an empty
string when none does.
available:
items:
$ref: '#/components/schemas/HwaccelFamily'
type: array
title: Available families
description: Every family this system's hardware can use, best first.
type: object
required:
- recommended
title: HwaccelRecommendation
description: The hardware decoding this system can do.
Last24HoursReview:
properties:
reviewed_alert:
+11 -1
View File
@@ -13,6 +13,7 @@ from frigate.api.auth import require_role
from frigate.api.defs.tags import Tags
from frigate.jobs.debug_replay import (
ExportDebugReplaySource,
NoRecordingsError,
RecordingDebugReplaySource,
start_debug_replay_job,
)
@@ -74,7 +75,8 @@ class DebugReplayStopResponse(BaseModel):
response_model=DebugReplayStartResponse,
status_code=202,
responses={
400: {"description": "Invalid camera, time range, or no recordings"},
400: {"description": "Invalid camera or time range"},
404: {"description": "No recordings in the requested time range"},
409: {"description": "A replay session is already active"},
},
dependencies=[Depends(require_role(["admin"]))],
@@ -113,6 +115,14 @@ async def start_debug_replay(request: Request, body: DebugReplayStartBody):
},
status_code=409,
)
except NoRecordingsError:
return JSONResponse(
content={
"success": False,
"message": "No recordings found in the selected time range",
},
status_code=404,
)
except ValueError:
logger.exception("Rejected debug replay start request")
return JSONResponse(
+61 -23
View File
@@ -6,11 +6,13 @@ import logging
import math
import os
import subprocess as sp
import tempfile
import time
from collections.abc import Iterator
from datetime import UTC, datetime, timedelta
from enum import Enum
from pathlib import Path as FilePath
from typing import Any
from typing import IO, Any
from urllib.parse import unquote
import cv2
@@ -47,6 +49,7 @@ from frigate.const import (
from frigate.models import Event, Previews, Recordings, Regions, ReviewSegment
from frigate.output.preview import get_most_recent_preview_frame
from frigate.track.object_processing import TrackedObjectProcessor
from frigate.util.ffmpeg import terminate_ffmpeg_stream
from frigate.util.file import (
get_event_snapshot_bytes,
get_event_snapshot_path,
@@ -70,6 +73,12 @@ logger = logging.getLogger(__name__)
# normal hour needs ~360, one clip per recording file
NGINX_VOD_MAX_CLIPS = 1080
# tail of ffmpeg's stderr kept for the clip download failure log
CLIP_STDERR_LOG_BYTES = 8192
# how long a drained clip download waits for ffmpeg to exit on its own
CLIP_FFMPEG_EXIT_TIMEOUT = 10
class VodStreamPreference(str, Enum):
"""Stream pin for the path-segment VOD route.
@@ -465,6 +474,53 @@ async def submit_recording_snapshot_to_plus(
)
def _read_stderr_tail(stderr_file: IO[bytes]) -> str:
"""Read back the last CLIP_STDERR_LOG_BYTES of a captured stderr file."""
stderr_file.seek(0, os.SEEK_END)
stderr_file.seek(max(0, stderr_file.tell() - CLIP_STDERR_LOG_BYTES))
return stderr_file.read().decode("utf-8", "replace")
def _run_clip_download(ffmpeg_cmd: list[str], file_path: str) -> Iterator[bytes]:
"""Stream an ffmpeg concat remux to the client, always cleaning up after it."""
stderr_file = None
ffmpeg = None
try:
stderr_file = tempfile.TemporaryFile()
ffmpeg = sp.Popen(ffmpeg_cmd, stdout=sp.PIPE, stderr=stderr_file)
while True:
data = ffmpeg.stdout.read(8192)
if not data:
break
yield data
try:
# wait rather than signal, so the real exit code survives
ffmpeg.wait(timeout=CLIP_FFMPEG_EXIT_TIMEOUT)
except sp.TimeoutExpired:
pass
finally:
if ffmpeg is not None:
# read before terminating: a None here is our teardown, not a failure
exit_code = ffmpeg.poll()
terminate_ffmpeg_stream(ffmpeg)
if exit_code:
logger.error(
"Failed to generate clip, ffmpeg logs: %s",
_read_stderr_tail(stderr_file),
)
if stderr_file is not None:
stderr_file.close()
FilePath(file_path).unlink(missing_ok=True)
@router.get(
"/{camera_name}/start/{start_ts}/end/{end_ts}/clip.mp4",
dependencies=[Depends(require_camera_access)],
@@ -476,26 +532,6 @@ async def recording_clip(
start_ts: float,
end_ts: float,
):
def run_download(ffmpeg_cmd: list[str], file_path: str):
with sp.Popen(
ffmpeg_cmd,
stderr=sp.PIPE,
stdout=sp.PIPE,
text=False,
) as ffmpeg:
while True:
data = ffmpeg.stdout.read(8192)
if data is not None and len(data) > 0:
yield data
else:
if ffmpeg.returncode and ffmpeg.returncode != 0:
logger.error(
f"Failed to generate clip, ffmpeg logs: {ffmpeg.stderr.read()}"
)
else:
FilePath(file_path).unlink(missing_ok=True)
break
def get_clip_query(stream_type: str):
return (
Recordings.select(
@@ -529,7 +565,9 @@ async def recording_clip(
status_code=400,
)
file_name = sanitize_filename(f"playlist_{camera_name}_{start_ts}-{end_ts}.txt")
file_name = sanitize_filename(
f"playlist_{camera_name}_{start_ts}-{end_ts}_{os.urandom(4).hex()}.txt"
)
file_path = os.path.join(CACHE_DIR, file_name)
with open(file_path, "w") as file:
clip: Recordings
@@ -577,7 +615,7 @@ async def recording_clip(
]
return StreamingResponse(
run_download(ffmpeg_cmd, file_path),
_run_clip_download(ffmpeg_cmd, file_path),
media_type="video/mp4",
)
+5 -1
View File
@@ -83,7 +83,9 @@ class WebPushClient(Communicator):
# notification and auth config updater
self.global_config_subscriber = ConfigSubscriber("config/")
self.config_subscriber = CameraConfigUpdateSubscriber(
self.config, self.config.cameras, [CameraConfigUpdateEnum.notifications]
self.config,
self.config.cameras,
[CameraConfigUpdateEnum.add, CameraConfigUpdateEnum.notifications],
)
self._refresh_user_cameras()
@@ -217,6 +219,8 @@ class WebPushClient(Communicator):
self.suspended_cameras[camera] = 0
self.last_camera_notification_time[camera] = 0
self._refresh_user_cameras()
if topic == "reviews":
decoded = json.loads(payload)
camera = decoded["before"]["camera"]
+2 -2
View File
@@ -245,7 +245,7 @@ class GeminiClient(GenAIClient):
)
gemini_messages.append(
types.Content(
role="function",
role="user",
parts=[
types.Part.from_function_response(
name=msg.get("name")
@@ -501,7 +501,7 @@ class GeminiClient(GenAIClient):
)
gemini_messages.append(
types.Content(
role="function",
role="user",
parts=[
types.Part.from_function_response(
name=msg.get("name")
+5 -1
View File
@@ -115,6 +115,10 @@ def query_recordings(source_camera: str, start_ts: float, end_ts: float) -> Mode
return cast(ModelSelect, query)
class NoRecordingsError(ValueError):
"""Raised when no recordings exist in the requested time range."""
class DebugReplaySource(ABC):
"""Abstract source for a debug replay session.
@@ -187,7 +191,7 @@ class RecordingDebugReplaySource(DebugReplaySource):
raise ValueError("End time must be after start time")
if not query_recordings(self._camera, self._start_ts, self._end_ts).count():
raise ValueError(
raise NoRecordingsError(
f"No recordings found for camera '{self._camera}' in the specified time range"
)
+2 -20
View File
@@ -17,6 +17,7 @@ import numpy as np
from frigate.config import CameraConfig
from frigate.ffmpeg_presets import parse_preset_hardware_acceleration_decode
from frigate.util.ffmpeg import terminate_ffmpeg_stream
from frigate.util.services import auto_detect_hwaccel
logger = logging.getLogger(__name__)
@@ -88,25 +89,6 @@ def _read_exact(stream: IO[bytes], size: int) -> bytes | None:
return bytes(buf)
def _terminate(proc: sp.Popen[bytes]) -> None:
"""Stop an ffmpeg decode process promptly."""
# Close the read end first so a blocked ffmpeg write unblocks (ffmpeg then
# sees a broken pipe), then signal it. The resulting ffmpeg write error is
# harmless and goes to the captured stderr.
if proc.stdout is not None:
try:
proc.stdout.close()
except OSError:
pass
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except sp.TimeoutExpired:
proc.kill()
proc.wait()
KEYFRAME_MAX_GAP_SECONDS = 2.0
@@ -222,7 +204,7 @@ def _run_vod_decode(
count += 1
yield frame
finally:
_terminate(proc)
terminate_ffmpeg_stream(proc)
stderr_file.close()
if count == 0 and software_retry and not should_stop():
@@ -2,6 +2,7 @@
from unittest.mock import patch
from frigate.jobs.debug_replay import NoRecordingsError
from frigate.models import Event, Recordings, ReviewSegment
from frigate.test.http_api.base_http_test import AuthTestClient, BaseTestHttp
@@ -66,6 +67,32 @@ class TestDebugReplayAPI(BaseTestHttp):
# (CodeQL: information exposure through an exception).
self.assertEqual(body["message"], "Invalid debug replay parameters")
def test_start_returns_404_when_no_recordings(self):
with patch(
"frigate.api.debug_replay.start_debug_replay_job",
side_effect=NoRecordingsError(
"No recordings found for camera 'front' in the specified time range"
),
):
with AuthTestClient(self.app) as client:
resp = client.post(
"/debug_replay/start",
json={
"camera": "front",
"start_time": 100,
"end_time": 200,
},
)
self.assertEqual(resp.status_code, 404)
body = resp.json()
self.assertFalse(body["success"])
# Message is hard-coded so we don't echo exception text back to clients
# (CodeQL: information exposure through an exception).
self.assertEqual(
body["message"], "No recordings found in the selected time range"
)
def test_start_returns_409_when_session_already_active(self):
with patch(
"frigate.api.debug_replay.start_debug_replay_job",
+171
View File
@@ -0,0 +1,171 @@
"""Tests for the recording clip download stream."""
import os
import subprocess as sp
import sys
import tempfile
import threading
import unittest
from unittest.mock import patch
from frigate.api.media import _run_clip_download
# more than the 64 KB a pipe holds, so an undrained stderr blocks ffmpeg
STDERR_FLOOD_BYTES = 256 * 1024
PAYLOAD = b"0123456789" * 512
def fake_ffmpeg(*statements: str) -> list[str]:
"""Build an argv that stands in for ffmpeg, running the given statements."""
return [sys.executable, "-c", "\n".join(("import sys, time", *statements))]
class TestRunClipDownload(unittest.TestCase):
def setUp(self):
handle, self.playlist_path = tempfile.mkstemp(suffix=".txt")
os.close(handle)
def tearDown(self):
if os.path.exists(self.playlist_path):
os.unlink(self.playlist_path)
def collect(self, ffmpeg_cmd: list[str], timeout: float = 30.0) -> bytes:
"""Drain the generator on a worker thread so a deadlock fails the test."""
chunks: list[bytes] = []
errors: list[BaseException] = []
def drain() -> None:
try:
chunks.extend(_run_clip_download(ffmpeg_cmd, self.playlist_path))
except BaseException as err:
errors.append(err)
thread = threading.Thread(target=drain, daemon=True)
thread.start()
thread.join(timeout)
self.assertFalse(
thread.is_alive(), "clip download did not finish, ffmpeg is deadlocked"
)
if errors:
raise errors[0]
return b"".join(chunks)
def test_streams_full_clip_when_ffmpeg_floods_stderr(self):
"""A warning flood past the pipe buffer must not stall the download."""
data = self.collect(
fake_ffmpeg(
f"sys.stderr.write('w' * {STDERR_FLOOD_BYTES})",
"sys.stderr.flush()",
f"sys.stdout.buffer.write({PAYLOAD!r})",
)
)
self.assertEqual(data, PAYLOAD)
self.assertFalse(os.path.exists(self.playlist_path))
def test_streams_clip_written_before_stderr_flood(self):
data = self.collect(
fake_ffmpeg(
f"sys.stdout.buffer.write({PAYLOAD!r})",
"sys.stdout.flush()",
f"sys.stderr.write('w' * {STDERR_FLOOD_BYTES})",
)
)
self.assertEqual(data, PAYLOAD)
def test_logs_ffmpeg_output_and_removes_playlist_on_failure(self):
with patch("frigate.api.media.logger") as logger:
data = self.collect(
fake_ffmpeg(
"sys.stderr.write('something went wrong')",
"sys.exit(1)",
)
)
self.assertEqual(data, b"")
logger.error.assert_called_once()
self.assertIn("something went wrong", logger.error.call_args.args[1])
self.assertFalse(os.path.exists(self.playlist_path))
def test_logs_only_the_tail_of_a_flooded_stderr(self):
with patch("frigate.api.media.logger") as logger:
self.collect(
fake_ffmpeg(
f"sys.stderr.write('w' * {STDERR_FLOOD_BYTES})",
"sys.exit(1)",
)
)
logged = logger.error.call_args.args[1]
self.assertLess(len(logged), STDERR_FLOOD_BYTES)
def test_does_not_log_a_successful_download(self):
with patch("frigate.api.media.logger") as logger:
self.collect(fake_ffmpeg(f"sys.stdout.buffer.write({PAYLOAD!r})"))
logger.error.assert_not_called()
def test_removes_playlist_when_ffmpeg_cannot_start(self):
with self.assertRaises(OSError):
self.collect(["/nonexistent-ffmpeg-binary"])
self.assertFalse(os.path.exists(self.playlist_path))
def test_closes_the_stdout_pipe_after_a_successful_download(self):
processes: list[sp.Popen] = []
real_popen = sp.Popen
def spy(*args, **kwargs):
process = real_popen(*args, **kwargs)
processes.append(process)
return process
with patch("subprocess.Popen", spy):
self.collect(fake_ffmpeg(f"sys.stdout.buffer.write({PAYLOAD!r})"))
self.assertTrue(processes[0].stdout.closed)
def test_terminating_a_lingering_ffmpeg_is_not_logged_as_a_failure(self):
"""A complete download whose ffmpeg overstays is a success, not an error."""
lingering = fake_ffmpeg(
"import os",
f"os.write(1, {PAYLOAD!r})",
"os.close(1)",
"time.sleep(30)",
)
with patch("frigate.api.media.CLIP_FFMPEG_EXIT_TIMEOUT", 0.5):
with patch("frigate.api.media.logger") as logger:
data = self.collect(lingering)
self.assertEqual(data, PAYLOAD)
logger.error.assert_not_called()
def test_client_disconnect_kills_ffmpeg_and_removes_playlist(self):
processes: list[sp.Popen] = []
real_popen = sp.Popen
def spy(*args, **kwargs):
process = real_popen(*args, **kwargs)
processes.append(process)
return process
forever = fake_ffmpeg(
"while True:",
" sys.stdout.buffer.write(b'x' * 4096)",
" sys.stdout.flush()",
)
with patch("subprocess.Popen", spy):
stream = _run_clip_download(forever, self.playlist_path)
self.assertTrue(next(stream))
# Starlette never closes the generator itself, so a real disconnect
# reaches this path only once the frame is finalized
stream.close()
self.assertIsNotNone(processes[0].poll(), "ffmpeg outlived the request")
self.assertFalse(os.path.exists(self.playlist_path))
+2 -1
View File
@@ -9,6 +9,7 @@ from unittest.mock import MagicMock, patch
from frigate.debug_replay import DebugReplayManager
from frigate.jobs.debug_replay import (
DebugReplayJob,
NoRecordingsError,
RecordingDebugReplaySource,
cancel_debug_replay_job,
get_active_runner,
@@ -129,7 +130,7 @@ class TestStartDebugReplayJob(unittest.TestCase):
empty_qs = MagicMock()
empty_qs.count.return_value = 0
with patch("frigate.jobs.debug_replay.query_recordings", return_value=empty_qs):
with self.assertRaises(ValueError):
with self.assertRaises(NoRecordingsError):
start_debug_replay_job(
source=RecordingDebugReplaySource(
source_camera="front",
+19
View File
@@ -24,6 +24,25 @@ def stop_ffmpeg(ffmpeg_process: sp.Popen[Any], logger: logging.Logger):
ffmpeg_process = None
def terminate_ffmpeg_stream(proc: sp.Popen[Any]) -> None:
"""Stop an ffmpeg process whose stdout is being read over a pipe."""
# Close the read end first so a blocked ffmpeg write unblocks (ffmpeg then
# sees a broken pipe), then signal it. The resulting ffmpeg write error is
# harmless and goes to the captured stderr.
if proc.stdout is not None:
try:
proc.stdout.close()
except OSError:
pass
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except sp.TimeoutExpired:
proc.kill()
proc.wait()
def start_or_restart_ffmpeg(
ffmpeg_cmd, logger, logpipe: LogPipe, frame_size=None, ffmpeg_process=None
) -> sp.Popen[Any]:
+1
View File
@@ -21,6 +21,7 @@
"toast": {
"error": "Failed to start debug replay: {{error}}",
"alreadyActive": "A replay session is already active",
"noRecordings": "No recordings found in the selected time range",
"stopError": "Failed to stop debug replay: {{error}}",
"goToReplay": "Go to Replay"
}
@@ -163,7 +163,13 @@ export default function ClassificationSelectionDialog({
<DropdownMenuLabel>
{dialogLabel ?? t("categorizeImageAs")}
</DropdownMenuLabel>
<div className={cn("flex flex-col", isMobile && "gap-2 pb-4")}>
<div
className={cn(
"flex flex-col",
isMobile &&
"max-h-[40dvh] gap-2 overflow-y-auto overflow-x-hidden pb-4",
)}
>
{filteredClasses
.sort((a, b) => {
if (a === "none") return 1;
@@ -217,7 +217,11 @@ export default function DebugReplayDialog({
error.response?.data?.detail ||
"Unknown error";
if (error.response?.status === 409) {
if (error.response?.status === 404) {
toast.error(t("dialog.toast.noRecordings"), {
position: "top-center",
});
} else if (error.response?.status === 409) {
toast.error(t("dialog.toast.alreadyActive"), {
position: "top-center",
closeButton: true,