Compare commits

..
Author SHA1 Message Date
Nicolas MowenandGitHub 4232cc483d Genai docs refactor & fixes (#22175)
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* Improve GenAI docs

* Clarify

* Fix config updating

* Implement streaming for other providers

* Set openai base url if applied

* Cast context size
2026-02-28 11:40:26 -06:00
Josh HawkinsandGitHub 6a21b2952d Masks and zones improvements (#22163)
* migrator and runtime config changes

* component changes to use rasterized_mask

* frontend

* convert none to empty string for config save

* i18n

* update tests

* add enabled config to zones

* zones frontend

* i18n

* docs

* tweaks

* use dashed stroke to indicate disabled

* allow toggle from icon

* use filelock to ensure atomic config updates from endpoint

* enforce atomic config update in the frontend

* toggle via mqtt

* fix global object masks

* correctly handle global object masks in dispatcher

* ws hooks

* render masks and zones based on ws enabled state

* use enabled_in_config for zones and masks

* frontend for enabled_in_config

* tweaks

* i18n

* publish websocket on config save

* i18n tweaks

* pydantic title and description

* i18n generation

* tweaks

* fix typing
2026-02-28 07:04:43 -07:00
Nicolas MowenandGitHub fa1f9a1fa4 Add GenAI Backend Streaming and Chat (#22152)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* Add basic chat page with entry

* Add chat history

* processing

* Add markdown

* Improvements

* Adjust timing format

* Reduce fields in response

* More time parsing improvements

* Show tool calls separately from message

* Add title

* Improve UI handling

* Support streaming

* Full streaming support

* Fix tool calling

* Add copy button

* Improvements to UI

* Improve default behavior

* Implement message editing

* Add sub label to event tool filtering

* Cleanup

* Cleanup UI and prompt

* Cleanup UI bubbles

* Fix loading

* Add support for markdown tables

* Add thumbnail images to object results

* Add a starting state for chat

* Clenaup
2026-02-27 09:07:30 -07:00
Josh HawkinsandGitHub e7250f24cb Full UI configuration (#22151)
* use react-jsonschema-form for UI config

* don't use properties wrapper when generating config i18n json

* configure for full i18n support

* section fields

* add descriptions to all fields for i18n

* motion i18n

* fix nullable fields

* sanitize internal fields

* add switches widgets and use friendly names

* fix nullable schema entries

* ensure update_topic is added to api calls

this needs further backend implementation to work correctly

* add global sections, camera config overrides, and reset button

* i18n

* add reset logic to global config view

* tweaks

* fix sections and live validation

* fix validation for schema objects that can be null

* generic and custom per-field validation

* improve generic error validation messages

* remove show advanced fields switch

* tweaks

* use shadcn theme

* fix array field template

* i18n tweaks

* remove collapsible around root section

* deep merge schema for advanced fields

* add array field item template and fix ffmpeg section

* add missing i18n keys

* tweaks

* comment out api call for testing

* add config groups as a separate i18n namespace

* add descriptions to all pydantic fields

* make titles more concise

* new titles as i18n

* update i18n config generation script to use json schema

* tweaks

* tweaks

* rebase

* clean up

* form tweaks

* add wildcards and fix object filter fields

* add field template for additionalproperties schema objects

* improve typing

* add section description from schema and clarify global vs camera level descriptions

* separate and consolidate global and camera i18n namespaces

* clean up now obsolete namespaces

* tweaks

* refactor sections and overrides

* add ability to render components before and after fields

* fix titles

* chore(sections): remove legacy single-section components replaced by template

* refactor configs to use individual files with a template

* fix review description

* apply hidden fields after ui schema

* move util

* remove unused i18n

* clean up error messages

* fix fast refresh

* add custom validation and use it for ffmpeg input roles

* update nav tree

* remove unused

* re-add override and modified indicators

* mark pending changes and add confirmation dialog for resets

* fix red unsaved dot

* tweaks

* add docs links, readonly keys, and restart required per field

* add special case and comments for global motion section

* add section form special cases

* combine review sections

* tweaks

* add audio labels endpoint

* add audio label switches and input to filter list

* fix type

* remove key from config when resetting to default/global

* don't show description for new key/val fields

* tweaks

* spacing tweaks

* add activity indicator and scrollbar tweaks

* add docs to filter fields

* wording changes

* fix global ffmpeg section

* add review classification zones to review form

* add backend endpoint and frontend widget for ffmpeg presets and manual args

* improve wording

* hide descriptions for additional properties arrays

* add warning log about incorrectly nested model config

* spacing and language tweaks

* fix i18n keys

* networking section docs and description

* small wording tweaks

* add layout grid field

* refactor with shared utilities

* field order

* add individual detectors to schema

add detector titles and descriptions (docstrings in pydantic are used for descriptions) and add i18n keys to globals

* clean up detectors section and i18n

* don't save model config back to yaml when saving detectors

* add full detectors config to api model dump

works around the way we use detector plugins so we can have the full detector config for the frontend

* add restart button to toast when restart is required

* add ui option to remove inner cards

* fix buttons

* section tweaks

* don't zoom into text on mobile

* make buttons sticky at bottom of sections

* small tweaks

* highlight label of changed fields

* add null to enum list when unwrapping

* refactor to shared utils and add save all button

* add undo all button

* add RJSF to dictionary

* consolidate utils

* preserve form data when changing cameras

* add mono fonts

* add popover to show what fields will be saved

* fix mobile menu not re-rendering with unsaved dots

* tweaks

* fix logger and env vars config section saving

use escaped periods in keys to retain them in the config file (eg "frigate.embeddings")

* add timezone widget

* role map field with validation

* fix validation for model section

* add another hidden field

* add footer message for required restart

* use rjsf for notifications view

* fix config saving

* add replace rules field

* default column layout and add field sizing

* clean up field template

* refactor profile settings to match rjsf forms

* tweaks

* refactor frigate+ view and make tweaks to sections

* show frigate+ model info in detection model settings when using a frigate+ model

* update restartRequired for all fields

* fix restart fields

* tweaks and add ability enable disabled cameras

more backend changes required

* require restart when enabling camera that is disabled in config

* disable save when form is invalid

* refactor ffmpeg section for readability

* change label

* clean up camera inputs fields

* misc tweaks to ffmpeg section

- add raw paths endpoint to ensure credentials get saved
- restart required tooltip

* maintenance settings tweaks

* don't mutate with lodash

* fix description re-rendering for nullable object fields

* hide reindex field

* update rjsf

* add frigate+ description to settings pane

* disable save all when any section is invalid

* show translated field name in validation error pane

* clean up

* remove unused

* fix genai merge

* fix genai
2026-02-27 08:55:36 -07:00
Nicolas MowenandGitHub eeefbf2bb5 Add support for multiple GenAI Providers (#22144)
* GenAI client manager

* Add config migration

* Convert to roles list

* Support getting client via manager

* Cleanup

* Fix import issues

* Set model in llama.cpp config

* Clenaup

* Use config update

* Clenaup

* Add new title and desc
2026-02-27 08:35:33 -07:00
Martin WeineltandGitHub ba0e7bbc1a Remove redundant tensorflow import in BirdRealTimeProcessor (#22127)
Was added in ae0c1ca (#21301) and then incompletely reverted in ec1d794
(#21320).
2026-02-27 05:37:17 -07:00
Martin WeineltandGitHub e16763cff9 Fallback from tflite-runtime to ai-edge-litert (#21876)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
The fallback to tensorflow was established back in 2023, because we could
not provide tflite-runtime downstream in nixpkgs.

By now we have ai-edge-litert available, which is the successor to the
tflite-runtime. It still provides the same entrypoints as tflite-runtime
and functionality has been verified in multiple deployments for the last
two weeks.
2026-02-26 21:55:29 -07:00
Felipe SantosandGitHub b88186983a Increase maximum stream timeout to 15s (#21936)
* Increase maximum stream timeout to 15s

* Use predefined intervals instead for the stream timeout
2026-02-26 21:54:00 -07:00
Martin WeineltandGitHub b4eac11cbd Clean up trailing whitespaces in cpu stats process cmdline (#22089)
The psutil library reads the process commandline as by opening
/proc/pid/cmdline which returns a buffer that is larger than just the
program cmdline due to rounded memory allocation sizes.

That means that if the library does not detect a Null-terminated string
it keeps appending empty strings which add up as whitespaces when joined.
2026-02-26 21:53:26 -07:00
Nicolas Mowen 9c3a74b4f5 Cleanup 2026-02-26 21:27:56 -07:00
Nicolas Mowen 91714b8743 Remove exceptions 2026-02-26 21:27:56 -07:00
Nicolas Mowen e5087b092d Fix frame time access 2026-02-26 21:27:56 -07:00
Nicolas Mowen 5f02e33e55 Adapt to new Gemini format 2026-02-26 21:27:56 -07:00
nulledyandNicolas Mowen 84760c42cb ffmpeg Preview Segment Optimization for "high" and "very_high" (#21996)
* Introduce qmax parameter for ffmpeg preview encoding

Added PREVIEW_QMAX_PARAM to control ffmpeg encoding quality.

* formatting

* Fix spacing in qmax parameters for preview quality
2026-02-26 21:27:56 -07:00
nulledyandNicolas Mowen bb6e889449 Allow API Events to be Detections or Alerts, depending on the Event Label (#21923)
* - API created events will be alerts OR detections, depending on the event label, defaulting to alerts
- Indefinite API events will extend the recording segment until those events are ended
- API event start time is the actual start time, instead of having a pre-buffer of record.event_pre_capture

* Instead of checking for indefinite events on a camera before deciding if we should end the segment, only update last_detection_time and last_alert_time if frame_time is greater, which should have the same effect

* Add the ability to set a pre_capture number of seconds when creating a manual event via the API. Default behavior unchanged

* Remove unnecessary _publish_segment_start() call

* Formatting

* handle last_alert_time or last_detection_time being None when checking them against the frame_time

* comment manual_info["label"].split(": ")[0] for clarity
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen 12506f8c80 Improve jsmpeg player websocket handling (#21943)
* improve jsmpeg player websocket handling

prevent websocket console messages from appearing when player is destroyed

* reformat files after ruff upgrade
2026-02-26 21:27:56 -07:00
fef1fb36cc feat: add X-Frame-Time when returning snapshot (#21932)
Co-authored-by: Florent MORICONI <170678386+fmcloudconsulting@users.noreply.github.com>
2026-02-26 21:27:56 -07:00
Eric WorkandNicolas Mowen 2db0269825 Add networking options for configuring listening ports (#21779) 2026-02-26 21:27:56 -07:00
Nicolas Mowen a4362caa0a Add live context tool to LLM (#21754)
* Add live context tool

* Improve handling of images in request

* Improve prompt caching
2026-02-26 21:27:56 -07:00
Nicolas Mowen fa0feebd03 Update to ROCm 7.2.0 (#21753)
* Update to ROCm 7.2.0

* ROCm now works properly with JinaV1

* Arcface has compilation error
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen c78ab2dc87 Offline preview image (#21752)
* use latest preview frame for latest image when camera is offline

* remove frame extraction logic

* tests

* frontend

* add description to api endpoint
2026-02-26 21:27:56 -07:00
Nicolas Mowen e76b48f98b Implement LLM Chat API with tool calling support (#21731)
* Implement initial tools definiton APIs

* Add initial chat completion API with tool support

* Implement other providers

* Cleanup
2026-02-26 21:27:56 -07:00
John ShawandNicolas Mowen af2339b35c Remove parents in remove_empty_directories (#21726)
The original implementation did a full directory tree walk to find and remove
empty directories, so this implementation should remove the parents as well,
like the original did.
2026-02-26 21:27:56 -07:00
Nicolas Mowen 9b7cee18db Implement llama.cpp GenAI Provider (#21690)
* Implement llama.cpp GenAI Provider

* Add docs

* Update links

* Fix broken mqtt links

* Fix more broken anchors
2026-02-26 21:27:56 -07:00
John ShawandNicolas Mowen d3260e34b6 Optimize empty directory cleanup for recordings (#21695)
The previous empty directory cleanup did a full recursive directory
walk, which can be extremely slow. This new implementation only removes
directories which have a chance of being empty due to a recent file
deletion.
2026-02-26 21:27:56 -07:00
Nicolas Mowen ee2c96c793 Refactor Time-Lapse Export (#21668)
* refactor time lapse creation to be a separate API call with ability to pass arbitrary ffmpeg args

* Add CPU fallback
2026-02-26 21:27:56 -07:00
542295dcb3 Update go2rtc to v1.9.13 (#21648)
Co-authored-by: Eugeny Tulupov <eugeny.tulupov@spirent.com>
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen 56c7a13fbe Fix incorrect counting in sync_recordings (#21626) 2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen 88348bf535 use same logging pattern in sync_recordings as the other sync functions (#21625) 2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen b66e69efc9 Media sync API refactor and UI (#21542)
* generic job infrastructure

* types and dispatcher changes for jobs

* save data in memory only for completed jobs

* implement media sync job and endpoints

* change logs to debug

* websocket hook and types

* frontend

* i18n

* docs tweaks

* endpoint descriptions

* tweak docs
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen 63e7bf8b28 Add media sync API endpoint (#21526)
* add media cleanup functions

* add endpoint

* remove scheduled sync recordings from cleanup

* move to utils dir

* tweak import

* remove sync_recordings and add config migrator

* remove sync_recordings

* docs

* remove key

* clean up docs

* docs fix

* docs tweak
2026-02-26 21:27:56 -07:00
Nicolas Mowen 39ad565f81 Add API to handle deleting recordings (#21520)
* Add recording delete API

* Re-organize recordings apis

* Fix import

* Consolidate query types
2026-02-26 21:27:56 -07:00
Nicolas Mowen 9ef8b70208 Exports Improvements (#21521)
* Add images to case folder view

* Add ability to select case in export dialog

* Add to mobile review too
2026-02-26 21:27:56 -07:00
Nicolas Mowen 6b77952b72 Add support for GPU and NPU temperatures (#21495)
* Add rockchip temps

* Add support for GPU and NPU temperatures in the frontend

* Add support for Nvidia temperature

* Improve separation

* Adjust graph scaling
2026-02-26 21:27:56 -07:00
Andrew RobertsandNicolas Mowen 3745f5ff93 Camera-specific hwaccel settings for timelapse exports (correct base) (#21386)
* added hwaccel_args to camera.record.export config struct

* populate camera.record.export.hwaccel_args with a cascade up to camera then global if 'auto'

* use new hwaccel args in export

* added documentation for camera-specific hwaccel export

* fix c/p error

* missed an import

* fleshed out the docs and comments a bit

* ruff lint

* separated out the tips in the doc

* fix documentation

* fix and simplify reference config doc
2026-02-26 21:27:56 -07:00
Nicolas Mowenandtigattack 3297cab347 Refactor temperature reporting for detectors and implement Hailo temp reading (#21395)
* Add Hailo temperature retrieval

* Refactor `get_hailo_temps()` to use ctxmanager

* Show Hailo temps in system UI

* Move hailo_platform import to get_hailo_temps

* Refactor temperatures calculations to use within detector block

* Adjust webUI to handle new location

---------

Co-authored-by: tigattack <10629864+tigattack@users.noreply.github.com>
2026-02-26 21:27:56 -07:00
Nicolas Mowen fc3545310c Export filter UI (#21322)
* Get started on export filters

* implement basic filter

* Implement filtering and adjust api

* Improve filter handling

* Improve navigation

* Cleanup

* handle scrolling
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen dde738cfdc Camera connection quality indicator (#21297)
* add camera connection quality metrics and indicator

* formatting

* move stall calcs to watchdog

* clean up

* change watchdog to 1s and separately track time for ffmpeg retry_interval

* implement status caching to reduce message volume
2026-02-26 21:27:56 -07:00
Nicolas Mowen 004bb7d80d Case management UI (#21299)
* Refactor export cards to match existing cards in other UI pages

* Show cases separately from exports

* Add proper filtering and display of cases

* Add ability to edit and select cases for exports

* Cleanup typing

* Hide if no unassigned

* Cleanup hiding logic

* fix scrolling

* Improve layout
2026-02-26 21:27:56 -07:00
Josh HawkinsandNicolas Mowen 85feb4edcb refactor vainfo to search for first GPU (#21296)
use existing LibvaGpuSelector to pick appropritate libva device
2026-02-26 21:27:56 -07:00
Nicolas Mowen cffa54c80d implement case management for export apis (#21295) 2026-02-26 21:27:56 -07:00
Nicolas Mowen 48164f6dfc Create scaffolding for case management (#21293) 2026-02-26 21:27:56 -07:00
Nicolas Mowen bc457743b6 Update version 2026-02-26 21:27:56 -07:00
Nicolas MowenandGitHub 451d6f5c22 Revert "Early 0.18 work (#22138)" (#22142)
This reverts commit d24b96d3bb.
2026-02-26 21:27:31 -07:00
+2 d24b96d3bb Early 0.18 work (#22138)
* Update version

* Create scaffolding for case management (#21293)

* implement case management for export apis (#21295)

* refactor vainfo to search for first GPU (#21296)

use existing LibvaGpuSelector to pick appropritate libva device

* Case management UI (#21299)

* Refactor export cards to match existing cards in other UI pages

* Show cases separately from exports

* Add proper filtering and display of cases

* Add ability to edit and select cases for exports

* Cleanup typing

* Hide if no unassigned

* Cleanup hiding logic

* fix scrolling

* Improve layout

* Camera connection quality indicator (#21297)

* add camera connection quality metrics and indicator

* formatting

* move stall calcs to watchdog

* clean up

* change watchdog to 1s and separately track time for ffmpeg retry_interval

* implement status caching to reduce message volume

* Export filter UI (#21322)

* Get started on export filters

* implement basic filter

* Implement filtering and adjust api

* Improve filter handling

* Improve navigation

* Cleanup

* handle scrolling

* Refactor temperature reporting for detectors and implement Hailo temp reading (#21395)

* Add Hailo temperature retrieval

* Refactor `get_hailo_temps()` to use ctxmanager

* Show Hailo temps in system UI

* Move hailo_platform import to get_hailo_temps

* Refactor temperatures calculations to use within detector block

* Adjust webUI to handle new location

---------

Co-authored-by: tigattack <10629864+tigattack@users.noreply.github.com>

* Camera-specific hwaccel settings for timelapse exports (correct base) (#21386)

* added hwaccel_args to camera.record.export config struct

* populate camera.record.export.hwaccel_args with a cascade up to camera then global if 'auto'

* use new hwaccel args in export

* added documentation for camera-specific hwaccel export

* fix c/p error

* missed an import

* fleshed out the docs and comments a bit

* ruff lint

* separated out the tips in the doc

* fix documentation

* fix and simplify reference config doc

* Add support for GPU and NPU temperatures (#21495)

* Add rockchip temps

* Add support for GPU and NPU temperatures in the frontend

* Add support for Nvidia temperature

* Improve separation

* Adjust graph scaling

* Exports Improvements (#21521)

* Add images to case folder view

* Add ability to select case in export dialog

* Add to mobile review too

* Add API to handle deleting recordings  (#21520)

* Add recording delete API

* Re-organize recordings apis

* Fix import

* Consolidate query types

* Add media sync API endpoint (#21526)

* add media cleanup functions

* add endpoint

* remove scheduled sync recordings from cleanup

* move to utils dir

* tweak import

* remove sync_recordings and add config migrator

* remove sync_recordings

* docs

* remove key

* clean up docs

* docs fix

* docs tweak

* Media sync API refactor and UI (#21542)

* generic job infrastructure

* types and dispatcher changes for jobs

* save data in memory only for completed jobs

* implement media sync job and endpoints

* change logs to debug

* websocket hook and types

* frontend

* i18n

* docs tweaks

* endpoint descriptions

* tweak docs

* use same logging pattern in sync_recordings as the other sync functions (#21625)

* Fix incorrect counting in sync_recordings (#21626)

* Update go2rtc to v1.9.13 (#21648)

Co-authored-by: Eugeny Tulupov <eugeny.tulupov@spirent.com>

* Refactor Time-Lapse Export (#21668)

* refactor time lapse creation to be a separate API call with ability to pass arbitrary ffmpeg args

* Add CPU fallback

* Optimize empty directory cleanup for recordings (#21695)

The previous empty directory cleanup did a full recursive directory
walk, which can be extremely slow. This new implementation only removes
directories which have a chance of being empty due to a recent file
deletion.

* Implement llama.cpp GenAI Provider (#21690)

* Implement llama.cpp GenAI Provider

* Add docs

* Update links

* Fix broken mqtt links

* Fix more broken anchors

* Remove parents in remove_empty_directories (#21726)

The original implementation did a full directory tree walk to find and remove
empty directories, so this implementation should remove the parents as well,
like the original did.

* Implement LLM Chat API with tool calling support (#21731)

* Implement initial tools definiton APIs

* Add initial chat completion API with tool support

* Implement other providers

* Cleanup

* Offline preview image (#21752)

* use latest preview frame for latest image when camera is offline

* remove frame extraction logic

* tests

* frontend

* add description to api endpoint

* Update to ROCm 7.2.0 (#21753)

* Update to ROCm 7.2.0

* ROCm now works properly with JinaV1

* Arcface has compilation error

* Add live context tool to LLM (#21754)

* Add live context tool

* Improve handling of images in request

* Improve prompt caching

* Add networking options for configuring listening ports (#21779)

* feat: add X-Frame-Time when returning snapshot (#21932)

Co-authored-by: Florent MORICONI <170678386+fmcloudconsulting@users.noreply.github.com>

* Improve jsmpeg player websocket handling (#21943)

* improve jsmpeg player websocket handling

prevent websocket console messages from appearing when player is destroyed

* reformat files after ruff upgrade

* Allow API Events to be Detections or Alerts, depending on the Event Label (#21923)

* - API created events will be alerts OR detections, depending on the event label, defaulting to alerts
- Indefinite API events will extend the recording segment until those events are ended
- API event start time is the actual start time, instead of having a pre-buffer of record.event_pre_capture

* Instead of checking for indefinite events on a camera before deciding if we should end the segment, only update last_detection_time and last_alert_time if frame_time is greater, which should have the same effect

* Add the ability to set a pre_capture number of seconds when creating a manual event via the API. Default behavior unchanged

* Remove unnecessary _publish_segment_start() call

* Formatting

* handle last_alert_time or last_detection_time being None when checking them against the frame_time

* comment manual_info["label"].split(": ")[0] for clarity

* ffmpeg Preview Segment Optimization for "high" and "very_high" (#21996)

* Introduce qmax parameter for ffmpeg preview encoding

Added PREVIEW_QMAX_PARAM to control ffmpeg encoding quality.

* formatting

* Fix spacing in qmax parameters for preview quality

* Adapt to new Gemini format

* Fix frame time access

* Remove exceptions

* Cleanup

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Co-authored-by: tigattack <10629864+tigattack@users.noreply.github.com>
Co-authored-by: Andrew Roberts <adroberts@gmail.com>
Co-authored-by: Eugeny Tulupov <zhekka3@gmail.com>
Co-authored-by: Eugeny Tulupov <eugeny.tulupov@spirent.com>
Co-authored-by: John Shaw <1753078+johnshaw@users.noreply.github.com>
Co-authored-by: Eric Work <work.eric@gmail.com>
Co-authored-by: FL42 <46161216+fl42@users.noreply.github.com>
Co-authored-by: Florent MORICONI <170678386+fmcloudconsulting@users.noreply.github.com>
Co-authored-by: nulledy <254504350+nulledy@users.noreply.github.com>
2026-02-26 21:16:10 -07:00
404 changed files with 35049 additions and 8470 deletions
+1
View File
@@ -229,6 +229,7 @@ Reolink
restream
restreamed
restreaming
RJSF
rkmpp
rknn
rkrga
+3 -2
View File
@@ -1,7 +1,7 @@
default_target: local
COMMIT_HASH := $(shell git log -1 --pretty=format:"%h"|tail -1)
VERSION = 0.17.2
VERSION = 0.18.0
IMAGE_REPO ?= ghcr.io/blakeblackshear/frigate
GITHUB_REF_NAME ?= $(shell git rev-parse --abbrev-ref HEAD)
BOARDS= #Initialized empty
@@ -49,7 +49,8 @@ push: push-boards
--push
run: local
docker run --rm --publish=5000:5000 --volume=${PWD}/config:/config frigate:latest
docker run --rm --publish=5000:5000 --publish=8971:8971 \
--volume=${PWD}/config:/config frigate:latest
run_tests: local
docker run --rm --workdir=/opt/frigate --entrypoint= frigate:latest \
+1 -1
View File
@@ -55,7 +55,7 @@ RUN --mount=type=tmpfs,target=/tmp --mount=type=tmpfs,target=/var/cache/apt \
FROM scratch AS go2rtc
ARG TARGETARCH
WORKDIR /rootfs/usr/local/go2rtc/bin
ADD --link --chmod=755 "https://github.com/AlexxIT/go2rtc/releases/download/v1.9.10/go2rtc_linux_${TARGETARCH}" go2rtc
ADD --link --chmod=755 "https://github.com/AlexxIT/go2rtc/releases/download/v1.9.13/go2rtc_linux_${TARGETARCH}" go2rtc
FROM wget AS tempio
ARG TARGETARCH
@@ -10,7 +10,8 @@ echo "[INFO] Starting certsync..."
lefile="/etc/letsencrypt/live/frigate/fullchain.pem"
tls_enabled=`python3 /usr/local/nginx/get_listen_settings.py | jq -r .tls.enabled`
tls_enabled=`python3 /usr/local/nginx/get_nginx_settings.py | jq -r .tls.enabled`
listen_external_port=`python3 /usr/local/nginx/get_nginx_settings.py | jq -r .listen.external_port`
while true
do
@@ -34,7 +35,7 @@ do
;;
esac
liveprint=`echo | openssl s_client -showcerts -connect 127.0.0.1:8971 2>&1 | openssl x509 -fingerprint 2>&1 | grep -i fingerprint || echo 'failed'`
liveprint=`echo | openssl s_client -showcerts -connect 127.0.0.1:$listen_external_port 2>&1 | openssl x509 -fingerprint 2>&1 | grep -i fingerprint || echo 'failed'`
case "$liveprint" in
*Fingerprint*)
@@ -55,4 +56,4 @@ do
done
exit 0
exit 0
@@ -55,7 +55,7 @@ function setup_homekit_config() {
if [[ ! -f "${config_path}" ]]; then
echo "[INFO] Creating empty config file for HomeKit..."
: > "${config_path}"
echo '{}' > "${config_path}"
fi
# Convert YAML to JSON for jq processing
@@ -65,25 +65,23 @@ function setup_homekit_config() {
return 0
}
# Use jq to extract the homekit section, if it exists
local homekit_json
homekit_json=$(jq '
if has("homekit") then {homekit: .homekit} else null end
' "${temp_json}" 2>/dev/null) || homekit_json="null"
# Use jq to filter and keep only the homekit section
local cleaned_json="/tmp/cache/homekit_cleaned.json"
jq '
# Keep only the homekit section if it exists, otherwise empty object
if has("homekit") then {homekit: .homekit} else {} end
' "${temp_json}" > "${cleaned_json}" 2>/dev/null || {
echo '{}' > "${cleaned_json}"
}
# If no homekit section, write an empty config file
if [[ "${homekit_json}" == "null" ]]; then
: > "${config_path}"
else
# Convert homekit JSON back to YAML and write to the config file
echo "${homekit_json}" | yq eval -P - > "${config_path}" 2>/dev/null || {
echo "[WARNING] Failed to convert cleaned config to YAML, creating minimal config"
: > "${config_path}"
}
fi
# Convert back to YAML and write to the config file
yq eval -P "${cleaned_json}" > "${config_path}" 2>/dev/null || {
echo "[WARNING] Failed to convert cleaned config to YAML, creating minimal config"
echo '{}' > "${config_path}"
}
# Clean up temp files
rm -f "${temp_json}"
rm -f "${temp_json}" "${cleaned_json}"
}
set_libva_version
@@ -80,14 +80,14 @@ if [ ! \( -f "$letsencrypt_path/privkey.pem" -a -f "$letsencrypt_path/fullchain.
fi
# build templates for optional FRIGATE_BASE_PATH environment variable
python3 /usr/local/nginx/get_base_path.py | \
python3 /usr/local/nginx/get_nginx_settings.py | \
tempio -template /usr/local/nginx/templates/base_path.gotmpl \
-out /usr/local/nginx/conf/base_path.conf
-out /usr/local/nginx/conf/base_path.conf
# build templates for optional TLS support
python3 /usr/local/nginx/get_listen_settings.py | \
tempio -template /usr/local/nginx/templates/listen.gotmpl \
-out /usr/local/nginx/conf/listen.conf
# build templates for additional network settings
python3 /usr/local/nginx/get_nginx_settings.py | \
tempio -template /usr/local/nginx/templates/listen.gotmpl \
-out /usr/local/nginx/conf/listen.conf
# Replace the bash process with the NGINX process, redirecting stderr to stdout
exec 2>&1
@@ -17,15 +17,36 @@ from frigate.const import (
)
from frigate.ffmpeg_presets import parse_preset_hardware_acceleration_encode
from frigate.util.config import find_config_file
from frigate.util.services import (
is_go2rtc_arbitrary_exec_allowed,
is_restricted_go2rtc_source,
)
sys.path.remove("/opt/frigate")
yaml = YAML()
# Check if arbitrary exec sources are allowed (defaults to False for security)
allow_arbitrary_exec = None
if "GO2RTC_ALLOW_ARBITRARY_EXEC" in os.environ:
allow_arbitrary_exec = os.environ.get("GO2RTC_ALLOW_ARBITRARY_EXEC")
elif (
os.path.isdir("/run/secrets")
and os.access("/run/secrets", os.R_OK)
and "GO2RTC_ALLOW_ARBITRARY_EXEC" in os.listdir("/run/secrets")
):
allow_arbitrary_exec = (
Path(os.path.join("/run/secrets", "GO2RTC_ALLOW_ARBITRARY_EXEC"))
.read_text()
.strip()
)
# check for the add-on options file
elif os.path.isfile("/data/options.json"):
with open("/data/options.json") as f:
raw_options = f.read()
options = json.loads(raw_options)
allow_arbitrary_exec = options.get("go2rtc_allow_arbitrary_exec")
ALLOW_ARBITRARY_EXEC = allow_arbitrary_exec is not None and str(
allow_arbitrary_exec
).lower() in ("true", "1", "yes")
FRIGATE_ENV_VARS = {k: v for k, v in os.environ.items() if k.startswith("FRIGATE_")}
# read docker secret files as env vars too
if os.path.isdir("/run/secrets"):
@@ -114,13 +135,18 @@ if LIBAVFORMAT_VERSION_MAJOR < 59:
go2rtc_config["ffmpeg"]["rtsp"] = rtsp_args
def is_restricted_source(stream_source: str) -> bool:
"""Check if a stream source is restricted (echo, expr, or exec)."""
return stream_source.strip().startswith(("echo:", "expr:", "exec:"))
for name in list(go2rtc_config.get("streams", {})):
stream = go2rtc_config["streams"][name]
if isinstance(stream, str):
try:
formatted_stream = stream.format(**FRIGATE_ENV_VARS)
if is_restricted_go2rtc_source(formatted_stream):
if not ALLOW_ARBITRARY_EXEC and is_restricted_source(formatted_stream):
print(
f"[ERROR] Stream '{name}' uses a restricted source (echo/expr/exec) which is disabled by default for security. "
f"Set GO2RTC_ALLOW_ARBITRARY_EXEC=true to enable arbitrary exec sources."
@@ -139,7 +165,7 @@ for name in list(go2rtc_config.get("streams", {})):
for i, stream_item in enumerate(stream):
try:
formatted_stream = stream_item.format(**FRIGATE_ENV_VARS)
if is_restricted_go2rtc_source(formatted_stream):
if not ALLOW_ARBITRARY_EXEC and is_restricted_source(formatted_stream):
print(
f"[ERROR] Stream '{name}' item {i + 1} uses a restricted source (echo/expr/exec) which is disabled by default for security. "
f"Set GO2RTC_ALLOW_ARBITRARY_EXEC=true to enable arbitrary exec sources."
@@ -162,20 +188,6 @@ for name in list(go2rtc_config.get("streams", {})):
)
del go2rtc_config["streams"][name]
elif isinstance(stream, dict):
# The map form ({"url": ...}) lets go2rtc resolve the source
# recursively, so it is effectively a dynamic way to generate the URL
# for a stream. That can only be backed by an exec source, so it cannot
# be allowed unless arbitrary exec is explicitly enabled. When it is
# enabled, leave the map untouched for go2rtc to resolve.
if not is_go2rtc_arbitrary_exec_allowed():
print(
f"[ERROR] Stream '{name}' uses a dynamic source format which is disabled by default for security. "
f"Set GO2RTC_ALLOW_ARBITRARY_EXEC=true to enable arbitrary exec sources."
)
del go2rtc_config["streams"][name]
continue
# add birdseye restream stream if enabled
if config.get("birdseye", {}).get("restream", False):
birdseye: dict[str, Any] = config.get("birdseye")
@@ -259,7 +259,6 @@ http {
include proxy.conf;
proxy_cache api_cache;
proxy_cache_key "$scheme$proxy_host$request_uri|$role|$groups|$user";
proxy_cache_lock on;
proxy_cache_use_stale updating;
proxy_cache_valid 200 5s;
@@ -1,11 +0,0 @@
"""Prints the base path as json to stdout."""
import json
import os
from typing import Any
base_path = os.environ.get("FRIGATE_BASE_PATH", "")
result: dict[str, Any] = {"base_path": base_path}
print(json.dumps(result))
@@ -1,35 +0,0 @@
"""Prints the tls config as json to stdout."""
import json
import sys
from typing import Any
from ruamel.yaml import YAML
sys.path.insert(0, "/opt/frigate")
from frigate.util.config import find_config_file
sys.path.remove("/opt/frigate")
yaml = YAML()
config_file = find_config_file()
try:
with open(config_file) as f:
raw_config = f.read()
if config_file.endswith((".yaml", ".yml")):
config: dict[str, Any] = yaml.load(raw_config)
elif config_file.endswith(".json"):
config: dict[str, Any] = json.loads(raw_config)
except FileNotFoundError:
config: dict[str, Any] = {}
tls_config: dict[str, any] = config.get("tls", {"enabled": True})
networking_config = config.get("networking", {})
ipv6_config = networking_config.get("ipv6", {"enabled": False})
output = {"tls": tls_config, "ipv6": ipv6_config}
print(json.dumps(output))
@@ -0,0 +1,62 @@
"""Prints the nginx settings as json to stdout."""
import json
import os
import sys
from typing import Any
from ruamel.yaml import YAML
sys.path.insert(0, "/opt/frigate")
from frigate.util.config import find_config_file
sys.path.remove("/opt/frigate")
yaml = YAML()
config_file = find_config_file()
try:
with open(config_file) as f:
raw_config = f.read()
if config_file.endswith((".yaml", ".yml")):
config: dict[str, Any] = yaml.load(raw_config)
elif config_file.endswith(".json"):
config: dict[str, Any] = json.loads(raw_config)
except FileNotFoundError:
config: dict[str, Any] = {}
tls_config: dict[str, Any] = config.get("tls", {})
tls_config.setdefault("enabled", True)
networking_config: dict[str, Any] = config.get("networking", {})
ipv6_config: dict[str, Any] = networking_config.get("ipv6", {})
ipv6_config.setdefault("enabled", False)
listen_config: dict[str, Any] = networking_config.get("listen", {})
listen_config.setdefault("internal", 5000)
listen_config.setdefault("external", 8971)
# handle case where internal port is a string with ip:port
internal_port = listen_config["internal"]
if type(internal_port) is str:
internal_port = int(internal_port.split(":")[-1])
listen_config["internal_port"] = internal_port
# handle case where external port is a string with ip:port
external_port = listen_config["external"]
if type(external_port) is str:
external_port = int(external_port.split(":")[-1])
listen_config["external_port"] = external_port
base_path = os.environ.get("FRIGATE_BASE_PATH", "")
result: dict[str, Any] = {
"tls": tls_config,
"ipv6": ipv6_config,
"listen": listen_config,
"base_path": base_path,
}
print(json.dumps(result))
@@ -7,7 +7,7 @@ location ^~ {{ .base_path }}/ {
# remove base_url from the path before passing upstream
rewrite ^{{ .base_path }}/(.*) /$1 break;
proxy_pass $scheme://127.0.0.1:8971;
proxy_pass $scheme://127.0.0.1:{{ .listen.external_port }};
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
@@ -1,45 +1,36 @@
# Internal (IPv4 always; IPv6 optional)
listen 5000;
{{ if .ipv6 }}{{ if .ipv6.enabled }}listen [::]:5000;{{ end }}{{ end }}
listen {{ .listen.internal }};
{{ if .ipv6.enabled }}listen [::]:{{ .listen.internal_port }};{{ end }}
# intended for external traffic, protected by auth
{{ if .tls }}
{{ if .tls.enabled }}
# external HTTPS (IPv4 always; IPv6 optional)
listen 8971 ssl;
{{ if .ipv6 }}{{ if .ipv6.enabled }}listen [::]:8971 ssl;{{ end }}{{ end }}
{{ if .tls.enabled }}
# external HTTPS (IPv4 always; IPv6 optional)
listen {{ .listen.external }} ssl;
{{ if .ipv6.enabled }}listen [::]:{{ .listen.external_port }} ssl;{{ end }}
ssl_certificate /etc/letsencrypt/live/frigate/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/frigate/privkey.pem;
ssl_certificate /etc/letsencrypt/live/frigate/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/frigate/privkey.pem;
# generated 2024-06-01, Mozilla Guideline v5.7, nginx 1.25.3, OpenSSL 1.1.1w, modern configuration, no OCSP
# https://ssl-config.mozilla.org/#server=nginx&version=1.25.3&config=modern&openssl=1.1.1w&ocsp=false&guideline=5.7
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m; # about 40000 sessions
ssl_session_tickets off;
# generated 2024-06-01, Mozilla Guideline v5.7, nginx 1.25.3, OpenSSL 1.1.1w, modern configuration, no OCSP
# https://ssl-config.mozilla.org/#server=nginx&version=1.25.3&config=modern&openssl=1.1.1w&ocsp=false&guideline=5.7
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m; # about 40000 sessions
ssl_session_tickets off;
# modern configuration
ssl_protocols TLSv1.3;
ssl_prefer_server_ciphers off;
# modern configuration
ssl_protocols TLSv1.3;
ssl_prefer_server_ciphers off;
# HSTS (ngx_http_headers_module is required) (63072000 seconds)
add_header Strict-Transport-Security "max-age=63072000" always;
# HSTS (ngx_http_headers_module is required) (63072000 seconds)
add_header Strict-Transport-Security "max-age=63072000" always;
# ACME challenge location
location /.well-known/acme-challenge/ {
default_type "text/plain";
root /etc/letsencrypt/www;
}
{{ else }}
# external HTTP (IPv4 always; IPv6 optional)
listen 8971;
{{ if .ipv6 }}{{ if .ipv6.enabled }}listen [::]:8971;{{ end }}{{ end }}
{{ end }}
# ACME challenge location
location /.well-known/acme-challenge/ {
default_type "text/plain";
root /etc/letsencrypt/www;
}
{{ else }}
# (No tls section) default to HTTP (IPv4 always; IPv6 optional)
listen 8971;
{{ if .ipv6 }}{{ if .ipv6.enabled }}listen [::]:8971;{{ end }}{{ end }}
# (No tls) default to HTTP (IPv4 always; IPv6 optional)
listen {{ .listen.external }};
{{ if .ipv6.enabled }}listen [::]:{{ .listen.external_port }};{{ end }}
{{ end }}
+3 -1
View File
@@ -13,7 +13,7 @@ ARG ROCM
RUN apt update -qq && \
apt install -y wget gpg && \
wget -O rocm.deb https://repo.radeon.com/amdgpu-install/7.1.1/ubuntu/jammy/amdgpu-install_7.1.1.70101-1_all.deb && \
wget -O rocm.deb https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb && \
apt install -y ./rocm.deb && \
apt update && \
apt install -qq -y rocm
@@ -56,6 +56,8 @@ FROM scratch AS rocm-dist
ARG ROCM
# Copy HIP headers required for MIOpen JIT (BuildHip) / HIPRTC at runtime
COPY --from=rocm /opt/rocm-${ROCM}/include/ /opt/rocm-${ROCM}/include/
COPY --from=rocm /opt/rocm-$ROCM/bin/rocminfo /opt/rocm-$ROCM/bin/migraphx-driver /opt/rocm-$ROCM/bin/
# Copy MIOpen database files for gfx10xx and gfx11xx only (RDNA2/RDNA3)
COPY --from=rocm /opt/rocm-$ROCM/share/miopen/db/*gfx10* /opt/rocm-$ROCM/share/miopen/db/
+1 -1
View File
@@ -1 +1 @@
onnxruntime-migraphx @ https://github.com/NickM-27/frigate-onnxruntime-rocm/releases/download/v7.1.0/onnxruntime_migraphx-1.23.1-cp311-cp311-linux_x86_64.whl
onnxruntime-migraphx @ https://github.com/NickM-27/frigate-onnxruntime-rocm/releases/download/v7.2.0/onnxruntime_migraphx-1.23.1-cp311-cp311-linux_x86_64.whl
+1 -1
View File
@@ -1,5 +1,5 @@
variable "ROCM" {
default = "7.1.1"
default = "7.2.0"
}
variable "HSA_OVERRIDE_GFX_VERSION" {
default = ""
+23 -32
View File
@@ -44,21 +44,13 @@ go2rtc:
### `environment_vars`
This section can be used to set environment variables for those unable to modify the environment of the container, like within Home Assistant OS. Docker users should set environment variables in their `docker run` command (`-e FRIGATE_MQTT_PASSWORD=secret`) or `docker-compose.yml` file (`environment:` section) instead. Note that values set here are stored in plain text in your config file, so if the goal is to keep credentials out of your configuration, use Docker environment variables or Docker secrets instead.
Variables prefixed with `FRIGATE_` can be referenced in config fields that support environment variable substitution (such as MQTT host and credentials, camera stream URLs, and ONVIF host and credentials) using the `{FRIGATE_VARIABLE_NAME}` syntax.
This section can be used to set environment variables for those unable to modify the environment of the container, like within Home Assistant OS.
Example:
```yaml
environment_vars:
FRIGATE_MQTT_USER: my_mqtt_user
FRIGATE_MQTT_PASSWORD: my_mqtt_password
mqtt:
host: "{FRIGATE_MQTT_HOST}"
user: "{FRIGATE_MQTT_USER}"
password: "{FRIGATE_MQTT_PASSWORD}"
VARIABLE_NAME: variable_value
```
#### TensorFlow Thread Configuration
@@ -163,34 +155,33 @@ services:
### Enabling IPv6
IPv6 is disabled by default, to enable IPv6 listen.gotmpl needs to be bind mounted with IPv6 enabled. For example:
IPv6 is disabled by default, to enable IPv6 modify your Frigate configuration as follows:
```
{{ if not .enabled }}
# intended for external traffic, protected by auth
listen 8971;
{{ else }}
# intended for external traffic, protected by auth
listen 8971 ssl;
# intended for internal traffic, not protected by auth
listen 5000;
```yaml
networking:
ipv6:
enabled: True
```
becomes
### Listen on different ports
```
{{ if not .enabled }}
# intended for external traffic, protected by auth
listen [::]:8971 ipv6only=off;
{{ else }}
# intended for external traffic, protected by auth
listen [::]:8971 ipv6only=off ssl;
You can change the ports Nginx uses for listening using Frigate's configuration file. The internal port (unauthenticated) and external port (authenticated) can be changed independently. You can also specify an IP address using the format `ip:port` if you wish to bind the port to a specific interface. This may be useful for example to prevent exposing the internal port outside the container.
# intended for internal traffic, not protected by auth
listen [::]:5000 ipv6only=off;
For example:
```yaml
networking:
listen:
internal: 127.0.0.1:5000
external: 8971
```
:::warning
This setting is for advanced users. For the majority of use cases it's recommended to change the `ports` section of your Docker compose file or use the Docker `run` `--publish` option instead, e.g. `-p 443:8971`. Changing Frigate's ports may break some integrations.
:::
## Base path
By default, Frigate runs at the root path (`/`). However some setups require to run Frigate under a custom path prefix (e.g. `/frigate`), especially when Frigate is located behind a reverse proxy that requires path-based routing.
@@ -242,7 +233,7 @@ To do this:
### Custom go2rtc version
Frigate currently includes go2rtc v1.9.10, there may be certain cases where you want to run a different version of go2rtc.
Frigate currently includes go2rtc v1.9.13, there may be certain cases where you want to run a different version of go2rtc.
To do this:
+2 -2
View File
@@ -86,7 +86,7 @@ Frigate looks for a JWT token secret in the following order:
1. An environment variable named `FRIGATE_JWT_SECRET`
2. A file named `FRIGATE_JWT_SECRET` in the directory specified by the `CREDENTIALS_DIRECTORY` environment variable (defaults to the Docker Secrets directory: `/run/secrets/`)
3. A `jwt_secret` option from the Home Assistant App options
3. A `jwt_secret` option from the Home Assistant Add-on options
4. A `.jwt_secret` file in the config directory
If no secret is found on startup, Frigate generates one and stores it in a `.jwt_secret` file in the config directory.
@@ -232,7 +232,7 @@ The viewer role provides read-only access to all cameras in the UI and API. Cust
### Role Configuration Example
```yaml {11-16}
```yaml
cameras:
front_door:
# ... camera config
+3 -6
View File
@@ -24,7 +24,7 @@ A custom icon can be added to the birdseye background by providing a 180x180 ima
If you want to include a camera in Birdseye view only for specific circumstances, or just don't include it at all, the Birdseye setting can be set at the camera level.
```yaml {8-10,12-14}
```yaml
# Include all cameras by default in Birdseye view
birdseye:
enabled: True
@@ -48,7 +48,6 @@ By default birdseye shows all cameras that have had the configured activity in t
```yaml
birdseye:
enabled: True
# highlight-next-line
inactivity_threshold: 15
```
@@ -79,11 +78,9 @@ birdseye:
cameras:
front:
birdseye:
# highlight-next-line
order: 1
back:
birdseye:
# highlight-next-line
order: 2
```
@@ -95,7 +92,7 @@ It is possible to limit the number of cameras shown on birdseye at one time. Whe
For example, this can be configured to only show the most recently active camera.
```yaml {3-4}
```yaml
birdseye:
enabled: True
layout:
@@ -106,7 +103,7 @@ birdseye:
By default birdseye tries to fit 2 cameras in each row and then double in size until a suitable layout is found. The scaling can be configured with a value between 1.0 and 5.0 depending on use case.
```yaml {3-4}
```yaml
birdseye:
enabled: True
layout:
+3 -5
View File
@@ -23,7 +23,6 @@ Some cameras support h265 with different formats, but Safari only supports the a
cameras:
h265_cam: # <------ Doesn't matter what the camera is called
ffmpeg:
# highlight-next-line
apple_compatibility: true # <- Adds compatibility with MacOS and iPhone
```
@@ -31,7 +30,7 @@ cameras:
Note that mjpeg cameras require encoding the video into h264 for recording, and restream roles. This will use significantly more CPU than if the cameras supported h264 feeds directly. It is recommended to use the restream role to create an h264 restream and then use that as the source for ffmpeg.
```yaml {3,10}
```yaml
go2rtc:
streams:
mjpeg_cam: "ffmpeg:http://your_mjpeg_stream_url#video=h264#hardware" # <- use hardware acceleration to create an h264 stream usable for other components.
@@ -97,7 +96,6 @@ This camera is H.265 only. To be able to play clips on some devices (like MacOs
cameras:
annkec800: # <------ Name the camera
ffmpeg:
# highlight-next-line
apple_compatibility: true # <- Adds compatibility with MacOS and iPhone
output_args:
record: preset-record-generic-audio-aac
@@ -246,7 +244,7 @@ go2rtc:
- rtspx://192.168.1.1:7441/abcdefghijk
```
[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#source-rtsp)
[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#source-rtsp)
In the Unifi 2.0 update Unifi Protect Cameras had a change in audio sample rate which causes issues for ffmpeg. The input rate needs to be set for record if used directly with unifi protect.
@@ -276,7 +274,7 @@ To use a USB camera (webcam) with Frigate, the recommendation is to use go2rtc's
- In your Frigate Configuration File, add the go2rtc stream and roles as appropriate:
```yaml {4,11-12}
```
go2rtc:
streams:
usb_camera:
+1 -1
View File
@@ -66,7 +66,7 @@ Not every PTZ supports ONVIF, which is the standard protocol Frigate uses to com
Add the onvif section to your camera in your configuration file:
```yaml {4-8}
```yaml
cameras:
back:
ffmpeg: ...
@@ -7,11 +7,11 @@ Object classification allows you to train a custom MobileNetV2 classification mo
## Minimum System Requirements
Object classification models are lightweight and run very fast on CPU.
Object classification models are lightweight and run very fast on CPU. Inference should be usable on virtually any machine that can run Frigate.
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
A CPU with AVX + AVX2 instructions is required for training and inference.
A CPU with AVX instructions is required for training and inference.
## Classes
@@ -27,6 +27,7 @@ For object classification:
### Classification Type
- **Sub label**:
- Applied to the objects `sub_label` field.
- Ideal for a single, more specific identity or type.
- Example: `cat``Leo`, `Charlie`, `None`.
@@ -118,7 +119,6 @@ Enable debug logs for classification models by adding `frigate.data_processing.r
logger:
default: info
logs:
# highlight-next-line
frigate.data_processing.real_time.custom_classification: debug
```
@@ -7,11 +7,11 @@ State classification allows you to train a custom MobileNetV2 classification mod
## Minimum System Requirements
State classification models are lightweight and run very fast on CPU.
State classification models are lightweight and run very fast on CPU. Inference should be usable on virtually any machine that can run Frigate.
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
A CPU with AVX + AVX2 instructions is required for training and inference.
A CPU with AVX instructions is required for training and inference.
## Classes
@@ -85,7 +85,6 @@ Enable debug logs for classification models by adding `frigate.data_processing.r
logger:
default: info
logs:
# highlight-next-line
frigate.data_processing.real_time.custom_classification: debug
```
+4 -3
View File
@@ -9,7 +9,7 @@ Face recognition identifies known individuals by matching detected faces with pr
### Face Detection
When running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/index.md#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
When running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
When running a default COCO model or another model that does not include `face` as a detectable label, face detection will run via CV2 using a lightweight DNN model that runs on the CPU. In this case, you should _not_ define `face` in your list of objects to track.
@@ -32,8 +32,6 @@ All of these features run locally on your system.
## Minimum System Requirements
A CPU with AVX + AVX2 instructions is required to run Face Recognition.
The `small` model is optimized for efficiency and runs on the CPU, most CPUs should run the model efficiently.
The `large` model is optimized for accuracy, an integrated or discrete GPU / NPU is required. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.
@@ -145,14 +143,17 @@ Start with the [Usage](#usage) section and re-read the [Model Requirements](#mod
1. Ensure `person` is being _detected_. A `person` will automatically be scanned by Frigate for a face. Any detected faces will appear in the Recent Recognitions tab in the Frigate UI's Face Library.
If you are using a Frigate+ or `face` detecting model:
- Watch the debug view (Settings --> Debug) to ensure that `face` is being detected along with `person`.
- You may need to adjust the `min_score` for the `face` object if faces are not being detected.
If you are **not** using a Frigate+ or `face` detecting model:
- Check your `detect` stream resolution and ensure it is sufficiently high enough to capture face details on `person` objects.
- You may need to lower your `detection_threshold` if faces are not being detected.
2. Any detected faces will then be _recognized_.
- Make sure you have trained at least one face per the recommendations above.
- Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration).
+168 -113
View File
@@ -5,39 +5,31 @@ title: Configuring Generative AI
## Configuration
A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently 3 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI section below.
A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently 4 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI-Compatible section below.
To use Generative AI, you must define a single provider at the global level of your Frigate configuration. If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
## Ollama
## Local Providers
Local providers run on your own hardware and keep all data processing private. These require a GPU or dedicated hardware for best performance.
:::warning
Using Ollama on CPU is not recommended, high inference times make using Generative AI impractical.
Running Generative AI models on CPU is not recommended, as high inference times make using Generative AI impractical.
:::
[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
### Recommended Local Models
Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available.
You must use a vision-capable model with Frigate. The following models are recommended for local deployment:
Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://docs.ollama.com/faq#how-does-ollama-handle-concurrent-requests).
### Model Types: Instruct vs Thinking
Most vision-language models are available as **instruct** models, which are fine-tuned to follow instructions and respond concisely to prompts. However, some models (such as certain Qwen-VL or minigpt variants) offer both **instruct** and **thinking** versions.
- **Instruct models** are always recommended for use with Frigate. These models generate direct, relevant, actionable descriptions that best fit Frigate's object and event summary use case.
- **Thinking models** are fine-tuned for more free-form, open-ended, and speculative outputs, which are typically not concise and may not provide the practical summaries Frigate expects. For this reason, Frigate does **not** recommend or support using thinking models.
Some models are labeled as **hybrid** (capable of both thinking and instruct tasks). In these cases, Frigate will always use instruct-style prompts and specifically disables thinking-mode behaviors to ensure concise, useful responses.
**Recommendation:**
Always select the `-instruct` or documented instruct/tagged variant of any model you use in your Frigate configuration. If in doubt, refer to your model providers documentation or model library for guidance on the correct model variant to use.
### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their model library](https://ollama.com/library). Note that Frigate will not automatically download the model you specify in your config, Ollama will try to download the model but it may take longer than the timeout, it is recommended to pull the model beforehand by running `ollama pull your_model` on your Ollama server/Docker container. Note that the model specified in Frigate's config must match the downloaded model tag.
| Model | Notes |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | Strong visual and situational understanding, strong ability to identify smaller objects and interactions with object. |
| `qwen3.5` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `Intern3.5VL` | Relatively fast with good vision comprehension |
| `gemma3` | Slower model with good vision and temporal understanding |
| `qwen2.5-vl` | Fast but capable model with good vision comprehension |
:::info
@@ -45,114 +37,85 @@ Each model is available in multiple parameter sizes (3b, 4b, 8b, etc.). Larger s
:::
:::note
You should have at least 8 GB of RAM available (or VRAM if running on GPU) to run the 7B models, 16 GB to run the 13B models, and 24 GB to run the 33B models.
:::
### Model Types: Instruct vs Thinking
Most vision-language models are available as **instruct** models, which are fine-tuned to follow instructions and respond concisely to prompts. However, some models (such as certain Qwen-VL or minigpt variants) offer both **instruct** and **thinking** versions.
- **Instruct models** are always recommended for use with Frigate. These models generate direct, relevant, actionable descriptions that best fit Frigate's object and event summary use case.
- **Reasoning / Thinking models** are fine-tuned for more free-form, open-ended, and speculative outputs, which are typically not concise and may not provide the practical summaries Frigate expects. For this reason, Frigate does **not** recommend or support using thinking models.
Some models are labeled as **hybrid** (capable of both thinking and instruct tasks). In these cases, it is recommended to disable reasoning / thinking, which is generally model specific (see your models documentation).
**Recommendation:**
Always select the `-instruct` or documented instruct/tagged variant of any model you use in your Frigate configuration. If in doubt, refer to your model provider's documentation or model library for guidance on the correct model variant to use.
### llama.cpp
[llama.cpp](https://github.com/ggml-org/llama.cpp) is a C++ implementation of LLaMA that provides a high-performance inference server.
It is highly recommended to host the llama.cpp server on a machine with a discrete graphics card, or on an Apple silicon Mac for best performance.
#### Supported Models
You must use a vision capable model with Frigate. The llama.cpp server supports various vision models in GGUF format.
#### Configuration
All llama.cpp native options can be passed through `provider_options`, including `temperature`, `top_k`, `top_p`, `min_p`, `repeat_penalty`, `repeat_last_n`, `seed`, `grammar`, and more. See the [llama.cpp server documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) for a complete list of available parameters.
```yaml
genai:
provider: llamacpp
base_url: http://localhost:8080
model: your-model-name
provider_options:
context_size: 16000 # Tell Frigate your context size so it can send the appropriate amount of information.
```
### Ollama
[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available.
Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://docs.ollama.com/faq#how-does-ollama-handle-concurrent-requests).
:::tip
If you are trying to use a single model for Frigate and HomeAssistant, it will need to support vision and tools calling. qwen3-VL supports vision and tools simultaneously in Ollama.
:::
The following models are recommended:
Note that Frigate will not automatically download the model you specify in your config. Ollama will try to download the model but it may take longer than the timeout, so it is recommended to pull the model beforehand by running `ollama pull your_model` on your Ollama server/Docker container. The model specified in Frigate's config must match the downloaded model tag.
| Model | Notes |
| ------------- | -------------------------------------------------------------------- |
| `qwen3-vl` | Strong visual and situational understanding, higher vram requirement |
| `Intern3.5VL` | Relatively fast with good vision comprehension |
| `gemma3` | Strong frame-to-frame understanding, slower inference times |
| `qwen2.5-vl` | Fast but capable model with good vision comprehension |
:::note
You should have at least 8 GB of RAM available (or VRAM if running on GPU) to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
:::
#### Ollama Cloud models
Ollama also supports [cloud models](https://ollama.com/cloud), where your local Ollama instance handles requests from Frigate, but model inference is performed in the cloud. Set up Ollama locally, sign in with your Ollama account, and specify the cloud model name in your Frigate config. For more details, see the Ollama cloud model [docs](https://docs.ollama.com/cloud).
### Configuration
#### Configuration
```yaml
genai:
provider: ollama
base_url: http://localhost:11434
model: qwen3-vl:4b
provider_options: # other Ollama client options can be defined
keep_alive: -1
options:
num_ctx: 8192 # make sure the context matches other services that are using ollama
```
## Google Gemini
### OpenAI-Compatible
Google Gemini has a [free tier](https://ai.google.dev/pricing) for the API, however the limits may not be sufficient for standard Frigate usage. Choose a plan appropriate for your installation.
### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://ai.google.dev/gemini-api/docs/models/gemini).
### Get API Key
To start using Gemini, you must first get an API key from [Google AI Studio](https://aistudio.google.com).
1. Accept the Terms of Service
2. Click "Get API Key" from the right hand navigation
3. Click "Create API key in new project"
4. Copy the API key for use in your config
### Configuration
```yaml
genai:
provider: gemini
api_key: "{FRIGATE_GEMINI_API_KEY}"
model: gemini-2.5-flash
```
:::note
To use a different Gemini-compatible API endpoint, set the `provider_options` with the `base_url` key to your provider's API URL. For example:
```yaml {4,5}
genai:
provider: gemini
...
provider_options:
base_url: https://...
```
Other HTTP options are available, see the [python-genai documentation](https://github.com/googleapis/python-genai).
:::
## OpenAI
OpenAI does not have a free tier for their API. With the release of gpt-4o, pricing has been reduced and each generation should cost fractions of a cent if you choose to go this route.
### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://platform.openai.com/docs/models).
### Get API Key
To start using OpenAI, you must first [create an API key](https://platform.openai.com/api-keys) and [configure billing](https://platform.openai.com/settings/organization/billing/overview).
### Configuration
```yaml
genai:
provider: openai
api_key: "{FRIGATE_OPENAI_API_KEY}"
model: gpt-4o
```
:::note
To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
:::
Frigate supports any provider that implements the OpenAI API standard. This includes self-hosted solutions like [vLLM](https://docs.vllm.ai/), [LocalAI](https://localai.io/), and other OpenAI-compatible servers.
:::tip
For OpenAI-compatible servers (such as llama.cpp) that don't expose the configured context size in the API response, you can manually specify the context size in `provider_options`:
```yaml {5,6}
```yaml
genai:
provider: openai
base_url: http://your-llama-server
@@ -165,19 +128,111 @@ This ensures Frigate uses the correct context window size when generating prompt
:::
## Azure OpenAI
#### Configuration
```yaml
genai:
provider: openai
base_url: http://your-server:port
api_key: your-api-key # May not be required for local servers
model: your-model-name
```
To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
## Cloud Providers
Cloud providers run on remote infrastructure and require an API key for authentication. These services handle all model inference on their servers.
### Ollama Cloud
Ollama also supports [cloud models](https://ollama.com/cloud), where your local Ollama instance handles requests from Frigate, but model inference is performed in the cloud. Set up Ollama locally, sign in with your Ollama account, and specify the cloud model name in your Frigate config. For more details, see the Ollama cloud model [docs](https://docs.ollama.com/cloud).
#### Configuration
```yaml
genai:
provider: ollama
base_url: http://localhost:11434
model: cloud-model-name
```
### Google Gemini
Google Gemini has a [free tier](https://ai.google.dev/pricing) for the API, however the limits may not be sufficient for standard Frigate usage. Choose a plan appropriate for your installation.
#### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://ai.google.dev/gemini-api/docs/models/gemini).
#### Get API Key
To start using Gemini, you must first get an API key from [Google AI Studio](https://aistudio.google.com).
1. Accept the Terms of Service
2. Click "Get API Key" from the right hand navigation
3. Click "Create API key in new project"
4. Copy the API key for use in your config
#### Configuration
```yaml
genai:
provider: gemini
api_key: "{FRIGATE_GEMINI_API_KEY}"
model: gemini-2.5-flash
```
:::note
To use a different Gemini-compatible API endpoint, set the `provider_options` with the `base_url` key to your provider's API URL. For example:
```
genai:
provider: gemini
...
provider_options:
base_url: https://...
```
Other HTTP options are available, see the [python-genai documentation](https://github.com/googleapis/python-genai).
:::
### OpenAI
OpenAI does not have a free tier for their API. With the release of gpt-4o, pricing has been reduced and each generation should cost fractions of a cent if you choose to go this route.
#### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://platform.openai.com/docs/models).
#### Get API Key
To start using OpenAI, you must first [create an API key](https://platform.openai.com/api-keys) and [configure billing](https://platform.openai.com/settings/organization/billing/overview).
#### Configuration
```yaml
genai:
provider: openai
api_key: "{FRIGATE_OPENAI_API_KEY}"
model: gpt-4o
```
### Azure OpenAI
Microsoft offers several vision models through Azure OpenAI. A subscription is required.
### Supported Models
#### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
### Create Resource and Get API Key
#### Create Resource and Get API Key
To start using Azure OpenAI, you must first [create a resource](https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource). You'll need your API key, model name, and resource URL, which must include the `api-version` parameter (see the example below).
### Configuration
#### Configuration
```yaml
genai:
+2 -2
View File
@@ -11,7 +11,7 @@ By default, descriptions will be generated for all tracked objects and all zones
Optionally, you can generate the description using a snapshot (if enabled) by setting `use_snapshot` to `True`. By default, this is set to `False`, which sends the uncompressed images from the `detect` stream collected over the object's lifetime to the model. Once the object lifecycle ends, only a single compressed and cropped thumbnail is saved with the tracked object. Using a snapshot might be useful when you want to _regenerate_ a tracked object's description as it will provide the AI with a higher-quality image (typically downscaled by the AI itself) than the cropped/compressed thumbnail. Using a snapshot otherwise has a trade-off in that only a single image is sent to your provider, which will limit the model's ability to determine object movement or direction.
Generative AI object descriptions can also be toggled dynamically for a camera via MQTT with the topic `frigate/<camera_name>/object_descriptions/set`. See the [MQTT documentation](/integrations/mqtt/#frigatecamera_nameobjectdescriptionsset).
Generative AI object descriptions can also be toggled dynamically for a camera via MQTT with the topic `frigate/<camera_name>/object_descriptions/set`. See the [MQTT documentation](/integrations/mqtt#frigatecamera_nameobject_descriptionsset).
## Usage and Best Practices
@@ -75,4 +75,4 @@ Many providers also have a public facing chat interface for their models. Downlo
- OpenAI - [ChatGPT](https://chatgpt.com)
- Gemini - [Google AI Studio](https://aistudio.google.com)
- Ollama - [Open WebUI](https://docs.openwebui.com/)
- Ollama - [Open WebUI](https://docs.openwebui.com/)
@@ -7,7 +7,7 @@ Generative AI can be used to automatically generate structured summaries of revi
Requests for a summary are requested automatically to your AI provider for alert review items when the activity has ended, they can also be optionally enabled for detections as well.
Generative AI review summaries can also be toggled dynamically for a [camera via MQTT](/integrations/mqtt/#frigatecamera_namereviewdescriptionsset).
Generative AI review summaries can also be toggled dynamically for a [camera via MQTT](/integrations/mqtt#frigatecamera_namereview_descriptionsset).
## Review Summary Usage and Best Practices
@@ -80,7 +80,6 @@ By default, review summaries use preview images (cached preview frames) which ha
review:
genai:
enabled: true
# highlight-next-line
image_source: recordings # Options: "preview" (default) or "recordings"
```
@@ -105,7 +104,7 @@ If recordings are not available for a given time period, the system will automat
Along with the concern of suspicious activity or immediate threat, you may have concerns such as animals in your garden or a gate being left open. These concerns can be configured so that the review summaries will make note of them if the activity requires additional review. For example:
```yaml {4,5}
```yaml
review:
genai:
enabled: true
@@ -117,7 +116,7 @@ review:
By default, review summaries are generated in English. You can configure Frigate to generate summaries in your preferred language by setting the `preferred_language` option:
```yaml {4}
```yaml
review:
genai:
enabled: true
@@ -10,7 +10,6 @@ import CommunityBadge from '@site/src/components/CommunityBadge';
It is highly recommended to use an integrated or discrete GPU for hardware acceleration video decoding in Frigate.
Some types of hardware acceleration are detected and used automatically, but you may need to update your configuration to enable hardware accelerated decoding in ffmpeg. To verify that hardware acceleration is working:
- Check the logs: A message will either say that hardware acceleration was automatically detected, or there will be a warning that no hardware acceleration was automatically detected
- If hardware acceleration is specified in the config, verification can be done by ensuring the logs are free from errors. There is no CPU fallback for hardware acceleration.
@@ -68,7 +67,7 @@ Frigate can utilize most Intel integrated GPUs and Arc GPUs to accelerate video
:::note
The default driver is `iHD`. You may need to change the driver to `i965` by adding the following environment variable `LIBVA_DRIVER_NAME=i965` to your docker-compose file or [in the `config.yml` for HA App users](advanced.md#environment_vars).
The default driver is `iHD`. You may need to change the driver to `i965` by adding the following environment variable `LIBVA_DRIVER_NAME=i965` to your docker-compose file or [in the `config.yml` for HA Add-on users](advanced.md#environment_vars).
See [The Intel Docs](https://www.intel.com/content/www/us/en/support/articles/000005505/processors.html) to figure out what generation your CPU is.
@@ -117,13 +116,12 @@ services:
frigate:
...
image: ghcr.io/blakeblackshear/frigate:stable
# highlight-next-line
privileged: true
```
##### Docker Run CLI - Privileged
```bash {4}
```bash
docker run -d \
--name frigate \
...
@@ -137,7 +135,7 @@ Only recent versions of Docker support the `CAP_PERFMON` capability. You can tes
##### Docker Compose - CAP_PERFMON
```yaml {5,6}
```yaml
services:
frigate:
...
@@ -148,7 +146,7 @@ services:
##### Docker Run CLI - CAP_PERFMON
```bash {4}
```bash
docker run -d \
--name frigate \
...
@@ -190,7 +188,7 @@ Frigate can utilize modern AMD integrated GPUs and AMD GPUs to accelerate video
### Configuring Radeon Driver
You need to change the driver to `radeonsi` by adding the following environment variable `LIBVA_DRIVER_NAME=radeonsi` to your docker-compose file or [in the `config.yml` for HA App users](advanced.md#environment_vars).
You need to change the driver to `radeonsi` by adding the following environment variable `LIBVA_DRIVER_NAME=radeonsi` to your docker-compose file or [in the `config.yml` for HA Add-on users](advanced.md#environment_vars).
### Via VAAPI
@@ -215,7 +213,7 @@ Additional configuration is needed for the Docker container to be able to access
#### Docker Compose - Nvidia GPU
```yaml {5-12}
```yaml
services:
frigate:
...
@@ -232,7 +230,7 @@ services:
#### Docker Run CLI - Nvidia GPU
```bash {4}
```bash
docker run -d \
--name frigate \
...
@@ -294,7 +292,7 @@ These instructions were originally based on the [Jellyfin documentation](https:/
## Raspberry Pi 3/4
Ensure you increase the allocated RAM for your GPU to at least 128 (`raspi-config` > Performance Options > GPU Memory).
If you are using the HA App, you may need to use the full access variant and turn off _Protection mode_ for hardware acceleration.
If you are using the HA Add-on, you may need to use the full access variant and turn off _Protection mode_ for hardware acceleration.
```yaml
# if you want to decode a h264 stream
@@ -311,7 +309,7 @@ ffmpeg:
If running Frigate through Docker, you either need to run in privileged mode or
map the `/dev/video*` devices to Frigate. With Docker Compose add:
```yaml {4-5}
```yaml
services:
frigate:
...
@@ -321,7 +319,7 @@ services:
Or with `docker run`:
```bash {4}
```bash
docker run -d \
--name frigate \
...
@@ -353,7 +351,7 @@ You will need to use the image with the nvidia container runtime:
### Docker Run CLI - Jetson
```bash {3}
```bash
docker run -d \
...
--runtime nvidia
@@ -362,7 +360,7 @@ docker run -d \
### Docker Compose - Jetson
```yaml {5}
```yaml
services:
frigate:
...
@@ -453,14 +451,14 @@ Restarting ffmpeg...
you should try to uprade to FFmpeg 7. This can be done using this config option:
```yaml
```
ffmpeg:
path: "7.0"
```
You can set this option globally to use FFmpeg 7 for all cameras or on camera level to use it only for specific cameras. Do not confuse this option with:
```yaml
```
cameras:
name:
ffmpeg:
@@ -482,7 +480,7 @@ Make sure to follow the [Synaptics specific installation instructions](/frigate/
Add one of the following FFmpeg presets to your `config.yml` to enable hardware video processing:
```yaml {2}
```yaml
ffmpeg:
hwaccel_args: -c:v h264_v4l2m2m
input_args: preset-rtsp-restream
+21 -13
View File
@@ -3,7 +3,7 @@ id: index
title: Frigate Configuration
---
For Home Assistant App installations, the config file should be at `/addon_configs/<addon_directory>/config.yml`, where `<addon_directory>` is specific to the variant of the Frigate App you are running. See the list of directories [here](#accessing-app-config-dir).
For Home Assistant Add-on installations, the config file should be at `/addon_configs/<addon_directory>/config.yml`, where `<addon_directory>` is specific to the variant of the Frigate Add-on you are running. See the list of directories [here](#accessing-add-on-config-dir).
For all other installation types, the config file should be mapped to `/config/config.yml` inside the container.
@@ -25,11 +25,11 @@ cameras:
- detect
```
## Accessing the Home Assistant App configuration directory {#accessing-app-config-dir}
## Accessing the Home Assistant Add-on configuration directory {#accessing-add-on-config-dir}
When running Frigate through the HA App, the Frigate `/config` directory is mapped to `/addon_configs/<addon_directory>` in the host, where `<addon_directory>` is specific to the variant of the Frigate App you are running.
When running Frigate through the HA Add-on, the Frigate `/config` directory is mapped to `/addon_configs/<addon_directory>` in the host, where `<addon_directory>` is specific to the variant of the Frigate Add-on you are running.
| App Variant | Configuration directory |
| Add-on Variant | Configuration directory |
| -------------------------- | ----------------------------------------- |
| Frigate | `/addon_configs/ccab4aaf_frigate` |
| Frigate (Full Access) | `/addon_configs/ccab4aaf_frigate-fa` |
@@ -38,11 +38,11 @@ When running Frigate through the HA App, the Frigate `/config` directory is mapp
**Whenever you see `/config` in the documentation, it refers to this directory.**
If for example you are running the standard App variant and use the [VS Code App](https://github.com/hassio-addons/addon-vscode) to browse your files, you can click _File_ > _Open folder..._ and navigate to `/addon_configs/ccab4aaf_frigate` to access the Frigate `/config` directory and edit the `config.yaml` file. You can also use the built-in file editor in the Frigate UI to edit the configuration file.
If for example you are running the standard Add-on variant and use the [VS Code Add-on](https://github.com/hassio-addons/addon-vscode) to browse your files, you can click _File_ > _Open folder..._ and navigate to `/addon_configs/ccab4aaf_frigate` to access the Frigate `/config` directory and edit the `config.yaml` file. You can also use the built-in file editor in the Frigate UI to edit the configuration file.
## VS Code Configuration Schema
VS Code supports JSON schemas for automatically validating configuration files. You can enable this feature by adding `# yaml-language-server: $schema=http://frigate_host:5000/api/config/schema.json` to the beginning of the configuration file. Replace `frigate_host` with the IP address or hostname of your Frigate server. If you're using both VS Code and Frigate as an App, you should use `ccab4aaf-frigate` instead. Make sure to expose the internal unauthenticated port `5000` when accessing the config from VS Code on another machine.
VS Code supports JSON schemas for automatically validating configuration files. You can enable this feature by adding `# yaml-language-server: $schema=http://frigate_host:5000/api/config/schema.json` to the beginning of the configuration file. Replace `frigate_host` with the IP address or hostname of your Frigate server. If you're using both VS Code and Frigate as an Add-on, you should use `ccab4aaf-frigate` instead. Make sure to expose the internal unauthenticated port `5000` when accessing the config from VS Code on another machine.
## Environment Variable Substitution
@@ -50,7 +50,6 @@ Frigate supports the use of environment variables starting with `FRIGATE_` **onl
```yaml
mqtt:
host: "{FRIGATE_MQTT_HOST}"
user: "{FRIGATE_MQTT_USER}"
password: "{FRIGATE_MQTT_PASSWORD}"
```
@@ -61,7 +60,7 @@ mqtt:
```yaml
onvif:
host: "192.168.1.12"
host: 10.0.10.10
port: 8000
user: "{FRIGATE_RTSP_USER}"
password: "{FRIGATE_RTSP_PASSWORD}"
@@ -83,10 +82,10 @@ genai:
Here are some common starter configuration examples. Refer to the [reference config](./reference.md) for detailed information about all the config values.
### Raspberry Pi Home Assistant App with USB Coral
### Raspberry Pi Home Assistant Add-on with USB Coral
- Single camera with 720p, 5fps stream for detect
- MQTT connected to the Home Assistant Mosquitto App
- MQTT connected to the Home Assistant Mosquitto Add-on
- Hardware acceleration for decoding video
- USB Coral detector
- Save all video with any detectable motion for 7 days regardless of whether any objects were detected or not
@@ -139,7 +138,10 @@ cameras:
- detect
motion:
mask:
- 0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400
timestamp:
friendly_name: "Camera timestamp"
enabled: true
coordinates: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400"
```
### Standalone Intel Mini PC with USB Coral
@@ -196,7 +198,10 @@ cameras:
- detect
motion:
mask:
- 0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400
timestamp:
friendly_name: "Camera timestamp"
enabled: true
coordinates: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400"
```
### Home Assistant integrated Intel Mini PC with OpenVino
@@ -263,5 +268,8 @@ cameras:
- detect
motion:
mask:
- 0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400
timestamp:
friendly_name: "Camera timestamp"
enabled: true
coordinates: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400"
```
@@ -30,7 +30,7 @@ In the default mode, Frigate's LPR needs to first detect a `car` or `motorcycle`
## Minimum System Requirements
License plate recognition works by running AI models locally on your system. The YOLOv9 plate detector model and the OCR models ([PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)) are relatively lightweight and can run on your CPU or GPU, depending on your configuration. At least 4GB of RAM and a CPU with AVX + AVX2 instructions is required.
License plate recognition works by running AI models locally on your system. The YOLOv9 plate detector model and the OCR models ([PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)) are relatively lightweight and can run on your CPU or GPU, depending on your configuration. At least 4GB of RAM is required.
## Configuration
@@ -43,7 +43,7 @@ lpr:
Like other enrichments in Frigate, LPR **must be enabled globally** to use the feature. You should disable it for specific cameras at the camera level if you don't want to run LPR on cars on those cameras:
```yaml {4,5}
```yaml
cameras:
garage:
...
@@ -375,6 +375,7 @@ Use `match_distance` to allow small character mismatches. Alternatively, define
Start with ["Why isn't my license plate being detected and recognized?"](#why-isnt-my-license-plate-being-detected-and-recognized). If you are still having issues, work through these steps.
1. Start with a simplified LPR config.
- Remove or comment out everything in your LPR config, including `min_area`, `min_plate_length`, `format`, `known_plates`, or `enhancement` values so that the only values left are `enabled` and `debug_save_plates`. This will run LPR with Frigate's default values.
```yaml
@@ -385,28 +386,31 @@ Start with ["Why isn't my license plate being detected and recognized?"](#why-is
```
2. Enable debug logs to see exactly what Frigate is doing.
- Enable debug logs for LPR by adding `frigate.data_processing.common.license_plate: debug` to your `logger` configuration. These logs are _very_ verbose, so only keep this enabled when necessary. Restart Frigate after this change.
```yaml
logger:
default: info
logs:
# highlight-next-line
frigate.data_processing.common.license_plate: debug
```
3. Ensure your plates are being _detected_.
If you are using a Frigate+ or `license_plate` detecting model:
- Watch the debug view (Settings --> Debug) to ensure that `license_plate` is being detected.
- View MQTT messages for `frigate/events` to verify detected plates.
- You may need to adjust your `min_score` and/or `threshold` for the `license_plate` object if your plates are not being detected.
If you are **not** using a Frigate+ or `license_plate` detecting model:
- Watch the debug logs for messages from the YOLOv9 plate detector.
- You may need to adjust your `detection_threshold` if your plates are not being detected.
4. Ensure the characters on detected plates are being _recognized_.
- Enable `debug_save_plates` to save images of detected text on plates to the clips directory (`/media/frigate/clips/lpr`). Ensure these images are readable and the text is clear.
- Watch the debug view to see plates recognized in real-time. For non-dedicated LPR cameras, the `car` or `motorcycle` label will change to the recognized plate when LPR is enabled and working.
- Adjust `recognition_threshold` settings per the suggestions [above](#advanced-configuration).
+19 -8
View File
@@ -15,7 +15,7 @@ The jsmpeg live view will use more browser and client GPU resources. Using go2rt
| ------ | ------------------------------------- | ---------- | ---------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| jsmpeg | same as `detect -> fps`, capped at 10 | 720p | no | no | Resolution is configurable, but go2rtc is recommended if you want higher resolutions and better frame rates. jsmpeg is Frigate's default without go2rtc configured. |
| mse | native | native | yes (depends on audio codec) | yes | iPhone requires iOS 17.1+, Firefox is h.264 only. This is Frigate's default when go2rtc is configured. |
| webrtc | native | native | yes (depends on audio codec) | yes | Requires extra configuration. Frigate attempts to use WebRTC when MSE fails or when using a camera's two-way talk feature. |
| webrtc | native | native | yes (depends on audio codec) | yes | Requires extra configuration. Frigate attempts to use WebRTC when MSE fails or when using a camera's two-way talk feature. |
### Camera Settings Recommendations
@@ -77,7 +77,7 @@ Configure the `streams` option with a "friendly name" for your stream followed b
Using Frigate's internal version of go2rtc is required to use this feature. You cannot specify paths in the `streams` configuration, only go2rtc stream names.
```yaml {3,6,8,25-29}
```yaml
go2rtc:
streams:
test_cam:
@@ -114,9 +114,9 @@ cameras:
WebRTC works by creating a TCP or UDP connection on port `8555`. However, it requires additional configuration:
- For external access, over the internet, setup your router to forward port `8555` to port `8555` on the Frigate device, for both TCP and UDP.
- For internal/local access, unless you are running through the HA App, you will also need to set the WebRTC candidates list in the go2rtc config. For example, if `192.168.1.10` is the local IP of the device running Frigate:
- For internal/local access, unless you are running through the HA Add-on, you will also need to set the WebRTC candidates list in the go2rtc config. For example, if `192.168.1.10` is the local IP of the device running Frigate:
```yaml title="config.yml" {4-7}
```yaml title="config.yml"
go2rtc:
streams:
test_cam: ...
@@ -128,13 +128,13 @@ WebRTC works by creating a TCP or UDP connection on port `8555`. However, it req
- For access through Tailscale, the Frigate system's Tailscale IP must be added as a WebRTC candidate. Tailscale IPs all start with `100.`, and are reserved within the `100.64.0.0/10` CIDR block.
- Note that some browsers may not support H.265 (HEVC). You can check your browser's current version for H.265 compatibility [here](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness).
- Note that some browsers may not support H.265 (HEVC). You can check your browser's current version for H.265 compatibility [here](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness).
:::tip
This extra configuration may not be required if Frigate has been installed as a Home Assistant App, as Frigate uses the Supervisor's API to generate a WebRTC candidate.
This extra configuration may not be required if Frigate has been installed as a Home Assistant Add-on, as Frigate uses the Supervisor's API to generate a WebRTC candidate.
However, it is recommended if issues occur to define the candidates manually. You should do this if the Frigate App fails to generate a valid candidate. If an error occurs you will see some warnings like the below in the App logs page during the initialization:
However, it is recommended if issues occur to define the candidates manually. You should do this if the Frigate Add-on fails to generate a valid candidate. If an error occurs you will see some warnings like the below in the Add-on logs page during the initialization:
```log
[WARN] Failed to get IP address from supervisor
@@ -154,7 +154,7 @@ If not running in host mode, port 8555 will need to be mapped for the container:
docker-compose.yml
```yaml {4-6}
```yaml
services:
frigate:
...
@@ -222,28 +222,34 @@ Note that disabling a camera through the config file (`enabled: False`) removes
When your browser runs into problems playing back your camera streams, it will log short error messages to the browser console. They indicate playback, codec, or network issues on the client/browser side, not something server side with Frigate itself. Below are the common messages you may see and simple actions you can take to try to resolve them.
- **startup**
- What it means: The player failed to initialize or connect to the live stream (network or startup error).
- What to try: Reload the Live view or click _Reset_. Verify `go2rtc` is running and the camera stream is reachable. Try switching to a different stream from the Live UI dropdown (if available) or use a different browser.
- Possible console messages from the player code:
- `Error opening MediaSource.`
- `Browser reported a network error.`
- `Max error count ${errorCount} exceeded.` (the numeric value will vary)
- **mse-decode**
- What it means: The browser reported a decoding error while trying to play the stream, which usually is a result of a codec incompatibility or corrupted frames.
- What to try: Check the browser console for the supported and negotiated codecs. Ensure your camera/restream is using H.264 video and AAC audio (these are the most compatible). If your camera uses a non-standard audio codec, configure `go2rtc` to transcode the stream to AAC. Try another browser (some browsers have stricter MSE/codec support) and, for iPhone, ensure you're on iOS 17.1 or newer.
- Possible console messages from the player code:
- `Safari cannot open MediaSource.`
- `Safari reported InvalidStateError.`
- `Safari reported decoding errors.`
- **stalled**
- What it means: Playback has stalled because the player has fallen too far behind live (extended buffering or no data arriving).
- What to try: This is usually indicative of the browser struggling to decode too many high-resolution streams at once. Try selecting a lower-bandwidth stream (substream), reduce the number of live streams open, improve the network connection, or lower the camera resolution. Also check your camera's keyframe (I-frame) interval — shorter intervals make playback start and recover faster. You can also try increasing the timeout value in the UI pane of Frigate's settings.
- Possible console messages from the player code:
- `Buffer time (10 seconds) exceeded, browser may not be playing media correctly.`
- `Media playback has stalled after <n> seconds due to insufficient buffering or a network interruption.` (the seconds value will vary)
@@ -264,18 +270,21 @@ When your browser runs into problems playing back your camera streams, it will l
If you are using continuous streaming or you are loading more than a few high resolution streams at once on the dashboard, your browser may struggle to begin playback of your streams before the timeout. Frigate always prioritizes showing a live stream as quickly as possible, even if it is a lower quality jsmpeg stream. You can use the "Reset" link/button to try loading your high resolution stream again.
Errors in stream playback (e.g., connection failures, codec issues, or buffering timeouts) that cause the fallback to low bandwidth mode (jsmpeg) are logged to the browser console for easier debugging. These errors may include:
- Network issues (e.g., MSE or WebRTC network connection problems).
- Unsupported codecs or stream formats (e.g., H.265 in WebRTC, which is not supported in some browsers).
- Buffering timeouts or low bandwidth conditions causing fallback to jsmpeg.
- Browser compatibility problems (e.g., iOS Safari limitations with MSE).
To view browser console logs:
1. Open the Frigate Live View in your browser.
2. Open the browser's Developer Tools (F12 or right-click > Inspect > Console tab).
3. Reproduce the error (e.g., load a problematic stream or simulate network issues).
4. Look for messages prefixed with the camera name.
These logs help identify if the issue is player-specific (MSE vs. WebRTC) or related to camera configuration (e.g., go2rtc streams, codecs). If you see frequent errors:
- Verify your camera's H.264/AAC settings (see [Frigate's camera settings recommendations](#camera_settings_recommendations)).
- Check go2rtc configuration for transcoding (e.g., audio to AAC/OPUS).
- Test with a different stream via the UI dropdown (if `live -> streams` is configured).
@@ -315,7 +324,9 @@ When your browser runs into problems playing back your camera streams, it will l
To prevent this, make the `detect` stream match the go2rtc live stream's aspect ratio (resolution does not need to match, just the aspect ratio). You can either adjust the camera's output resolution or set the `width` and `height` values in your config's `detect` section to a resolution with an aspect ratio that matches.
Example: Resolutions from two streams
- Mismatched (may cause aspect ratio switching on the dashboard):
- Live/go2rtc stream: 1920x1080 (16:9)
- Detect stream: 640x352 (~1.82:1, not 16:9)
+41 -26
View File
@@ -3,8 +3,6 @@ id: masks
title: Masks
---
Frigate has two kinds of masks: motion masks and object filter masks. Both are narrow tools for fine-tuning, **not for hiding an area from Frigate**. Masks should be used sparingly; in most cases where users reach for one, a [zone](zones.md) with `required_zones` is the right tool instead. See [Which tool do I need?](#which-tool-do-i-need) and [Common mistakes](#common-mistakes) below if you're new to Frigate's mask behavior.
## Motion masks
Motion masks are used to prevent unwanted types of motion from triggering detection. Try watching the Debug feed (Settings --> Debug) with `Motion Boxes` enabled to see what may be regularly detected as motion. For example, you want to mask out your timestamp, the sky, rooftops, etc. Keep in mind that this mask only prevents motion from being detected and does not prevent objects from being detected if object detection was started due to motion in unmasked areas. Motion is also used during object tracking to refine the object detection area in the next frame. _Over-masking will make it more difficult for objects to be tracked._
@@ -19,15 +17,6 @@ Object filter masks can be used to filter out stubborn false positives in fixed
![object mask](/img/bottom-center-mask.jpg)
## Which tool do I need?
| What you're trying to do | Recommended tool | How it works |
| ------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Don't get alerts or recordings for activity in an area (e.g., the sidewalk in front of your house) | A [zone](zones.md) combined with `review.alerts.required_zones` (and/or `review.detections.required_zones`) | Frigate keeps detecting and tracking activity in the area, but a review item is only created once the bottom-center of an object's bounding box enters a required zone. |
| Stop a stubborn false positive at a specific fixed spot (e.g., a tree base that keeps being detected as a person) | An **object filter mask** for that object type | Any detection of that object type whose bounding-box bottom-center lands inside the mask is treated as a false positive and discarded. |
| Ignore motion in an area that obviously isn't an object of interest (e.g., the camera timestamp, sky, flags, treetops swaying) | A **motion mask** | Motion inside the mask is ignored when deciding whether to run object detection. Objects can still be detected in a motion masked area if motion elsewhere in the frame triggers detection. |
| Stop tracking an object type altogether on this camera (e.g., you never care about cats) | Remove the object from the camera's [`objects.track`](objects.md) list | Frigate skips this object type entirely on this camera, regardless of where it appears. |
## Using the mask creator
To create a poly mask:
@@ -44,18 +33,55 @@ Your config file will be updated with the relative coordinates of the mask/zone:
```yaml
motion:
mask: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400"
mask:
# Motion mask name (required)
mask1:
# Optional: A friendly name for the mask
friendly_name: "Timestamp area"
# Optional: Whether this mask is active (default: true)
enabled: true
# Required: Coordinates polygon for the mask
coordinates: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456,0.700,0.424,0.701,0.311,0.507,0.294,0.453,0.347,0.451,0.400"
```
Multiple masks can be listed in your config.
Multiple motion masks can be listed in your config:
```yaml
motion:
mask:
- 0.239,1.246,0.175,0.901,0.165,0.805,0.195,0.802
- 0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456
mask1:
friendly_name: "Timestamp area"
enabled: true
coordinates: "0.239,1.246,0.175,0.901,0.165,0.805,0.195,0.802"
mask2:
friendly_name: "Tree area"
enabled: true
coordinates: "0.000,0.427,0.002,0.000,0.999,0.000,0.999,0.781,0.885,0.456"
```
Object filter masks can also be created through the UI or manually in the config. They are configured under the object filters section for each object type:
```yaml
objects:
filters:
person:
mask:
person_filter1:
friendly_name: "Roof area"
enabled: true
coordinates: "0.000,0.000,1.000,0.000,1.000,0.400,0.000,0.400"
car:
mask:
car_filter1:
friendly_name: "Sidewalk area"
enabled: true
coordinates: "0.000,0.700,1.000,0.700,1.000,1.000,0.000,1.000"
```
## Enabling/Disabling Masks
Both motion masks and object filter masks can be toggled on or off without removing them from the configuration. Disabled masks are completely ignored at runtime - they will not affect motion detection or object filtering. This is useful for temporarily disabling a mask during certain seasons or times of day without modifying the configuration.
### Further Clarification
This is a response to a [question posed on reddit](https://www.reddit.com/r/homeautomation/comments/ppxdve/replacing_my_doorbell_with_a_security_camera_a_6/hd876w4?utm_source=share&utm_medium=web2x&context=3):
@@ -93,14 +119,3 @@ This is what `required_zones` are for. You should define a zone (remember this i
> Maybe my specific situation just warrants this. I've just been having a hard time understanding the relevance of this information - it seems to be that it's exactly what would be expected when "masking out" an area of ANY image.
That may be the case for you. Frigate will definitely work harder tracking people on the sidewalk to make sure it doesn't miss anyone who steps foot on your stoop. The trade off with the way you have it now is slower recognition of objects and potential misses. That may be acceptable based on your needs. Also, if your resolution is low enough on the detect stream, your regions may already be so big that they grab the entire object anyway.
## Common mistakes
**"I added a motion mask to ignore my driveway/sidewalk."**
A motion mask doesn't hide an area from Frigate. Objects can still be detected and tracked inside a masked area. The mask only stops motion _in that area_ from triggering object detection. If you want activity on the sidewalk to never produce a review item, define a [zone](zones.md) over the area you DO care about (your stoop, your driveway) and add it to `review.alerts.required_zones`. Frigate will still see people on the sidewalk, but it won't create an alert until they cross into the zone.
**"I added an object filter mask because I don't care about cars in my yard."**
Object filter masks are for stubborn false positives at fixed locations, not for filtering whole areas or whole object types. If you only want alerts when a car enters the driveway, use a [zone](zones.md) with `required_zones`. If you don't care about a whole object type on this camera, remove it from [`objects.track`](objects.md).
**"I masked everything except a thin strip on my stoop."**
Heavy masking hurts tracking. Frigate uses motion near a tracked object's previous bounding box to decide where to look in the next frame; with most of the frame masked, an object walking from an unmasked area into a masked one effectively disappears and gets picked up as a "new" object when it reappears. For example: someone walks down your sidewalk, stops under a tree (masked area) to tie their shoe, then continues. Frigate sees that as two separate people and can create two separate review items. Because Frigate needs several consecutive frames above the confidence threshold to commit to a detection, each re-appearance can also delay or miss alerts. Use `required_zones` for "only alert me about this spot" and leave the surrounding area unmasked so tracking stays intact.
+23 -133
View File
@@ -161,7 +161,7 @@ YOLOv9 models that are compiled for TensorFlow Lite and properly quantized are s
:::tip
**Frigate+ Users:** Follow the [instructions](/integrations/plus#use-models) to set a model ID in your config file.
**Frigate+ Users:** Follow the [instructions](../integrations/plus#use-models) to set a model ID in your config file.
:::
@@ -330,7 +330,7 @@ detectors:
| [YOLO-NAS](#yolo-nas) | ✅ | ✅ | |
| [MobileNet v2](#ssdlite-mobilenet-v2) | ✅ | ✅ | Fast and lightweight model, less accurate than larger models |
| [YOLOX](#yolox) | ✅ | ? | |
| [D-FINE / DEIMv2](#d-fine--deimv2) | ❌ | ❌ | |
| [D-FINE](#d-fine) | ❌ | ❌ | |
#### SSDLite MobileNet v2
@@ -464,13 +464,13 @@ model:
</details>
#### D-FINE / DEIMv2
#### D-FINE
[D-FINE](https://github.com/Peterande/D-FINE) and [DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) are DETR based models that share the same ONNX input/output format. The ONNX exported models are supported, but not included by default. See the models section for downloading [D-FINE](#downloading-d-fine-model) or [DEIMv2](#downloading-deimv2-model) for use in Frigate.
[D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.
:::warning
Currently D-FINE / DEIMv2 models only run on OpenVINO in CPU mode, GPUs currently fail to compile the model
Currently D-FINE models only run on OpenVINO in CPU mode, GPUs currently fail to compile the model
:::
@@ -499,31 +499,6 @@ Note that the labelmap uses a subset of the complete COCO label set that has onl
</details>
<details>
<summary>DEIMv2 Setup & Config</summary>
After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:
```yaml
detectors:
ov:
type: openvino
device: CPU
model:
model_type: dfine
width: 640
height: 640
input_tensor: nchw
input_dtype: float
path: /config/model_cache/deimv2_hgnetv2_n.onnx
labelmap_path: /labelmap/coco-80.txt
```
Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.
</details>
## Apple Silicon detector
The NPU in Apple Silicon can't be accessed from within a container, so the [Apple Silicon detector client](https://github.com/frigate-nvr/apple-silicon-detector) must first be setup. It is recommended to use the Frigate docker image with `-standard-arm64` suffix, for example `ghcr.io/blakeblackshear/frigate:stable-standard-arm64`.
@@ -597,7 +572,7 @@ $ docker run --device=/dev/kfd --device=/dev/dri \
When using Docker Compose:
```yaml {4-6}
```yaml
services:
frigate:
...
@@ -628,7 +603,7 @@ $ docker run -e HSA_OVERRIDE_GFX_VERSION=10.0.0 \
When using Docker Compose:
```yaml {4-5}
```yaml
services:
frigate:
...
@@ -673,7 +648,7 @@ The AMD GPU kernel is known problematic especially when converting models to mxr
See [ONNX supported models](#supported-models) for supported models, there are some caveats:
- D-FINE / DEIMv2 models are not supported
- D-FINE models are not supported
- YOLO-NAS models are known to not run well on integrated GPUs
## ONNX
@@ -718,7 +693,7 @@ detectors:
| [RF-DETR](#rf-detr) | ✅ | ❌ | Supports CUDA Graphs for optimal Nvidia performance |
| [YOLO-NAS](#yolo-nas-1) | ⚠️ | ⚠️ | Not supported by CUDA Graphs |
| [YOLOX](#yolox-1) | ✅ | ✅ | Supports CUDA Graphs for optimal Nvidia performance |
| [D-FINE / DEIMv2](#d-fine--deimv2-1) | ⚠️ | ❌ | Not supported by CUDA Graphs |
| [D-FINE](#d-fine) | ⚠️ | ❌ | Not supported by CUDA Graphs |
There is no default model provided, the following formats are supported:
@@ -847,9 +822,9 @@ model:
</details>
#### D-FINE / DEIMv2
#### D-FINE
[D-FINE](https://github.com/Peterande/D-FINE) and [DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) are DETR based models that share the same ONNX input/output format. The ONNX exported models are supported, but not included by default. See the models section for downloading [D-FINE](#downloading-d-fine-model) or [DEIMv2](#downloading-deimv2-model) for use in Frigate.
[D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.
<details>
<summary>D-FINE Setup & Config</summary>
@@ -873,28 +848,6 @@ model:
</details>
<details>
<summary>DEIMv2 Setup & Config</summary>
After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:
```yaml
detectors:
onnx:
type: onnx
model:
model_type: dfine
width: 640
height: 640
input_tensor: nchw
input_dtype: float
path: /config/model_cache/deimv2_hgnetv2_n.onnx
labelmap_path: /labelmap/coco-80.txt
```
</details>
Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.
## CPU Detector (not recommended)
@@ -994,7 +947,7 @@ MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the
#### YOLO-NAS
The [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model included in this detector is downloaded from the [Models Section](#downloading-yolo-nas-model) and compiled to DFP with [mx_nc](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage).
The [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model included in this detector is downloaded from the [Models Section](#downloading-yolo-nas-model) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
**Note:** The default model for the MemryX detector is YOLO-NAS 320x320.
@@ -1028,7 +981,7 @@ model:
#### YOLOv9
The YOLOv9s model included in this detector is downloaded from [the original GitHub](https://github.com/WongKinYiu/yolov9) like in the [Models Section](#yolov9-1) and compiled to DFP with [mx_nc](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage).
The YOLOv9s model included in this detector is downloaded from [the original GitHub](https://github.com/WongKinYiu/yolov9) like in the [Models Section](#yolov9-1) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
##### Configuration
@@ -1110,39 +1063,19 @@ model:
#### Using a Custom Model
To use your own custom model, first compile it into a [.dfp](https://developer.memryx.com/2p1/specs/files.html#dataflow-program) file, which is the format used by MemryX.
To use your own model:
#### Compile the Model
1. Package your compiled model into a `.zip` file.
Custom models must be compiled using **MemryX SDK 2.1**.
2. The `.zip` must contain the compiled `.dfp` file.
Before compiling your model, install the MemryX Neural Compiler tools from the
[Install Tools](https://developer.memryx.com/2p1/get_started/install_tools.html) page on the **host**.
3. Depending on the model, the compiler may also generate a cropped post-processing network. If present, it will be named with the suffix `_post.onnx`.
> **Note:** It is recommended to compile the model on the host machine, or on another separate machine, rather than inside the Frigate Docker container. Installing the compiler inside Docker may conflict with container packages. It is recommended to create a Python virtual environment and install the compiler there.
4. Bind-mount the `.zip` file into the container and specify its path using `model.path` in your config.
Once the SDK 2.1 environment is set up, follow the
[MemryX Compiler](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage) documentation to compile your model.
5. Update the `labelmap_path` to match your custom model's labels.
Example:
```bash
mx_nc -m yolonas.onnx -c 4 --autocrop -v --dfp_fname yolonas.dfp
```
For detailed instructions on compiling models, refer to the [MemryX Compiler](https://developer.memryx.com/2p1/tools/neural_compiler.html#usage) docs and [Tutorials](https://developer.memryx.com/2p1/tutorials/tutorials.html).
#### Package the Compiled Model
1. Package your compiled model into a `.zip` file.
2. The `.zip` file must contain the compiled `.dfp` file.
3. Depending on the model, the compiler may also generate a cropped post-processing network. If present, it will be named with the suffix `_post.onnx`.
4. Bind-mount the `.zip` file into the container and specify its path using `model.path` in your config.
5. Update `labelmap_path` to match your custom model's labels.
For detailed instructions on compiling models, refer to the [MemryX Compiler](https://developer.memryx.com/tools/neural_compiler.html#usage) docs and [Tutorials](https://developer.memryx.com/tutorials/tutorials.html).
```yaml
# The detector automatically selects the default model if nothing is provided in the config.
@@ -1579,49 +1512,6 @@ COPY --from=build /dfine/output/dfine_${MODEL_SIZE}_obj2coco.onnx /dfine-${MODEL
EOF
```
### Downloading DEIMv2 Model
[DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2) can be exported as ONNX by running the command below. Pretrained weights are available on Hugging Face for two backbone families:
- **HGNetv2** (smaller/faster): `atto`, `femto`, `pico`, `n`
- **DINOv3** (larger/more accurate): `s`, `m`, `l`, `x`
Set `BACKBONE` and `MODEL_SIZE` in the first line to match your desired variant. Hugging Face model names use uppercase (e.g. `HGNetv2_N`, `DINOv3_S`), while config files use lowercase (e.g. `hgnetv2_n`, `dinov3_s`).
```sh
docker build . --rm --build-arg BACKBONE=hgnetv2 --build-arg MODEL_SIZE=n --output . -f- <<'EOF'
FROM python:3.11-slim AS build
RUN apt-get update && apt-get install --no-install-recommends -y git libgl1 libglib2.0-0 && rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:0.8.0 /uv /bin/
WORKDIR /deimv2
RUN git clone https://github.com/Intellindust-AI-Lab/DEIMv2.git .
# Install CPU-only PyTorch first to avoid pulling CUDA variant
RUN uv pip install --no-cache --system torch torchvision --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache --system -r requirements.txt
RUN uv pip install --no-cache --system onnx safetensors huggingface_hub
RUN mkdir -p output
ARG BACKBONE
ARG MODEL_SIZE
# Download from Hugging Face and convert safetensors to pth
RUN python3 -c "\
from huggingface_hub import hf_hub_download; \
from safetensors.torch import load_file; \
import torch; \
backbone = '${BACKBONE}'.replace('hgnetv2','HGNetv2').replace('dinov3','DINOv3'); \
size = '${MODEL_SIZE}'.upper(); \
st = load_file(hf_hub_download('Intellindust/DEIMv2_' + backbone + '_' + size + '_COCO', 'model.safetensors')); \
torch.save({'model': st}, 'output/deimv2.pth')"
RUN sed -i "s/data = torch.rand(2/data = torch.rand(1/" tools/deployment/export_onnx.py
# HuggingFace safetensors omits frozen constants that the model constructor initializes
RUN sed -i "s/cfg.model.load_state_dict(state)/cfg.model.load_state_dict(state, strict=False)/" tools/deployment/export_onnx.py
RUN python3 tools/deployment/export_onnx.py -c configs/deimv2/deimv2_${BACKBONE}_${MODEL_SIZE}_coco.yml -r output/deimv2.pth
FROM scratch
ARG BACKBONE
ARG MODEL_SIZE
COPY --from=build /deimv2/output/deimv2.onnx /deimv2_${BACKBONE}_${MODEL_SIZE}.onnx
EOF
```
### Downloading RF-DETR Model
RF-DETR can be exported as ONNX by running the command below. You can copy and paste the whole thing to your terminal and execute, altering `MODEL_SIZE=Nano` in the first line to `Nano`, `Small`, or `Medium` size.
@@ -1681,12 +1571,12 @@ YOLOv9 model can be exported as ONNX using the command below. You can copy and p
```sh
docker build . --build-arg MODEL_SIZE=t --build-arg IMG_SIZE=320 --output . -f- <<'EOF'
FROM python:3.11 AS build
RUN apt-get update && apt-get install --no-install-recommends -y cmake libgl1 && rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:0.10.4 /uv /bin/
RUN apt-get update && apt-get install --no-install-recommends -y libgl1 && rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:0.8.0 /uv /bin/
WORKDIR /yolov9
ADD https://github.com/WongKinYiu/yolov9.git .
RUN uv pip install --system -r requirements.txt
RUN uv pip install --system onnx==1.18.0 onnxruntime onnx-simplifier==0.4.* onnxscript
RUN uv pip install --system onnx==1.18.0 onnxruntime onnx-simplifier>=0.4.1 onnxscript
ARG MODEL_SIZE
ARG IMG_SIZE
ADD https://github.com/WongKinYiu/yolov9/releases/download/v0.1/yolov9-${MODEL_SIZE}-converted.pt yolov9-${MODEL_SIZE}.pt
+14 -11
View File
@@ -68,7 +68,7 @@ record:
## Will Frigate delete old recordings if my storage runs out?
If there is less than an hour left of storage, the oldest hour of recordings will be deleted and a message will be printed in the Frigate logs. This emergency cleanup deletes the oldest recordings first regardless of retention settings to reclaim space as quickly as possible.
As of Frigate 0.12 if there is less than an hour left of storage, the oldest 2 hours of recordings will be deleted.
## Configuring Recording Retention
@@ -130,7 +130,7 @@ When exporting a time-lapse the default speed-up is 25x with 30 FPS. This means
To configure the speed-up factor, the frame rate and further custom settings, the configuration parameter `timelapse_args` can be used. The below configuration example would change the time-lapse speed to 60x (for fitting 1 hour of recording into 1 minute of time-lapse) with 25 FPS:
```yaml {3-4}
```yaml
record:
enabled: True
export:
@@ -139,7 +139,13 @@ record:
:::tip
When using `hwaccel_args` globally hardware encoding is used for time lapse generation. The encoder determines its own behavior so the resulting file size may be undesirably large.
When using `hwaccel_args`, hardware encoding is used for timelapse generation. This setting can be overridden for a specific camera (e.g., when camera resolution exceeds hardware encoder limits); set `cameras.<camera>.record.export.hwaccel_args` with the appropriate settings. Using an unrecognized value or empty string will fall back to software encoding (libx264).
:::
:::tip
The encoder determines its own behavior so the resulting file size may be undesirably large.
To reduce the output file size the ffmpeg parameter `-qp n` can be utilized (where `n` stands for the value of the quantisation parameter). The value can be adjusted to get an acceptable tradeoff between quality and file size for the given scenario.
:::
@@ -148,19 +154,16 @@ To reduce the output file size the ffmpeg parameter `-qp n` can be utilized (whe
Apple devices running the Safari browser may fail to playback h.265 recordings. The [apple compatibility option](../configuration/camera_specific.md#h265-cameras-via-safari) should be used to ensure seamless playback on Apple devices.
## Syncing Recordings With Disk
## Syncing Media Files With Disk
In some cases the recordings files may be deleted but Frigate will not know this has happened. Recordings sync can be enabled which will tell Frigate to check the file system and delete any db entries for files which don't exist.
Media files (event snapshots, event thumbnails, review thumbnails, previews, exports, and recordings) can become orphaned when database entries are deleted but the corresponding files remain on disk.
```yaml
record:
sync_recordings: True
```
Normal operation may leave small numbers of orphaned files until Frigate's scheduled cleanup, but crashes, configuration changes, or upgrades may cause more orphaned files that Frigate does not clean up. This feature checks the file system for media files and removes any that are not referenced in the database.
This feature is meant to fix variations in files, not completely delete entries in the database. If you delete all of your media, don't use `sync_recordings`, just stop Frigate, delete the `frigate.db` database, and restart.
The Maintenance pane in the Frigate UI or an API endpoint `POST /api/media/sync` can be used to trigger a media sync. When using the API, a job ID is returned and the operation continues on the server. Status can be checked with the `/api/media/sync/status/{job_id}` endpoint.
:::warning
The sync operation uses considerable CPU resources and in most cases is not needed, only enable when necessary.
This operation uses considerable CPU resources and includes a safety threshold that aborts if more than 50% of files would be deleted. Only run when necessary. If you set `force: true` the safety threshold will be bypassed; do not use `force` unless you are certain the deletions are intended.
:::
+47 -11
View File
@@ -16,8 +16,6 @@ mqtt:
# Optional: Enable mqtt server (default: shown below)
enabled: True
# Required: host name
# NOTE: MQTT host can be specified with an environment variable or docker secrets that must begin with 'FRIGATE_'.
# e.g. host: '{FRIGATE_MQTT_HOST}'
host: mqtt.server.com
# Optional: port (default: shown below)
port: 1883
@@ -75,11 +73,19 @@ tls:
# Optional: Enable TLS for port 8971 (default: shown below)
enabled: True
# Optional: IPv6 configuration
# Optional: Networking configuration
networking:
# Optional: Enable IPv6 on 5000, and 8971 if tls is configured (default: shown below)
ipv6:
enabled: False
# Optional: Override ports Frigate uses for listening (defaults: shown below)
# An IP address may also be provided to bind to a specific interface, e.g. ip:port
# NOTE: This setting is for advanced users and may break some integrations. The majority
# of users should change ports in the docker compose file
# or use the docker run `--publish` option to select a different port.
listen:
internal: 5000
external: 8971
# Optional: Proxy configuration
proxy:
@@ -339,7 +345,15 @@ objects:
# Optional: mask to prevent all object types from being detected in certain areas (default: no mask)
# Checks based on the bottom center of the bounding box of the object.
# NOTE: This mask is COMBINED with the object type specific mask below
mask: 0.000,0.000,0.781,0.000,0.781,0.278,0.000,0.278
mask:
# Object filter mask name (required)
mask1:
# Optional: A friendly name for the mask
friendly_name: "Object filter mask area"
# Optional: Whether this mask is active (default: true)
enabled: true
# Required: Coordinates polygon for the mask
coordinates: "0.000,0.000,0.781,0.000,0.781,0.278,0.000,0.278"
# Optional: filters to reduce false positives for specific object types
filters:
person:
@@ -359,7 +373,15 @@ objects:
threshold: 0.7
# Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
# Checks based on the bottom center of the bounding box of the object
mask: 0.000,0.000,0.781,0.000,0.781,0.278,0.000,0.278
mask:
# Object filter mask name (required)
mask1:
# Optional: A friendly name for the mask
friendly_name: "Object filter mask area"
# Optional: Whether this mask is active (default: true)
enabled: true
# Required: Coordinates polygon for the mask
coordinates: "0.000,0.000,0.781,0.000,0.781,0.278,0.000,0.278"
# Optional: Configuration for AI generated tracked object descriptions
genai:
# Optional: Enable AI object description generation (default: shown below)
@@ -483,7 +505,15 @@ motion:
frame_height: 100
# Optional: motion mask
# NOTE: see docs for more detailed info on creating masks
mask: 0.000,0.469,1.000,0.469,1.000,1.000,0.000,1.000
mask:
# Motion mask name (required)
mask1:
# Optional: A friendly name for the mask
friendly_name: "Motion mask area"
# Optional: Whether this mask is active (default: true)
enabled: true
# Required: Coordinates polygon for the mask
coordinates: "0.000,0.469,1.000,0.469,1.000,1.000,0.000,1.000"
# Optional: improve contrast (default: shown below)
# Enables dynamic contrast improvement. This should help improve night detections at the cost of making motion detection more sensitive
# for daytime.
@@ -512,8 +542,6 @@ record:
# Optional: Number of minutes to wait between cleanup runs (default: shown below)
# This can be used to reduce the frequency of deleting recording segments from disk if you want to minimize i/o
expire_interval: 60
# Optional: Two-way sync recordings database with disk on startup and once a day (default: shown below).
sync_recordings: False
# Optional: Continuous retention settings
continuous:
# Optional: Number of days to retain recordings regardless of tracked objects or motion (default: shown below)
@@ -536,6 +564,8 @@ record:
# The -r (framerate) dictates how smooth the output video is.
# So the args would be -vf setpts=0.02*PTS -r 30 in that case.
timelapse_args: "-vf setpts=0.04*PTS -r 30"
# Optional: Global hardware acceleration settings for timelapse exports. (default: inherit)
hwaccel_args: auto
# Optional: Recording Preview Settings
preview:
# Optional: Quality of recording preview (default: shown below).
@@ -754,7 +784,7 @@ classification:
interval: None
# Optional: Restream configuration
# Uses https://github.com/AlexxIT/go2rtc (v1.9.10)
# Uses https://github.com/AlexxIT/go2rtc (v1.9.13)
# NOTE: The default go2rtc API port (1984) must be used,
# changing this port for the integrated go2rtc instance is not supported.
go2rtc:
@@ -840,6 +870,11 @@ cameras:
# Optional: camera specific output args (default: inherit)
# output_args:
# Optional: camera specific hwaccel args for timelapse export (default: inherit)
# record:
# export:
# hwaccel_args:
# Optional: timeout for highest scoring image before allowing it
# to be replaced by a newer image. (default: shown below)
best_image_timeout: 60
@@ -855,6 +890,9 @@ cameras:
front_steps:
# Optional: A friendly name or descriptive text for the zones
friendly_name: ""
# Optional: Whether this zone is active (default: shown below)
# Disabled zones are completely ignored at runtime - no object tracking or debug drawing
enabled: True
# Required: List of x,y coordinates to define the polygon of the zone.
# NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
coordinates: 0.033,0.306,0.324,0.138,0.439,0.185,0.042,0.428
@@ -908,8 +946,6 @@ cameras:
onvif:
# Required: host of the camera being connected to.
# NOTE: HTTP is assumed by default; HTTPS is supported if you specify the scheme, ex: "https://0.0.0.0".
# NOTE: ONVIF user, and password can be specified with environment variables or docker secrets
# that must begin with 'FRIGATE_'. e.g. host: '{FRIGATE_ONVIF_USERNAME}'
host: 0.0.0.0
# Optional: ONVIF port for device (default: shown below).
port: 8000
+10 -7
View File
@@ -7,7 +7,7 @@ title: Restream
Frigate can restream your video feed as an RTSP feed for other applications such as Home Assistant to utilize it at `rtsp://<frigate_host>:8554/<camera_name>`. Port 8554 must be open. [This allows you to use a video feed for detection in Frigate and Home Assistant live view at the same time without having to make two separate connections to the camera](#reduce-connections-to-camera). The video feed is copied from the original video feed directly to avoid re-encoding. This feed does not include any annotation by Frigate.
Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.10) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#configuration) for more advanced configurations and features.
Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.13) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#configuration) for more advanced configurations and features.
:::note
@@ -34,7 +34,7 @@ To improve connection speed when using Birdseye via restream you can enable a sm
The go2rtc restream can be secured with RTSP based username / password authentication. Ex:
```yaml {2-4}
```yaml
go2rtc:
rtsp:
username: "admin"
@@ -147,7 +147,6 @@ For example:
```yaml
go2rtc:
streams:
# highlight-error-line
my_camera: rtsp://username:$@foo%@192.168.1.100
```
@@ -156,7 +155,6 @@ becomes
```yaml
go2rtc:
streams:
# highlight-next-line
my_camera: rtsp://username:$%40foo%25@192.168.1.100
```
@@ -208,7 +206,7 @@ Enabling arbitrary exec sources allows execution of arbitrary commands through g
## Advanced Restream Configurations
The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#source-exec) source in go2rtc can be used for custom ffmpeg commands and other applications. An example is below:
The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below:
:::warning
@@ -216,11 +214,16 @@ The `exec:`, `echo:`, and `expr:` sources are disabled by default for security.
:::
NOTE: RTSP output will need to be passed with two curly braces `{{output}}`, whereas pipe output must be passed without curly braces.
:::warning
The `exec:`, `echo:`, and `expr:` sources are disabled by default for security. You must set `GO2RTC_ALLOW_ARBITRARY_EXEC=true` to use them. See [Security: Restricted Stream Sources](#security-restricted-stream-sources) for more information.
:::
NOTE: The output will need to be passed with two curly braces `{{output}}`
```yaml
go2rtc:
streams:
stream1: exec:ffmpeg -hide_banner -re -stream_loop -1 -i /media/BigBuckBunny.mp4 -c copy -rtsp_transport tcp -f rtsp {{output}}
stream2: exec:rpicam-vid -t 0 --libav-format h264 -o -
```
+1 -1
View File
@@ -71,7 +71,7 @@ To exclude a specific camera from alerts or detections, simply provide an empty
For example, to exclude objects on the camera _gatecamera_ from any detections, include this in your config:
```yaml {3-5}
```yaml
cameras:
gatecamera:
review:
+1 -1
View File
@@ -13,7 +13,7 @@ Semantic Search is accessed via the _Explore_ view in the Frigate UI.
Semantic Search works by running a large AI model locally on your system. Small or underpowered systems like a Raspberry Pi will not run Semantic Search reliably or at all.
A minimum of 8GB of RAM is required to use Semantic Search. A CPU with AVX + AVX2 instructions is required to run Semantic Search. A GPU is not strictly required but will provide a significant performance increase over CPU-only systems.
A minimum of 8GB of RAM is required to use Semantic Search. A GPU is not strictly required but will provide a significant performance increase over CPU-only systems.
For best performance, 16GB or more of RAM and a dedicated GPU are recommended.
+3 -3
View File
@@ -20,7 +20,7 @@ tls:
TLS certificates can be mounted at `/etc/letsencrypt/live/frigate` using a bind mount or docker volume.
```yaml {3-4}
```yaml
frigate:
...
volumes:
@@ -32,7 +32,7 @@ Within the folder, the private key is expected to be named `privkey.pem` and the
Note that certbot uses symlinks, and those can't be followed by the container unless it has access to the targets as well, so if using certbot you'll also have to mount the `archive` folder for your domain, e.g.:
```yaml {3-5}
```yaml
frigate:
...
volumes:
@@ -46,7 +46,7 @@ Frigate automatically compares the fingerprint of the certificate at `/etc/letse
If you issue Frigate valid certificates you will likely want to configure it to run on port 443 so you can access it without a port number like `https://your-frigate-domain.com` by mapping 8971 to 443.
```yaml {3-4}
```yaml
frigate:
...
ports:
+6 -6
View File
@@ -10,6 +10,10 @@ For example, the cat in this image is currently in Zone 1, but **not** Zone 2.
Zones cannot have the same name as a camera. If desired, a single zone can include multiple cameras if you have multiple cameras covering the same area by configuring zones with the same name for each camera.
## Enabling/Disabling Zones
Zones can be toggled on or off without removing them from the configuration. Disabled zones are completely ignored at runtime - objects will not be tracked for zone presence, and zones will not appear in the debug view. This is useful for temporarily disabling a zone during certain seasons or times of day without modifying the configuration.
During testing, enable the Zones option for the Debug view of your camera (Settings --> Debug) so you can adjust as needed. The zone line will increase in thickness when any object enters the zone.
To create a zone, follow [the steps for a "Motion mask"](masks.md), but use the section of the web UI for creating a zone instead.
@@ -18,7 +22,7 @@ To create a zone, follow [the steps for a "Motion mask"](masks.md), but use the
Often you will only want alerts to be created when an object enters areas of interest. This is done using zones along with setting required_zones. Let's say you only want to have an alert created when an object enters your entire_yard zone, the config would be:
```yaml {6,8}
```yaml
cameras:
name_of_your_camera:
review:
@@ -86,7 +90,6 @@ cameras:
Only car objects can trigger the `front_yard_street` zone and only person can trigger the `entire_yard`. Objects will be tracked for any `person` that enter anywhere in the yard, and for cars only if they enter the street.
### Zone Loitering
Sometimes objects are expected to be passing through a zone, but an object loitering in an area is unexpected. Zones can be configured to have a minimum loitering time after which the object will be considered in the zone.
@@ -94,6 +97,7 @@ Sometimes objects are expected to be passing through a zone, but an object loite
:::note
When using loitering zones, a review item will behave in the following way:
- When a person is in a loitering zone, the review item will remain active until the person leaves the loitering zone, regardless of if they are stationary.
- When any other object is in a loitering zone, the review item will remain active until the loitering time is met. Then if the object is stationary the review item will end.
@@ -104,7 +108,6 @@ cameras:
name_of_your_camera:
zones:
sidewalk:
# highlight-next-line
loitering_time: 4 # unit is in seconds
objects:
- person
@@ -119,7 +122,6 @@ cameras:
name_of_your_camera:
zones:
front_yard:
# highlight-next-line
inertia: 3
objects:
- person
@@ -132,7 +134,6 @@ cameras:
name_of_your_camera:
zones:
driveway_entrance:
# highlight-next-line
inertia: 1
objects:
- car
@@ -195,6 +196,5 @@ cameras:
coordinates: ...
distances: ...
inertia: 1
# highlight-next-line
speed_threshold: 20 # unit is in kph or mph, depending on how unit_system is set (see above)
```
+3 -33
View File
@@ -17,15 +17,15 @@ From here, follow the guides for:
- [Web Interface](#web-interface)
- [Documentation](#documentation)
### Frigate Home Assistant App
### Frigate Home Assistant Add-on
This repository holds the Home Assistant App, for use with Home Assistant OS and compatible installations. It is the piece that allows you to run Frigate from your Home Assistant Supervisor tab.
This repository holds the Home Assistant Add-on, for use with Home Assistant OS and compatible installations. It is the piece that allows you to run Frigate from your Home Assistant Supervisor tab.
Fork [blakeblackshear/frigate-hass-addons](https://github.com/blakeblackshear/frigate-hass-addons) to your own Github profile, then clone the forked repo to your local machine.
### Frigate Home Assistant Integration
This repository holds the custom integration that allows your Home Assistant installation to automatically create entities for your Frigate instance, whether you are running Frigate as a standalone Docker container or as a [Home Assistant App](#frigate-home-assistant-app).
This repository holds the custom integration that allows your Home Assistant installation to automatically create entities for your Frigate instance, whether you are running Frigate as a standalone Docker container or as a [Home Assistant Add-on](#frigate-home-assistant-add-on).
Fork [blakeblackshear/frigate-hass-integration](https://github.com/blakeblackshear/frigate-hass-integration) to your own GitHub profile, then clone the forked repo to your local machine.
@@ -89,14 +89,6 @@ After closing VS Code, you may still have containers running. To close everythin
### Testing
#### Unit Tests
GitHub will execute unit tests on new PRs. You must ensure that all tests pass.
```shell
python3 -u -m unittest
```
#### FFMPEG Hardware Acceleration
The following commands are used inside the container to ensure hardware acceleration is working properly.
@@ -133,28 +125,6 @@ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format
ffmpeg -c:v h264_qsv -re -stream_loop -1 -i https://streams.videolan.org/ffmpeg/incoming/720p60.mp4 -f rawvideo -pix_fmt yuv420p pipe: > /dev/null
```
### Submitting a pull request
Code must be formatted, linted and type-tested. GitHub will run these checks on pull requests, so it is advised to run them yourself prior to opening.
**Formatting**
```shell
ruff format frigate migrations docker *.py
```
**Linting**
```shell
ruff check frigate migrations docker *.py
```
**MyPy Static Typing**
```shell
python3 -u -m mypy --config-file frigate/mypy.ini frigate
```
## Web Interface
### Prerequisites
+1 -1
View File
@@ -26,7 +26,7 @@ I may earn a small commission for my endorsement, recommendation, testimonial, o
## Server
My current favorite is the Beelink EQ13 because of the efficient N100 CPU and dual NICs that allow you to setup a dedicated private network for your cameras where they can be blocked from accessing the internet. There are many used workstation options on eBay that work very well. Anything with an Intel CPU (with AVX + AVX2 instructions) and capable of running Debian should work fine. As a bonus, you may want to look for devices with a M.2 or PCIe express slot that is compatible with the Google Coral, Hailo, or other AI accelerators.
My current favorite is the Beelink EQ13 because of the efficient N100 CPU and dual NICs that allow you to setup a dedicated private network for your cameras where they can be blocked from accessing the internet. There are many used workstation options on eBay that work very well. Anything with an Intel CPU and capable of running Debian should work fine. As a bonus, you may want to look for devices with a M.2 or PCIe express slot that is compatible with the Google Coral, Hailo, or other AI accelerators.
Note that many of these mini PCs come with Windows pre-installed, and you will need to install Linux according to the [getting started guide](../guides/getting_started.md).
+14 -20
View File
@@ -3,11 +3,11 @@ id: installation
title: Installation
---
Frigate is a Docker container that can be run on any Docker host including as a [Home Assistant App](https://www.home-assistant.io/apps/). Note that the Home Assistant App is **not** the same thing as the integration. The [integration](/integrations/home-assistant) is required to integrate Frigate into Home Assistant, whether you are running Frigate as a standalone Docker container or as a Home Assistant App.
Frigate is a Docker container that can be run on any Docker host including as a [Home Assistant Add-on](https://www.home-assistant.io/addons/). Note that the Home Assistant Add-on is **not** the same thing as the integration. The [integration](/integrations/home-assistant) is required to integrate Frigate into Home Assistant, whether you are running Frigate as a standalone Docker container or as a Home Assistant Add-on.
:::tip
If you already have Frigate installed as a Home Assistant App, check out the [getting started guide](../guides/getting_started.md#configuring-frigate) to configure Frigate.
If you already have Frigate installed as a Home Assistant Add-on, check out the [getting started guide](../guides/getting_started#configuring-frigate) to configure Frigate.
:::
@@ -92,7 +92,7 @@ $ python -c 'print("{:.2f}MB".format(((1280 * 720 * 1.5 * 20 + 270480) / 1048576
253MB
```
The shm size cannot be set per container for Home Assistant Apps. However, this is probably not required since by default Home Assistant Supervisor allocates `/dev/shm` with half the size of your total memory. If your machine has 8GB of memory, chances are that Frigate will have access to up to 4GB without any additional configuration.
The shm size cannot be set per container for Home Assistant add-ons. However, this is probably not required since by default Home Assistant Supervisor allocates `/dev/shm` with half the size of your total memory. If your machine has 8GB of memory, chances are that Frigate will have access to up to 4GB without any additional configuration.
## Extra Steps for Specific Hardware
@@ -297,7 +297,7 @@ The MemryX MX3 Accelerator is available in the M.2 2280 form factor (like an NVM
#### Installation
To get started with MX3 hardware setup for your system, refer to the [Hardware Setup Guide](https://developer.memryx.com/2p1/get_started/install_hardware.html).
To get started with MX3 hardware setup for your system, refer to the [Hardware Setup Guide](https://developer.memryx.com/get_started/hardware_setup.html).
Then follow these steps for installing the correct driver/runtime configuration:
@@ -306,12 +306,6 @@ Then follow these steps for installing the correct driver/runtime configuration:
3. Run the script with `./user_installation.sh`
4. **Restart your computer** to complete driver installation.
:::warning
For manual setup, use **MemryX SDK 2.1** only. Other SDK versions are not supported for this setup. See the [SDK 2.1 documentation](https://developer.memryx.com/2p1/index.html)
:::
#### Setup
To set up Frigate, follow the default installation instructions, for example: `ghcr.io/blakeblackshear/frigate:stable`
@@ -516,7 +510,7 @@ The community supported docker image tags for the current stable version are:
- `stable-tensorrt-jp6` - Frigate build optimized for Nvidia Jetson devices running Jetpack 6
- `stable-rk` - Frigate build for SBCs with Rockchip SoC
## Home Assistant App
## Home Assistant Add-on
:::warning
@@ -527,7 +521,7 @@ There are important limitations in HA OS to be aware of:
- Separate local storage for media is not yet supported by Home Assistant
- AMD GPUs are not supported because HA OS does not include the mesa driver.
- Intel NPUs are not supported because HA OS does not include the NPU firmware.
- Nvidia GPUs are not supported because HA Apps do not support the Nvidia runtime.
- Nvidia GPUs are not supported because addons do not support the Nvidia runtime.
:::
@@ -537,27 +531,27 @@ See [the network storage guide](/guides/ha_network_storage.md) for instructions
:::
Home Assistant OS users can install via the App repository.
Home Assistant OS users can install via the Add-on repository.
1. In Home Assistant, navigate to _Settings_ > _Apps_ > _App Store_ > _Repositories_
1. In Home Assistant, navigate to _Settings_ > _Add-ons_ > _Add-on Store_ > _Repositories_
2. Add `https://github.com/blakeblackshear/frigate-hass-addons`
3. Install the desired variant of the Frigate App (see below)
3. Install the desired variant of the Frigate Add-on (see below)
4. Setup your network configuration in the `Configuration` tab
5. Start the App
5. Start the Add-on
6. Use the _Open Web UI_ button to access the Frigate UI, then click in the _cog icon_ > _Configuration editor_ and configure Frigate to your liking
There are several variants of the App available:
There are several variants of the Add-on available:
| App Variant | Description |
| Add-on Variant | Description |
| -------------------------- | ---------------------------------------------------------- |
| Frigate | Current release with protection mode on |
| Frigate (Full Access) | Current release with the option to disable protection mode |
| Frigate Beta | Beta release with protection mode on |
| Frigate Beta (Full Access) | Beta release with the option to disable protection mode |
If you are using hardware acceleration for ffmpeg, you **may** need to use the _Full Access_ variant of the App. This is because the Frigate App runs in a container with limited access to the host system. The _Full Access_ variant allows you to disable _Protection mode_ and give Frigate full access to the host system.
If you are using hardware acceleration for ffmpeg, you **may** need to use the _Full Access_ variant of the Add-on. This is because the Frigate Add-on runs in a container with limited access to the host system. The _Full Access_ variant allows you to disable _Protection mode_ and give Frigate full access to the host system.
You can also edit the Frigate configuration file through the [VS Code App](https://github.com/hassio-addons/addon-vscode) or similar. In that case, the configuration file will be at `/addon_configs/<addon_directory>/config.yml`, where `<addon_directory>` is specific to the variant of the Frigate App you are running. See the list of directories [here](../configuration/index.md#accessing-app-config-dir).
You can also edit the Frigate configuration file through the [VS Code Add-on](https://github.com/hassio-addons/addon-vscode) or similar. In that case, the configuration file will be at `/addon_configs/<addon_directory>/config.yml`, where `<addon_directory>` is specific to the variant of the Frigate Add-on you are running. See the list of directories [here](../configuration/index.md#accessing-add-on-config-dir).
## Kubernetes
+2 -5
View File
@@ -34,14 +34,11 @@ For commercial installations it is important to verify the number of supported c
There are many different hardware options for object detection depending on priorities and available hardware. See [the recommended hardware page](./hardware.md#detectors) for more specifics on what hardware is recommended for object detection.
### CPU
Frigate requires a CPU with AVX + AVX2 instructions. Most modern CPUs (post-2011) support AVX and AVX2, but it is generally absent in low-power or budget-oriented processors, particularly older Intel Pentium, Celeron, and Atom-based chips. Specifically, Intel Celeron and Pentium models prior to the 2020 Tiger Lake generation typically lack AVX. Older Intel Xeon models may have AVX, but may lack AVX2.
### Storage
Storage is an important consideration when planning a new installation. To get a more precise estimate of your storage requirements, you can use an IP camera storage calculator. Websites like [IPConfigure Storage Calculator](https://calculator.ipconfigure.com/) can help you determine the necessary disk space based on your camera settings.
#### SSDs (Solid State Drives)
SSDs are an excellent choice for Frigate, offering high speed and responsiveness. The older concern that SSDs would quickly "wear out" from constant video recording is largely no longer valid for modern consumer and enterprise-grade SSDs.
@@ -74,4 +71,4 @@ While supported, using network-attached storage (NAS) for recordings can introdu
- **Basic Minimum: 4GB RAM**: This is generally sufficient for a very basic Frigate setup with a few cameras and a dedicated object detection accelerator, without running any enrichments. Performance might be tight, especially with higher resolution streams or numerous detections.
- **Minimum for Enrichments: 8GB RAM**: If you plan to utilize Frigate's enrichment features (e.g., facial recognition, license plate recognition, or other AI models that run alongside standard object detection), 8GB of RAM should be considered the minimum. Enrichments require additional memory to load and process their respective models and data.
- **Recommended: 16GB RAM**: For most users, especially those with many cameras (8+) or who plan to heavily leverage enrichments, 16GB of RAM is highly recommended. This provides ample headroom for smooth operation, reduces the likelihood of swapping to disk (which can impact performance), and allows for future expansion.
- **Recommended: 16GB RAM**: For most users, especially those with many cameras (8+) or who plan to heavily leverage enrichments, 16GB of RAM is highly recommended. This provides ample headroom for smooth operation, reduces the likelihood of swapping to disk (which can impact performance), and allows for future expansion.
+18 -18
View File
@@ -5,9 +5,9 @@ title: Updating
# Updating Frigate
The current stable version of Frigate is **0.17.2**. The release notes and any breaking changes for this version can be found on the [Frigate GitHub releases page](https://github.com/blakeblackshear/frigate/releases/tag/v0.17.2).
The current stable version of Frigate is **0.17.0**. The release notes and any breaking changes for this version can be found on the [Frigate GitHub releases page](https://github.com/blakeblackshear/frigate/releases/tag/v0.17.0).
Keeping Frigate up to date ensures you benefit from the latest features, performance improvements, and bug fixes. The update process varies slightly depending on your installation method (Docker, Home Assistant App, etc.). Below are instructions for the most common setups.
Keeping Frigate up to date ensures you benefit from the latest features, performance improvements, and bug fixes. The update process varies slightly depending on your installation method (Docker, Home Assistant Addon, etc.). Below are instructions for the most common setups.
## Before You Begin
@@ -31,21 +31,21 @@ If youre running Frigate via Docker (recommended method), follow these steps:
2. **Update and Pull the Latest Image**:
- If using Docker Compose:
- Edit your `docker-compose.yml` file to specify the desired version tag (e.g., `0.17.2` instead of `0.16.4`). For example:
- Edit your `docker-compose.yml` file to specify the desired version tag (e.g., `0.17.0` instead of `0.16.4`). For example:
```yaml
services:
frigate:
image: ghcr.io/blakeblackshear/frigate:0.17.2
image: ghcr.io/blakeblackshear/frigate:0.17.0
```
- Then pull the image:
```bash
docker pull ghcr.io/blakeblackshear/frigate:0.17.2
docker pull ghcr.io/blakeblackshear/frigate:0.17.0
```
- **Note for `stable` Tag Users**: If your `docker-compose.yml` uses the `stable` tag (e.g., `ghcr.io/blakeblackshear/frigate:stable`), you dont need to update the tag manually. The `stable` tag always points to the latest stable release after pulling.
- If using `docker run`:
- Pull the image with the appropriate tag (e.g., `0.17.2`, `0.17.2-tensorrt`, or `stable`):
- Pull the image with the appropriate tag (e.g., `0.17.0`, `0.17.0-tensorrt`, or `stable`):
```bash
docker pull ghcr.io/blakeblackshear/frigate:0.17.2
docker pull ghcr.io/blakeblackshear/frigate:0.17.0
```
3. **Start the Container**:
@@ -67,30 +67,30 @@ If youre running Frigate via Docker (recommended method), follow these steps:
- If youve customized other settings (e.g., `shm-size`), ensure theyre still appropriate after the update.
- Docker will automatically use the updated image when you restart the container, as long as you pulled the correct version.
## Updating the Home Assistant App (formerly Addon)
## Updating the Home Assistant Addon
For users running Frigate as a Home Assistant App:
For users running Frigate as a Home Assistant Addon:
1. **Check for Updates**:
- Navigate to **Settings > Apps** in Home Assistant.
- Find your installed Frigate app (e.g., "Frigate NVR" or "Frigate NVR (Full Access)").
- Navigate to **Settings > Add-ons** in Home Assistant.
- Find your installed Frigate addon (e.g., "Frigate NVR" or "Frigate NVR (Full Access)").
- If an update is available, youll see an "Update" button.
2. **Update the App**:
- Click the "Update" button next to the Frigate app.
2. **Update the Addon**:
- Click the "Update" button next to the Frigate addon.
- Wait for the process to complete. Home Assistant will handle downloading and installing the new version.
3. **Restart the App**:
- After updating, go to the apps page and click "Restart" to apply the changes.
3. **Restart the Addon**:
- After updating, go to the addons page and click "Restart" to apply the changes.
4. **Verify the Update**:
- Check the app logs (under the "Log" tab) to ensure Frigate starts without errors.
- Check the addon logs (under the "Log" tab) to ensure Frigate starts without errors.
- Access the Frigate Web UI to confirm the new version is running.
### Notes
- Ensure your `/config/frigate.yml` is compatible with the new version by reviewing the [Release notes](https://github.com/blakeblackshear/frigate/releases).
- If using custom hardware (e.g., Coral or GPU), verify that configurations still work, as app updates dont modify your hardware settings.
- If using custom hardware (e.g., Coral or GPU), verify that configurations still work, as addon updates dont modify your hardware settings.
## Rolling Back
@@ -101,7 +101,7 @@ If an update causes issues:
3. Revert to the previous image version:
- For Docker: Specify an older tag (e.g., `ghcr.io/blakeblackshear/frigate:0.16.4`) in your `docker run` command.
- For Docker Compose: Edit your `docker-compose.yml`, specify the older version tag (e.g., `ghcr.io/blakeblackshear/frigate:0.16.4`), and re-run `docker compose up -d`.
- For Home Assistant: Restore from the app/addon backup you took before you updated.
- For Home Assistant: Reinstall the previous addon version manually via the repository if needed and restart the addon.
4. Verify the old version is running again.
## Troubleshooting
+8 -8
View File
@@ -37,18 +37,18 @@ The following diagram adds a lot more detail than the simple view explained befo
%%{init: {"themeVariables": {"edgeLabelBackground": "transparent"}}}%%
flowchart TD
RecStore[(Recording<br>store)]
SnapStore[(Snapshot<br>store)]
RecStore[(Recording\nstore)]
SnapStore[(Snapshot\nstore)]
subgraph Acquisition
Cam["Camera"] -->|FFmpeg supported| Stream
Cam -->|"Other streaming<br>protocols"| go2rtc
Cam -->|"Other streaming\nprotocols"| go2rtc
go2rtc("go2rtc") --> Stream
Stream[Capture main and<br>sub streams] --> |detect stream|Decode(Decode and<br>downscale)
Stream[Capture main and\nsub streams] --> |detect stream|Decode(Decode and\ndownscale)
end
subgraph Motion
Decode --> MotionM(Apply<br>motion masks)
MotionM --> MotionD(Motion<br>detection)
Decode --> MotionM(Apply\nmotion masks)
MotionM --> MotionD(Motion\ndetection)
end
subgraph Detection
MotionD --> |motion regions| ObjectD(Object detection)
@@ -60,8 +60,8 @@ flowchart TD
MotionD --> |motion event|Birdseye
ObjectZ --> |object event|Birdseye
MotionD --> |"video segments<br>(retain motion)"|RecStore
MotionD --> |"video segments\n(retain motion)"|RecStore
ObjectZ --> |detection clip|RecStore
Stream -->|"video segments<br>(retain all)"| RecStore
Stream -->|"video segments\n(retain all)"| RecStore
ObjectZ --> |detection snapshot|SnapStore
```
+9 -5
View File
@@ -11,7 +11,7 @@ Use of the bundled go2rtc is optional. You can still configure FFmpeg to connect
## Setup a go2rtc stream
First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#module-streams), not just rtsp.
First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#module-streams), not just rtsp.
:::tip
@@ -33,19 +33,22 @@ After adding this to the config, restart Frigate and try to watch the live strea
### What if my video doesn't play?
- Check Logs:
- Access the go2rtc logs in the Frigate UI under Logs in the sidebar.
- If go2rtc is having difficulty connecting to your camera, you should see some error messages in the log.
- Check go2rtc Web Interface: if you don't see any errors in the logs, try viewing the camera through go2rtc's web interface.
- Navigate to port 1984 in your browser to access go2rtc's web interface.
- If using Frigate through Home Assistant, enable the web interface at port 1984.
- If using Docker, forward port 1984 before accessing the web interface.
- Click `stream` for the specific camera to see if the camera's stream is being received.
- Check Video Codec:
- If the camera stream works in go2rtc but not in your browser, the video codec might be unsupported.
- If using H265, switch to H264. Refer to [video codec compatibility](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#codecs-madness) in go2rtc documentation.
- If unable to switch from H265 to H264, or if the stream format is different (e.g., MJPEG), re-encode the video using [FFmpeg parameters](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#source-ffmpeg). It supports rotating and resizing video feeds and hardware acceleration. Keep in mind that transcoding video from one format to another is a resource intensive task and you may be better off using the built-in jsmpeg view.
- If using H265, switch to H264. Refer to [video codec compatibility](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#codecs-madness) in go2rtc documentation.
- If unable to switch from H265 to H264, or if the stream format is different (e.g., MJPEG), re-encode the video using [FFmpeg parameters](https://github.com/AlexxIT/go2rtc/tree/v1.9.13#source-ffmpeg). It supports rotating and resizing video feeds and hardware acceleration. Keep in mind that transcoding video from one format to another is a resource intensive task and you may be better off using the built-in jsmpeg view.
```yaml
go2rtc:
streams:
@@ -55,6 +58,7 @@ After adding this to the config, restart Frigate and try to watch the live strea
```
- Switch to FFmpeg if needed:
- Some camera streams may need to use the ffmpeg module in go2rtc. This has the downside of slower startup times, but has compatibility with more stream types.
```yaml
@@ -97,9 +101,9 @@ After adding this to the config, restart Frigate and try to watch the live strea
:::warning
To access the go2rtc stream externally when utilizing the Frigate App (for
To access the go2rtc stream externally when utilizing the Frigate Add-On (for
instance through VLC), you must first enable the RTSP Restream port.
You can do this by visiting the Frigate App configuration page within Home
You can do this by visiting the Frigate Add-On configuration page within Home
Assistant and revealing the hidden options under the "Show disabled ports"
section.
+12 -51
View File
@@ -9,7 +9,7 @@ title: Getting started
If you already have an environment with Linux and Docker installed, you can continue to [Installing Frigate](#installing-frigate) below.
If you already have Frigate installed through Docker or through a Home Assistant App, you can continue to [Configuring Frigate](#configuring-frigate) below.
If you already have Frigate installed through Docker or through a Home Assistant Add-on, you can continue to [Configuring Frigate](#configuring-frigate) below.
:::
@@ -81,7 +81,7 @@ Now you have a minimal Debian server that requires very little maintenance.
## Installing Frigate
This section shows how to create a minimal directory structure for a Docker installation on Debian. If you have installed Frigate as a Home Assistant App or another way, you can continue to [Configuring Frigate](#configuring-frigate).
This section shows how to create a minimal directory structure for a Docker installation on Debian. If you have installed Frigate as a Home Assistant Add-on or another way, you can continue to [Configuring Frigate](#configuring-frigate).
### Setup directories
@@ -150,7 +150,7 @@ Here is an example configuration with hardware acceleration configured to work w
`docker-compose.yml` (after modifying, you will need to run `docker compose up -d` to apply changes)
```yaml {4,5}
```yaml
services:
frigate:
...
@@ -168,57 +168,17 @@ cameras:
name_of_your_camera:
ffmpeg:
inputs: ...
# highlight-next-line
hwaccel_args: preset-vaapi
detect: ...
```
### Step 4: Configure detectors
By default, Frigate will use a single CPU detector.
In many cases, the integrated graphics on Intel CPUs provides sufficient performance for typical Frigate setups. If you have an Intel processor, you can follow the configuration below.
<details>
<summary>Use Intel OpenVINO detector</summary>
You need to refer to **Configure hardware acceleration** above to enable the container to use the GPU.
```yaml {3-6,9-15,20-21}
mqtt: ...
detectors: # <---- add detectors
ov:
type: openvino # <---- use openvino detector
device: GPU
# We will use the default MobileNet_v2 model from OpenVINO.
model:
width: 300
height: 300
input_tensor: nhwc
input_pixel_format: bgr
path: /openvino-model/ssdlite_mobilenet_v2.xml
labelmap_path: /openvino-model/coco_91cl_bkgr.txt
cameras:
name_of_your_camera:
ffmpeg: ...
detect:
enabled: True # <---- turn on detection
...
```
</details>
If you have a USB Coral, you will need to add a detectors section to your config.
<details>
<summary>Use USB Coral detector</summary>
By default, Frigate will use a single CPU detector. If you have a USB Coral, you will need to add a detectors section to your config.
`docker-compose.yml` (after modifying, you will need to run `docker compose up -d` to apply changes)
```yaml {4-6}
```yaml
services:
frigate:
...
@@ -228,7 +188,7 @@ services:
...
```
```yaml {3-6,11-12}
```yaml
mqtt: ...
detectors: # <---- add detectors
@@ -244,8 +204,6 @@ cameras:
...
```
</details>
More details on available detectors can be found [here](../configuration/object_detectors.md).
Restart Frigate and you should start seeing detections for `person`. If you want to track other objects, they will need to be added according to the [configuration file reference](../configuration/reference.md).
@@ -264,7 +222,7 @@ Note that motion masks should not be used to mark out areas where you do not wan
Your configuration should look similar to this now.
```yaml {16-18}
```yaml
mqtt:
enabled: False
@@ -282,7 +240,10 @@ cameras:
- detect
motion:
mask:
- 0,461,3,0,1919,0,1919,843,1699,492,1344,458,1346,336,973,317,869,375,866,432
motion_area:
friendly_name: "Motion mask"
enabled: true
coordinates: "0,461,3,0,1919,0,1919,843,1699,492,1344,458,1346,336,973,317,869,375,866,432"
```
### Step 6: Enable recordings
@@ -291,7 +252,7 @@ In order to review activity in the Frigate UI, recordings need to be enabled.
To enable recording video, add the `record` role to a stream and enable it in the config. If record is disabled in the config, it won't be possible to enable it in the UI.
```yaml {16-17}
```yaml
mqtt: ...
detectors: ...
+3 -3
View File
@@ -3,7 +3,7 @@ id: ha_network_storage
title: Home Assistant network storage
---
As of Home Assistant 2023.6, Network Mounted Storage is supported for Apps.
As of Home Assistant 2023.6, Network Mounted Storage is supported for Add-ons.
## Setting Up Remote Storage For Frigate
@@ -14,7 +14,7 @@ As of Home Assistant 2023.6, Network Mounted Storage is supported for Apps.
### Initial Setup
1. Stop the Frigate App
1. Stop the Frigate Add-on
### Move current data
@@ -37,4 +37,4 @@ Keeping the current data is optional, but the data will need to be moved regardl
4. Fill out the additional required info for your particular NAS
5. Connect
6. Move files from `/media/frigate_tmp` to `/media/frigate` if they were kept in previous step
7. Start the Frigate App
7. Start the Frigate Add-on
+3 -3
View File
@@ -99,11 +99,11 @@ services:
...
```
### Home Assistant App
### Home Assistant Add-on
If you are using Home Assistant App, the URL should be one of the following depending on which App variant you are using. Note that if you are using the Proxy App, you should NOT point the integration at the proxy URL. Just enter the same URL used to access Frigate directly from your network.
If you are using Home Assistant Add-on, the URL should be one of the following depending on which Add-on variant you are using. Note that if you are using the Proxy Add-on, you should NOT point the integration at the proxy URL. Just enter the same URL used to access Frigate directly from your network.
| App Variant | URL |
| Add-on Variant | URL |
| -------------------------- | -------------------------------------- |
| Frigate | `http://ccab4aaf-frigate:5000` |
| Frigate (Full Access) | `http://ccab4aaf-frigate-fa:5000` |
+24
View File
@@ -429,6 +429,30 @@ Topic to adjust motion contour area for a camera. Expected value is an integer.
Topic with current motion contour area for a camera. Published value is an integer.
### `frigate/<camera_name>/motion_mask/<mask_name>/set`
Topic to turn a specific motion mask for a camera on and off. Expected values are `ON` and `OFF`.
### `frigate/<camera_name>/motion_mask/<mask_name>/state`
Topic with current state of a specific motion mask for a camera. Published values are `ON` and `OFF`.
### `frigate/<camera_name>/object_mask/<mask_name>/set`
Topic to turn a specific object mask for a camera on and off. Expected values are `ON` and `OFF`.
### `frigate/<camera_name>/object_mask/<mask_name>/state`
Topic with current state of a specific object mask for a camera. Published values are `ON` and `OFF`.
### `frigate/<camera_name>/zone/<zone_name>/set`
Topic to turn a specific zone for a camera on and off. Expected values are `ON` and `OFF`.
### `frigate/<camera_name>/zone/<zone_name>/state`
Topic with current state of a specific zone for a camera. Published values are `ON` and `OFF`.
### `frigate/<camera_name>/review_status`
Topic with current activity status of the camera. Possible values are `NONE`, `DETECTION`, or `ALERT`.
+2 -2
View File
@@ -19,11 +19,11 @@ Once logged in, you can generate an API key for Frigate in Settings.
### Set your API key
In Frigate, you can use an environment variable or a docker secret named `PLUS_API_KEY` to enable the `Frigate+` buttons on the Explore page. Home Assistant App users can set it under Settings > Apps > Frigate > Configuration > Options (be sure to toggle the "Show unused optional configuration options" switch).
In Frigate, you can use an environment variable or a docker secret named `PLUS_API_KEY` to enable the `Frigate+` buttons on the Explore page. Home Assistant Addon users can set it under Settings > Add-ons > Frigate > Configuration > Options (be sure to toggle the "Show unused optional configuration options" switch).
:::warning
You cannot use the `environment_vars` section of your Frigate configuration file to set this environment variable. It must be defined as an environment variable in the docker config or Home Assistant App config.
You cannot use the `environment_vars` section of your Frigate configuration file to set this environment variable. It must be defined as an environment variable in the docker config or Home Assistant Add-on config.
:::
@@ -42,7 +42,3 @@ This is a fork (with fixed errors and new features) of [original Double Take](ht
## [Scrypted - Frigate bridge plugin](https://github.com/apocaliss92/scrypted-frigate-bridge)
[Scrypted - Frigate bridge](https://github.com/apocaliss92/scrypted-frigate-bridge) is an plugin that allows to ingest Frigate detections, motion, videoclips on Scrypted as well as provide templates to export rebroadcast configurations on Frigate.
## [Strix](https://github.com/eduard256/Strix)
[Strix](https://github.com/eduard256/Strix) auto-discovers working stream URLs for IP cameras and generates ready-to-use Frigate configs. It tests thousands of URL patterns against your camera and supports cameras without RTSP or ONVIF. 67K+ camera models from 3.6K+ brands.
+1 -1
View File
@@ -32,7 +32,7 @@ The USB coral can draw up to 900mA and this can be too much for some on-device U
The USB coral has different IDs when it is uninitialized and initialized.
- When running Frigate in a VM, Proxmox lxc, etc. you must ensure both device IDs are mapped.
- When running through the Home Assistant OS you may need to run the Full Access variant of the Frigate App with the _Protection mode_ switch disabled so that the coral can be accessed.
- When running through the Home Assistant OS you may need to run the Full Access variant of the Frigate Add-on with the _Protection mode_ switch disabled so that the coral can be accessed.
### Synology 716+II running DSM 7.2.1-69057 Update 5
-16
View File
@@ -110,19 +110,3 @@ No. Frigate uses the TCP protocol to connect to your camera's RTSP URL. VLC auto
TCP ensures that all data packets arrive in the correct order. This is crucial for video recording, decoding, and stream processing, which is why Frigate enforces a TCP connection. UDP is faster but less reliable, as it does not guarantee packet delivery or order, and VLC does not have the same requirements as Frigate.
You can still configure Frigate to use UDP by using ffmpeg input args or the preset `preset-rtsp-udp`. See the [ffmpeg presets](/configuration/ffmpeg_presets) documentation.
### Why does Frigate keep creating new events for my parked car?
Stationary tracking is designed to _prevent_ this — a parked car should stay one tracked object and not generate new events. If you're getting repeated events for the same car, it's likely that Frigate is losing the tracked object and re-detecting it as a new one.
Open one of the events in Explore → **Tracking Details**. If the detection scores are low (< 70% or so), the model isn't confident the parked car is a car. This is common with the free [COCO-trained](https://cocodataset.org/#explore) object detection models on steep/top-down angles, partially occluded cars, foliage, or low-light footage. When detections fall below `min_score` for too many frames the tracker loses the object, and the next confident frame creates a brand new one.
What helps:
- **Improve the view** — even a small angle change that gets more of the car visible could lift scores enough to stabilize tracking.
- **Use a more accurate model** — switching from `mobiledet` to `yolov9`, or stepping up to a larger variant like `yolov9-s` over `yolov9-t`, can help (at the cost of inference time, and still on the COCO dataset). The biggest gains usually come from fine-tuning a model on images from your own cameras so it learns your specific scene. [Frigate+](https://frigate.video/plus) is a paid option that does this - models are trained on security-camera footage and can be fine-tuned on images you submit from your own setup.
- **Don't set `detect -> stationary -> max_frames` for `car`** — it artificially ends tracking and forces re-detection as a new object. See [Stationary Objects](../configuration/stationary_objects.md).
- **Restrict alerts to the areas you care about** with `required_zones` — see [Zones](../configuration/zones.md#restricting-alerts-and-detections-to-specific-zones). Make sure those zones use the default `loitering_time: 0` unless you specifically want the review item to stay open until the car leaves.
- **Filter impossible locations** with [object filter masks](../configuration/masks.md#object-filter-masks) if cars are being detected on rooftops, treetops, etc.
See [Object Filters](../configuration/object_filters.md) for more on tuning `min_score` and `threshold` — note that raising them too high will make this exact problem worse.
-11
View File
@@ -83,17 +83,6 @@ const config: Config = {
},
},
prism: {
magicComments:[
{
className: 'theme-code-block-highlighted-line',
line: 'highlight-next-line',
block: {start: 'highlight-start', end: 'highlight-end'},
},
{
className: 'code-block-error-line',
line: 'highlight-error-line',
},
],
additionalLanguages: ["bash", "json"],
},
languageTabs: [
+1 -1
View File
@@ -28,7 +28,7 @@ const sidebars: SidebarsConfig = {
{
type: "link",
label: "Go2RTC Configuration Reference",
href: "https://github.com/AlexxIT/go2rtc/tree/v1.9.10#configuration",
href: "https://github.com/AlexxIT/go2rtc/tree/v1.9.13#configuration",
} as PropSidebarItemLink,
],
Detectors: [
-8
View File
@@ -234,11 +234,3 @@
content: "schema";
color: var(--ifm-color-secondary-contrast-foreground);
}
.code-block-error-line {
background-color: #ff000020;
display: block;
margin: 0 calc(-1 * var(--ifm-pre-padding));
padding: 0 var(--ifm-pre-padding);
border-left: 3px solid #ff000080;
}
+60
View File
@@ -331,6 +331,59 @@ paths:
application/json:
schema:
$ref: "#/components/schemas/HTTPValidationError"
/media/sync:
post:
tags:
- App
summary: Start media sync job
description: |-
Start an asynchronous media sync job to find and (optionally) remove orphaned media files.
Returns 202 with job details when queued, or 409 if a job is already running.
operationId: sync_media_media_sync_post
requestBody:
required: true
content:
application/json:
responses:
"202":
description: Accepted - Job queued
"409":
description: Conflict - Job already running
"422":
description: Validation Error
/media/sync/current:
get:
tags:
- App
summary: Get current media sync job
description: |-
Retrieve the current running media sync job, if any. Returns the job details or null when no job is active.
operationId: get_media_sync_current_media_sync_current_get
responses:
"200":
description: Successful Response
"422":
description: Validation Error
/media/sync/status/{job_id}:
get:
tags:
- App
summary: Get media sync job status
description: |-
Get status and results for the specified media sync job id. Returns 200 with job details including results, or 404 if the job is not found.
operationId: get_media_sync_status_media_sync_status__job_id__get
parameters:
- name: job_id
in: path
responses:
"200":
description: Successful Response
"404":
description: Not Found - Job not found
"422":
description: Validation Error
/faces/train/{name}/classify:
post:
tags:
@@ -3147,6 +3200,7 @@ paths:
duration: 30
include_recording: true
draw: {}
pre_capture: null
responses:
"200":
description: Successful Response
@@ -4949,6 +5003,12 @@ components:
- type: "null"
title: Draw
default: {}
pre_capture:
anyOf:
- type: integer
- type: "null"
title: Pre Capture Seconds
default: null
type: object
title: EventsCreateBody
EventsDeleteBody:
+274 -99
View File
@@ -19,6 +19,7 @@ from fastapi import APIRouter, Body, Path, Request, Response
from fastapi.encoders import jsonable_encoder
from fastapi.params import Depends
from fastapi.responses import JSONResponse, PlainTextResponse, StreamingResponse
from filelock import FileLock, Timeout
from markupsafe import escape
from peewee import SQL, fn, operator
from pydantic import ValidationError
@@ -30,22 +31,31 @@ from frigate.api.auth import (
require_role,
)
from frigate.api.defs.query.app_query_parameters import AppTimelineHourlyQueryParameters
from frigate.api.defs.request.app_body import AppConfigSetBody
from frigate.api.defs.request.app_body import AppConfigSetBody, MediaSyncBody
from frigate.api.defs.tags import Tags
from frigate.config import FrigateConfig
from frigate.config.camera.updater import (
CameraConfigUpdateEnum,
CameraConfigUpdateTopic,
)
from frigate.ffmpeg_presets import FFMPEG_HWACCEL_VAAPI, _gpu_selector
from frigate.jobs.media_sync import (
get_current_media_sync_job,
get_media_sync_job_by_id,
start_media_sync_job,
)
from frigate.models import Event, Timeline
from frigate.stats.prometheus import get_metrics, update_metrics
from frigate.types import JobStatusTypesEnum
from frigate.util.builtin import (
clean_camera_user_pass,
flatten_config_data,
load_labels,
process_config_query_string,
update_yaml_file_bulk,
)
from frigate.util.config import find_config_file
from frigate.util.schema import get_config_schema
from frigate.util.services import (
get_nvidia_driver_info,
process_logs,
@@ -70,9 +80,7 @@ def is_healthy():
@router.get("/config/schema.json", dependencies=[Depends(allow_public())])
def config_schema(request: Request):
return Response(
content=request.app.frigate_config.schema_json(), media_type="application/json"
)
return JSONResponse(content=get_config_schema(FrigateConfig))
@router.get(
@@ -118,6 +126,10 @@ def config(request: Request):
config: dict[str, dict[str, Any]] = config_obj.model_dump(
mode="json", warnings="none", exclude_none=True
)
config["detectors"] = {
name: detector.model_dump(mode="json", warnings="none", exclude_none=True)
for name, detector in config_obj.detectors.items()
}
# remove the mqtt password
config["mqtt"].pop("password", None)
@@ -188,6 +200,54 @@ def config(request: Request):
return JSONResponse(content=config)
@router.get("/ffmpeg/presets", dependencies=[Depends(allow_any_authenticated())])
def ffmpeg_presets():
"""Return available ffmpeg preset keys for config UI usage."""
# Whitelist based on documented presets in ffmpeg_presets.md
hwaccel_presets = [
"preset-rpi-64-h264",
"preset-rpi-64-h265",
"preset-vaapi",
"preset-intel-qsv-h264",
"preset-intel-qsv-h265",
"preset-nvidia",
"preset-jetson-h264",
"preset-jetson-h265",
"preset-rkmpp",
]
input_presets = [
"preset-http-jpeg-generic",
"preset-http-mjpeg-generic",
"preset-http-reolink",
"preset-rtmp-generic",
"preset-rtsp-generic",
"preset-rtsp-restream",
"preset-rtsp-restream-low-latency",
"preset-rtsp-udp",
"preset-rtsp-blue-iris",
]
record_output_presets = [
"preset-record-generic",
"preset-record-generic-audio-copy",
"preset-record-generic-audio-aac",
"preset-record-mjpeg",
"preset-record-jpeg",
"preset-record-ubiquiti",
]
return JSONResponse(
content={
"hwaccel_args": hwaccel_presets,
"input_args": input_presets,
"output_args": {
"record": record_output_presets,
"detect": [],
},
}
)
@router.get("/config/raw_paths", dependencies=[Depends(require_role(["admin"]))])
def config_raw_paths(request: Request):
"""Admin-only endpoint that returns camera paths and go2rtc streams without credential masking."""
@@ -218,7 +278,7 @@ def config_raw_paths(request: Request):
return JSONResponse(content=raw_paths)
@router.get("/config/raw", dependencies=[Depends(require_role(["admin"]))])
@router.get("/config/raw", dependencies=[Depends(allow_any_authenticated())])
def config_raw():
config_file = find_config_file()
@@ -365,105 +425,136 @@ def config_save(save_option: str, body: Any = Body(media_type="text/plain")):
@router.put("/config/set", dependencies=[Depends(require_role(["admin"]))])
def config_set(request: Request, body: AppConfigSetBody):
config_file = find_config_file()
with open(config_file, "r") as f:
old_raw_config = f.read()
lock = FileLock(f"{config_file}.lock", timeout=5)
try:
updates = {}
with lock:
with open(config_file, "r") as f:
old_raw_config = f.read()
# process query string parameters (takes precedence over body.config_data)
parsed_url = urllib.parse.urlparse(str(request.url))
query_string = urllib.parse.parse_qs(parsed_url.query, keep_blank_values=True)
try:
updates = {}
# Filter out empty keys but keep blank values for non-empty keys
query_string = {k: v for k, v in query_string.items() if k}
# process query string parameters (takes precedence over body.config_data)
parsed_url = urllib.parse.urlparse(str(request.url))
query_string = urllib.parse.parse_qs(
parsed_url.query, keep_blank_values=True
)
if query_string:
updates = process_config_query_string(query_string)
elif body.config_data:
updates = flatten_config_data(body.config_data)
# Filter out empty keys but keep blank values for non-empty keys
query_string = {k: v for k, v in query_string.items() if k}
if not updates:
return JSONResponse(
content=(
{"success": False, "message": "No configuration data provided"}
),
status_code=400,
)
if query_string:
updates = process_config_query_string(query_string)
elif body.config_data:
updates = flatten_config_data(body.config_data)
# Convert None values to empty strings for deletion (e.g., when deleting masks)
updates = {k: ("" if v is None else v) for k, v in updates.items()}
# apply all updates in a single operation
update_yaml_file_bulk(config_file, updates)
if not updates:
return JSONResponse(
content=(
{
"success": False,
"message": "No configuration data provided",
}
),
status_code=400,
)
# validate the updated config
with open(config_file, "r") as f:
new_raw_config = f.read()
# apply all updates in a single operation
update_yaml_file_bulk(config_file, updates)
# validate the updated config
with open(config_file, "r") as f:
new_raw_config = f.read()
try:
config = FrigateConfig.parse(new_raw_config)
except Exception:
with open(config_file, "w") as f:
f.write(old_raw_config)
f.close()
logger.error(f"\nConfig Error:\n\n{str(traceback.format_exc())}")
return JSONResponse(
content=(
{
"success": False,
"message": "Error parsing config. Check logs for error message.",
}
),
status_code=400,
)
except Exception as e:
logging.error(f"Error updating config: {e}")
return JSONResponse(
content=({"success": False, "message": "Error updating config"}),
status_code=500,
)
if body.requires_restart == 0 or body.update_topic:
old_config: FrigateConfig = request.app.frigate_config
request.app.frigate_config = config
request.app.genai_manager.update_config(config)
if body.update_topic:
if body.update_topic.startswith("config/cameras/"):
_, _, camera, field = body.update_topic.split("/")
if field == "add":
settings = config.cameras[camera]
elif field == "remove":
settings = old_config.cameras[camera]
else:
settings = config.get_nested_object(body.update_topic)
request.app.config_publisher.publish_update(
CameraConfigUpdateTopic(
CameraConfigUpdateEnum[field], camera
),
settings,
)
else:
# Generic handling for global config updates
settings = config.get_nested_object(body.update_topic)
# Publish None for removal, actual config for add/update
request.app.config_publisher.publisher.publish(
body.update_topic, settings
)
try:
config = FrigateConfig.parse(new_raw_config)
except Exception:
with open(config_file, "w") as f:
f.write(old_raw_config)
f.close()
logger.error(f"\nConfig Error:\n\n{str(traceback.format_exc())}")
return JSONResponse(
content=(
{
"success": False,
"message": "Error parsing config. Check logs for error message.",
"success": True,
"message": "Config successfully updated, restart to apply",
}
),
status_code=400,
status_code=200,
)
except Exception as e:
logging.error(f"Error updating config: {e}")
except Timeout:
return JSONResponse(
content=({"success": False, "message": "Error updating config"}),
status_code=500,
content=(
{
"success": False,
"message": "Another process is currently updating the config. Please try again in a few seconds.",
}
),
status_code=503,
)
if body.requires_restart == 0 or body.update_topic:
old_config: FrigateConfig = request.app.frigate_config
request.app.frigate_config = config
if body.update_topic:
if body.update_topic.startswith("config/cameras/"):
_, _, camera, field = body.update_topic.split("/")
if field == "add":
settings = config.cameras[camera]
elif field == "remove":
settings = old_config.cameras[camera]
else:
settings = config.get_nested_object(body.update_topic)
request.app.config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum[field], camera),
settings,
)
else:
# Generic handling for global config updates
settings = config.get_nested_object(body.update_topic)
# Publish None for removal, actual config for add/update
request.app.config_publisher.publisher.publish(
body.update_topic, settings
)
return JSONResponse(
content=(
{
"success": True,
"message": "Config successfully updated, restart to apply",
}
),
status_code=200,
)
@router.get("/vainfo", dependencies=[Depends(allow_any_authenticated())])
def vainfo():
vainfo = vainfo_hwaccel()
# Use LibvaGpuSelector to pick an appropriate libva device (if available)
selected_gpu = ""
try:
selected_gpu = _gpu_selector.get_gpu_arg(FFMPEG_HWACCEL_VAAPI, 0) or ""
except Exception:
selected_gpu = ""
# If selected_gpu is empty, pass None to vainfo_hwaccel to run plain `vainfo`.
vainfo = vainfo_hwaccel(device_name=selected_gpu or None)
return JSONResponse(
content={
"return_code": vainfo.returncode,
@@ -598,6 +689,98 @@ def restart():
)
@router.post(
"/media/sync",
dependencies=[Depends(require_role(["admin"]))],
summary="Start media sync job",
description="""Start an asynchronous media sync job to find and (optionally) remove orphaned media files.
Returns 202 with job details when queued, or 409 if a job is already running.""",
)
def sync_media(body: MediaSyncBody = Body(...)):
"""Start async media sync job - remove orphaned files.
Syncs specified media types: event snapshots, event thumbnails, review thumbnails,
previews, exports, and/or recordings. Job runs in background; use /media/sync/current
or /media/sync/status/{job_id} to check status.
Args:
body: MediaSyncBody with dry_run flag and media_types list.
media_types can include: 'all', 'event_snapshots', 'event_thumbnails',
'review_thumbnails', 'previews', 'exports', 'recordings'
Returns:
202 Accepted with job_id, or 409 Conflict if job already running.
"""
job_id = start_media_sync_job(
dry_run=body.dry_run, media_types=body.media_types, force=body.force
)
if job_id is None:
# A job is already running
current = get_current_media_sync_job()
return JSONResponse(
content={
"error": "A media sync job is already running",
"current_job_id": current.id if current else None,
},
status_code=409,
)
return JSONResponse(
content={
"job": {
"job_type": "media_sync",
"status": JobStatusTypesEnum.queued,
"id": job_id,
}
},
status_code=202,
)
@router.get(
"/media/sync/current",
dependencies=[Depends(require_role(["admin"]))],
summary="Get current media sync job",
description="""Retrieve the current running media sync job, if any. Returns the job details
or null when no job is active.""",
)
def get_media_sync_current():
"""Get the current running media sync job, if any."""
job = get_current_media_sync_job()
if job is None:
return JSONResponse(content={"job": None}, status_code=200)
return JSONResponse(
content={"job": job.to_dict()},
status_code=200,
)
@router.get(
"/media/sync/status/{job_id}",
dependencies=[Depends(require_role(["admin"]))],
summary="Get media sync job status",
description="""Get status and results for the specified media sync job id. Returns 200 with
job details including results, or 404 if the job is not found.""",
)
def get_media_sync_status(job_id: str):
"""Get the status of a specific media sync job."""
job = get_media_sync_job_by_id(job_id)
if job is None:
return JSONResponse(
content={"error": "Job not found"},
status_code=404,
)
return JSONResponse(
content={"job": job.to_dict()},
status_code=200,
)
@router.get("/labels", dependencies=[Depends(allow_any_authenticated())])
def get_labels(camera: str = ""):
try:
@@ -647,6 +830,12 @@ def get_sub_labels(split_joined: Optional[int] = None):
return JSONResponse(content=sub_labels)
@router.get("/audio_labels", dependencies=[Depends(allow_any_authenticated())])
def get_audio_labels():
labels = load_labels("/audio-labelmap.txt", prefill=521)
return JSONResponse(content=labels)
@router.get("/plus/models", dependencies=[Depends(allow_any_authenticated())])
def plusModels(request: Request, filterByCurrentModelDetector: bool = False):
if not request.app.frigate_config.plus_api.is_active():
@@ -732,12 +921,7 @@ def get_recognized_license_plates(
@router.get("/timeline", dependencies=[Depends(allow_any_authenticated())])
def timeline(
camera: str = "all",
limit: int = 100,
source_id: Optional[str] = None,
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
def timeline(camera: str = "all", limit: int = 100, source_id: Optional[str] = None):
clauses = []
selected_columns = [
@@ -759,9 +943,6 @@ def timeline(
else:
clauses.append((Timeline.source_id.in_(source_ids)))
# Enforce per-camera access control
clauses.append((Timeline.camera << allowed_cameras))
if len(clauses) == 0:
clauses.append((True))
@@ -777,10 +958,7 @@ def timeline(
@router.get("/timeline/hourly", dependencies=[Depends(allow_any_authenticated())])
def hourly_timeline(
params: AppTimelineHourlyQueryParameters = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
def hourly_timeline(params: AppTimelineHourlyQueryParameters = Depends()):
"""Get hourly summary for timeline."""
cameras = params.cameras
labels = params.labels
@@ -798,9 +976,6 @@ def hourly_timeline(
camera_list = cameras.split(",")
clauses.append((Timeline.camera << camera_list))
# Enforce per-camera access control
clauses.append((Timeline.camera << allowed_cameras))
if labels != "all":
label_list = labels.split(",")
clauses.append((Timeline.data["label"] << label_list))
+32 -74
View File
@@ -26,7 +26,7 @@ from frigate.api.defs.request.app_body import (
AppPutRoleBody,
)
from frigate.api.defs.tags import Tags
from frigate.config import AuthConfig, ProxyConfig
from frigate.config import AuthConfig, NetworkingConfig, ProxyConfig
from frigate.const import CONFIG_DIR, JWT_SECRET_ENV_VAR, PASSWORD_HASH_ALGORITHM
from frigate.models import User
@@ -41,7 +41,7 @@ def require_admin_by_default():
endpoints require admin access unless explicitly overridden with
allow_public(), allow_any_authenticated(), or require_role().
Port 5000 (internal) always has admin role set by the /auth endpoint,
Internal port always has admin role set by the /auth endpoint,
so this check passes automatically for internal requests.
Certain paths are exempted from the global admin check because they must
@@ -67,6 +67,7 @@ def require_admin_by_default():
"/stats",
"/stats/history",
"/config",
"/config/raw",
"/vainfo",
"/nvinfo",
"/labels",
@@ -129,7 +130,7 @@ def require_admin_by_default():
pass
# For all other paths, require admin role
# Port 5000 (internal) requests have admin role set automatically
# Internal port requests have admin role set automatically
role = request.headers.get("remote-role")
if role == "admin":
return
@@ -142,6 +143,17 @@ def require_admin_by_default():
return admin_checker
def _is_authenticated(request: Request) -> bool:
"""
Helper to determine if a request is from an authenticated user.
Returns True if the request has a valid authenticated user (not anonymous).
Internal port requests are considered anonymous despite having admin role.
"""
username = request.headers.get("remote-user")
return username is not None and username != "anonymous"
def allow_public():
"""
Override dependency to allow unauthenticated access to an endpoint.
@@ -170,6 +182,7 @@ def allow_any_authenticated():
Rejects:
- Requests with no remote-user header (did not pass through /auth endpoint)
- External port requests with anonymous user (auth disabled, no proxy auth)
Example:
@router.get("/authenticated-endpoint", dependencies=[Depends(allow_any_authenticated())])
@@ -178,8 +191,14 @@ def allow_any_authenticated():
async def auth_checker(request: Request):
# Ensure a remote-user has been set by the /auth endpoint
username = request.headers.get("remote-user")
if username is None:
raise HTTPException(status_code=401, detail="Authentication required")
# Internal port requests have admin role and should be allowed
role = request.headers.get("remote-role")
if role != "admin":
if username is None or not _is_authenticated(request):
raise HTTPException(status_code=401, detail="Authentication required")
return
return auth_checker
@@ -569,12 +588,18 @@ def resolve_role(
def auth(request: Request):
auth_config: AuthConfig = request.app.frigate_config.auth
proxy_config: ProxyConfig = request.app.frigate_config.proxy
networking_config: NetworkingConfig = request.app.frigate_config.networking
success_response = Response("", status_code=202)
# handle case where internal port is a string with ip:port
internal_port = networking_config.listen.internal
if type(internal_port) is str:
internal_port = int(internal_port.split(":")[-1])
# dont require auth if the request is on the internal port
# this header is set by Frigate's nginx proxy, so it cant be spoofed
if int(request.headers.get("x-server-port", default=0)) == 5000:
if int(request.headers.get("x-server-port", default=0)) == internal_port:
success_response.headers["remote-user"] = "anonymous"
success_response.headers["remote-role"] = "admin"
return success_response
@@ -836,7 +861,6 @@ def create_user(
User.notification_tokens: [],
}
).execute()
request.app.config_publisher.publisher.publish("config/auth", None)
return JSONResponse(content={"username": body.username})
@@ -854,7 +878,6 @@ def delete_user(request: Request, username: str):
)
User.delete_by_id(username)
request.app.config_publisher.publisher.publish("config/auth", None)
return JSONResponse(content={"success": True})
@@ -974,7 +997,6 @@ async def update_role(
)
User.set_by_id(username, {User.role: body.role})
request.app.config_publisher.publisher.publish("config/auth", None)
return JSONResponse(content={"success": True})
@@ -988,16 +1010,7 @@ async def require_camera_access(
current_user = await get_current_user(request)
if isinstance(current_user, JSONResponse):
detail = "Authentication required"
try:
error_payload = json.loads(current_user.body)
detail = (
error_payload.get("message") or error_payload.get("detail") or detail
)
except Exception:
pass
raise HTTPException(status_code=current_user.status_code, detail=detail)
return current_user
role = current_user["role"]
all_camera_names = set(request.app.frigate_config.cameras.keys())
@@ -1015,61 +1028,6 @@ async def require_camera_access(
)
def _get_stream_owner_cameras(request: Request, stream_name: str) -> set[str]:
owner_cameras: set[str] = set()
for camera_name, camera in request.app.frigate_config.cameras.items():
if stream_name == camera_name:
owner_cameras.add(camera_name)
continue
if stream_name in camera.live.streams.values():
owner_cameras.add(camera_name)
return owner_cameras
async def require_go2rtc_stream_access(
stream_name: Optional[str] = None,
request: Request = None,
):
"""Dependency to enforce go2rtc stream access based on owning camera access."""
if stream_name is None:
return
current_user = await get_current_user(request)
if isinstance(current_user, JSONResponse):
detail = "Authentication required"
try:
error_payload = json.loads(current_user.body)
detail = (
error_payload.get("message") or error_payload.get("detail") or detail
)
except Exception:
pass
raise HTTPException(status_code=current_user.status_code, detail=detail)
role = current_user["role"]
all_camera_names = set(request.app.frigate_config.cameras.keys())
roles_dict = request.app.frigate_config.auth.roles
allowed_cameras = User.get_allowed_cameras(role, roles_dict, all_camera_names)
# Admin or full access bypasses
if role == "admin" or not roles_dict.get(role):
return
owner_cameras = _get_stream_owner_cameras(request, stream_name)
if owner_cameras & set(allowed_cameras):
return
raise HTTPException(
status_code=403,
detail=f"Access denied to camera '{stream_name}'. Allowed: {allowed_cameras}",
)
async def get_allowed_cameras_for_filter(request: Request):
"""Dependency to get allowed_cameras for filtering lists."""
current_user = await get_current_user(request)
+6 -32
View File
@@ -17,14 +17,14 @@ from zeep.transports import AsyncTransport
from frigate.api.auth import (
allow_any_authenticated,
require_go2rtc_stream_access,
require_camera_access,
require_role,
)
from frigate.api.defs.tags import Tags
from frigate.config.config import FrigateConfig
from frigate.util.builtin import clean_camera_user_pass
from frigate.util.image import run_ffmpeg_snapshot
from frigate.util.services import ffprobe_stream, is_restricted_go2rtc_source
from frigate.util.services import ffprobe_stream
logger = logging.getLogger(__name__)
@@ -71,27 +71,14 @@ def go2rtc_streams():
@router.get(
"/go2rtc/streams/{stream_name}",
dependencies=[Depends(require_go2rtc_stream_access)],
"/go2rtc/streams/{camera_name}", dependencies=[Depends(require_camera_access)]
)
def go2rtc_camera_stream(request: Request, stream_name: str):
def go2rtc_camera_stream(request: Request, camera_name: str):
r = requests.get(
"http://127.0.0.1:1984/api/streams",
params={
"src": stream_name,
"video": "all",
"audio": "all",
"microphone": "",
},
f"http://127.0.0.1:1984/api/streams?src={camera_name}&video=all&audio=all&microphone"
)
if not r.ok:
camera_config = request.app.frigate_config.cameras.get(stream_name)
if camera_config is None:
for camera_name, camera in request.app.frigate_config.cameras.items():
if stream_name in camera.live.streams.values():
camera_config = request.app.frigate_config.cameras.get(camera_name)
break
camera_config = request.app.frigate_config.cameras.get(camera_name)
if camera_config and camera_config.enabled:
logger.error("Failed to fetch streams from go2rtc")
@@ -111,19 +98,6 @@ def go2rtc_camera_stream(request: Request, stream_name: str):
)
def go2rtc_add_stream(request: Request, stream_name: str, src: str = ""):
"""Add or update a go2rtc stream configuration."""
if src and is_restricted_go2rtc_source(src):
logger.warning(
"Rejected go2rtc stream '%s' with restricted source type (echo/expr/exec)",
stream_name,
)
return JSONResponse(
content={
"success": False,
"message": "Restricted stream source type",
},
status_code=400,
)
try:
params = {"name": stream_name}
if src:
+821
View File
@@ -0,0 +1,821 @@
"""Chat and LLM tool calling APIs."""
import base64
import json
import logging
import time
from datetime import datetime
from typing import Any, Dict, Generator, List, Optional
import cv2
from fastapi import APIRouter, Body, Depends, Request
from fastapi.responses import JSONResponse, StreamingResponse
from pydantic import BaseModel
from frigate.api.auth import (
allow_any_authenticated,
get_allowed_cameras_for_filter,
)
from frigate.api.defs.query.events_query_parameters import EventsQueryParams
from frigate.api.defs.request.chat_body import ChatCompletionRequest
from frigate.api.defs.response.chat_response import (
ChatCompletionResponse,
ChatMessageResponse,
ToolCall,
)
from frigate.api.defs.tags import Tags
from frigate.api.event import events
from frigate.genai.utils import build_assistant_message_for_conversation
logger = logging.getLogger(__name__)
router = APIRouter(tags=[Tags.chat])
def _chunk_content(content: str, chunk_size: int = 80) -> Generator[str, None, None]:
"""Yield content in word-aware chunks for streaming."""
if not content:
return
words = content.split(" ")
current: List[str] = []
current_len = 0
for w in words:
current.append(w)
current_len += len(w) + 1
if current_len >= chunk_size:
yield " ".join(current) + " "
current = []
current_len = 0
if current:
yield " ".join(current)
def _format_events_with_local_time(
events_list: List[Dict[str, Any]],
) -> List[Dict[str, Any]]:
"""Add human-readable local start/end times to each event for the LLM."""
result = []
for evt in events_list:
if not isinstance(evt, dict):
result.append(evt)
continue
copy_evt = dict(evt)
try:
start_ts = evt.get("start_time")
end_ts = evt.get("end_time")
if start_ts is not None:
dt_start = datetime.fromtimestamp(start_ts)
copy_evt["start_time_local"] = dt_start.strftime("%Y-%m-%d %I:%M:%S %p")
if end_ts is not None:
dt_end = datetime.fromtimestamp(end_ts)
copy_evt["end_time_local"] = dt_end.strftime("%Y-%m-%d %I:%M:%S %p")
except (TypeError, ValueError, OSError):
pass
result.append(copy_evt)
return result
class ToolExecuteRequest(BaseModel):
"""Request model for tool execution."""
tool_name: str
arguments: Dict[str, Any]
def get_tool_definitions() -> List[Dict[str, Any]]:
"""
Get OpenAI-compatible tool definitions for Frigate.
Returns a list of tool definitions that can be used with OpenAI-compatible
function calling APIs.
"""
return [
{
"type": "function",
"function": {
"name": "search_objects",
"description": (
"Search for detected objects in Frigate by camera, object label, time range, "
"zones, and other filters. Use this to answer questions about when "
"objects were detected, what objects appeared, or to find specific object detections. "
"An 'object' in Frigate represents a tracked detection (e.g., a person, package, car). "
"When the user asks about a specific name (person, delivery company, animal, etc.), "
"filter by sub_label only and do not set label."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to filter by (optional).",
},
"label": {
"type": "string",
"description": "Object label to filter by (e.g., 'person', 'package', 'car').",
},
"sub_label": {
"type": "string",
"description": "Name of a person, delivery company, animal, etc. When filtering by a specific name, use only sub_label; do not set label.",
},
"after": {
"type": "string",
"description": "Start time in ISO 8601 format (e.g., '2024-01-01T00:00:00Z').",
},
"before": {
"type": "string",
"description": "End time in ISO 8601 format (e.g., '2024-01-01T23:59:59Z').",
},
"zones": {
"type": "array",
"items": {"type": "string"},
"description": "List of zone names to filter by.",
},
"limit": {
"type": "integer",
"description": "Maximum number of objects to return (default: 25).",
"default": 25,
},
},
},
"required": [],
},
},
{
"type": "function",
"function": {
"name": "get_live_context",
"description": (
"Get the current detection information for a camera: objects being tracked, "
"zones, timestamps. Use this to understand what is visible in the live view. "
"Call this when the user has included a live image (via include_live_image) or "
"when answering questions about what is happening right now on a specific camera."
),
"parameters": {
"type": "object",
"properties": {
"camera": {
"type": "string",
"description": "Camera name to get live context for.",
},
},
"required": ["camera"],
},
},
},
]
@router.get(
"/chat/tools",
dependencies=[Depends(allow_any_authenticated())],
summary="Get available tools",
description="Returns OpenAI-compatible tool definitions for function calling.",
)
def get_tools() -> JSONResponse:
"""Get list of available tools for LLM function calling."""
tools = get_tool_definitions()
return JSONResponse(content={"tools": tools})
async def _execute_search_objects(
arguments: Dict[str, Any],
allowed_cameras: List[str],
) -> JSONResponse:
"""
Execute the search_objects tool.
This searches for detected objects (events) in Frigate using the same
logic as the events API endpoint.
"""
# Parse after/before as server local time; convert to Unix timestamp
after = arguments.get("after")
before = arguments.get("before")
def _parse_as_local_timestamp(s: str):
s = s.replace("Z", "").strip()[:19]
dt = datetime.strptime(s, "%Y-%m-%dT%H:%M:%S")
return time.mktime(dt.timetuple())
if after:
try:
after = _parse_as_local_timestamp(after)
except (ValueError, AttributeError, TypeError):
logger.warning(f"Invalid 'after' timestamp format: {after}")
after = None
if before:
try:
before = _parse_as_local_timestamp(before)
except (ValueError, AttributeError, TypeError):
logger.warning(f"Invalid 'before' timestamp format: {before}")
before = None
# Convert zones array to comma-separated string if provided
zones = arguments.get("zones")
if isinstance(zones, list):
zones = ",".join(zones)
elif zones is None:
zones = "all"
# Build query parameters compatible with EventsQueryParams
query_params = EventsQueryParams(
cameras=arguments.get("camera", "all"),
labels=arguments.get("label", "all"),
sub_labels=arguments.get("sub_label", "all").lower(),
zones=zones,
zone=zones,
after=after,
before=before,
limit=arguments.get("limit", 25),
)
try:
# Call the events endpoint function directly
# The events function is synchronous and takes params and allowed_cameras
response = events(query_params, allowed_cameras)
# The response is already a JSONResponse with event data
# Return it as-is for the LLM
return response
except Exception as e:
logger.error(f"Error executing search_objects: {e}", exc_info=True)
return JSONResponse(
content={
"success": False,
"message": "Error searching objects",
},
status_code=500,
)
@router.post(
"/chat/execute",
dependencies=[Depends(allow_any_authenticated())],
summary="Execute a tool",
description="Execute a tool function call from an LLM.",
)
async def execute_tool(
body: ToolExecuteRequest = Body(...),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
) -> JSONResponse:
"""
Execute a tool function call.
This endpoint receives tool calls from LLMs and executes the corresponding
Frigate operations, returning results in a format the LLM can understand.
"""
tool_name = body.tool_name
arguments = body.arguments
logger.debug(f"Executing tool: {tool_name} with arguments: {arguments}")
if tool_name == "search_objects":
return await _execute_search_objects(arguments, allowed_cameras)
return JSONResponse(
content={
"success": False,
"message": f"Unknown tool: {tool_name}",
"tool": tool_name,
},
status_code=400,
)
async def _execute_get_live_context(
request: Request,
camera: str,
allowed_cameras: List[str],
) -> Dict[str, Any]:
if camera not in allowed_cameras:
return {
"error": f"Camera '{camera}' not found or access denied",
}
if camera not in request.app.frigate_config.cameras:
return {
"error": f"Camera '{camera}' not found",
}
try:
frame_processor = request.app.detected_frames_processor
camera_state = frame_processor.camera_states.get(camera)
if camera_state is None:
return {
"error": f"Camera '{camera}' state not available",
}
tracked_objects_dict = {}
with camera_state.current_frame_lock:
tracked_objects = camera_state.tracked_objects.copy()
frame_time = camera_state.current_frame_time
for obj_id, tracked_obj in tracked_objects.items():
obj_dict = tracked_obj.to_dict()
if obj_dict.get("frame_time") == frame_time:
tracked_objects_dict[obj_id] = {
"label": obj_dict.get("label"),
"zones": obj_dict.get("current_zones", []),
"sub_label": obj_dict.get("sub_label"),
"stationary": obj_dict.get("stationary", False),
}
return {
"camera": camera,
"timestamp": frame_time,
"detections": list(tracked_objects_dict.values()),
}
except Exception as e:
logger.error(f"Error executing get_live_context: {e}", exc_info=True)
return {
"error": "Error getting live context",
}
async def _get_live_frame_image_url(
request: Request,
camera: str,
allowed_cameras: List[str],
) -> Optional[str]:
"""
Fetch the current live frame for a camera as a base64 data URL.
Returns None if the frame cannot be retrieved. Used when include_live_image
is set to attach the image to the first user message.
"""
if (
camera not in allowed_cameras
or camera not in request.app.frigate_config.cameras
):
return None
try:
frame_processor = request.app.detected_frames_processor
if camera not in frame_processor.camera_states:
return None
frame = frame_processor.get_current_frame(camera, {})
if frame is None:
return None
height, width = frame.shape[:2]
max_dimension = 1024
if height > max_dimension or width > max_dimension:
scale = max_dimension / max(height, width)
frame = cv2.resize(
frame,
(int(width * scale), int(height * scale)),
interpolation=cv2.INTER_AREA,
)
_, img_encoded = cv2.imencode(".jpg", frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
b64 = base64.b64encode(img_encoded.tobytes()).decode("utf-8")
return f"data:image/jpeg;base64,{b64}"
except Exception as e:
logger.debug("Failed to get live frame for %s: %s", camera, e)
return None
async def _execute_tool_internal(
tool_name: str,
arguments: Dict[str, Any],
request: Request,
allowed_cameras: List[str],
) -> Dict[str, Any]:
"""
Internal helper to execute a tool and return the result as a dict.
This is used by the chat completion endpoint to execute tools.
"""
if tool_name == "search_objects":
response = await _execute_search_objects(arguments, allowed_cameras)
try:
if hasattr(response, "body"):
body_str = response.body.decode("utf-8")
return json.loads(body_str)
elif hasattr(response, "content"):
return response.content
else:
return {}
except (json.JSONDecodeError, AttributeError) as e:
logger.warning(f"Failed to extract tool result: {e}")
return {"error": "Failed to parse tool result"}
elif tool_name == "get_live_context":
camera = arguments.get("camera")
if not camera:
logger.error(
"Tool get_live_context failed: camera parameter is required. "
"Arguments: %s",
json.dumps(arguments),
)
return {"error": "Camera parameter is required"}
return await _execute_get_live_context(request, camera, allowed_cameras)
else:
logger.error(
"Tool call failed: unknown tool %r. Expected one of: search_objects, get_live_context. "
"Arguments received: %s",
tool_name,
json.dumps(arguments),
)
return {"error": f"Unknown tool: {tool_name}"}
async def _execute_pending_tools(
pending_tool_calls: List[Dict[str, Any]],
request: Request,
allowed_cameras: List[str],
) -> tuple[List[ToolCall], List[Dict[str, Any]]]:
"""
Execute a list of tool calls; return (ToolCall list for API response, tool result dicts for conversation).
"""
tool_calls_out: List[ToolCall] = []
tool_results: List[Dict[str, Any]] = []
for tool_call in pending_tool_calls:
tool_name = tool_call["name"]
tool_args = tool_call.get("arguments") or {}
tool_call_id = tool_call["id"]
logger.debug(
f"Executing tool: {tool_name} (id: {tool_call_id}) with arguments: {json.dumps(tool_args, indent=2)}"
)
try:
tool_result = await _execute_tool_internal(
tool_name, tool_args, request, allowed_cameras
)
if isinstance(tool_result, dict) and tool_result.get("error"):
logger.error(
"Tool call %s (id: %s) returned error: %s. Arguments: %s",
tool_name,
tool_call_id,
tool_result.get("error"),
json.dumps(tool_args),
)
if tool_name == "search_objects" and isinstance(tool_result, list):
tool_result = _format_events_with_local_time(tool_result)
_keys = {
"id",
"camera",
"label",
"zones",
"start_time_local",
"end_time_local",
"sub_label",
"event_count",
}
tool_result = [
{k: evt[k] for k in _keys if k in evt}
for evt in tool_result
if isinstance(evt, dict)
]
result_content = (
json.dumps(tool_result)
if isinstance(tool_result, (dict, list))
else (tool_result if isinstance(tool_result, str) else str(tool_result))
)
tool_calls_out.append(
ToolCall(name=tool_name, arguments=tool_args, response=result_content)
)
tool_results.append(
{
"role": "tool",
"tool_call_id": tool_call_id,
"content": result_content,
}
)
except Exception as e:
logger.error(
"Error executing tool %s (id: %s): %s. Arguments: %s",
tool_name,
tool_call_id,
e,
json.dumps(tool_args),
exc_info=True,
)
error_content = json.dumps({"error": f"Tool execution failed: {str(e)}"})
tool_calls_out.append(
ToolCall(name=tool_name, arguments=tool_args, response=error_content)
)
tool_results.append(
{
"role": "tool",
"tool_call_id": tool_call_id,
"content": error_content,
}
)
return (tool_calls_out, tool_results)
@router.post(
"/chat/completion",
dependencies=[Depends(allow_any_authenticated())],
summary="Chat completion with tool calling",
description=(
"Send a chat message to the configured GenAI provider with tool calling support. "
"The LLM can call Frigate tools to answer questions about your cameras and events."
),
)
async def chat_completion(
request: Request,
body: ChatCompletionRequest = Body(...),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""
Chat completion endpoint with tool calling support.
This endpoint:
1. Gets the configured GenAI client
2. Gets tool definitions
3. Sends messages + tools to LLM
4. Handles tool_calls if present
5. Executes tools and sends results back to LLM
6. Repeats until final answer
7. Returns response to user
"""
genai_client = request.app.genai_manager.tool_client
if not genai_client:
return JSONResponse(
content={
"error": "GenAI is not configured. Please configure a GenAI provider in your Frigate config.",
},
status_code=400,
)
tools = get_tool_definitions()
conversation = []
current_datetime = datetime.now()
current_date_str = current_datetime.strftime("%Y-%m-%d")
current_time_str = current_datetime.strftime("%I:%M:%S %p")
cameras_info = []
config = request.app.frigate_config
for camera_id in allowed_cameras:
if camera_id not in config.cameras:
continue
camera_config = config.cameras[camera_id]
friendly_name = (
camera_config.friendly_name
if camera_config.friendly_name
else camera_id.replace("_", " ").title()
)
cameras_info.append(f" - {friendly_name} (ID: {camera_id})")
cameras_section = ""
if cameras_info:
cameras_section = (
"\n\nAvailable cameras:\n"
+ "\n".join(cameras_info)
+ "\n\nWhen users refer to cameras by their friendly name (e.g., 'Back Deck Camera'), use the corresponding camera ID (e.g., 'back_deck_cam') in tool calls."
)
live_image_note = ""
if body.include_live_image:
live_image_note = (
f"\n\nThe first user message includes a live image from camera "
f"'{body.include_live_image}'. Use get_live_context for that camera to get "
"current detection details (objects, zones) to aid in understanding the image."
)
system_prompt = f"""You are a helpful assistant for Frigate, a security camera NVR system. You help users answer questions about their cameras, detected objects, and events.
Current server local date and time: {current_date_str} at {current_time_str}
Do not start your response with phrases like "I will check...", "Let me see...", or "Let me look...". Answer directly.
Always present times to the user in the server's local timezone. When tool results include start_time_local and end_time_local, use those exact strings when listing or describing detection times—do not convert or invent timestamps. Do not use UTC or ISO format with Z for the user-facing answer unless the tool result only provides Unix timestamps without local time fields.
When users ask about "today", "yesterday", "this week", etc., use the current date above as reference.
When searching for objects or events, use ISO 8601 format for dates (e.g., {current_date_str}T00:00:00Z for the start of today).
Always be accurate with time calculations based on the current date provided.{cameras_section}{live_image_note}"""
conversation.append(
{
"role": "system",
"content": system_prompt,
}
)
first_user_message_seen = False
for msg in body.messages:
msg_dict = {
"role": msg.role,
"content": msg.content,
}
if msg.tool_call_id:
msg_dict["tool_call_id"] = msg.tool_call_id
if msg.name:
msg_dict["name"] = msg.name
if (
msg.role == "user"
and not first_user_message_seen
and body.include_live_image
):
first_user_message_seen = True
image_url = await _get_live_frame_image_url(
request, body.include_live_image, allowed_cameras
)
if image_url:
msg_dict["content"] = [
{"type": "text", "text": msg.content},
{"type": "image_url", "image_url": {"url": image_url}},
]
conversation.append(msg_dict)
tool_iterations = 0
tool_calls: List[ToolCall] = []
max_iterations = body.max_tool_iterations
logger.debug(
f"Starting chat completion with {len(conversation)} message(s), "
f"{len(tools)} tool(s) available, max_iterations={max_iterations}"
)
# True LLM streaming when client supports it and stream requested
if body.stream and hasattr(genai_client, "chat_with_tools_stream"):
stream_tool_calls: List[ToolCall] = []
stream_iterations = 0
async def stream_body_llm():
nonlocal conversation, stream_tool_calls, stream_iterations
while stream_iterations < max_iterations:
logger.debug(
f"Streaming LLM (iteration {stream_iterations + 1}/{max_iterations}) "
f"with {len(conversation)} message(s)"
)
async for event in genai_client.chat_with_tools_stream(
messages=conversation,
tools=tools if tools else None,
tool_choice="auto",
):
kind, value = event
if kind == "content_delta":
yield (
json.dumps({"type": "content", "delta": value}).encode(
"utf-8"
)
+ b"\n"
)
elif kind == "message":
msg = value
if msg.get("finish_reason") == "error":
yield (
json.dumps(
{
"type": "error",
"error": "An error occurred while processing your request.",
}
).encode("utf-8")
+ b"\n"
)
return
pending = msg.get("tool_calls")
if pending:
stream_iterations += 1
conversation.append(
build_assistant_message_for_conversation(
msg.get("content"), pending
)
)
executed_calls, tool_results = await _execute_pending_tools(
pending, request, allowed_cameras
)
stream_tool_calls.extend(executed_calls)
conversation.extend(tool_results)
yield (
json.dumps(
{
"type": "tool_calls",
"tool_calls": [
tc.model_dump() for tc in stream_tool_calls
],
}
).encode("utf-8")
+ b"\n"
)
break
else:
yield (json.dumps({"type": "done"}).encode("utf-8") + b"\n")
return
else:
yield json.dumps({"type": "done"}).encode("utf-8") + b"\n"
return StreamingResponse(
stream_body_llm(),
media_type="application/x-ndjson",
headers={"X-Accel-Buffering": "no"},
)
try:
while tool_iterations < max_iterations:
logger.debug(
f"Calling LLM (iteration {tool_iterations + 1}/{max_iterations}) "
f"with {len(conversation)} message(s) in conversation"
)
response = genai_client.chat_with_tools(
messages=conversation,
tools=tools if tools else None,
tool_choice="auto",
)
if response.get("finish_reason") == "error":
logger.error("GenAI client returned an error")
return JSONResponse(
content={
"error": "An error occurred while processing your request.",
},
status_code=500,
)
conversation.append(
build_assistant_message_for_conversation(
response.get("content"), response.get("tool_calls")
)
)
pending_tool_calls = response.get("tool_calls")
if not pending_tool_calls:
logger.debug(
f"Chat completion finished with final answer (iterations: {tool_iterations})"
)
final_content = response.get("content") or ""
if body.stream:
async def stream_body() -> Any:
if tool_calls:
yield (
json.dumps(
{
"type": "tool_calls",
"tool_calls": [
tc.model_dump() for tc in tool_calls
],
}
).encode("utf-8")
+ b"\n"
)
# Stream content in word-sized chunks for smooth UX
for part in _chunk_content(final_content):
yield (
json.dumps({"type": "content", "delta": part}).encode(
"utf-8"
)
+ b"\n"
)
yield json.dumps({"type": "done"}).encode("utf-8") + b"\n"
return StreamingResponse(
stream_body(),
media_type="application/x-ndjson",
)
return JSONResponse(
content=ChatCompletionResponse(
message=ChatMessageResponse(
role="assistant",
content=final_content,
tool_calls=None,
),
finish_reason=response.get("finish_reason", "stop"),
tool_iterations=tool_iterations,
tool_calls=tool_calls,
).model_dump(),
)
tool_iterations += 1
logger.debug(
f"Tool calls detected (iteration {tool_iterations}/{max_iterations}): "
f"{len(pending_tool_calls)} tool(s) to execute"
)
executed_calls, tool_results = await _execute_pending_tools(
pending_tool_calls, request, allowed_cameras
)
tool_calls.extend(executed_calls)
conversation.extend(tool_results)
logger.debug(
f"Added {len(tool_results)} tool result(s) to conversation. "
f"Continuing with next LLM call..."
)
logger.warning(
f"Max tool iterations ({max_iterations}) reached. Returning partial response."
)
return JSONResponse(
content=ChatCompletionResponse(
message=ChatMessageResponse(
role="assistant",
content="I reached the maximum number of tool call iterations. Please try rephrasing your question.",
tool_calls=None,
),
finish_reason="length",
tool_iterations=tool_iterations,
tool_calls=tool_calls,
).model_dump(),
)
except Exception as e:
logger.error(f"Error in chat completion: {e}", exc_info=True)
return JSONResponse(
content={
"error": "An error occurred while processing your request.",
},
status_code=500,
)
@@ -1,8 +1,7 @@
from enum import Enum
from typing import Optional, Union
from typing import Optional
from pydantic import BaseModel
from pydantic.json_schema import SkipJsonSchema
class Extension(str, Enum):
@@ -48,15 +47,3 @@ class MediaMjpegFeedQueryParams(BaseModel):
mask: Optional[int] = None
motion: Optional[int] = None
regions: Optional[int] = None
class MediaRecordingsSummaryQueryParams(BaseModel):
timezone: str = "utc"
cameras: Optional[str] = "all"
class MediaRecordingsAvailabilityQueryParams(BaseModel):
cameras: str = "all"
before: Union[float, SkipJsonSchema[None]] = None
after: Union[float, SkipJsonSchema[None]] = None
scale: int = 30
@@ -0,0 +1,21 @@
from typing import Optional, Union
from pydantic import BaseModel
from pydantic.json_schema import SkipJsonSchema
class MediaRecordingsSummaryQueryParams(BaseModel):
timezone: str = "utc"
cameras: Optional[str] = "all"
class MediaRecordingsAvailabilityQueryParams(BaseModel):
cameras: str = "all"
before: Union[float, SkipJsonSchema[None]] = None
after: Union[float, SkipJsonSchema[None]] = None
scale: int = 30
class RecordingsDeleteQueryParams(BaseModel):
keep: Optional[str] = None
cameras: Optional[str] = "all"
+15 -2
View File
@@ -1,6 +1,6 @@
from typing import Any, Dict, Optional
from typing import Any, Dict, List, Optional
from pydantic import BaseModel
from pydantic import BaseModel, Field
class AppConfigSetBody(BaseModel):
@@ -27,3 +27,16 @@ class AppPostLoginBody(BaseModel):
class AppPutRoleBody(BaseModel):
role: str
class MediaSyncBody(BaseModel):
dry_run: bool = Field(
default=True, description="If True, only report orphans without deleting them"
)
media_types: List[str] = Field(
default=["all"],
description="Types of media to sync: 'all', 'event_snapshots', 'event_thumbnails', 'review_thumbnails', 'previews', 'exports', 'recordings'",
)
force: bool = Field(
default=False, description="If True, bypass safety threshold checks"
)
+45
View File
@@ -0,0 +1,45 @@
"""Chat API request models."""
from typing import Optional
from pydantic import BaseModel, Field
class ChatMessage(BaseModel):
"""A single message in a chat conversation."""
role: str = Field(
description="Message role: 'user', 'assistant', 'system', or 'tool'"
)
content: str = Field(description="Message content")
tool_call_id: Optional[str] = Field(
default=None, description="For tool messages, the ID of the tool call"
)
name: Optional[str] = Field(
default=None, description="For tool messages, the tool name"
)
class ChatCompletionRequest(BaseModel):
"""Request for chat completion with tool calling."""
messages: list[ChatMessage] = Field(
description="List of messages in the conversation"
)
max_tool_iterations: int = Field(
default=5,
ge=1,
le=10,
description="Maximum number of tool call iterations (default: 5)",
)
include_live_image: Optional[str] = Field(
default=None,
description=(
"If set, the current live frame from this camera is attached to the first "
"user message as multimodal content. Use with get_live_context for detection info."
),
)
stream: bool = Field(
default=False,
description="If true, stream the final assistant response in the body as newline-delimited JSON.",
)
+1
View File
@@ -41,6 +41,7 @@ class EventsCreateBody(BaseModel):
duration: Optional[int] = 30
include_recording: Optional[bool] = True
draw: Optional[dict] = {}
pre_capture: Optional[int] = None
class EventsEndBody(BaseModel):
@@ -0,0 +1,35 @@
from typing import Optional
from pydantic import BaseModel, Field
class ExportCaseCreateBody(BaseModel):
"""Request body for creating a new export case."""
name: str = Field(max_length=100, description="Friendly name of the export case")
description: Optional[str] = Field(
default=None, description="Optional description of the export case"
)
class ExportCaseUpdateBody(BaseModel):
"""Request body for updating an existing export case."""
name: Optional[str] = Field(
default=None,
max_length=100,
description="Updated friendly name of the export case",
)
description: Optional[str] = Field(
default=None, description="Updated description of the export case"
)
class ExportCaseAssignBody(BaseModel):
"""Request body for assigning or unassigning an export to a case."""
export_case_id: Optional[str] = Field(
default=None,
max_length=30,
description="Case ID to assign to the export, or null to unassign",
)
@@ -3,27 +3,47 @@ from typing import Optional, Union
from pydantic import BaseModel, Field
from pydantic.json_schema import SkipJsonSchema
from frigate.record.export import (
ChaptersEnum,
PlaybackFactorEnum,
PlaybackSourceEnum,
)
from frigate.record.export import PlaybackSourceEnum
class ExportRecordingsBody(BaseModel):
playback: PlaybackFactorEnum = Field(
default=PlaybackFactorEnum.realtime, title="Playback factor"
)
source: PlaybackSourceEnum = Field(
default=PlaybackSourceEnum.recordings, title="Playback source"
)
name: Optional[str] = Field(title="Friendly name", default=None, max_length=256)
image_path: Union[str, SkipJsonSchema[None]] = None
chapters: Optional[ChaptersEnum] = Field(
export_case_id: Optional[str] = Field(
default=None,
title="Chapter mode",
description=(
"Optional chapter metadata to embed in the export. When omitted, "
"no chapter track is added."
),
title="Export case ID",
max_length=30,
description="ID of the export case to assign this export to",
)
class ExportRecordingsCustomBody(BaseModel):
source: PlaybackSourceEnum = Field(
default=PlaybackSourceEnum.recordings, title="Playback source"
)
name: str = Field(title="Friendly name", default=None, max_length=256)
image_path: Union[str, SkipJsonSchema[None]] = None
export_case_id: Optional[str] = Field(
default=None,
title="Export case ID",
max_length=30,
description="ID of the export case to assign this export to",
)
ffmpeg_input_args: Optional[str] = Field(
default=None,
title="FFmpeg input arguments",
description="Custom FFmpeg input arguments. If not provided, defaults to timelapse input args.",
)
ffmpeg_output_args: Optional[str] = Field(
default=None,
title="FFmpeg output arguments",
description="Custom FFmpeg output arguments. If not provided, defaults to timelapse output args.",
)
cpu_fallback: bool = Field(
default=False,
title="CPU Fallback",
description="If true, retry export without hardware acceleration if the initial export fails.",
)
@@ -0,0 +1,54 @@
"""Chat API response models."""
from typing import Any, Optional
from pydantic import BaseModel, Field
class ToolCallInvocation(BaseModel):
"""A tool call requested by the LLM (before execution)."""
id: str = Field(description="Unique identifier for this tool call")
name: str = Field(description="Tool name to call")
arguments: dict[str, Any] = Field(description="Arguments for the tool call")
class ChatMessageResponse(BaseModel):
"""A message in the chat response."""
role: str = Field(description="Message role")
content: Optional[str] = Field(
default=None, description="Message content (None if tool calls present)"
)
tool_calls: Optional[list[ToolCallInvocation]] = Field(
default=None, description="Tool calls if LLM wants to call tools"
)
class ToolCall(BaseModel):
"""A tool that was executed during the completion, with its response."""
name: str = Field(description="Tool name that was called")
arguments: dict[str, Any] = Field(
default_factory=dict, description="Arguments passed to the tool"
)
response: str = Field(
default="",
description="The response or result returned from the tool execution",
)
class ChatCompletionResponse(BaseModel):
"""Response from chat completion."""
message: ChatMessageResponse = Field(description="The assistant's message")
finish_reason: str = Field(
description="Reason generation stopped: 'stop', 'tool_calls', 'length', 'error'"
)
tool_iterations: int = Field(
default=0, description="Number of tool call iterations performed"
)
tool_calls: list[ToolCall] = Field(
default_factory=list,
description="List of tool calls that were executed during this completion",
)
@@ -0,0 +1,22 @@
from typing import List, Optional
from pydantic import BaseModel, Field
class ExportCaseModel(BaseModel):
"""Model representing a single export case."""
id: str = Field(description="Unique identifier for the export case")
name: str = Field(description="Friendly name of the export case")
description: Optional[str] = Field(
default=None, description="Optional description of the export case"
)
created_at: float = Field(
description="Unix timestamp when the export case was created"
)
updated_at: float = Field(
description="Unix timestamp when the export case was last updated"
)
ExportCasesResponse = List[ExportCaseModel]
@@ -15,6 +15,9 @@ class ExportModel(BaseModel):
in_progress: bool = Field(
description="Whether the export is currently being processed"
)
export_case_id: Optional[str] = Field(
default=None, description="ID of the export case this export belongs to"
)
class StartExportResponse(BaseModel):
+7 -5
View File
@@ -3,13 +3,15 @@ from enum import Enum
class Tags(Enum):
app = "App"
auth = "Auth"
camera = "Camera"
preview = "Preview"
chat = "Chat"
events = "Events"
export = "Export"
classification = "Classification"
logs = "Logs"
media = "Media"
notifications = "Notifications"
preview = "Preview"
recordings = "Recordings"
review = "Review"
export = "Export"
events = "Events"
classification = "Classification"
auth = "Auth"
+1
View File
@@ -1782,6 +1782,7 @@ def create_event(
body.duration,
"api",
body.draw,
body.pre_capture,
),
EventMetadataTypeEnum.manual_event_create.value,
)
+331 -36
View File
@@ -4,10 +4,10 @@ import logging
import random
import string
from pathlib import Path
from typing import List
from typing import List, Optional
import psutil
from fastapi import APIRouter, Depends, Request
from fastapi import APIRouter, Depends, Query, Request
from fastapi.responses import JSONResponse
from pathvalidate import sanitize_filepath
from peewee import DoesNotExist
@@ -19,8 +19,20 @@ from frigate.api.auth import (
require_camera_access,
require_role,
)
from frigate.api.defs.request.export_recordings_body import ExportRecordingsBody
from frigate.api.defs.request.export_case_body import (
ExportCaseAssignBody,
ExportCaseCreateBody,
ExportCaseUpdateBody,
)
from frigate.api.defs.request.export_recordings_body import (
ExportRecordingsBody,
ExportRecordingsCustomBody,
)
from frigate.api.defs.request.export_rename_body import ExportRenameBody
from frigate.api.defs.response.export_case_response import (
ExportCaseModel,
ExportCasesResponse,
)
from frigate.api.defs.response.export_response import (
ExportModel,
ExportsResponse,
@@ -29,9 +41,9 @@ from frigate.api.defs.response.export_response import (
from frigate.api.defs.response.generic_response import GenericResponse
from frigate.api.defs.tags import Tags
from frigate.const import CLIPS_DIR, EXPORT_DIR
from frigate.models import Export, Previews, Recordings
from frigate.models import Export, ExportCase, Previews, Recordings
from frigate.record.export import (
PlaybackFactorEnum,
DEFAULT_TIME_LAPSE_FFMPEG_ARGS,
PlaybackSourceEnum,
RecordingExporter,
)
@@ -52,17 +64,182 @@ router = APIRouter(tags=[Tags.export])
)
def get_exports(
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
export_case_id: Optional[str] = None,
cameras: Optional[str] = Query(default="all"),
start_date: Optional[float] = None,
end_date: Optional[float] = None,
):
exports = (
Export.select()
.where(Export.camera << allowed_cameras)
.order_by(Export.date.desc())
.dicts()
.iterator()
)
query = Export.select().where(Export.camera << allowed_cameras)
if export_case_id is not None:
if export_case_id == "unassigned":
query = query.where(Export.export_case.is_null(True))
else:
query = query.where(Export.export_case == export_case_id)
if cameras and cameras != "all":
requested = set(cameras.split(","))
filtered_cameras = list(requested.intersection(allowed_cameras))
if not filtered_cameras:
return JSONResponse(content=[])
query = query.where(Export.camera << filtered_cameras)
if start_date is not None:
query = query.where(Export.date >= start_date)
if end_date is not None:
query = query.where(Export.date <= end_date)
exports = query.order_by(Export.date.desc()).dicts().iterator()
return JSONResponse(content=[e for e in exports])
@router.get(
"/cases",
response_model=ExportCasesResponse,
dependencies=[Depends(allow_any_authenticated())],
summary="Get export cases",
description="Gets all export cases from the database.",
)
def get_export_cases():
cases = (
ExportCase.select().order_by(ExportCase.created_at.desc()).dicts().iterator()
)
return JSONResponse(content=[c for c in cases])
@router.post(
"/cases",
response_model=ExportCaseModel,
dependencies=[Depends(require_role(["admin"]))],
summary="Create export case",
description="Creates a new export case.",
)
def create_export_case(body: ExportCaseCreateBody):
case = ExportCase.create(
id="".join(random.choices(string.ascii_lowercase + string.digits, k=12)),
name=body.name,
description=body.description,
created_at=Path().stat().st_mtime,
updated_at=Path().stat().st_mtime,
)
return JSONResponse(content=model_to_dict(case))
@router.get(
"/cases/{case_id}",
response_model=ExportCaseModel,
dependencies=[Depends(allow_any_authenticated())],
summary="Get a single export case",
description="Gets a specific export case by ID.",
)
def get_export_case(case_id: str):
try:
case = ExportCase.get(ExportCase.id == case_id)
return JSONResponse(content=model_to_dict(case))
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found"},
status_code=404,
)
@router.patch(
"/cases/{case_id}",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Update export case",
description="Updates an existing export case.",
)
def update_export_case(case_id: str, body: ExportCaseUpdateBody):
try:
case = ExportCase.get(ExportCase.id == case_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found"},
status_code=404,
)
if body.name is not None:
case.name = body.name
if body.description is not None:
case.description = body.description
case.save()
return JSONResponse(
content={"success": True, "message": "Successfully updated export case."}
)
@router.delete(
"/cases/{case_id}",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Delete export case",
description="""Deletes an export case.\n Exports that reference this case will have their export_case set to null.\n """,
)
def delete_export_case(case_id: str):
try:
case = ExportCase.get(ExportCase.id == case_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found"},
status_code=404,
)
# Unassign exports from this case but keep the exports themselves
Export.update(export_case=None).where(Export.export_case == case).execute()
case.delete_instance()
return JSONResponse(
content={"success": True, "message": "Successfully deleted export case."}
)
@router.patch(
"/export/{export_id}/case",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Assign export to case",
description=(
"Assigns an export to a case, or unassigns it if export_case_id is null."
),
)
async def assign_export_case(
export_id: str,
body: ExportCaseAssignBody,
request: Request,
):
try:
export: Export = Export.get(Export.id == export_id)
await require_camera_access(export.camera, request=request)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export not found."},
status_code=404,
)
if body.export_case_id is not None:
try:
ExportCase.get(ExportCase.id == body.export_case_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found."},
status_code=404,
)
export.export_case = body.export_case_id
else:
export.export_case = None
export.save()
return JSONResponse(
content={"success": True, "message": "Successfully updated export case."}
)
@router.post(
"/export/{camera_name}/start/{start_time}/end/{end_time}",
response_model=StartExportResponse,
@@ -88,28 +265,19 @@ def export_recording(
status_code=404,
)
playback_factor = body.playback
playback_source = body.source
friendly_name = body.name
# sanitize_filepath normalizes "\" to "/" but leaves ".." intact, so a path
# like "clips\..\..\etc/passwd" passes the CLIPS_DIR prefix check yet still
# escapes the directory once resolved. A valid snapshot path never uses "..".
if body.image_path and ".." in body.image_path:
return JSONResponse(
content=({"success": False, "message": "Invalid image path"}),
status_code=400,
)
existing_image = sanitize_filepath(body.image_path) if body.image_path else None
# a chapters value in the request body overrides the camera's export config
camera_config = request.app.frigate_config.cameras[camera_name]
chapters = (
body.chapters
if body.chapters is not None
else camera_config.record.export.chapters
)
export_case_id = body.export_case_id
if export_case_id is not None:
try:
ExportCase.get(ExportCase.id == export_case_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found"},
status_code=404,
)
# Ensure that existing_image is a valid path
if existing_image and not existing_image.startswith(CLIPS_DIR):
@@ -169,17 +337,12 @@ def export_recording(
existing_image,
int(start_time),
int(end_time),
(
PlaybackFactorEnum[playback_factor]
if playback_factor in PlaybackFactorEnum.__members__.values()
else PlaybackFactorEnum.realtime
),
(
PlaybackSourceEnum[playback_source]
if playback_source in PlaybackSourceEnum.__members__.values()
else PlaybackSourceEnum.recordings
),
chapters=chapters,
export_case_id,
)
exporter.start()
return JSONResponse(
@@ -290,6 +453,138 @@ async def export_delete(event_id: str, request: Request):
)
@router.post(
"/export/custom/{camera_name}/start/{start_time}/end/{end_time}",
response_model=StartExportResponse,
dependencies=[Depends(require_camera_access)],
summary="Start custom recording export",
description="""Starts an export of a recording for the specified time range using custom FFmpeg arguments.
The export can be from recordings or preview footage. Returns the export ID if
successful, or an error message if the camera is invalid or no recordings/previews
are found for the time range. If ffmpeg_input_args and ffmpeg_output_args are not provided,
defaults to timelapse export settings.""",
)
def export_recording_custom(
request: Request,
camera_name: str,
start_time: float,
end_time: float,
body: ExportRecordingsCustomBody,
):
if not camera_name or not request.app.frigate_config.cameras.get(camera_name):
return JSONResponse(
content=(
{"success": False, "message": f"{camera_name} is not a valid camera."}
),
status_code=404,
)
playback_source = body.source
friendly_name = body.name
existing_image = sanitize_filepath(body.image_path) if body.image_path else None
ffmpeg_input_args = body.ffmpeg_input_args
ffmpeg_output_args = body.ffmpeg_output_args
cpu_fallback = body.cpu_fallback
export_case_id = body.export_case_id
if export_case_id is not None:
try:
ExportCase.get(ExportCase.id == export_case_id)
except DoesNotExist:
return JSONResponse(
content={"success": False, "message": "Export case not found"},
status_code=404,
)
# Ensure that existing_image is a valid path
if existing_image and not existing_image.startswith(CLIPS_DIR):
return JSONResponse(
content=({"success": False, "message": "Invalid image path"}),
status_code=400,
)
if playback_source == "recordings":
recordings_count = (
Recordings.select()
.where(
Recordings.start_time.between(start_time, end_time)
| Recordings.end_time.between(start_time, end_time)
| (
(start_time > Recordings.start_time)
& (end_time < Recordings.end_time)
)
)
.where(Recordings.camera == camera_name)
.count()
)
if recordings_count <= 0:
return JSONResponse(
content=(
{"success": False, "message": "No recordings found for time range"}
),
status_code=400,
)
else:
previews_count = (
Previews.select()
.where(
Previews.start_time.between(start_time, end_time)
| Previews.end_time.between(start_time, end_time)
| ((start_time > Previews.start_time) & (end_time < Previews.end_time))
)
.where(Previews.camera == camera_name)
.count()
)
if not is_current_hour(start_time) and previews_count <= 0:
return JSONResponse(
content=(
{"success": False, "message": "No previews found for time range"}
),
status_code=400,
)
export_id = f"{camera_name}_{''.join(random.choices(string.ascii_lowercase + string.digits, k=6))}"
# Set default values if not provided (timelapse defaults)
if ffmpeg_input_args is None:
ffmpeg_input_args = ""
if ffmpeg_output_args is None:
ffmpeg_output_args = DEFAULT_TIME_LAPSE_FFMPEG_ARGS
exporter = RecordingExporter(
request.app.frigate_config,
export_id,
camera_name,
friendly_name,
existing_image,
int(start_time),
int(end_time),
(
PlaybackSourceEnum[playback_source]
if playback_source in PlaybackSourceEnum.__members__.values()
else PlaybackSourceEnum.recordings
),
export_case_id,
ffmpeg_input_args,
ffmpeg_output_args,
cpu_fallback,
)
exporter.start()
return JSONResponse(
content=(
{
"success": True,
"message": "Starting export of recording.",
"export_id": export_id,
}
),
status_code=200,
)
@router.get(
"/exports/{export_id}",
response_model=ExportModel,
+6
View File
@@ -16,12 +16,14 @@ from frigate.api import app as main_app
from frigate.api import (
auth,
camera,
chat,
classification,
event,
export,
media,
notification,
preview,
record,
review,
)
from frigate.api.auth import get_jwt_secret, limiter, require_admin_by_default
@@ -31,6 +33,7 @@ from frigate.comms.event_metadata_updater import (
from frigate.config import FrigateConfig
from frigate.config.camera.updater import CameraConfigUpdatePublisher
from frigate.embeddings import EmbeddingsContext
from frigate.genai import GenAIClientManager
from frigate.ptz.onvif import OnvifController
from frigate.stats.emitter import StatsEmitter
from frigate.storage import StorageMaintainer
@@ -120,6 +123,7 @@ def create_fastapi_app(
# Order of include_router matters: https://fastapi.tiangolo.com/tutorial/path-params/#order-matters
app.include_router(auth.router)
app.include_router(camera.router)
app.include_router(chat.router)
app.include_router(classification.router)
app.include_router(review.router)
app.include_router(main_app.router)
@@ -128,8 +132,10 @@ def create_fastapi_app(
app.include_router(export.router)
app.include_router(event.router)
app.include_router(media.router)
app.include_router(record.router)
# App Properties
app.frigate_config = frigate_config
app.genai_manager = GenAIClientManager(frigate_config)
app.embeddings = embeddings
app.detected_frames_processor = detected_frames_processor
app.storage_maintainer = storage_maintainer
+82 -437
View File
@@ -8,9 +8,8 @@ import os
import subprocess as sp
import time
from datetime import datetime, timedelta, timezone
from functools import reduce
from pathlib import Path as FilePath
from typing import Any, List
from typing import Any
from urllib.parse import unquote
import cv2
@@ -19,12 +18,11 @@ import pytz
from fastapi import APIRouter, Depends, Path, Query, Request, Response
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
from pathvalidate import sanitize_filename
from peewee import DoesNotExist, fn, operator
from peewee import DoesNotExist, fn
from tzlocal import get_localzone_name
from frigate.api.auth import (
allow_any_authenticated,
get_allowed_cameras_for_filter,
require_camera_access,
)
from frigate.api.defs.query.media_query_parameters import (
@@ -32,8 +30,6 @@ from frigate.api.defs.query.media_query_parameters import (
MediaEventsSnapshotQueryParams,
MediaLatestFrameQueryParams,
MediaMjpegFeedQueryParams,
MediaRecordingsAvailabilityQueryParams,
MediaRecordingsSummaryQueryParams,
)
from frigate.api.defs.tags import Tags
from frigate.camera.state import CameraState
@@ -44,18 +40,15 @@ from frigate.const import (
INSTALL_DIR,
MAX_SEGMENT_DURATION,
PREVIEW_FRAME_TYPE,
RECORD_DIR,
)
from frigate.models import Event, Previews, Recordings, Regions, ReviewSegment
from frigate.output.preview import get_most_recent_preview_frame
from frigate.track.object_processing import TrackedObjectProcessor
from frigate.util.file import get_event_thumbnail_bytes
from frigate.util.image import get_image_from_recording
from frigate.util.media import get_keyframe_before
from frigate.util.time import get_dst_transitions
logger = logging.getLogger(__name__)
router = APIRouter(tags=[Tags.media])
@@ -133,7 +126,9 @@ async def camera_ptz_info(request: Request, camera_name: str):
@router.get(
"/{camera_name}/latest.{extension}", dependencies=[Depends(require_camera_access)]
"/{camera_name}/latest.{extension}",
dependencies=[Depends(require_camera_access)],
description="Returns the latest frame from the specified camera in the requested format (jpg, png, webp). Falls back to preview frames if the camera is offline.",
)
async def latest_frame(
request: Request,
@@ -167,20 +162,37 @@ async def latest_frame(
or 10
)
is_offline = False
if frame is None or datetime.now().timestamp() > (
frame_processor.get_current_frame_time(camera_name) + retry_interval
):
if request.app.camera_error_image is None:
error_image = glob.glob(
os.path.join(INSTALL_DIR, "frigate/images/camera-error.jpg")
)
last_frame_time = frame_processor.get_current_frame_time(camera_name)
preview_path = get_most_recent_preview_frame(
camera_name, before=last_frame_time
)
if len(error_image) > 0:
request.app.camera_error_image = cv2.imread(
error_image[0], cv2.IMREAD_UNCHANGED
if preview_path:
logger.debug(f"Using most recent preview frame for {camera_name}")
frame = cv2.imread(preview_path, cv2.IMREAD_UNCHANGED)
if frame is not None:
is_offline = True
if frame is None or not is_offline:
logger.debug(
f"No live or preview frame available for {camera_name}. Using error image."
)
if request.app.camera_error_image is None:
error_image = glob.glob(
os.path.join(INSTALL_DIR, "frigate/images/camera-error.jpg")
)
frame = request.app.camera_error_image
if len(error_image) > 0:
request.app.camera_error_image = cv2.imread(
error_image[0], cv2.IMREAD_UNCHANGED
)
frame = request.app.camera_error_image
height = int(params.height or str(frame.shape[0]))
width = int(height * frame.shape[1] / frame.shape[0])
@@ -202,14 +214,18 @@ async def latest_frame(
frame = cv2.resize(frame, dsize=(width, height), interpolation=cv2.INTER_AREA)
_, img = cv2.imencode(f".{extension.value}", frame, quality_params)
headers = {
"Cache-Control": "no-store" if not params.store else "private, max-age=60",
}
if is_offline:
headers["X-Frigate-Offline"] = "true"
return Response(
content=img.tobytes(),
media_type=extension.get_mime_type(),
headers={
"Cache-Control": "no-store"
if not params.store
else "private, max-age=60",
},
headers=headers,
)
elif (
camera_name == "birdseye"
@@ -399,333 +415,6 @@ async def submit_recording_snapshot_to_plus(
)
@router.get("/recordings/storage", dependencies=[Depends(allow_any_authenticated())])
def get_recordings_storage_usage(request: Request):
recording_stats = request.app.stats_emitter.get_latest_stats()["service"][
"storage"
][RECORD_DIR]
if not recording_stats:
return JSONResponse({})
total_mb = recording_stats["total"]
camera_usages: dict[str, dict] = (
request.app.storage_maintainer.calculate_camera_usages()
)
for camera_name in camera_usages.keys():
if camera_usages.get(camera_name, {}).get("usage"):
camera_usages[camera_name]["usage_percent"] = (
camera_usages.get(camera_name, {}).get("usage", 0) / total_mb
) * 100
return JSONResponse(content=camera_usages)
@router.get("/recordings/summary", dependencies=[Depends(allow_any_authenticated())])
def all_recordings_summary(
request: Request,
params: MediaRecordingsSummaryQueryParams = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""Returns true/false by day indicating if recordings exist"""
cameras = params.cameras
if cameras != "all":
requested = set(unquote(cameras).split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(content={})
camera_list = list(filtered)
else:
camera_list = allowed_cameras
time_range_query = (
Recordings.select(
fn.MIN(Recordings.start_time).alias("min_time"),
fn.MAX(Recordings.start_time).alias("max_time"),
)
.where(Recordings.camera << camera_list)
.dicts()
.get()
)
min_time = time_range_query.get("min_time")
max_time = time_range_query.get("max_time")
if min_time is None or max_time is None:
return JSONResponse(content={})
dst_periods = get_dst_transitions(params.timezone, min_time, max_time)
days: dict[str, bool] = {}
for period_start, period_end, period_offset in dst_periods:
hours_offset = int(period_offset / 60 / 60)
minutes_offset = int(period_offset / 60 - hours_offset * 60)
period_hour_modifier = f"{hours_offset} hour"
period_minute_modifier = f"{minutes_offset} minute"
period_query = (
Recordings.select(
fn.strftime(
"%Y-%m-%d",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("day")
)
.where(
(Recordings.camera << camera_list)
& (Recordings.end_time >= period_start)
& (Recordings.start_time <= period_end)
)
.group_by(
fn.strftime(
"%Y-%m-%d",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
)
)
.order_by(Recordings.start_time.desc())
.namedtuples()
)
for g in period_query:
days[g.day] = True
return JSONResponse(content=dict(sorted(days.items())))
@router.get(
"/{camera_name}/recordings/summary", dependencies=[Depends(require_camera_access)]
)
async def recordings_summary(camera_name: str, timezone: str = "utc"):
"""Returns hourly summary for recordings of given camera"""
time_range_query = (
Recordings.select(
fn.MIN(Recordings.start_time).alias("min_time"),
fn.MAX(Recordings.start_time).alias("max_time"),
)
.where(Recordings.camera == camera_name)
.dicts()
.get()
)
min_time = time_range_query.get("min_time")
max_time = time_range_query.get("max_time")
days: dict[str, dict] = {}
if min_time is None or max_time is None:
return JSONResponse(content=list(days.values()))
dst_periods = get_dst_transitions(timezone, min_time, max_time)
for period_start, period_end, period_offset in dst_periods:
hours_offset = int(period_offset / 60 / 60)
minutes_offset = int(period_offset / 60 - hours_offset * 60)
period_hour_modifier = f"{hours_offset} hour"
period_minute_modifier = f"{minutes_offset} minute"
recording_groups = (
Recordings.select(
fn.strftime(
"%Y-%m-%d %H",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("hour"),
fn.SUM(Recordings.duration).alias("duration"),
fn.SUM(Recordings.motion).alias("motion"),
fn.SUM(Recordings.objects).alias("objects"),
)
.where(
(Recordings.camera == camera_name)
& (Recordings.end_time >= period_start)
& (Recordings.start_time <= period_end)
)
.group_by((Recordings.start_time + period_offset).cast("int") / 3600)
.order_by(Recordings.start_time.desc())
.namedtuples()
)
event_groups = (
Event.select(
fn.strftime(
"%Y-%m-%d %H",
fn.datetime(
Event.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("hour"),
fn.COUNT(Event.id).alias("count"),
)
.where(Event.camera == camera_name, Event.has_clip)
.where(
(Event.start_time >= period_start) & (Event.start_time <= period_end)
)
.group_by((Event.start_time + period_offset).cast("int") / 3600)
.namedtuples()
)
event_map = {g.hour: g.count for g in event_groups}
for recording_group in recording_groups:
parts = recording_group.hour.split()
hour = parts[1]
day = parts[0]
events_count = event_map.get(recording_group.hour, 0)
hour_data = {
"hour": hour,
"events": events_count,
"motion": recording_group.motion,
"objects": recording_group.objects,
"duration": round(recording_group.duration),
}
if day in days:
# merge counts if already present (edge-case at DST boundary)
days[day]["events"] += events_count or 0
days[day]["hours"].append(hour_data)
else:
days[day] = {
"events": events_count or 0,
"hours": [hour_data],
"day": day,
}
return JSONResponse(content=list(days.values()))
@router.get("/{camera_name}/recordings", dependencies=[Depends(require_camera_access)])
async def recordings(
camera_name: str,
after: float = (datetime.now() - timedelta(hours=1)).timestamp(),
before: float = datetime.now().timestamp(),
):
"""Return specific camera recordings between the given 'after'/'end' times. If not provided the last hour will be used"""
recordings = (
Recordings.select(
Recordings.id,
Recordings.start_time,
Recordings.end_time,
Recordings.segment_size,
Recordings.motion,
Recordings.objects,
Recordings.duration,
)
.where(
Recordings.camera == camera_name,
Recordings.end_time >= after,
Recordings.start_time <= before,
)
.order_by(Recordings.start_time)
.dicts()
.iterator()
)
return JSONResponse(content=list(recordings))
@router.get(
"/recordings/unavailable",
response_model=list[dict],
dependencies=[Depends(allow_any_authenticated())],
)
async def no_recordings(
request: Request,
params: MediaRecordingsAvailabilityQueryParams = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""Get time ranges with no recordings."""
cameras = params.cameras
if cameras != "all":
requested = set(unquote(cameras).split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(content=[])
cameras = ",".join(filtered)
else:
cameras = allowed_cameras
before = params.before or datetime.datetime.now().timestamp()
after = (
params.after
or (datetime.datetime.now() - datetime.timedelta(hours=1)).timestamp()
)
scale = params.scale
clauses = [(Recordings.end_time >= after) & (Recordings.start_time <= before)]
if cameras != "all":
camera_list = cameras.split(",")
clauses.append((Recordings.camera << camera_list))
else:
camera_list = allowed_cameras
# Get recording start times
data: list[Recordings] = (
Recordings.select(Recordings.start_time, Recordings.end_time)
.where(reduce(operator.and_, clauses))
.order_by(Recordings.start_time.asc())
.dicts()
.iterator()
)
# Convert recordings to list of (start, end) tuples
recordings = [(r["start_time"], r["end_time"]) for r in data]
# Iterate through time segments and check if each has any recording
no_recording_segments = []
current = after
current_gap_start = None
while current < before:
segment_end = min(current + scale, before)
# Check if this segment overlaps with any recording
has_recording = any(
rec_start < segment_end and rec_end > current
for rec_start, rec_end in recordings
)
if not has_recording:
# This segment has no recordings
if current_gap_start is None:
current_gap_start = current # Start a new gap
else:
# This segment has recordings
if current_gap_start is not None:
# End the current gap and append it
no_recording_segments.append(
{"start_time": int(current_gap_start), "end_time": int(current)}
)
current_gap_start = None
current = segment_end
# Append the last gap if it exists
if current_gap_start is not None:
no_recording_segments.append(
{"start_time": int(current_gap_start), "end_time": int(before)}
)
return JSONResponse(content=no_recording_segments)
@router.get(
"/{camera_name}/start/{start_ts}/end/{end_ts}/clip.mp4",
dependencies=[Depends(require_camera_access)],
@@ -902,33 +591,6 @@ async def vod_ts(
if recording.end_time > end_ts:
duration -= int((recording.end_time - end_ts) * 1000)
# nginx-vod-module pushes clipFrom forward to the next keyframe,
# which can leave too few frames and produce an empty/unplayable
# segment. Snap clipFrom back to the preceding keyframe so the
# segment always starts with a decodable frame.
if "clipFrom" in clip:
keyframe_ms = get_keyframe_before(recording.path, clip["clipFrom"])
if keyframe_ms is not None:
gained = clip["clipFrom"] - keyframe_ms
clip["clipFrom"] = keyframe_ms
duration += gained
logger.debug(
"VOD: snapped clipFrom to keyframe at %sms for %s, duration now %sms",
keyframe_ms,
recording.path,
duration,
)
else:
# could not read keyframes, remove clipFrom to use full recording
logger.debug(
"VOD: no keyframe info for %s, removing clipFrom to use full recording",
recording.path,
)
del clip["clipFrom"]
duration = int(recording.duration * 1000)
if recording.end_time > end_ts:
duration -= int((recording.end_time - end_ts) * 1000)
if duration < min_duration_ms:
# skip if the clip has no valid duration (too short to contain frames)
logger.debug(
@@ -1075,6 +737,7 @@ async def event_snapshot(
):
event_complete = False
jpg_bytes = None
frame_time = 0
try:
event = Event.get(Event.id == event_id, Event.end_time != None)
event_complete = True
@@ -1099,7 +762,7 @@ async def event_snapshot(
if event_id in camera_state.tracked_objects:
tracked_obj = camera_state.tracked_objects.get(event_id)
if tracked_obj is not None:
jpg_bytes = tracked_obj.get_img_bytes(
jpg_bytes, frame_time = tracked_obj.get_img_bytes(
ext="jpg",
timestamp=params.timestamp,
bounding_box=params.bbox,
@@ -1128,6 +791,7 @@ async def event_snapshot(
headers = {
"Content-Type": "image/jpeg",
"Cache-Control": "private, max-age=31536000" if event_complete else "no-store",
"X-Frame-Time": str(frame_time),
}
if params.download:
@@ -1142,6 +806,7 @@ async def event_snapshot(
@router.get(
"/events/{event_id}/thumbnail.{extension}",
dependencies=[Depends(require_camera_access)],
)
async def event_thumbnail(
request: Request,
@@ -1185,12 +850,11 @@ async def event_thumbnail(
status_code=404,
)
img_as_np = np.frombuffer(thumbnail_bytes, dtype=np.uint8)
img = cv2.imdecode(img_as_np, flags=1)
# android notifications prefer a 2:1 ratio
if format == "android":
img = cv2.copyMakeBorder(
img_as_np = np.frombuffer(thumbnail_bytes, dtype=np.uint8)
img = cv2.imdecode(img_as_np, flags=1)
thumbnail = cv2.copyMakeBorder(
img,
0,
0,
@@ -1200,14 +864,14 @@ async def event_thumbnail(
(0, 0, 0),
)
quality_params = None
if extension in (Extension.jpg, Extension.jpeg):
quality_params = [int(cv2.IMWRITE_JPEG_QUALITY), 70]
elif extension == Extension.webp:
quality_params = [int(cv2.IMWRITE_WEBP_QUALITY), 60]
quality_params = None
if extension in (Extension.jpg, Extension.jpeg):
quality_params = [int(cv2.IMWRITE_JPEG_QUALITY), 70]
elif extension == Extension.webp:
quality_params = [int(cv2.IMWRITE_WEBP_QUALITY), 60]
_, encoded = cv2.imencode(f".{extension.value}", img, quality_params)
thumbnail_bytes = encoded.tobytes()
_, img = cv2.imencode(f".{extension.value}", thumbnail, quality_params)
thumbnail_bytes = img.tobytes()
return Response(
thumbnail_bytes,
@@ -1343,12 +1007,12 @@ def grid_snapshot(
@router.get(
"/events/{event_id}/snapshot-clean.webp",
dependencies=[Depends(require_camera_access)],
)
async def event_snapshot_clean(request: Request, event_id: str, download: bool = False):
def event_snapshot_clean(request: Request, event_id: str, download: bool = False):
webp_bytes = None
try:
event = Event.get(Event.id == event_id)
await require_camera_access(event.camera, request=request)
snapshot_config = request.app.frigate_config.cameras[event.camera].snapshots
if not (snapshot_config.enabled and event.has_snapshot):
return JSONResponse(
@@ -1469,7 +1133,7 @@ async def event_snapshot_clean(request: Request, event_id: str, download: bool =
@router.get(
"/events/{event_id}/clip.mp4",
"/events/{event_id}/clip.mp4", dependencies=[Depends(require_camera_access)]
)
async def event_clip(
request: Request,
@@ -1483,8 +1147,6 @@ async def event_clip(
content={"success": False, "message": "Event not found"}, status_code=404
)
await require_camera_access(event.camera, request=request)
if not event.has_clip:
return JSONResponse(
content={"success": False, "message": "Clip not available"}, status_code=404
@@ -1501,9 +1163,9 @@ async def event_clip(
@router.get(
"/events/{event_id}/preview.gif",
"/events/{event_id}/preview.gif", dependencies=[Depends(require_camera_access)]
)
async def event_preview(request: Request, event_id: str):
def event_preview(request: Request, event_id: str):
try:
event: Event = Event.get(Event.id == event_id)
except DoesNotExist:
@@ -1511,8 +1173,6 @@ async def event_preview(request: Request, event_id: str):
content={"success": False, "message": "Event not found"}, status_code=404
)
await require_camera_access(event.camera, request=request)
start_ts = event.start_time
end_ts = start_ts + (
min(event.end_time - event.start_time, 20) if event.end_time else 20
@@ -1535,25 +1195,25 @@ def preview_gif(
):
if datetime.fromtimestamp(start_ts) < datetime.now().replace(minute=0, second=0):
# has preview mp4
try:
preview: Previews = (
Previews.select(
Previews.camera,
Previews.path,
Previews.duration,
Previews.start_time,
Previews.end_time,
)
.where(
Previews.start_time.between(start_ts, end_ts)
| Previews.end_time.between(start_ts, end_ts)
| ((start_ts > Previews.start_time) & (end_ts < Previews.end_time))
)
.where(Previews.camera == camera_name)
.limit(1)
.get()
preview: Previews = (
Previews.select(
Previews.camera,
Previews.path,
Previews.duration,
Previews.start_time,
Previews.end_time,
)
except DoesNotExist:
.where(
Previews.start_time.between(start_ts, end_ts)
| Previews.end_time.between(start_ts, end_ts)
| ((start_ts > Previews.start_time) & (end_ts < Previews.end_time))
)
.where(Previews.camera == camera_name)
.limit(1)
.get()
)
if not preview:
return JSONResponse(
content={"success": False, "message": "Preview not found"},
status_code=404,
@@ -1857,8 +1517,8 @@ def preview_mp4(
)
@router.get("/review/{event_id}/preview")
async def review_preview(
@router.get("/review/{event_id}/preview", dependencies=[Depends(require_camera_access)])
def review_preview(
request: Request,
event_id: str,
format: str = Query(default="gif", enum=["gif", "mp4"]),
@@ -1871,8 +1531,6 @@ async def review_preview(
status_code=404,
)
await require_camera_access(review.camera, request=request)
padding = 8
start_ts = review.start_time - padding
end_ts = (
@@ -1886,14 +1544,12 @@ async def review_preview(
@router.get(
"/preview/{file_name}/thumbnail.jpg",
dependencies=[Depends(allow_any_authenticated())],
"/preview/{file_name}/thumbnail.jpg", dependencies=[Depends(require_camera_access)]
)
@router.get(
"/preview/{file_name}/thumbnail.webp",
dependencies=[Depends(allow_any_authenticated())],
"/preview/{file_name}/thumbnail.webp", dependencies=[Depends(require_camera_access)]
)
async def preview_thumbnail(request: Request, file_name: str):
def preview_thumbnail(file_name: str):
"""Get a thumbnail from the cached preview frames."""
if len(file_name) > 1000:
return JSONResponse(
@@ -1903,17 +1559,6 @@ async def preview_thumbnail(request: Request, file_name: str):
status_code=403,
)
# Extract camera name from preview filename (format: preview_{camera}-{timestamp}.ext)
if not file_name.startswith("preview_"):
return JSONResponse(
content={"success": False, "message": "Invalid preview filename"},
status_code=400,
)
# Use rsplit to handle camera names containing dashes (e.g. front-door)
name_part = file_name[len("preview_") :].rsplit(".", 1)[0] # strip extension
camera_name = name_part.rsplit("-", 1)[0] # split off timestamp
await require_camera_access(camera_name, request=request)
safe_file_name_current = sanitize_filename(file_name)
preview_dir = os.path.join(CACHE_DIR, "preview_frames")
+479
View File
@@ -0,0 +1,479 @@
"""Recording APIs."""
import logging
from datetime import datetime, timedelta
from functools import reduce
from pathlib import Path
from typing import List
from urllib.parse import unquote
from fastapi import APIRouter, Depends, Request
from fastapi import Path as PathParam
from fastapi.responses import JSONResponse
from peewee import fn, operator
from frigate.api.auth import (
allow_any_authenticated,
get_allowed_cameras_for_filter,
require_camera_access,
require_role,
)
from frigate.api.defs.query.recordings_query_parameters import (
MediaRecordingsAvailabilityQueryParams,
MediaRecordingsSummaryQueryParams,
RecordingsDeleteQueryParams,
)
from frigate.api.defs.response.generic_response import GenericResponse
from frigate.api.defs.tags import Tags
from frigate.const import RECORD_DIR
from frigate.models import Event, Recordings
from frigate.util.time import get_dst_transitions
logger = logging.getLogger(__name__)
router = APIRouter(tags=[Tags.recordings])
@router.get("/recordings/storage", dependencies=[Depends(allow_any_authenticated())])
def get_recordings_storage_usage(request: Request):
recording_stats = request.app.stats_emitter.get_latest_stats()["service"][
"storage"
][RECORD_DIR]
if not recording_stats:
return JSONResponse({})
total_mb = recording_stats["total"]
camera_usages: dict[str, dict] = (
request.app.storage_maintainer.calculate_camera_usages()
)
for camera_name in camera_usages.keys():
if camera_usages.get(camera_name, {}).get("usage"):
camera_usages[camera_name]["usage_percent"] = (
camera_usages.get(camera_name, {}).get("usage", 0) / total_mb
) * 100
return JSONResponse(content=camera_usages)
@router.get("/recordings/summary", dependencies=[Depends(allow_any_authenticated())])
def all_recordings_summary(
request: Request,
params: MediaRecordingsSummaryQueryParams = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""Returns true/false by day indicating if recordings exist"""
cameras = params.cameras
if cameras != "all":
requested = set(unquote(cameras).split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(content={})
camera_list = list(filtered)
else:
camera_list = allowed_cameras
time_range_query = (
Recordings.select(
fn.MIN(Recordings.start_time).alias("min_time"),
fn.MAX(Recordings.start_time).alias("max_time"),
)
.where(Recordings.camera << camera_list)
.dicts()
.get()
)
min_time = time_range_query.get("min_time")
max_time = time_range_query.get("max_time")
if min_time is None or max_time is None:
return JSONResponse(content={})
dst_periods = get_dst_transitions(params.timezone, min_time, max_time)
days: dict[str, bool] = {}
for period_start, period_end, period_offset in dst_periods:
hours_offset = int(period_offset / 60 / 60)
minutes_offset = int(period_offset / 60 - hours_offset * 60)
period_hour_modifier = f"{hours_offset} hour"
period_minute_modifier = f"{minutes_offset} minute"
period_query = (
Recordings.select(
fn.strftime(
"%Y-%m-%d",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("day")
)
.where(
(Recordings.camera << camera_list)
& (Recordings.end_time >= period_start)
& (Recordings.start_time <= period_end)
)
.group_by(
fn.strftime(
"%Y-%m-%d",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
)
)
.order_by(Recordings.start_time.desc())
.namedtuples()
)
for g in period_query:
days[g.day] = True
return JSONResponse(content=dict(sorted(days.items())))
@router.get(
"/{camera_name}/recordings/summary", dependencies=[Depends(require_camera_access)]
)
async def recordings_summary(camera_name: str, timezone: str = "utc"):
"""Returns hourly summary for recordings of given camera"""
time_range_query = (
Recordings.select(
fn.MIN(Recordings.start_time).alias("min_time"),
fn.MAX(Recordings.start_time).alias("max_time"),
)
.where(Recordings.camera == camera_name)
.dicts()
.get()
)
min_time = time_range_query.get("min_time")
max_time = time_range_query.get("max_time")
days: dict[str, dict] = {}
if min_time is None or max_time is None:
return JSONResponse(content=list(days.values()))
dst_periods = get_dst_transitions(timezone, min_time, max_time)
for period_start, period_end, period_offset in dst_periods:
hours_offset = int(period_offset / 60 / 60)
minutes_offset = int(period_offset / 60 - hours_offset * 60)
period_hour_modifier = f"{hours_offset} hour"
period_minute_modifier = f"{minutes_offset} minute"
recording_groups = (
Recordings.select(
fn.strftime(
"%Y-%m-%d %H",
fn.datetime(
Recordings.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("hour"),
fn.SUM(Recordings.duration).alias("duration"),
fn.SUM(Recordings.motion).alias("motion"),
fn.SUM(Recordings.objects).alias("objects"),
)
.where(
(Recordings.camera == camera_name)
& (Recordings.end_time >= period_start)
& (Recordings.start_time <= period_end)
)
.group_by((Recordings.start_time + period_offset).cast("int") / 3600)
.order_by(Recordings.start_time.desc())
.namedtuples()
)
event_groups = (
Event.select(
fn.strftime(
"%Y-%m-%d %H",
fn.datetime(
Event.start_time,
"unixepoch",
period_hour_modifier,
period_minute_modifier,
),
).alias("hour"),
fn.COUNT(Event.id).alias("count"),
)
.where(Event.camera == camera_name, Event.has_clip)
.where(
(Event.start_time >= period_start) & (Event.start_time <= period_end)
)
.group_by((Event.start_time + period_offset).cast("int") / 3600)
.namedtuples()
)
event_map = {g.hour: g.count for g in event_groups}
for recording_group in recording_groups:
parts = recording_group.hour.split()
hour = parts[1]
day = parts[0]
events_count = event_map.get(recording_group.hour, 0)
hour_data = {
"hour": hour,
"events": events_count,
"motion": recording_group.motion,
"objects": recording_group.objects,
"duration": round(recording_group.duration),
}
if day in days:
# merge counts if already present (edge-case at DST boundary)
days[day]["events"] += events_count or 0
days[day]["hours"].append(hour_data)
else:
days[day] = {
"events": events_count or 0,
"hours": [hour_data],
"day": day,
}
return JSONResponse(content=list(days.values()))
@router.get("/{camera_name}/recordings", dependencies=[Depends(require_camera_access)])
async def recordings(
camera_name: str,
after: float = (datetime.now() - timedelta(hours=1)).timestamp(),
before: float = datetime.now().timestamp(),
):
"""Return specific camera recordings between the given 'after'/'end' times. If not provided the last hour will be used"""
recordings = (
Recordings.select(
Recordings.id,
Recordings.start_time,
Recordings.end_time,
Recordings.segment_size,
Recordings.motion,
Recordings.objects,
Recordings.duration,
)
.where(
Recordings.camera == camera_name,
Recordings.end_time >= after,
Recordings.start_time <= before,
)
.order_by(Recordings.start_time)
.dicts()
.iterator()
)
return JSONResponse(content=list(recordings))
@router.get(
"/recordings/unavailable",
response_model=list[dict],
dependencies=[Depends(allow_any_authenticated())],
)
async def no_recordings(
request: Request,
params: MediaRecordingsAvailabilityQueryParams = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""Get time ranges with no recordings."""
cameras = params.cameras
if cameras != "all":
requested = set(unquote(cameras).split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(content=[])
cameras = ",".join(filtered)
else:
cameras = allowed_cameras
before = params.before or datetime.datetime.now().timestamp()
after = (
params.after
or (datetime.datetime.now() - datetime.timedelta(hours=1)).timestamp()
)
scale = params.scale
clauses = [(Recordings.end_time >= after) & (Recordings.start_time <= before)]
if cameras != "all":
camera_list = cameras.split(",")
clauses.append((Recordings.camera << camera_list))
else:
camera_list = allowed_cameras
# Get recording start times
data: list[Recordings] = (
Recordings.select(Recordings.start_time, Recordings.end_time)
.where(reduce(operator.and_, clauses))
.order_by(Recordings.start_time.asc())
.dicts()
.iterator()
)
# Convert recordings to list of (start, end) tuples
recordings = [(r["start_time"], r["end_time"]) for r in data]
# Iterate through time segments and check if each has any recording
no_recording_segments = []
current = after
current_gap_start = None
while current < before:
segment_end = min(current + scale, before)
# Check if this segment overlaps with any recording
has_recording = any(
rec_start < segment_end and rec_end > current
for rec_start, rec_end in recordings
)
if not has_recording:
# This segment has no recordings
if current_gap_start is None:
current_gap_start = current # Start a new gap
else:
# This segment has recordings
if current_gap_start is not None:
# End the current gap and append it
no_recording_segments.append(
{"start_time": int(current_gap_start), "end_time": int(current)}
)
current_gap_start = None
current = segment_end
# Append the last gap if it exists
if current_gap_start is not None:
no_recording_segments.append(
{"start_time": int(current_gap_start), "end_time": int(before)}
)
return JSONResponse(content=no_recording_segments)
@router.delete(
"/recordings/start/{start}/end/{end}",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Delete recordings",
description="""Deletes recordings within the specified time range.
Recordings can be filtered by cameras and kept based on motion, objects, or audio attributes.
""",
)
async def delete_recordings(
start: float = PathParam(..., description="Start timestamp (unix)"),
end: float = PathParam(..., description="End timestamp (unix)"),
params: RecordingsDeleteQueryParams = Depends(),
allowed_cameras: List[str] = Depends(get_allowed_cameras_for_filter),
):
"""Delete recordings in the specified time range."""
if start >= end:
return JSONResponse(
content={
"success": False,
"message": "Start time must be less than end time.",
},
status_code=400,
)
cameras = params.cameras
if cameras != "all":
requested = set(cameras.split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(
content={
"success": False,
"message": "No valid cameras found in the request.",
},
status_code=400,
)
camera_list = list(filtered)
else:
camera_list = allowed_cameras
# Parse keep parameter
keep_set = set()
if params.keep:
keep_set = set(params.keep.split(","))
# Build query to find overlapping recordings
clauses = [
(
Recordings.start_time.between(start, end)
| Recordings.end_time.between(start, end)
| ((start > Recordings.start_time) & (end < Recordings.end_time))
),
(Recordings.camera << camera_list),
]
keep_clauses = []
if "motion" in keep_set:
keep_clauses.append(Recordings.motion.is_null(False) & (Recordings.motion > 0))
if "object" in keep_set:
keep_clauses.append(
Recordings.objects.is_null(False) & (Recordings.objects > 0)
)
if "audio" in keep_set:
keep_clauses.append(Recordings.dBFS.is_null(False))
if keep_clauses:
keep_condition = reduce(operator.or_, keep_clauses)
clauses.append(~keep_condition)
recordings_to_delete = (
Recordings.select(Recordings.id, Recordings.path)
.where(reduce(operator.and_, clauses))
.dicts()
.iterator()
)
recording_ids = []
deleted_count = 0
error_count = 0
for recording in recordings_to_delete:
recording_ids.append(recording["id"])
try:
Path(recording["path"]).unlink(missing_ok=True)
deleted_count += 1
except Exception as e:
logger.error(f"Failed to delete recording file {recording['path']}: {e}")
error_count += 1
if recording_ids:
max_deletes = 100000
recording_ids_list = list(recording_ids)
for i in range(0, len(recording_ids_list), max_deletes):
Recordings.delete().where(
Recordings.id << recording_ids_list[i : i + max_deletes]
).execute()
message = f"Successfully deleted {deleted_count} recording(s)."
if error_count > 0:
message += f" {error_count} file deletion error(s) occurred."
return JSONResponse(
content={"success": True, "message": message},
status_code=200,
)
+1 -4
View File
@@ -33,7 +33,6 @@ from frigate.api.defs.response.review_response import (
ReviewSummaryResponse,
)
from frigate.api.defs.tags import Tags
from frigate.config import FrigateConfig
from frigate.embeddings import EmbeddingsContext
from frigate.models import Recordings, ReviewSegment, UserReviewStatus
from frigate.review.types import SeverityEnum
@@ -747,9 +746,7 @@ async def set_not_reviewed(
description="Use GenAI to summarize review items over a period of time.",
)
def generate_review_summary(request: Request, start_ts: float, end_ts: float):
config: FrigateConfig = request.app.frigate_config
if not config.genai.provider:
if not request.app.genai_manager.vision_client:
return JSONResponse(
content=(
{
+4
View File
@@ -19,6 +19,8 @@ class CameraMetrics:
process_pid: Synchronized
capture_process_pid: Synchronized
ffmpeg_pid: Synchronized
reconnects_last_hour: Synchronized
stalls_last_hour: Synchronized
def __init__(self, manager: SyncManager):
self.camera_fps = manager.Value("d", 0)
@@ -35,6 +37,8 @@ class CameraMetrics:
self.process_pid = manager.Value("i", 0)
self.capture_process_pid = manager.Value("i", 0)
self.ffmpeg_pid = manager.Value("i", 0)
self.reconnects_last_hour = manager.Value("i", 0)
self.stalls_last_hour = manager.Value("i", 0)
class PTZMetrics:
+5 -1
View File
@@ -65,7 +65,7 @@ class CameraState:
frame_copy = cv2.cvtColor(frame_copy, cv2.COLOR_YUV2BGR_I420)
# draw on the frame
if draw_options.get("mask"):
mask_overlay = np.where(self.camera_config.motion.mask == [0])
mask_overlay = np.where(self.camera_config.motion.rasterized_mask == [0])
frame_copy[mask_overlay] = [0, 0, 0]
if draw_options.get("bounding_boxes"):
@@ -197,6 +197,10 @@ class CameraState:
if draw_options.get("zones"):
for name, zone in self.camera_config.zones.items():
# skip disabled zones
if not zone.enabled:
continue
thickness = (
8
if any(
+186 -2
View File
@@ -15,6 +15,7 @@ from frigate.config.camera.updater import (
CameraConfigUpdatePublisher,
CameraConfigUpdateTopic,
)
from frigate.config.config import RuntimeFilterConfig, RuntimeMotionConfig
from frigate.const import (
CLEAR_ONGOING_REVIEW_SEGMENTS,
EXPIRE_AUDIO_ACTIVITY,
@@ -28,6 +29,7 @@ from frigate.const import (
UPDATE_CAMERA_ACTIVITY,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
UPDATE_EVENT_DESCRIPTION,
UPDATE_JOB_STATE,
UPDATE_MODEL_STATE,
UPDATE_REVIEW_DESCRIPTION,
UPSERT_REVIEW_SEGMENT,
@@ -60,6 +62,7 @@ class Dispatcher:
self.camera_activity = CameraActivityManager(config, self.publish)
self.audio_activity = AudioActivityManager(config, self.publish)
self.model_state: dict[str, ModelStatusTypesEnum] = {}
self.job_state: dict[str, dict[str, Any]] = {} # {job_type: job_data}
self.embeddings_reindex: dict[str, Any] = {}
self.birdseye_layout: dict[str, Any] = {}
self.audio_transcription_state: str = "idle"
@@ -82,6 +85,9 @@ class Dispatcher:
"review_detections": self._on_detections_command,
"object_descriptions": self._on_object_description_command,
"review_descriptions": self._on_review_description_command,
"motion_mask": self._on_motion_mask_command,
"object_mask": self._on_object_mask_command,
"zone": self._on_zone_command,
}
self._global_settings_handlers: dict[str, Callable] = {
"notifications": self._on_global_notification_command,
@@ -98,11 +104,20 @@ class Dispatcher:
"""Handle receiving of payload from communicators."""
def handle_camera_command(
command_type: str, camera_name: str, command: str, payload: str
command_type: str,
camera_name: str,
command: str,
payload: str,
sub_command: str | None = None,
) -> None:
try:
if command_type == "set":
self._camera_settings_handlers[command](camera_name, payload)
if sub_command:
self._camera_settings_handlers[command](
camera_name, sub_command, payload
)
else:
self._camera_settings_handlers[command](camera_name, payload)
elif command_type == "ptz":
self._on_ptz_command(camera_name, payload)
except KeyError:
@@ -180,6 +195,19 @@ class Dispatcher:
def handle_model_state() -> None:
self.publish("model_state", json.dumps(self.model_state.copy()))
def handle_update_job_state() -> None:
if payload and isinstance(payload, dict):
job_type = payload.get("job_type")
if job_type:
self.job_state[job_type] = payload
self.publish(
"job_state",
json.dumps(self.job_state),
)
def handle_job_state() -> None:
self.publish("job_state", json.dumps(self.job_state.copy()))
def handle_update_audio_transcription_state() -> None:
if payload:
self.audio_transcription_state = payload
@@ -277,6 +305,7 @@ class Dispatcher:
UPDATE_EVENT_DESCRIPTION: handle_update_event_description,
UPDATE_REVIEW_DESCRIPTION: handle_update_review_description,
UPDATE_MODEL_STATE: handle_update_model_state,
UPDATE_JOB_STATE: handle_update_job_state,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS: handle_update_embeddings_reindex_progress,
UPDATE_BIRDSEYE_LAYOUT: handle_update_birdseye_layout,
UPDATE_AUDIO_TRANSCRIPTION_STATE: handle_update_audio_transcription_state,
@@ -284,6 +313,7 @@ class Dispatcher:
"restart": handle_restart,
"embeddingsReindexProgress": handle_embeddings_reindex_progress,
"modelState": handle_model_state,
"jobState": handle_job_state,
"audioTranscriptionState": handle_audio_transcription_state,
"birdseyeLayout": handle_birdseye_layout,
"onConnect": handle_on_connect,
@@ -297,6 +327,14 @@ class Dispatcher:
camera_name = parts[-3]
command = parts[-2]
handle_camera_command("set", camera_name, command, payload)
elif len(parts) == 4 and topic.endswith("set"):
# example /cam_name/motion_mask/mask_name/set payload=ON|OFF
camera_name = parts[-4]
command = parts[-3]
sub_command = parts[-2]
handle_camera_command(
"set", camera_name, command, payload, sub_command
)
elif len(parts) == 2 and topic.endswith("set"):
command = parts[-2]
self._global_settings_handlers[command](payload)
@@ -841,3 +879,149 @@ class Dispatcher:
genai_settings,
)
self.publish(f"{camera_name}/review_descriptions/state", payload, retain=True)
def _on_motion_mask_command(
self, camera_name: str, mask_name: str, payload: str
) -> None:
"""Callback for motion mask topic."""
if payload not in ["ON", "OFF"]:
logger.error(f"Invalid payload for motion mask {mask_name}: {payload}")
return
motion_settings = self.config.cameras[camera_name].motion
if mask_name not in motion_settings.mask:
logger.error(f"Unknown motion mask: {mask_name}")
return
mask = motion_settings.mask[mask_name]
if not mask:
logger.error(f"Motion mask {mask_name} is None")
return
if payload == "ON":
if not mask.enabled_in_config:
logger.error(
f"Motion mask {mask_name} must be enabled in the config to be turned on via MQTT."
)
return
mask.enabled = payload == "ON"
# Recreate RuntimeMotionConfig to update rasterized_mask
motion_settings = RuntimeMotionConfig(
frame_shape=self.config.cameras[camera_name].frame_shape,
**motion_settings.model_dump(exclude_unset=True),
)
# Update the dispatcher's own config
self.config.cameras[camera_name].motion = motion_settings
self.config_updater.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.motion, camera_name),
motion_settings,
)
self.publish(
f"{camera_name}/motion_mask/{mask_name}/state", payload, retain=True
)
def _on_object_mask_command(
self, camera_name: str, mask_name: str, payload: str
) -> None:
"""Callback for object mask topic."""
if payload not in ["ON", "OFF"]:
logger.error(f"Invalid payload for object mask {mask_name}: {payload}")
return
object_settings = self.config.cameras[camera_name].objects
# Check if this is a global mask
mask_found = False
if mask_name in object_settings.mask:
mask = object_settings.mask[mask_name]
if mask:
if payload == "ON":
if not mask.enabled_in_config:
logger.error(
f"Object mask {mask_name} must be enabled in the config to be turned on via MQTT."
)
return
mask.enabled = payload == "ON"
mask_found = True
# Check if this is a per-object filter mask
for object_name, filter_config in object_settings.filters.items():
if mask_name in filter_config.mask:
mask = filter_config.mask[mask_name]
if mask:
if payload == "ON":
if not mask.enabled_in_config:
logger.error(
f"Object mask {mask_name} must be enabled in the config to be turned on via MQTT."
)
return
mask.enabled = payload == "ON"
mask_found = True
if not mask_found:
logger.error(f"Unknown object mask: {mask_name}")
return
# Recreate RuntimeFilterConfig for each object filter to update rasterized_mask
for object_name, filter_config in object_settings.filters.items():
# Merge global object masks with per-object filter masks
merged_mask = dict(filter_config.mask) # Copy filter-specific masks
# Add global object masks if they exist
if object_settings.mask:
for global_mask_id, global_mask_config in object_settings.mask.items():
# Use a global prefix to avoid key collisions
global_mask_id_prefixed = f"global_{global_mask_id}"
merged_mask[global_mask_id_prefixed] = global_mask_config
object_settings.filters[object_name] = RuntimeFilterConfig(
frame_shape=self.config.cameras[camera_name].frame_shape,
mask=merged_mask,
**filter_config.model_dump(
exclude_unset=True, exclude={"mask", "raw_mask"}
),
)
# Update the dispatcher's own config
self.config.cameras[camera_name].objects = object_settings
self.config_updater.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.objects, camera_name),
object_settings,
)
self.publish(
f"{camera_name}/object_mask/{mask_name}/state", payload, retain=True
)
def _on_zone_command(self, camera_name: str, zone_name: str, payload: str) -> None:
"""Callback for zone topic."""
if payload not in ["ON", "OFF"]:
logger.error(f"Invalid payload for zone {zone_name}: {payload}")
return
camera_config = self.config.cameras[camera_name]
if zone_name not in camera_config.zones:
logger.error(f"Unknown zone: {zone_name}")
return
if payload == "ON":
if not camera_config.zones[zone_name].enabled_in_config:
logger.error(
f"Zone {zone_name} must be enabled in the config to be turned on via MQTT."
)
return
camera_config.zones[zone_name].enabled = payload == "ON"
self.config_updater.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum.zones, camera_name),
camera_config.zones,
)
self.publish(f"{camera_name}/zone/{zone_name}/state", payload, retain=True)
+41
View File
@@ -133,6 +133,29 @@ class MqttClient(Communicator):
retain=True,
)
for mask_name, motion_mask in camera.motion.mask.items():
if motion_mask:
self.publish(
f"{camera_name}/motion_mask/{mask_name}/state",
"ON" if motion_mask.enabled else "OFF",
retain=True,
)
for mask_name, object_mask in camera.objects.mask.items():
if object_mask:
self.publish(
f"{camera_name}/object_mask/{mask_name}/state",
"ON" if object_mask.enabled else "OFF",
retain=True,
)
for zone_name, zone in camera.zones.items():
self.publish(
f"{camera_name}/zone/{zone_name}/state",
"ON" if zone.enabled else "OFF",
retain=True,
)
if self.config.notifications.enabled_in_config:
self.publish(
"notifications/state",
@@ -242,6 +265,24 @@ class MqttClient(Communicator):
self.on_mqtt_command,
)
for mask_name in self.config.cameras[name].motion.mask.keys():
self.client.message_callback_add(
f"{self.mqtt_config.topic_prefix}/{name}/motion_mask/{mask_name}/set",
self.on_mqtt_command,
)
for mask_name in self.config.cameras[name].objects.mask.keys():
self.client.message_callback_add(
f"{self.mqtt_config.topic_prefix}/{name}/object_mask/{mask_name}/set",
self.on_mqtt_command,
)
for zone_name in self.config.cameras[name].zones.keys():
self.client.message_callback_add(
f"{self.mqtt_config.topic_prefix}/{name}/zone/{zone_name}/set",
self.on_mqtt_command,
)
if self.config.notifications.enabled_in_config:
self.client.message_callback_add(
f"{self.mqtt_config.topic_prefix}/notifications/set",
+11 -59
View File
@@ -17,7 +17,6 @@ from titlecase import titlecase
from frigate.comms.base_communicator import Communicator
from frigate.comms.config_updater import ConfigSubscriber
from frigate.config import FrigateConfig
from frigate.config.auth import AuthConfig
from frigate.config.camera.updater import (
CameraConfigUpdateEnum,
CameraConfigUpdateSubscriber,
@@ -59,7 +58,6 @@ class WebPushClient(Communicator):
for c in self.config.cameras.values()
}
self.last_notification_time: float = 0
self.user_cameras: dict[str, set[str]] = {}
self.notification_queue: queue.Queue[PushNotification] = queue.Queue()
self.notification_thread = threading.Thread(
target=self._process_notifications, daemon=True
@@ -80,12 +78,13 @@ class WebPushClient(Communicator):
for sub in user["notification_tokens"]:
self.web_pushers[user["username"]].append(WebPusher(sub))
# notification and auth config updater
self.global_config_subscriber = ConfigSubscriber("config/")
# notification config updater
self.global_config_subscriber = ConfigSubscriber(
"config/notifications", exact=True
)
self.config_subscriber = CameraConfigUpdateSubscriber(
self.config, self.config.cameras, [CameraConfigUpdateEnum.notifications]
)
self._refresh_user_cameras()
def subscribe(self, receiver: Callable) -> None:
"""Wrapper for allowing dispatcher to subscribe."""
@@ -165,19 +164,13 @@ class WebPushClient(Communicator):
def publish(self, topic: str, payload: Any, retain: bool = False) -> None:
"""Wrapper for publishing when client is in valid state."""
# check for updated global config (notifications, auth)
while True:
config_topic, config_payload = (
self.global_config_subscriber.check_for_update()
)
if config_topic is None:
break
if config_topic == "config/notifications" and config_payload:
self.config.notifications = config_payload
elif config_topic == "config/auth":
if isinstance(config_payload, AuthConfig):
self.config.auth = config_payload
self._refresh_user_cameras()
# check for updated notification config
_, updated_notification_config = (
self.global_config_subscriber.check_for_update()
)
if updated_notification_config:
self.config.notifications = updated_notification_config
updates = self.config_subscriber.check_for_updates()
@@ -298,31 +291,6 @@ class WebPushClient(Communicator):
except Exception as e:
logger.error(f"Error processing notification: {str(e)}")
def _refresh_user_cameras(self) -> None:
"""Rebuild the user-to-cameras access cache from the database."""
all_camera_names = set(self.config.cameras.keys())
roles_dict = self.config.auth.roles
updated: dict[str, set[str]] = {}
for user in User.select(User.username, User.role).dicts().iterator():
allowed = User.get_allowed_cameras(
user["role"], roles_dict, all_camera_names
)
updated[user["username"]] = set(allowed)
logger.debug(
"User %s has access to cameras: %s",
user["username"],
", ".join(allowed),
)
self.user_cameras = updated
def _user_has_camera_access(self, username: str, camera: str) -> bool:
"""Check if a user has access to a specific camera based on cached roles."""
allowed = self.user_cameras.get(username)
if allowed is None:
logger.debug(f"No camera access information found for user {username}")
return False
return camera in allowed
def _within_cooldown(self, camera: str) -> bool:
now = datetime.datetime.now().timestamp()
if now - self.last_notification_time < self.config.notifications.cooldown:
@@ -450,14 +418,6 @@ class WebPushClient(Communicator):
logger.debug(f"Sending push notification for {camera}, review ID {reviewId}")
for user in self.web_pushers:
if not self._user_has_camera_access(user, camera):
logger.debug(
"Skipping notification for user %s - no access to camera %s",
user,
camera,
)
continue
self.send_push_notification(
user=user,
payload=payload,
@@ -505,14 +465,6 @@ class WebPushClient(Communicator):
)
for user in self.web_pushers:
if not self._user_has_camera_access(user, camera):
logger.debug(
"Skipping notification for user %s - no access to camera %s",
user,
camera,
)
continue
self.send_push_notification(
user=user,
payload=payload,
+3 -99
View File
@@ -17,90 +17,9 @@ from ws4py.websocket import WebSocket as WebSocket_
from frigate.comms.base_communicator import Communicator
from frigate.config import FrigateConfig
from frigate.const import (
CLEAR_ONGOING_REVIEW_SEGMENTS,
EXPIRE_AUDIO_ACTIVITY,
INSERT_MANY_RECORDINGS,
INSERT_PREVIEW,
NOTIFICATION_TEST,
REQUEST_REGION_GRID,
UPDATE_AUDIO_ACTIVITY,
UPDATE_AUDIO_TRANSCRIPTION_STATE,
UPDATE_BIRDSEYE_LAYOUT,
UPDATE_CAMERA_ACTIVITY,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
UPDATE_EVENT_DESCRIPTION,
UPDATE_MODEL_STATE,
UPDATE_REVIEW_DESCRIPTION,
UPSERT_REVIEW_SEGMENT,
)
logger = logging.getLogger(__name__)
# Internal IPC topics — NEVER allowed from WebSocket, regardless of role
_WS_BLOCKED_TOPICS = frozenset(
{
INSERT_MANY_RECORDINGS,
INSERT_PREVIEW,
REQUEST_REGION_GRID,
UPSERT_REVIEW_SEGMENT,
CLEAR_ONGOING_REVIEW_SEGMENTS,
UPDATE_CAMERA_ACTIVITY,
UPDATE_AUDIO_ACTIVITY,
EXPIRE_AUDIO_ACTIVITY,
UPDATE_EVENT_DESCRIPTION,
UPDATE_REVIEW_DESCRIPTION,
UPDATE_MODEL_STATE,
UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
UPDATE_BIRDSEYE_LAYOUT,
UPDATE_AUDIO_TRANSCRIPTION_STATE,
NOTIFICATION_TEST,
}
)
# Read-only topics any authenticated user (including viewer) can send
_WS_VIEWER_TOPICS = frozenset(
{
"onConnect",
"modelState",
"audioTranscriptionState",
"birdseyeLayout",
"embeddingsReindexProgress",
}
)
def _check_ws_authorization(
topic: str,
role_header: str | None,
separator: str,
) -> bool:
"""Check if a WebSocket message is authorized.
Args:
topic: The message topic.
role_header: The HTTP_REMOTE_ROLE header value, or None.
separator: The role separator character from proxy config.
Returns:
True if authorized, False if blocked.
"""
# Block IPC-only topics unconditionally
if topic in _WS_BLOCKED_TOPICS:
return False
# No role header: default to viewer (fail-closed)
if role_header is None:
return topic in _WS_VIEWER_TOPICS
# Check if any role is admin
roles = [r.strip() for r in role_header.split(separator)]
if "admin" in roles:
return True
# Non-admin: only viewer topics allowed
return topic in _WS_VIEWER_TOPICS
class WebSocket(WebSocket_): # type: ignore[misc]
def unhandled_error(self, error: Any) -> None:
@@ -130,7 +49,6 @@ class WebSocketClient(Communicator):
class _WebSocketHandler(WebSocket):
receiver = self._dispatcher
role_separator = self.config.proxy.separator or ","
def received_message(self, message: WebSocket.received_message) -> None: # type: ignore[name-defined]
try:
@@ -145,25 +63,11 @@ class WebSocketClient(Communicator):
)
return
topic = json_message["topic"]
# Authorization check (skip when environ is None — direct internal connection)
role_header = (
self.environ.get("HTTP_REMOTE_ROLE") if self.environ else None
logger.debug(
f"Publishing mqtt message from websockets at {json_message['topic']}."
)
if self.environ is not None and not _check_ws_authorization(
topic, role_header, self.role_separator
):
logger.warning(
"Blocked unauthorized WebSocket message: topic=%s, role=%s",
topic,
role_header,
)
return
logger.debug(f"Publishing mqtt message from websockets at {topic}.")
self.receiver(
topic,
json_message["topic"],
json_message["payload"],
)
+1
View File
@@ -8,6 +8,7 @@ from .config import * # noqa: F403
from .database import * # noqa: F403
from .logger import * # noqa: F403
from .mqtt import * # noqa: F403
from .network import * # noqa: F403
from .proxy import * # noqa: F403
from .telemetry import * # noqa: F403
from .tls import * # noqa: F403
+35 -11
View File
@@ -8,39 +8,63 @@ __all__ = ["AuthConfig"]
class AuthConfig(FrigateBaseModel):
enabled: bool = Field(default=True, title="Enable authentication")
enabled: bool = Field(
default=True,
title="Enable authentication",
description="Enable native authentication for the Frigate UI.",
)
reset_admin_password: bool = Field(
default=False, title="Reset the admin password on startup"
default=False,
title="Reset admin password",
description="If true, reset the admin user's password on startup and print the new password in logs.",
)
cookie_name: str = Field(
default="frigate_token", title="Name for jwt token cookie", pattern=r"^[a-z_]+$"
default="frigate_token",
title="JWT cookie name",
description="Name of the cookie used to store the JWT token for native authentication.",
pattern=r"^[a-z_]+$",
)
cookie_secure: bool = Field(
default=False,
title="Secure cookie flag",
description="Set the secure flag on the auth cookie; should be true when using TLS.",
)
cookie_secure: bool = Field(default=False, title="Set secure flag on cookie")
session_length: int = Field(
default=86400, title="Session length for jwt session tokens", ge=60
default=86400,
title="Session length",
description="Session duration in seconds for JWT-based sessions.",
ge=60,
)
refresh_time: int = Field(
default=1800,
title="Refresh the session if it is going to expire in this many seconds",
title="Session refresh window",
description="When a session is within this many seconds of expiring, refresh it back to full length.",
ge=30,
)
failed_login_rate_limit: Optional[str] = Field(
default=None,
title="Rate limits for failed login attempts.",
title="Failed login limits",
description="Rate limiting rules for failed login attempts to reduce brute-force attacks.",
)
trusted_proxies: list[str] = Field(
default=[],
title="Trusted proxies for determining IP address to rate limit",
title="Trusted proxies",
description="List of trusted proxy IPs used when determining client IP for rate limiting.",
)
# As of Feb 2023, OWASP recommends 600000 iterations for PBKDF2-SHA256
hash_iterations: int = Field(default=600000, title="Password hash iterations")
hash_iterations: int = Field(
default=600000,
title="Hash iterations",
description="Number of PBKDF2-SHA256 iterations to use when hashing user passwords.",
)
roles: Dict[str, List[str]] = Field(
default_factory=dict,
title="Role to camera mappings. Empty list grants access to all cameras.",
title="Role mappings",
description="Map roles to camera lists. An empty list grants access to all cameras for the role.",
)
admin_first_time_login: Optional[bool] = Field(
default=False,
title="Internal field to expose first-time admin login flag to the UI",
title="First-time admin flag",
description=(
"When true the UI may show a help link on the login page informing users how to sign in after an admin password reset. "
),
+28 -8
View File
@@ -17,25 +17,45 @@ class AudioFilterConfig(FrigateBaseModel):
default=0.8,
ge=AUDIO_MIN_CONFIDENCE,
lt=1.0,
title="Minimum detection confidence threshold for audio to be counted.",
title="Minimum audio confidence",
description="Minimum confidence threshold for the audio event to be counted.",
)
class AudioConfig(FrigateBaseModel):
enabled: bool = Field(default=False, title="Enable audio events.")
enabled: bool = Field(
default=False,
title="Enable audio detection",
description="Enable or disable audio event detection for all cameras; can be overridden per-camera.",
)
max_not_heard: int = Field(
default=30, title="Seconds of not hearing the type of audio to end the event."
default=30,
title="End timeout",
description="Amount of seconds without the configured audio type before the audio event is ended.",
)
min_volume: int = Field(
default=500, title="Min volume required to run audio detection."
default=500,
title="Minimum volume",
description="Minimum RMS volume threshold required to run audio detection; lower values increase sensitivity (e.g., 200 high, 500 medium, 1000 low).",
)
listen: list[str] = Field(
default=DEFAULT_LISTEN_AUDIO, title="Audio to listen for."
default=DEFAULT_LISTEN_AUDIO,
title="Listen types",
description="List of audio event types to detect (for example: bark, fire_alarm, scream, speech, yell).",
)
filters: Optional[dict[str, AudioFilterConfig]] = Field(
None, title="Audio filters."
None,
title="Audio filters",
description="Per-audio-type filter settings such as confidence thresholds used to reduce false positives.",
)
enabled_in_config: Optional[bool] = Field(
None, title="Keep track of original state of audio detection."
None,
title="Original audio state",
description="Indicates whether audio detection was originally enabled in the static config file.",
)
num_threads: int = Field(
default=2,
title="Detection threads",
description="Number of threads to use for audio detection processing.",
ge=1,
)
num_threads: int = Field(default=2, title="Number of detection threads", ge=1)
+57 -14
View File
@@ -29,45 +29,88 @@ class BirdseyeModeEnum(str, Enum):
class BirdseyeLayoutConfig(FrigateBaseModel):
scaling_factor: float = Field(
default=2.0, title="Birdseye Scaling Factor", ge=1.0, le=5.0
default=2.0,
title="Scaling factor",
description="Scaling factor used by the layout calculator (range 1.0 to 5.0).",
ge=1.0,
le=5.0,
)
max_cameras: Optional[int] = Field(
default=None,
title="Max cameras",
description="Maximum number of cameras to display at once in Birdseye; shows the most recent cameras.",
)
max_cameras: Optional[int] = Field(default=None, title="Max cameras")
class BirdseyeConfig(FrigateBaseModel):
enabled: bool = Field(default=True, title="Enable birdseye view.")
enabled: bool = Field(
default=True,
title="Enable Birdseye",
description="Enable or disable the Birdseye view feature.",
)
mode: BirdseyeModeEnum = Field(
default=BirdseyeModeEnum.objects, title="Tracking mode."
default=BirdseyeModeEnum.objects,
title="Tracking mode",
description="Mode for including cameras in Birdseye: 'objects', 'motion', or 'continuous'.",
)
restream: bool = Field(default=False, title="Restream birdseye via RTSP.")
width: int = Field(default=1280, title="Birdseye width.")
height: int = Field(default=720, title="Birdseye height.")
restream: bool = Field(
default=False,
title="Restream RTSP",
description="Re-stream the Birdseye output as an RTSP feed; enabling this will keep Birdseye running continuously.",
)
width: int = Field(
default=1280,
title="Width",
description="Output width (pixels) of the composed Birdseye frame.",
)
height: int = Field(
default=720,
title="Height",
description="Output height (pixels) of the composed Birdseye frame.",
)
quality: int = Field(
default=8,
title="Encoding quality.",
title="Encoding quality",
description="Encoding quality for the Birdseye mpeg1 feed (1 highest quality, 31 lowest).",
ge=1,
le=31,
)
inactivity_threshold: int = Field(
default=30, title="Birdseye Inactivity Threshold", gt=0
default=30,
title="Inactivity threshold",
description="Seconds of inactivity after which a camera will stop being shown in Birdseye.",
gt=0,
)
layout: BirdseyeLayoutConfig = Field(
default_factory=BirdseyeLayoutConfig, title="Birdseye Layout Config"
default_factory=BirdseyeLayoutConfig,
title="Layout",
description="Layout options for the Birdseye composition.",
)
idle_heartbeat_fps: float = Field(
default=0.0,
ge=0.0,
le=10.0,
title="Idle heartbeat FPS (0 disables, max 10)",
title="Idle heartbeat FPS",
description="Frames-per-second to resend the last composed Birdseye frame when idle; set to 0 to disable.",
)
# uses BaseModel because some global attributes are not available at the camera level
class BirdseyeCameraConfig(BaseModel):
enabled: bool = Field(default=True, title="Enable birdseye view for camera.")
enabled: bool = Field(
default=True,
title="Enable Birdseye",
description="Enable or disable the Birdseye view feature.",
)
mode: BirdseyeModeEnum = Field(
default=BirdseyeModeEnum.objects, title="Tracking mode for camera."
default=BirdseyeModeEnum.objects,
title="Tracking mode",
description="Mode for including cameras in Birdseye: 'objects', 'motion', or 'continuous'.",
)
order: int = Field(default=0, title="Position of the camera in the birdseye view.")
order: int = Field(
default=0,
title="Position",
description="Numeric position controlling the camera's ordering in the Birdseye layout.",
)
+83 -27
View File
@@ -50,10 +50,17 @@ class CameraTypeEnum(str, Enum):
class CameraConfig(FrigateBaseModel):
name: Optional[str] = Field(None, title="Camera name.", pattern=REGEX_CAMERA_NAME)
name: Optional[str] = Field(
None,
title="Camera name",
description="Camera name is required",
pattern=REGEX_CAMERA_NAME,
)
friendly_name: Optional[str] = Field(
None, title="Camera friendly name used in the Frigate UI."
None,
title="Friendly name",
description="Camera friendly name used in the Frigate UI",
)
@model_validator(mode="before")
@@ -63,80 +70,129 @@ class CameraConfig(FrigateBaseModel):
pass
return values
enabled: bool = Field(default=True, title="Enable camera.")
enabled: bool = Field(default=True, title="Enabled", description="Enabled")
# Options with global fallback
audio: AudioConfig = Field(
default_factory=AudioConfig, title="Audio events configuration."
default_factory=AudioConfig,
title="Audio events",
description="Settings for audio-based event detection for this camera.",
)
audio_transcription: CameraAudioTranscriptionConfig = Field(
default_factory=CameraAudioTranscriptionConfig,
title="Audio transcription config.",
title="Audio transcription",
description="Settings for live and speech audio transcription used for events and live captions.",
)
birdseye: BirdseyeCameraConfig = Field(
default_factory=BirdseyeCameraConfig, title="Birdseye camera configuration."
default_factory=BirdseyeCameraConfig,
title="Birdseye",
description="Settings for the Birdseye composite view that composes multiple camera feeds into a single layout.",
)
detect: DetectConfig = Field(
default_factory=DetectConfig, title="Object detection configuration."
default_factory=DetectConfig,
title="Object Detection",
description="Settings for the detection/detect role used to run object detection and initialize trackers.",
)
face_recognition: CameraFaceRecognitionConfig = Field(
default_factory=CameraFaceRecognitionConfig, title="Face recognition config."
default_factory=CameraFaceRecognitionConfig,
title="Face recognition",
description="Settings for face detection and recognition for this camera.",
)
ffmpeg: CameraFfmpegConfig = Field(
title="FFmpeg",
description="FFmpeg settings including binary path, args, hwaccel options, and per-role output args.",
)
ffmpeg: CameraFfmpegConfig = Field(title="FFmpeg configuration for the camera.")
live: CameraLiveConfig = Field(
default_factory=CameraLiveConfig, title="Live playback settings."
default_factory=CameraLiveConfig,
title="Live playback",
description="Settings used by the Web UI to control live stream selection, resolution and quality.",
)
lpr: CameraLicensePlateRecognitionConfig = Field(
default_factory=CameraLicensePlateRecognitionConfig, title="LPR config."
default_factory=CameraLicensePlateRecognitionConfig,
title="License Plate Recognition",
description="License plate recognition settings including detection thresholds, formatting, and known plates.",
)
motion: MotionConfig = Field(
None,
title="Motion detection",
description="Default motion detection settings for this camera.",
)
motion: MotionConfig = Field(None, title="Motion detection configuration.")
objects: ObjectConfig = Field(
default_factory=ObjectConfig, title="Object configuration."
default_factory=ObjectConfig,
title="Objects",
description="Object tracking defaults including which labels to track and per-object filters.",
)
record: RecordConfig = Field(
default_factory=RecordConfig, title="Record configuration."
default_factory=RecordConfig,
title="Recording",
description="Recording and retention settings for this camera.",
)
review: ReviewConfig = Field(
default_factory=ReviewConfig, title="Review configuration."
default_factory=ReviewConfig,
title="Review",
description="Settings that control alerts, detections, and GenAI review summaries used by the UI and storage for this camera.",
)
semantic_search: CameraSemanticSearchConfig = Field(
default_factory=CameraSemanticSearchConfig,
title="Semantic search configuration.",
title="Semantic Search",
description="Settings for semantic search which builds and queries object embeddings to find similar items.",
)
snapshots: SnapshotsConfig = Field(
default_factory=SnapshotsConfig, title="Snapshot configuration."
default_factory=SnapshotsConfig,
title="Snapshots",
description="Settings for saved JPEG snapshots of tracked objects for this camera.",
)
timestamp_style: TimestampStyleConfig = Field(
default_factory=TimestampStyleConfig, title="Timestamp style configuration."
default_factory=TimestampStyleConfig,
title="Timestamp style",
description="Styling options for in-feed timestamps applied to recordings and snapshots.",
)
# Options without global fallback
best_image_timeout: int = Field(
default=60,
title="How long to wait for the image with the highest confidence score.",
title="Best image timeout",
description="How long to wait for the image with the highest confidence score.",
)
mqtt: CameraMqttConfig = Field(
default_factory=CameraMqttConfig, title="MQTT configuration."
default_factory=CameraMqttConfig,
title="MQTT",
description="MQTT image publishing settings.",
)
notifications: NotificationConfig = Field(
default_factory=NotificationConfig, title="Notifications configuration."
default_factory=NotificationConfig,
title="Notifications",
description="Settings to enable and control notifications for this camera.",
)
onvif: OnvifConfig = Field(
default_factory=OnvifConfig, title="Camera Onvif Configuration."
default_factory=OnvifConfig,
title="ONVIF",
description="ONVIF connection and PTZ autotracking settings for this camera.",
)
type: CameraTypeEnum = Field(
default=CameraTypeEnum.generic,
title="Camera type",
description="Camera Type",
)
type: CameraTypeEnum = Field(default=CameraTypeEnum.generic, title="Camera Type")
ui: CameraUiConfig = Field(
default_factory=CameraUiConfig, title="Camera UI Modifications."
default_factory=CameraUiConfig,
title="Camera UI",
description="Display ordering and visibility for this camera in the UI. Ordering affects the default dashboard. For more granular control, use camera groups.",
)
webui_url: Optional[str] = Field(
None,
title="URL to visit the camera directly from system page",
title="Camera URL",
description="URL to visit the camera directly from system page",
)
zones: dict[str, ZoneConfig] = Field(
default_factory=dict, title="Zone configuration."
default_factory=dict,
title="Zones",
description="Zones allow you to define a specific area of the frame so you can determine whether or not an object is within a particular area.",
)
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of camera."
default=None,
title="Original camera state",
description="Keep track of original state of camera.",
)
_ffmpeg_cmds: list[dict[str, list[str]]] = PrivateAttr()
+40 -14
View File
@@ -8,56 +8,82 @@ __all__ = ["DetectConfig", "StationaryConfig", "StationaryMaxFramesConfig"]
class StationaryMaxFramesConfig(FrigateBaseModel):
default: Optional[int] = Field(default=None, title="Default max frames.", ge=1)
default: Optional[int] = Field(
default=None,
title="Default max frames",
description="Default maximum frames to track a stationary object before stopping.",
ge=1,
)
objects: dict[str, int] = Field(
default_factory=dict, title="Object specific max frames."
default_factory=dict,
title="Object max frames",
description="Per-object overrides for maximum frames to track stationary objects.",
)
class StationaryConfig(FrigateBaseModel):
interval: Optional[int] = Field(
default=None,
title="Frame interval for checking stationary objects.",
title="Stationary interval",
description="How often (in frames) to run a detection check to confirm a stationary object.",
gt=0,
)
threshold: Optional[int] = Field(
default=None,
title="Number of frames without a position change for an object to be considered stationary",
title="Stationary threshold",
description="Number of frames with no position change required to mark an object as stationary.",
ge=1,
)
max_frames: StationaryMaxFramesConfig = Field(
default_factory=StationaryMaxFramesConfig,
title="Max frames for stationary objects.",
title="Max frames",
description="Limits how long stationary objects are tracked before being discarded.",
)
classifier: bool = Field(
default=True,
title="Enable visual classifier for determing if objects with jittery bounding boxes are stationary.",
title="Enable visual classifier",
description="Use a visual classifier to detect truly stationary objects even when bounding boxes jitter.",
)
class DetectConfig(FrigateBaseModel):
enabled: bool = Field(default=False, title="Detection Enabled.")
enabled: bool = Field(
default=False,
title="Detection enabled",
description="Enable or disable object detection for all cameras; can be overridden per-camera. Detection must be enabled for object tracking to run.",
)
height: Optional[int] = Field(
default=None, title="Height of the stream for the detect role."
default=None,
title="Detect height",
description="Height (pixels) of frames used for the detect stream; leave empty to use the native stream resolution.",
)
width: Optional[int] = Field(
default=None, title="Width of the stream for the detect role."
default=None,
title="Detect width",
description="Width (pixels) of frames used for the detect stream; leave empty to use the native stream resolution.",
)
fps: int = Field(
default=5, title="Number of frames per second to process through detection."
default=5,
title="Detect FPS",
description="Desired frames per second to run detection on; lower values reduce CPU usage (recommended value is 5, only set higher - at most 10 - if tracking extremely fast moving objects).",
)
min_initialized: Optional[int] = Field(
default=None,
title="Minimum number of consecutive hits for an object to be initialized by the tracker.",
title="Minimum initialization frames",
description="Number of consecutive detection hits required before creating a tracked object. Increase to reduce false initializations. Default value is fps divided by 2.",
)
max_disappeared: Optional[int] = Field(
default=None,
title="Maximum number of frames the object can disappear before detection ends.",
title="Maximum disappeared frames",
description="Number of frames without a detection before a tracked object is considered gone.",
)
stationary: StationaryConfig = Field(
default_factory=StationaryConfig,
title="Stationary objects config.",
title="Stationary objects config",
description="Settings to detect and manage objects that remain stationary for a period of time.",
)
annotation_offset: int = Field(
default=0, title="Milliseconds to offset detect annotations by."
default=0,
title="Annotation offset",
description="Milliseconds to shift detect annotations to better align timeline bounding boxes with recordings; can be positive or negative.",
)
+50 -16
View File
@@ -35,39 +35,58 @@ DETECT_FFMPEG_OUTPUT_ARGS_DEFAULT = [
class FfmpegOutputArgsConfig(FrigateBaseModel):
detect: Union[str, list[str]] = Field(
default=DETECT_FFMPEG_OUTPUT_ARGS_DEFAULT,
title="Detect role FFmpeg output arguments.",
title="Detect output arguments",
description="Default output arguments for detect role streams.",
)
record: Union[str, list[str]] = Field(
default=RECORD_FFMPEG_OUTPUT_ARGS_DEFAULT,
title="Record role FFmpeg output arguments.",
title="Record output arguments",
description="Default output arguments for record role streams.",
)
class FfmpegConfig(FrigateBaseModel):
path: str = Field(default="default", title="FFmpeg path")
path: str = Field(
default="default",
title="FFmpeg path",
description='Path to the FFmpeg binary to use or a version alias ("5.0" or "7.0").',
)
global_args: Union[str, list[str]] = Field(
default=FFMPEG_GLOBAL_ARGS_DEFAULT, title="Global FFmpeg arguments."
default=FFMPEG_GLOBAL_ARGS_DEFAULT,
title="FFmpeg global arguments",
description="Global arguments passed to FFmpeg processes.",
)
hwaccel_args: Union[str, list[str]] = Field(
default="auto", title="FFmpeg hardware acceleration arguments."
default="auto",
title="Hardware acceleration arguments",
description="Hardware acceleration arguments for FFmpeg. Provider-specific presets are recommended.",
)
input_args: Union[str, list[str]] = Field(
default=FFMPEG_INPUT_ARGS_DEFAULT, title="FFmpeg input arguments."
default=FFMPEG_INPUT_ARGS_DEFAULT,
title="Input arguments",
description="Input arguments applied to FFmpeg input streams.",
)
output_args: FfmpegOutputArgsConfig = Field(
default_factory=FfmpegOutputArgsConfig,
title="FFmpeg output arguments per role.",
title="Output arguments",
description="Default output arguments used for different FFmpeg roles such as detect and record.",
)
retry_interval: float = Field(
default=10.0,
title="Time in seconds to wait before FFmpeg retries connecting to the camera.",
title="FFmpeg retry time",
description="Seconds to wait before attempting to reconnect a camera stream after failure. Default is 10.",
gt=0.0,
)
apple_compatibility: bool = Field(
default=False,
title="Set tag on HEVC (H.265) recording stream to improve compatibility with Apple players.",
title="Apple compatibility",
description="Enable HEVC tagging for better Apple player compatibility when recording H.265.",
)
gpu: int = Field(
default=0,
title="GPU index",
description="Default GPU index used for hardware acceleration if available.",
)
gpu: int = Field(default=0, title="GPU index to use for hardware acceleration.")
@property
def ffmpeg_path(self) -> str:
@@ -95,21 +114,36 @@ class CameraRoleEnum(str, Enum):
class CameraInput(FrigateBaseModel):
path: EnvString = Field(title="Camera input path.")
roles: list[CameraRoleEnum] = Field(title="Roles assigned to this input.")
path: EnvString = Field(
title="Input path",
description="Camera input stream URL or path.",
)
roles: list[CameraRoleEnum] = Field(
title="Input roles",
description="Roles for this input stream.",
)
global_args: Union[str, list[str]] = Field(
default_factory=list, title="FFmpeg global arguments."
default_factory=list,
title="FFmpeg global arguments",
description="FFmpeg global arguments for this input stream.",
)
hwaccel_args: Union[str, list[str]] = Field(
default_factory=list, title="FFmpeg hardware acceleration arguments."
default_factory=list,
title="Hardware acceleration arguments",
description="Hardware acceleration arguments for this input stream.",
)
input_args: Union[str, list[str]] = Field(
default_factory=list, title="FFmpeg input arguments."
default_factory=list,
title="Input arguments",
description="Input arguments specific to this stream.",
)
class CameraFfmpegConfig(FfmpegConfig):
inputs: list[CameraInput] = Field(title="Camera inputs.")
inputs: list[CameraInput] = Field(
title="Camera inputs",
description="List of input stream definitions (paths and roles) for this camera.",
)
@field_validator("inputs")
@classmethod
+45 -7
View File
@@ -6,7 +6,7 @@ from pydantic import Field
from ..base import FrigateBaseModel
from ..env import EnvString
__all__ = ["GenAIConfig", "GenAIProviderEnum"]
__all__ = ["GenAIConfig", "GenAIProviderEnum", "GenAIRoleEnum"]
class GenAIProviderEnum(str, Enum):
@@ -14,18 +14,56 @@ class GenAIProviderEnum(str, Enum):
azure_openai = "azure_openai"
gemini = "gemini"
ollama = "ollama"
llamacpp = "llamacpp"
class GenAIRoleEnum(str, Enum):
tools = "tools"
vision = "vision"
embeddings = "embeddings"
class GenAIConfig(FrigateBaseModel):
"""Primary GenAI Config to define GenAI Provider."""
api_key: Optional[EnvString] = Field(default=None, title="Provider API key.")
base_url: Optional[str] = Field(default=None, title="Provider base url.")
model: str = Field(default="gpt-4o", title="GenAI model.")
provider: GenAIProviderEnum | None = Field(default=None, title="GenAI provider.")
api_key: Optional[EnvString] = Field(
default=None,
title="API key",
description="API key required by some providers (can also be set via environment variables).",
)
base_url: Optional[str] = Field(
default=None,
title="Base URL",
description="Base URL for self-hosted or compatible providers (for example an Ollama instance).",
)
model: str = Field(
default="gpt-4o",
title="Model",
description="The model to use from the provider for generating descriptions or summaries.",
)
provider: GenAIProviderEnum | None = Field(
default=None,
title="Provider",
description="The GenAI provider to use (for example: ollama, gemini, openai).",
)
roles: list[GenAIRoleEnum] = Field(
default_factory=lambda: [
GenAIRoleEnum.embeddings,
GenAIRoleEnum.vision,
GenAIRoleEnum.tools,
],
title="Roles",
description="GenAI roles (tools, vision, embeddings); one provider per role.",
)
provider_options: dict[str, Any] = Field(
default={}, title="GenAI Provider extra options."
default={},
title="Provider options",
description="Additional provider-specific options to pass to the GenAI client.",
json_schema_extra={"additionalProperties": {"type": "string"}},
)
runtime_options: dict[str, Any] = Field(
default={}, title="Options to pass during inference calls."
default={},
title="Runtime options",
description="Runtime options passed to the provider for each inference call.",
json_schema_extra={"additionalProperties": {"type": "string"}},
)
+14 -3
View File
@@ -10,7 +10,18 @@ __all__ = ["CameraLiveConfig"]
class CameraLiveConfig(FrigateBaseModel):
streams: Dict[str, str] = Field(
default_factory=list,
title="Friendly names and restream names to use for live view.",
title="Live stream names",
description="Mapping of configured stream names to restream/go2rtc names used for live playback.",
)
height: int = Field(
default=720,
title="Live height",
description="Height (pixels) to render the jsmpeg live stream in the Web UI; must be <= detect stream height.",
)
quality: int = Field(
default=8,
ge=1,
le=31,
title="Live quality",
description="Encoding quality for the jsmpeg stream (1 highest, 31 lowest).",
)
height: int = Field(default=720, title="Live camera view height")
quality: int = Field(default=8, ge=1, le=31, title="Live camera view quality")
+85
View File
@@ -0,0 +1,85 @@
"""Mask configuration for motion and object masks."""
from typing import Any, Optional, Union
from pydantic import Field, field_serializer
from ..base import FrigateBaseModel
__all__ = ["MotionMaskConfig", "ObjectMaskConfig"]
class MotionMaskConfig(FrigateBaseModel):
"""Configuration for a single motion mask."""
friendly_name: Optional[str] = Field(
default=None,
title="Friendly name",
description="A friendly name for this motion mask used in the Frigate UI",
)
enabled: bool = Field(
default=True,
title="Enabled",
description="Enable or disable this motion mask",
)
coordinates: Union[str, list[str]] = Field(
default="",
title="Coordinates",
description="Ordered x,y coordinates defining the motion mask polygon used to include/exclude areas.",
)
raw_coordinates: Union[str, list[str]] = ""
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of motion mask."
)
def get_formatted_name(self, mask_id: str) -> str:
"""Return the friendly name if set, otherwise return a formatted version of the mask ID."""
if self.friendly_name:
return self.friendly_name
return mask_id.replace("_", " ").title()
@field_serializer("coordinates", when_used="json")
def serialize_coordinates(self, value: Any, info):
return self.raw_coordinates if self.raw_coordinates else value
@field_serializer("raw_coordinates", when_used="json")
def serialize_raw_coordinates(self, value: Any, info):
return None
class ObjectMaskConfig(FrigateBaseModel):
"""Configuration for a single object mask."""
friendly_name: Optional[str] = Field(
default=None,
title="Friendly name",
description="A friendly name for this object mask used in the Frigate UI",
)
enabled: bool = Field(
default=True,
title="Enabled",
description="Enable or disable this object mask",
)
coordinates: Union[str, list[str]] = Field(
default="",
title="Coordinates",
description="Ordered x,y coordinates defining the object mask polygon used to include/exclude areas.",
)
raw_coordinates: Union[str, list[str]] = ""
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of object mask."
)
@field_serializer("coordinates", when_used="json")
def serialize_coordinates(self, value: Any, info):
return self.raw_coordinates if self.raw_coordinates else value
@field_serializer("raw_coordinates", when_used="json")
def serialize_raw_coordinates(self, value: Any, info):
return None
def get_formatted_name(self, mask_id: str) -> str:
"""Return the friendly name if set, otherwise return a formatted version of the mask ID."""
if self.friendly_name:
return self.friendly_name
return mask_id.replace("_", " ").title()
+54 -15
View File
@@ -1,43 +1,82 @@
from typing import Any, Optional, Union
from typing import Any, Optional
from pydantic import Field, field_serializer
from ..base import FrigateBaseModel
from .mask import MotionMaskConfig
__all__ = ["MotionConfig"]
class MotionConfig(FrigateBaseModel):
enabled: bool = Field(default=True, title="Enable motion on all cameras.")
enabled: bool = Field(
default=True,
title="Enable motion detection",
description="Enable or disable motion detection for all cameras; can be overridden per-camera.",
)
threshold: int = Field(
default=30,
title="Motion detection threshold (1-255).",
title="Motion threshold",
description="Pixel difference threshold used by the motion detector; higher values reduce sensitivity (range 1-255).",
ge=1,
le=255,
)
lightning_threshold: float = Field(
default=0.8, title="Lightning detection threshold (0.3-1.0).", ge=0.3, le=1.0
default=0.8,
title="Lightning threshold",
description="Threshold to detect and ignore brief lighting spikes (lower is more sensitive, values between 0.3 and 1.0).",
ge=0.3,
le=1.0,
)
improve_contrast: bool = Field(default=True, title="Improve Contrast")
contour_area: Optional[int] = Field(default=10, title="Contour Area")
delta_alpha: float = Field(default=0.2, title="Delta Alpha")
frame_alpha: float = Field(default=0.01, title="Frame Alpha")
frame_height: Optional[int] = Field(default=100, title="Frame Height")
mask: Union[str, list[str]] = Field(
default="", title="Coordinates polygon for the motion mask."
improve_contrast: bool = Field(
default=True,
title="Improve contrast",
description="Apply contrast improvement to frames before motion analysis to help detection.",
)
contour_area: Optional[int] = Field(
default=10,
title="Contour area",
description="Minimum contour area in pixels required for a motion contour to be counted.",
)
delta_alpha: float = Field(
default=0.2,
title="Delta alpha",
description="Alpha blending factor used in frame differencing for motion calculation.",
)
frame_alpha: float = Field(
default=0.01,
title="Frame alpha",
description="Alpha value used when blending frames for motion preprocessing.",
)
frame_height: Optional[int] = Field(
default=100,
title="Frame height",
description="Height in pixels to scale frames to when computing motion.",
)
mask: dict[str, Optional[MotionMaskConfig]] = Field(
default_factory=dict,
title="Mask coordinates",
description="Ordered x,y coordinates defining the motion mask polygon used to include/exclude areas.",
)
mqtt_off_delay: int = Field(
default=30,
title="Delay for updating MQTT with no motion detected.",
title="MQTT off delay",
description="Seconds to wait after last motion before publishing an MQTT 'off' state.",
)
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of motion detection."
default=None,
title="Original motion state",
description="Indicates whether motion detection was enabled in the original static configuration.",
)
raw_mask: dict[str, Optional[MotionMaskConfig]] = Field(
default_factory=dict, exclude=True
)
raw_mask: Union[str, list[str]] = ""
@field_serializer("mask", when_used="json")
def serialize_mask(self, value: Any, info):
return self.raw_mask
if self.raw_mask:
return self.raw_mask
return value
@field_serializer("raw_mask", when_used="json")
def serialize_raw_mask(self, value: Any, info):
+29 -7
View File
@@ -6,18 +6,40 @@ __all__ = ["CameraMqttConfig"]
class CameraMqttConfig(FrigateBaseModel):
enabled: bool = Field(default=True, title="Send image over MQTT.")
timestamp: bool = Field(default=True, title="Add timestamp to MQTT image.")
bounding_box: bool = Field(default=True, title="Add bounding box to MQTT image.")
crop: bool = Field(default=True, title="Crop MQTT image to detected object.")
height: int = Field(default=270, title="MQTT image height.")
enabled: bool = Field(
default=True,
title="Send image",
description="Enable publishing image snapshots for objects to MQTT topics for this camera.",
)
timestamp: bool = Field(
default=True,
title="Add timestamp",
description="Overlay a timestamp on images published to MQTT.",
)
bounding_box: bool = Field(
default=True,
title="Add bounding box",
description="Draw bounding boxes on images published over MQTT.",
)
crop: bool = Field(
default=True,
title="Crop image",
description="Crop images published to MQTT to the detected object's bounding box.",
)
height: int = Field(
default=270,
title="Image height",
description="Height (pixels) to resize images published over MQTT.",
)
required_zones: list[str] = Field(
default_factory=list,
title="List of required zones to be entered in order to send the image.",
title="Required zones",
description="Zones that an object must enter for an MQTT image to be published.",
)
quality: int = Field(
default=70,
title="Quality of the encoded jpeg (0-100).",
title="JPEG quality",
description="JPEG quality for images published to MQTT (0-100).",
ge=0,
le=100,
)
+17 -4
View File
@@ -8,11 +8,24 @@ __all__ = ["NotificationConfig"]
class NotificationConfig(FrigateBaseModel):
enabled: bool = Field(default=False, title="Enable notifications")
email: Optional[str] = Field(default=None, title="Email required for push.")
enabled: bool = Field(
default=False,
title="Enable notifications",
description="Enable or disable notifications for all cameras; can be overridden per-camera.",
)
email: Optional[str] = Field(
default=None,
title="Notification email",
description="Email address used for push notifications or required by certain notification providers.",
)
cooldown: int = Field(
default=0, ge=0, title="Cooldown period for notifications (time in seconds)."
default=0,
ge=0,
title="Cooldown period",
description="Cooldown (seconds) between notifications to avoid spamming recipients.",
)
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of notifications."
default=None,
title="Original notifications state",
description="Indicates whether notifications were enabled in the original static configuration.",
)
+81 -26
View File
@@ -3,6 +3,7 @@ from typing import Any, Optional, Union
from pydantic import Field, PrivateAttr, field_serializer, field_validator
from ..base import FrigateBaseModel
from .mask import ObjectMaskConfig
__all__ = ["ObjectConfig", "GenAIObjectConfig", "FilterConfig"]
@@ -13,36 +14,48 @@ DEFAULT_TRACKED_OBJECTS = ["person"]
class FilterConfig(FrigateBaseModel):
min_area: Union[int, float] = Field(
default=0,
title="Minimum area of bounding box for object to be counted. Can be pixels (int) or percentage (float between 0.000001 and 0.99).",
title="Minimum object area",
description="Minimum bounding box area (pixels or percentage) required for this object type. Can be pixels (int) or percentage (float between 0.000001 and 0.99).",
)
max_area: Union[int, float] = Field(
default=24000000,
title="Maximum area of bounding box for object to be counted. Can be pixels (int) or percentage (float between 0.000001 and 0.99).",
title="Maximum object area",
description="Maximum bounding box area (pixels or percentage) allowed for this object type. Can be pixels (int) or percentage (float between 0.000001 and 0.99).",
)
min_ratio: float = Field(
default=0,
title="Minimum ratio of bounding box's width/height for object to be counted.",
title="Minimum aspect ratio",
description="Minimum width/height ratio required for the bounding box to qualify.",
)
max_ratio: float = Field(
default=24000000,
title="Maximum ratio of bounding box's width/height for object to be counted.",
title="Maximum aspect ratio",
description="Maximum width/height ratio allowed for the bounding box to qualify.",
)
threshold: float = Field(
default=0.7,
title="Average detection confidence threshold for object to be counted.",
title="Confidence threshold",
description="Average detection confidence threshold required for the object to be considered a true positive.",
)
min_score: float = Field(
default=0.5, title="Minimum detection confidence for object to be counted."
default=0.5,
title="Minimum confidence",
description="Minimum single-frame detection confidence required for the object to be counted.",
)
mask: Optional[Union[str, list[str]]] = Field(
default=None,
title="Detection area polygon mask for this filter configuration.",
mask: dict[str, Optional[ObjectMaskConfig]] = Field(
default_factory=dict,
title="Filter mask",
description="Polygon coordinates defining where this filter applies within the frame.",
)
raw_mask: dict[str, Optional[ObjectMaskConfig]] = Field(
default_factory=dict, exclude=True
)
raw_mask: Union[str, list[str]] = ""
@field_serializer("mask", when_used="json")
def serialize_mask(self, value: Any, info):
return self.raw_mask
if self.raw_mask:
return self.raw_mask
return value
@field_serializer("raw_mask", when_used="json")
def serialize_raw_mask(self, value: Any, info):
@@ -51,46 +64,64 @@ class FilterConfig(FrigateBaseModel):
class GenAIObjectTriggerConfig(FrigateBaseModel):
tracked_object_end: bool = Field(
default=True, title="Send once the object is no longer tracked."
default=True,
title="Send on end",
description="Send a request to GenAI when the tracked object ends.",
)
after_significant_updates: Optional[int] = Field(
default=None,
title="Send an early request to generative AI when X frames accumulated.",
title="Early GenAI trigger",
description="Send a request to GenAI after a specified number of significant updates for the tracked object.",
ge=1,
)
class GenAIObjectConfig(FrigateBaseModel):
enabled: bool = Field(default=False, title="Enable GenAI for camera.")
enabled: bool = Field(
default=False,
title="Enable GenAI",
description="Enable GenAI generation of descriptions for tracked objects by default.",
)
use_snapshot: bool = Field(
default=False, title="Use snapshots for generating descriptions."
default=False,
title="Use snapshots",
description="Use object snapshots instead of thumbnails for GenAI description generation.",
)
prompt: str = Field(
default="Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next.",
title="Default caption prompt.",
title="Caption prompt",
description="Default prompt template used when generating descriptions with GenAI.",
)
object_prompts: dict[str, str] = Field(
default_factory=dict, title="Object specific prompts."
default_factory=dict,
title="Object prompts",
description="Per-object prompts to customize GenAI outputs for specific labels.",
)
objects: Union[str, list[str]] = Field(
default_factory=list,
title="List of objects to run generative AI for.",
title="GenAI objects",
description="List of object labels to send to GenAI by default.",
)
required_zones: Union[str, list[str]] = Field(
default_factory=list,
title="List of required zones to be entered in order to run generative AI.",
title="Required zones",
description="Zones that must be entered for objects to qualify for GenAI description generation.",
)
debug_save_thumbnails: bool = Field(
default=False,
title="Save thumbnails sent to generative AI for debugging purposes.",
title="Save thumbnails",
description="Save thumbnails sent to GenAI for debugging and review.",
)
send_triggers: GenAIObjectTriggerConfig = Field(
default_factory=GenAIObjectTriggerConfig,
title="What triggers to use to send frames to generative AI for a tracked object.",
title="GenAI triggers",
description="Defines when frames should be sent to GenAI (on end, after updates, etc.).",
)
enabled_in_config: Optional[bool] = Field(
default=None, title="Keep track of original state of generative AI."
default=None,
title="Original GenAI state",
description="Indicates whether GenAI was enabled in the original static config.",
)
@field_validator("required_zones", mode="before")
@@ -103,14 +134,28 @@ class GenAIObjectConfig(FrigateBaseModel):
class ObjectConfig(FrigateBaseModel):
track: list[str] = Field(default=DEFAULT_TRACKED_OBJECTS, title="Objects to track.")
track: list[str] = Field(
default=DEFAULT_TRACKED_OBJECTS,
title="Objects to track",
description="List of object labels to track for all cameras; can be overridden per-camera.",
)
filters: dict[str, FilterConfig] = Field(
default_factory=dict, title="Object filters."
default_factory=dict,
title="Object filters",
description="Filters applied to detected objects to reduce false positives (area, ratio, confidence).",
)
mask: dict[str, Optional[ObjectMaskConfig]] = Field(
default_factory=dict,
title="Object mask",
description="Mask polygon used to prevent object detection in specified areas.",
)
raw_mask: dict[str, Optional[ObjectMaskConfig]] = Field(
default_factory=dict, exclude=True
)
mask: Union[str, list[str]] = Field(default="", title="Object mask.")
genai: GenAIObjectConfig = Field(
default_factory=GenAIObjectConfig,
title="Config for using genai to analyze objects.",
title="GenAI object config",
description="GenAI options for describing tracked objects and sending frames for generation.",
)
_all_objects: list[str] = PrivateAttr()
@@ -129,3 +174,13 @@ class ObjectConfig(FrigateBaseModel):
enabled_labels.update(camera.objects.track)
self._all_objects = list(enabled_labels)
@field_serializer("mask", when_used="json")
def serialize_mask(self, value: Any, info):
if self.raw_mask:
return self.raw_mask
return value
@field_serializer("raw_mask", when_used="json")
def serialize_raw_mask(self, value: Any, info):
return None

Some files were not shown because too many files have changed in this diff Show More