* feat(deepx): add DEEPX NPU detector and runtime integration.
* feat(deepx): enforce model_format requirement when ppu is enabled and add integrity checks for driver installation
* Update frigate/detectors/plugins/deepx.py
Public method lacks docstring
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Refactor DEEPX detector tests, support SSD and DAMO-YOLO
* feat(deepx): add anchor-free output decoding and corresponding tests
* Add tests and updates for DEEPX detector and refactor DEEPX accelerator code structure.
* fix: enhance model type validation and update documentation for DEEPX detector
* fix: add support for customizable score and NMS thresholds
* refactor: infer YOLO layout from the model, drop per-detector options and the dxrtd placeholder
* fix: keep only the anchor-free PPU verdict, re-read anchor-based each frame
* Update latency data for DEEPX NPU
* Expanding PPU support for DEEPX and set yolo-generic as default.
* enhance scale count resolution logic
* Extend PPU layout handling and YOLOX support to DEEPX detector
* fix: assume the largest PPU anchor table when the .dxnn has no layout
* Improve PPU decoding and introduce strides handling
* Improve PPU scope and fix box format mismatch
* Fix unnamed node issue that breaks traversal
* fix: update object detection model type description to remove outdated architecture
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* don't display audio transcription provider message as health notice
* show remote provider for audio transcription in health pane
* adjust trigger and notifications messages to be consistent with the rest of the settings UI
* disable save buttons when there are no changes in config editor
* fix audio manager crash when a camera is added at runtime
The audio processor and the camera maintainer both poll the same `add` config update on their own one second timers, and the maintainer is what creates `camera_metrics[name]`. When the audio processor got there first it looked the new camera up before that entry existed, and the `KeyError` took down the whole `frigate.audio_manager` process. Whether it happens depends purely on which poll fires first, so cloning a camera from the UI fails or succeeds at random. `spawn_if_needed` now skips a camera whose metrics aren't there yet and picks it up on the next poll, the same way it already waits on a late ffmpeg update.
`AudioEventMaintainer` holds the `CameraMetrics` object now instead of indexing the manager dict on every audio chunk, which drops the IPC round trips and means a removed camera can't `KeyError` out of `detect_audio` after the maintainer pops the entry. The audio process is also registered with the watchdog, since a crash there previously left audio detection dead for every camera until a full restart, and it now receives the shared `DataProcessorMetrics` so `AudioTranscriptionRealTimeProcessor` gets the same type as the other real time processors.
* fix stationary max_frames dropping other tracked objects
When `max_frames` was set for a label, deregistering one object rebuilt norfair's list with a filter that kept an object only if it was both not the target and already on its way out, so every other healthy object of that label was dropped along with it. Any car leaving the frame took the rest of the cars with it and they came back as new tracked objects a few frames later. The filter now removes only the target, and objects that are expiring are still reaped by norfair on the next update.
* fix test
* fix skip_motion_threshold permanently disabling motion detection
The skip check returned before the two `accumulateWeighted` calls at the end of `detect`, so a skipped frame never made it into the background and setting `calibrating` there only picked a faster alpha for calls that never ran. `avg_frame` starts as an all zero image and a normally lit scene differs from black across nearly the whole frame, so the cameras I tested measure 0.84 to 0.98 against it. Any `skip_motion_threshold` below that number skips the first frame, leaves the background black, and skips every frame after it. Motion detection is dead for that camera until the setting is removed or Frigate restarts, with no motion boxes, no motion recordings, and no regions for the tracker since the detector stays calibrating.
Startup isn't the only way in. `update_mask` zeroes the background on any motion config change, and once a camera has calibrated the first IR switch or PTZ move freezes the background on the old scene, so it can't transition to the new one, which is the case the option exists for. The frame is now blended in before the early return at the same 0.2 alpha the calibrating path uses elsewhere, so a large scene change is still suppressed while the background catches up, about a second on a 5 fps camera, and then motion comes back.
* dump ffmpeg logs on every restart
The record watchdog restarted ffmpeg without flushing its `LogPipe`, so a camera whose recording segments went stale never showed a single line of ffmpeg output. The dump now happens in `start_or_restart_ffmpeg` right after the stop, which covers the stale record path, the record crash path, and the audio restart. `reset_capture_thread` and the audio `log_and_restart` fallback keep their own dumps since both pass `ffmpeg_process=None`.
* dump ffmpeg logs once per restart
The audio restart path dumped the log pipe itself before calling the helper, so the restart dump printed a second "last 100 lines" heading over an already drained deque and split the tail that `stop_ffmpeg` flushed into its own section. The heading is now only printed when there's something under it, and the audio path leaves the dump to the restart so each failure produces one section.
* keep all logpipe dumps consistent
* check for a valid frame before using its shape
With the camera offline, no preview frame, and `camera-error.jpg` missing, `latest_frame` read `frame.shape` before its `frame is None` check, so it raised `AttributeError` and answered 500 with a traceback instead of the intended "Unable to get valid frame". The check now runs first.
* fix the has_clip self-heal for events with no recordings
`vod_event` looked for a `(body, 404)` tuple, but `vod_ts` returns a `JSONResponse`, so the check never matched and an old event whose recordings are gone kept offering a clip that can't play. It now checks the response status code.
* return 403 for a snapshot or thumbnail on another camera
The broad `except Exception` handlers in `event_snapshot` and `event_thumbnail` caught the `HTTPException` from `require_camera_access`, so a restricted user asking for another camera's snapshot got a 404 instead of a 403, and for an object still being tracked the snapshot was rendered before the check ran. Both endpoints now look up the event and check access in their own block, the way the other endpoints do, so a denial propagates.
* find DST transitions to the second
`get_dst_transitions` probed the offset once every 24 hours from the start time and reported a change at the first probe after it, up to a day late, so events, review items and recordings near a transition were grouped into days with the old offset. A transition after the last daily probe wasn't found at all. The end of the range is probed too now, and a probe that sees the offset change bisects the interval to the second of the transition.
* don't run page shortcuts for keys a dialog already handled
Radix dismisses a dialog on Escape from a capture-phase keydown listener and calls `preventDefault()` without stopping propagation, so `useKeyboardListener` still ran the page's Escape shortcut: cancelling the delete dialog in the face library or a classification model also cleared the whole selection. Keys another shortcut hook handled still get through, since their listener order changes with every render.
* fix train image filtering for a class with a dash
The backend writes a class with a `-` as `_` in train file names, since it splits those names on `-`, while a dataset folder keeps the dash. Filtering the Train grid by `half-open` compared it with `half_open` and hid every attempt. Both sides are normalized the same way now.
* don't edit a chat message while a reply streams
The edit button stayed active while a reply streamed. `submitConversation` returns early while loading, but the message bubble still closed its editor, so the edit was silently lost. The edit button is hidden while a reply streams, and an editor that's already open keeps its draft with send disabled until the reply ends.
* fix restart failing under non-root
restart_frigate() called psutil.Process(1).terminate() to signal s6-svscan, but s6-svscan runs as root while frigate runs as uid 1000, so the call raised AccessDenied. That exception escaped every caller: the UI restart button dropped its websocket client, MQTT restart and Save & Restart just logged and did nothing, and the watchdog crashed its own monitoring thread on a dead detector. This catches AccessDenied and falls through to the existing SIGINT branch, which exits the process for s6 to restart it.
* show runtime overrides in the settings form
The settings form read a camera section's saved config value, but its dependent warnings (audio transcription requiring audio detection, snapshots requiring detect, etc.) read the live config instead. A runtime toggle from the live view, MQTT, or an active profile can turn a section off without touching yaml, and that override persists across restarts, so the Enable switch showed on while the warning said the feature wasn't enabled. This adds an "Overridden (Live)" badge to any field whose live value differs from what's saved, and swaps the affected warnings to runtime-specific wording when a runtime override is the actual cause instead of the config.
* fix mobile overflowing icons in system due to new health pane
* fix genai settings keeping a stale model and dropping roles after save
Switching a GenAI entry's provider left the previous provider's model selected, so saving wrote a model the new provider doesn't serve. llama.cpp can't find that model in `/v1/models`, so the backend reported every capability as false for the entry, and once the save refetched `genai/models` the roles widget stripped `transcribe` from the form on its own. The section showed unsaved changes right after saving, and saving again would have dropped the role. Switching provider now clears the model, and the roles widget only strips a role for a model or provider picked in the form, since the entry-level capability flags only describe the saved model. A selected role stays visible when the provider can't confirm it, so it can still be switched off. The llama.cpp model list also no longer repeats a model whose alias matches its id, which is what `--alias` produces.
* close onvif sessions on shutdown
`OnvifController.close()` only stopped its event loop, so the aiohttp sessions each `ONVIFCamera` holds and the `_poll_config_updates` task were left to be garbage collected during interpreter shutdown, when their warnings can no longer be logged. Every restart ended with a run of `Unclosed client session` and `Task was destroyed but it is pending!` logging errors, which only became visible once restart started exiting the process itself under non-root. `close()` now closes each camera's client and cancels the tasks on the loop before stopping it.
* fixes
* fixes
Validate config exists 0 regardless of if the config is valid or not.
This makes it not very useful for CI
Tiny fix to bail non-zero if the config is invalid
* Add support for running transcription with GenAI
* Improve audio joining
* Fix GenAI model capability reporting
* Support language correctly
* Migrate existing users to keep english selected
* Fix models
* Fix tests
* Fix accepted null model
* Handle slwo providers
* Implement annotated frames mode for GenAI reviews to improve models with lacking temporal understanding
* Updates
* Improve debug sharing
* Do not number objects
* Fix assumptions
* Remove unhelpful content
* Improve object data sent as part of prompt
* Cleanup ollama dumbness
* Bind db
* Fixes
* Cleanup
* improve zone renaming
Zone rename now saves as one JSON body via config/set, moving required_zones and profile overrides instead of leaving stale references
* fixes
* allow users to select live streaming technology
* fix webrtc being downgraded to mse on load
`useUserPersistence` seeds state with the default and loads asynchronously, so the first render always used `mse` instead of the saved choice, and `useWebRTCGloballyAvailable` reports `checking` until the probe settles and re-enters that state on every consumer mount, so a saved `webrtc` was rewritten to `mse` even after the probe had already passed. On Safari the MSE player then timed out and latched the jsmpeg fallback. A pending probe now counts as available, the player waits on `autoLive` until the stored preferences load, and `handleError` gates on the mode in use since the fallback flag no longer implies webrtc is untried. A rejected IndexedDB read also resolves `loaded` now, so a blocked store can't leave the player waiting forever.
* add support for configurable ICE servers in WebRTC player
* add mic error state, fix dialog overwriting saved choice and dashboard ignoring stream
* tweaks
fix duplicated styles, move fonts to src/assets/fonts for vite to bundle (nginx already rewrites correctly), and fix fast refresh for PreviewController, auth context/provider, and statusbar context
* update date-fns, i18next, react-dropzone, react-markdown and @types/node
react-i18next 17 peers `i18next >= 26.2.0`, so the two move together. react-day-picker already depends on date-fns 4, so date-fns now dedupes to a single copy. react-dropzone 20 declares `node >=22` in `engines`, but npm only warns on Node 20 and nothing in the build needs Node 22.
* update js-yaml, @hookform/resolvers and prettier
js-yaml 5 has no default export, so `DictAsYamlField` imports `dump`, `load` and `YAMLException` by name. @hookform/resolvers 5 types `zodResolver` with the schema's input and output types separately, and fields with defaults are optional on input, so the zone and classification model forms pass both types to `useForm`. zod's range now starts at 3.25, which resolvers 5 requires.
* format with prettier 3.9
* remove unused web test deps
Nothing runs vitest. The CI step that called `npm run test` is commented out, `web/__test__/` was deleted in https://github.com/blakeblackshear/frigate/pull/8983 so `setupFiles` points at a missing file, and there are no unit tests, so `npx vitest run` only picks up the Playwright specs and fails. jsdom, `@testing-library/jest-dom`, msw and fake-indexeddb were only there for vitest.
* update contributing docs
* remove unused deps
konva 10.5 removed the private `Node._lastPos`, so `PolygonCanvas` now reads the dragged point from `getAbsolutePosition()`, which returns the position konva just applied. monaco-yaml 5.5 takes formatter options instead of a boolean for `format`. The radix packages move together so every shared primitive stays a single copy under the `react-slot` and `compose-refs` overrides.
* fix recordings unavailable endpoint when no params are provided
we already import the datetime class directly, so those attribute lookups raised AttributeError and the request returned 500
* reject JWTs whose role is no longer in the config
`/auth` trusted the role inside the JWT and re-signed it on refresh without checking the config, so a user whose restricted role was deleted kept a session carrying a role that isn't in `auth.roles`. The media, clip, recording, export and go2rtc checks treat a missing role the same as a role with no camera list, so that session could open every camera. `/auth` now returns 401 for a token whose role isn't configured, which sends the user back through login, and login already falls back to `viewer` for a role that's gone.
* delete the deleted camera group's layout, not the open one's
Deleting a camera group removed the layout of the group being viewed, because the dialog's delete was bound to `${activeGroup}-draggable-layout`. It also cleared the saved group, and both ran before the `config/set` request whether or not it succeeded, so deleting one group while viewing another lost the open group's layout and left the deleted group's layout behind. The deleted group's own layout is now removed after a successful save, and the saved group is only cleared when it was the open group.
* don't block API when querying PTZ info
camera_ptz_info is async but waited on the ONVIF controller's future with future.result(), blocking the API event loop for as long as a slow or unreachable ONVIF camera took to answer (including reconnect attempts), so every other async request stalled with it. Await the future with asyncio.wrap_future instead. The coroutine runs on the controller's own loop and thread, so this cannot deadlock.
* for custom exports, only allow admin users to add to existing cases
follows the existing convention where attaching an export to an existing case is admin-only on `POST /export/{camera}/...` and `POST /exports/batch`
* drop pending edits for a camera or profile that no longer exists
* match cached preview frames to their camera exactly
https://github.com/blakeblackshear/frigate/pull/22594 added a trailing `-` to the `preview_{camera}` prefix so `camera` stopped matching `camera2`'s frames, but camera names can contain `-`, so `front` still matched `front-door`'s. After a restart, `front`'s preview recorder deleted this hour's frames of a matching camera that sorted before it and added the timestamps of one that sorted after it, so ffmpeg was asked for files that don't exist and that hour's preview was lost. The offline fallback for `latest.jpg` could also return `front-door`'s frame for `front`, even to a user without access to `front-door`, and a short export could take its fallback thumbnail from the other camera. These now compare the full camera name taken from the file name.
* migrate web to eslint 10 flat config
ESLint 10 dropped `.eslintrc` support, so `.eslintrc.cjs` is replaced with `eslint.config.js` and the lint scripts no longer pass `--ext` or `--ignore-path`. typescript-eslint moves to 8, react-hooks to 7, and react-refresh to 0.5, and the unused jest and vitest-globals plugins are removed. Lint behaves as it did before: catch variables aren't checked, unused disable directives aren't reported, and rules newly added to the recommended sets are off until the code passes them. typescript-eslint 8 flags constants used only in `typeof`, so those are now exported, or replaced with a union type where the export would trip react-refresh.
* fix lint findings from the eslint 10 recommended rules
Remove the rule overrides from the flat config migration and fix what they were hiding. Unused catch bindings are dropped, 20 disable directives that suppressed nothing are removed (react-hooks 5.2 and 7.1.1 report identical exhaustive-deps findings with inline config ignored), dead initial values are dropped, short-circuit calls become if statements or optional calls, rethrown errors pass `cause`, and the disabled "No recordings" tooltip in `ReviewTimeline` is removed along with its memo and the `getRecordingAvailability` prop. The 3 react-refresh warnings for files that export contexts or classes are left for a later refactor.
* update docusaurus to 3.10.2
* bump @types/node to 25.9.6 and ES2022
target ES2022, which already includes ES2020 and ES2021.String, so the lib list was also trimmed
* bump vite to 8.3.0 and vitest to 4.1.11
Swap `@vitejs/plugin-react-swc` for `@vitejs/plugin-react` and add `esbuild` as a devDependency, since `vite-plugin-monaco-editor` requires it and Vite 8 no longer ships it. `keepNames` moves to `build.rolldownOptions.output` because Vite 8 ignores the `esbuild` block. Rolldown's minifier now writes the preload helper's base path as a template literal instead of a double-quoted string, so the nginx `sub_filter` for `return"/BASE_PATH/"` stopped matching and lazy-loaded chunks and their CSS were requested from a literal `/BASE_PATH/` under Home Assistant ingress. The rule now matches the backtick form.
* bump apexcharts to 7.3.0 and react-apexcharts to 2.1.1
Under Vite 8, a default import from a CommonJS package resolves to its whole `module.exports` when `package.json` has `"type": "module"`, so react-apexcharts 1.4.1 handed React an object and every chart crashed. 2.x ships an ESM build. apexcharts 7 no longer sets `window.ApexCharts`, so the chart components now import it for `ApexCharts.exec`, and `ApexAxisChartSeries` is derived in `types/graph.ts` because it's no longer a global type.
* remove unused immer dep
* remove unused cython pin from tensorrt requirements
The pin was added alongside tensorrt 8.5.3 and cuda-python 11.8, which needed Cython to build, and both have since been dropped from this file. Nothing imports Cython at runtime, and `pip3 wheel` runs with build isolation, so any source build gets its own build dependencies. The pin only installed an unused Cython wheel into the TensorRT image.
* require node 20.19 for docs
types-peewee 4.0 types model fields precisely, so 17 `type: ignore` comments and 2 `cast(str, ...)` calls are no longer needed. Its stubs type `.namedtuples()` and `.dicts()` queries as returning model instances, so the review cleanup reads namedtuple fields by name and the storage usage query casts its dict rows. `start_time` is declared `DateTimeField` but stores unix timestamps, so two reads cast it like `debug_replay.py` already does. `Export` gets an annotation for the `export_case_id` attribute peewee adds at runtime.
* add error boundary
on frontend errors, show a page-level recovery panel and a separate compact strip for the sidebar/status/bottombar so failures there don't take down the rest of the page
* tweaks
* fix test
* Rename face model to face recognizer
* Refactor face detection into own module
* Return face and face landmarks
* Align faces with 5 points instead of just the eyes
* Add landmark validation to throw out images which do not fit a landmark
* Fix circular import and lock face detector for concurrent runs across threads
* Fix mypy
* Support multi resolution exports
* Fix decoder text
* Add dropdown and ability to select export stream selection
* Fix for review comments
* Fix mypy
* Cleanup wording
* rename migration
* use ASC composite index for event camera and start_time
the planner still picks for newest-first single-camera queries but doesn't slow down full per-camera scans on a cold cache
* pause playback while the timeline range handles are up
* reduce multi camera seeded export range to 30m
allows both handlebars to fit within most desktop windows
* Add additional indicies on event and review tables. Every events or timeline endpoint filters on event start time and camera, this should speed things up by avoiding a range scan on the table.
* Rewrite to use a CTE to leverage speedups by using sqllite internal optimization to do a single query instead of a starter query to get distinct labels and a subsequent loop of querys per distinct event labels.
Frigate is currently shipping sqlite 3.46.1, which is above the minimum version 3.25 needed for CTEs.
* Collapse a few sequential queries into a single one.
* Use peewee instead of rw sql for the CTE query.
* Slightly simplify review logic and avoid duplicating the json response for empty review IDs.
* Rerun ruff formatting.
* Remove 2x unnecessary index on reviewsegment, remove reference to prior code implementation in comment in event.py
* Editor fail, re-ruff format.
* Remove the CTE and restore the generator with sub-queries, which is more performance (thanks Nick and Blake for testing against your larger DB!)
* Update peewee index migration description
* Add on_conflict_ignore, replacing the try/catch/pass on IntegrityError
* Add a testcase for validating that on_conflict_ignore bypasses what was formerly an IntegrityError
* Change testcase to clarify that it covers the peewee behavior of on_conflict_ignore
---------
Co-authored-by: Greg <{ID}+{username}@users.noreply.github.com>
* add a notice registry and System Health tab
Problems Frigate detects on its own (ffmpeg crash loops, stuck detectors, failed model downloads, recordings deleted before their retention period) only ever existed as log lines. This adds a `NoticeRegistry` in the main process backed by two tables, an `update_notice` IPC topic so producers in other processes can reach it through the dispatcher, an admin-only API and websocket topic, and a Health tab that lists them. Kinds declare their own mode, severity, and category in one place: state notices are resolved by their producer, event notices are dismissed by the user.
* treat a prerelease as behind its final release
* fix notices clearing early
* rename menu items and update docs
* don't resolve the update notice on a failed version lookup
isLoading and isBuffering only clear on playback progress, and scrubbing holds the player paused, so a source rebuild during a timeline selection left them set with nothing able to clear them. Cancelling made them visible as a spinner over an already-loaded frame, which stayed until the next manual seek. Leaving a scrub now clears them when the source is loaded and the element holds a frame.
* Refactor hardware stats to have consolidated ffmpeg, detector, and enrichments running.
* Cleanup hardware access that is not passed into the container
* Remove network stats from hardware refactor
`after` and `before` defaulted to `datetime.now()` in the function signature, so they were evaluated once at import. Requests that omitted them got a window ending at process start. They now resolve in the handler.
* Support read-only rootfs with self-signed certs in /config/tls
* Support read-only rootfs in s6 and pre-compile bytecode
* Assert read-only rootfs support in CI
* Document hardened read-only deployment
* keep certsync's cert selection identical to nginx's
* note the uid trade-off in user: mode
* fail fast when EXTRA_GROUPS or a missing media volume meets read_only
* keep nosuid and nodev on the /run tmpfs
* support read_only in the default mode
* don't take go2rtc down when the homekit file isn't writable
* lead with the hardware consequence of switching to user:
* refuse to write TLS material through a symlink as root
* note that memryx writes models to the root filesystem
* certsync watches whichever cert path nginx loaded
* Run the frigate service as the frigate user
* Run go2rtc as its own restricted user
* Run nginx as the frigate user with writable state in /tmp/nginx
* Disable bandwidth stats gracefully when not running as root
* Hand TensorRT model cache ownership to the runtime user
* Document non-root operation and per-hardware device access
* Create /media/frigate after the ownership sweep
* Assert non-root services, JWT migration, and escape hatch in CI
* only write the sweep sentinel when a media volume is mounted
* tolerate homekit config chown failures in the go2rtc run script
* chown the s6 log pipe so non-root nginx can reopen /dev/stdout
* set HOME to /config for non-root services
* run smoke nginx -t and the write probe as the runtime user
* re-own the nginx shm cache on service restart
* discard stdout for the unprivileged smoke nginx -t
* unwrap hard-wrapped prose in the installation docs
* report progress during the ownership sweep
* document EXTRA_GROUPS as the only device access path for dropped services
* expand the non-root device access docs with diagnosis steps and udev rules
* document network storage ownership and the remaining detector hardware
* skip lost+found during the ownership sweep
* hand /tmp/cache to the runtime user before services start
* make bundled models readable by the runtime user
* reload nginx by signaling the master instead of parsing its config as root
* harden root writes into unprivileged-owned paths
Restrict the sweep sentinel to a mount at or below /media/frigate so a
parent /media mount cannot bless a later-shadowed volume. Rebuild
/tmp/nginx root-owned each start so root's cp and tempio writes cannot
follow a symlink an unprivileged nginx planted in the previous run.
* collapse the duplicated sentinel comment
* add a service-runs-as-root helper for granular root services
* validate FRIGATE_ROOT_SERVICES and fail fast on unknown names
* let services listed in FRIGATE_ROOT_SERVICES skip the privilege drop
* record the root-services mode in the sentinel and sweep small trees each boot
* cache the runtime ids in the ownership helper
* chown recordings, previews, and exports to the runtime user at create
* chown the database files after init
* recommend FRIGATE_ROOT_SERVICES in the bandwidth stats warning
* assert granular root services in CI
* document FRIGATE_ROOT_SERVICES
* own every directory level created for a recording segment
* clear the cached runtime ids when ownership tests finish
* skip missing media paths in the per-boot ownership sweep
* clarify granular root services docs
* clean up
* install acl for device access grants
* grant runtime users access to mapped device nodes at boot
* assert device access grants in CI
* document automatic device access grants
* stop telling users device access needs host side setup
* clarify the non-root docs
* link the migration script to the repo
* group the manual device setup under one section
* harden against symlink attacks
/config is owned by the unprivileged runtime user after the ownership sweep, so root operations on files there could be redirected by a planted symlink.
- go2rtc HomeKit setup: replace the root yq/jq normalization and chown with an O_NOFOLLOW helper (prepare_homekit.py), so a symlink at go2rtc_homekit.yml can't redirect a root write or chown onto another file
- go2rtc binary override: ignore /config/go2rtc whenever the service runs as root, so a planted binary can't exec as root under FRIGATE_ROOT_SERVICES
- sweep sentinel: read and write it through safe-sentinel, which trusts only a root-owned regular file and never follows a symlink, so it can't be forged to skip the migration or symlinked to clobber a root file
- ownership sweep: chown with -execdir so a parent directory swapped for a symlink mid-walk can't redirect the chown out of the volume
- validate inputs: restrict DEVICE_ACL_PATHS to /dev, require nonzero numeric EXTRA_GROUPS, and reject PUID/PGID that collide with the go2rtc ids
- docs: correct the TLS key ownership note to match what actually happens
* tweak docs
* stop the ownership sweep chasing entries other mechanisms own
* keep custom binaries out of root services only under granular root
The `v1.x` line is the same muxed fMP4 code as Kaltura's 1.31 with fixes backported, so the mapping JSON, manifest routes, and ffmpeg consumers are unchanged. The `MAX_CLIPS` patch applies as-is and the HEVC workaround is rewritten for the fork's reformatted source.
`vod_hls_version 6` is now explicit because the fork replaced Kaltura's automatic version calculation with a directive that defaults to 4 and only warns when fmp4 needs 6, so playlists were being stamped `EXT-X-VERSION:4` while carrying `EXT-X-MAP`. The `error_page 502 =404` hack is gone: https://github.com/kaltura/nginx-vod-module/issues/468 is a `vod_mode remote` bug and we're `mapped`, so those 502s were really `_vod_response` returning 404 upstream. The fork maps that through now, and the hack was also turning real 5xx into "no recordings".
* use the camera's model for the lpr vehicle check
`FrigateConfig.model` became `models[]` in the detector refactor, so this didn't compile on 0.19. Each camera has its own detector/model pair now, so the check resolves the camera's scene with `getModelForCamera` instead of looking at every model.
* use camera config model
* fix export test
* add onboarding wizard for new users
* resolve hwaccel per camera and clarify recording retention
The hwaccel step listed every preset Frigate ships, so an Intel box was offered Raspberry Pi and Rockchip decoding, and the codec specific presets (`preset-intel-qsv-h264` vs `-h265`) were offered as global values that break as soon as two cameras use different codecs. `/hardware/hwaccel` now returns the decoding families the probed hardware can actually use, each carrying a preset per codec, and the wizard resolves the family against the detect stream codec the camera wizard already probed: one global `ffmpeg.hwaccel_args` when every camera agrees, per-camera `cameras.<name>.ffmpeg.hwaccel_args` when they don't. The global stays on `auto` in that case so cameras added later still resolve at startup. A gen13+ Intel machine keeps its QuickSync recommendation with mixed h264 and h265 cameras instead of dropping to vaapi.
The recording step's "Days to retain recordings" only wrote alert and detection retention, and the storage estimate under it assumed continuous recording. It now asks what to record in plain language, writes `record.continuous.days` to match, shows the estimate only for continuous, and drops the spinner arrows on the number input.
* clean up
* add light/dark mode icon switcher
* use yml as default config file extension when not found
* i18n tweaks
* gate the setup wizard on cameras instead of a config key
* render setup wizard steps by key
* share the setup wizard e2e helpers and mock users
* add an account step to the setup wizard
* add setup wizard account step e2e coverage
* cover the account step's restart behavior
* button consistency
* fix test
* docs
* fixes
* fix inconsistent export download filenames
Zip entries in a case download were named from `Export.name`, the friendly display name, while an individual download uses the file name on disk. The two have always been formatted differently, so one export came out as `front_door_20260823_020615-20260823_020734_abc123.mp4` on its own and `front door 2026-08-23 020615 2026-08-23 020734.mp4` inside a zip. Zip entries now use the on-disk file name, and renaming an export renames its file, so there's only one name to download under. The rename is blocked while ffmpeg still holds the file.
* cap filename length and catch duplicate names
* fix export rename and stop blocking the event loop
* move the rename rollback off the event loop
* no awaits
* pin genai review frames to the main stream
* retain previews as long as either stream has recordings
* watch sub stream recording health separately from main
* reject record_sub on the same input as record and document the role
* derive recording paths from the cache segment timestamp
Recording paths carry one second of resolution, but since sub stream recording start times are resolved to fractional wall clock, anchored to the cache file mtime and chained to the previous segment's end. A stream cutting segments faster than once a second resolves consecutive segments into the same second, so two rows collide on the unique path index and the batch insert fails. The cache segment name is unique per camera stream and second by construction because ffmpeg names segments with strftime, so the recording path is now built from that timestamp while the row keeps the resolved start time. This also restores the path semantics from before sub stream recording, when start times came straight from the cache filename.
Nothing derives times from recording paths: playback offsets, stream switching, and export all use the row's start time, which is unchanged, and the recordings sync matches files by exact path string.
* keep the rest of a recording batch when one row conflicts
* only publish record_sub status when a sub stream is configured
* don't shadow camera_cfg when publishing empty cache streams
* back off restarts when a recording stream goes stale
* give the shared sub stream grace on any capture thread reset
* include segment details in recording discard warnings
* Create frigate and go2rtc runtime users in the image
* Add single fix-ownership helper for volume permission migration
* Add init-usermod oneshot for PUID and PGID remapping
* Chown newly created runtime directories to the frigate user
* Run sentinel-guarded ownership sweep during prepare
* Add host-side volume permission migration script
* Guard log directory ownership for user-mode startup
* Fall back to plain s6-log when running without root
* Assert PUID remapping and sweep sentinel in CI smoke test
* Skip the ownership sweep in the devcontainer
* Pin FRIGATE_RUN_AS_ROOT in ownership tests
* Do not record the sweep as complete when a chown failed
* Validate PUID and PGID in the migration script
* Treat a failed ownership scan as an incomplete sweep
* Reject PUID and PGID of 0 during remapping
* Handle symlinks, dry runs, and sentinel write failures in the sweep
* Treat an absent sweep root as an incomplete sweep
* Verify s6-overlay downloads against pinned checksums
* Verify go2rtc download against pinned checksums
The v1.9.14 release publishes no checksums file, just the bare per-platform binaries, so these digests come from a one-time fetch rather than upstream. That pins the artifact against later substitution, which is the realistic threat for a version we stay on for months, but it does not verify the original download. The stage moves from `ADD --link` to a script because `ADD --checksum` can't express an architecture-dependent URL.
* Verify main image downloads against pinned checksums
Covers everything the main image downloads on the default path: tempio, the hailort runtime tarball and wheel, the six ffmpeg builds, the libedgetpu deb, and the thirteen Intel driver debs. The hailort tarball was streamed straight into `tar`, which can't be verified before extraction, so it downloads to `/tmp` first. The three ffmpeg blocks per arch collapse into one `install_ffmpeg` helper since they only differed by URL and install dir, and the Intel debs go through a `fetch_intel_deb` helper for the same reason.
The Intel debs are the ones that mattered most here. They're installed as root with `dpkg` on the default amd64 path and had no verification at all. compute-runtime publishes a `ww<week>.sum` asset with every release and npu-driver published `checksum.sha256` on v1.19.0, so those eight digests came from upstream rather than from us. intel-graphics-compiler and level-zero publish none, so those five and everything else here come from a one-time fetch, which pins the artifact against later substitution but doesn't verify the original download. The comment above the map says which is which and how to refresh them, since npu-driver has stopped publishing sums since v1.19.0 and that provenance won't survive the next bump.
Still unpinned: `get-pip.py`, which is a rolling URL where a digest would just break the build on pypa's next edit, and the per-variant artifacts for Axera, Synaptics, and Jetson. apt repositories are out of scope since apt already verifies signatures.
* Restrict generated TLS key permissions
OpenSSL 3.x already writes the key at 600 on its own, so this pins the guarantee rather than fixing an observed leak: the mode no longer depends on the openssl version or the umask the service happens to start with. Only the generated pair is touched. User-mounted certs take the other branch and are never chmod'd, which matters when they're mounted read-only.
* Add security headers and server_tokens off
Adds `X-Content-Type-Options: nosniff` and `Referrer-Policy: strict-origin-when-cross-origin`, and turns off nginx version disclosure.
No `X-Frame-Options` and no CSP `frame-ancestors`. HA's Webpage card and iframe panels frame Frigate's own address cross-origin, and either header would break them silently with nothing in Frigate's logs to explain it. Ingress is same-origin and would survive `SAMEORIGIN`, but Frigate can't tell the two apart from inside the container. `security_headers.conf` is a plain file in the image rather than a generated one, so anyone who does want framing restrictions can bind-mount it.
`add_header` doesn't inherit into a block that declares its own, so the include goes in per block, all nine of them, including the four nested static-asset locations that serve the JS bundles. Those are the ones nosniff actually matters for.
The run script now reads `get_nginx_settings.py` once into a variable instead of shelling out per template. That script imports the frigate config machinery, which is noticeable on an SBC.
Not fixed here: `listen.conf` is included at server level and carries `Strict-Transport-Security`, so those same nine blocks already drop HSTS under TLS today. Folding it into this file would change existing TLS behavior on nine paths, so it needs its own PR.
* Restrict go2rtc config file permissions
* Log failed login attempts with source address
Failed logins returned a bare 401 and left nothing behind, so credential stuffing was invisible unless you were already watching nginx access logs. Both failure branches now log a warning with the attempted username and the client address.
The address comes from `get_remote_addr()`, the same helper the login rate limiter keys on, so the two agree on who the client is and the trusted-proxy handling is consistent. Logging a raw `x-forwarded-for` instead would let an attacker forge the source address in the very log line meant to catch them.
The response is unchanged and identical either way. Which factor failed is only visible in the log, never to the client, and the password is never logged.
* Recommend least-privilege container options in install docs
The compose generator pushed `privileged: true` into every file it produced, no matter what hardware you picked, and it's the default tab on the install page so it's what most people copy. It now emits `security_opt: no-new-privileges:true` instead, and only adds `privileged: true` for hardware that actually needs it, with the reason inline. MemryX is the only one today, since it needs to reach the max-manager. Rockchip and Synaptics only want privileged during initial setup and their documented end state is device mappings, so neither gets it.
`no-new-privileges` merges into the same `security_opt` block as any device-specific entries, so Rockchip still gets its `apparmor=unconfined` and `systempaths=unconfined` without a duplicate key.
The static example now has `privileged` commented out, and there's a short section on the options worth adding, with a note that `cap_drop: ALL` breaks `telemetry.stats.network_bandwidth` since nethogs needs NET_ADMIN/NET_RAW.
* Add amd64 container smoke test to CI
Boots the built amd64 image against a minimal config and asserts the two security headers, that the Server header no longer carries a version, that no frame-ancestors is present, that nginx accepts its own config, and the two file modes. This is also the harness the rest of the hardening work extends.
The two negative assertions are written as `if grep; then exit 1; fi` rather than `! grep`. Bash exempts a negated command from `set -e`, so the `!` form would have passed even with the version and frame-ancestors both present, which is the opposite of what a regression net is for.
* improve keyframes messages
* don't pad the labelmap with unknown
`load_labels()` prefilled 91 `unknown` entries before reading the label file, so any model with fewer than 91 classes kept that padding in `merged_labelmap` and `unknown` showed up as a selectable object type in the objects settings UI. The padding only existed so `RemoteObjectDetector.detect` could index the labelmap without a KeyError, and it didn't even cover the empty-file case or Frigate+, which never had a prefill. Both lookups now skip class ids the labelmap doesn't name and warn once per id.
* add secrets.yaml and merge substitution sources by precedence
FRIGATE_ENV_VARS was built once at import from container env and /run/secrets, and the environment_vars validator overwrote it unconditionally, so the block beat the deployment and nothing could be re-read. Sources are now separate dicts merged lowest to highest (environment_vars, secrets.yaml, container env, credentials directory), re-read at the top of every parse, and a collision warns once naming the winner. An undefined {FRIGATE_*} raises a ValueError subclass so pydantic reports the field instead of a KeyError traceback.
* use the shared substitution namespace in go2rtc config
The generator rebuilt the namespace itself from os.environ and a hardcoded /run/secrets, so it never saw environment_vars or CREDENTIALS_DIRECTORY, and str.format made any stray brace fatal. It now installs the FRIGATE_ names from environment_vars and substitutes streams the same way every other field does.
* read the exec override from an import time snapshot
environment_vars is exported into os.environ, and is_go2rtc_arbitrary_exec_allowed read os.environ live, so the config file could enable exec sources. Snapshot the variable at import, which runs before any config is loaded.
* docs
* clarify docs
ffmpeg's stderr was piped but never read, so recording segments that generate more than 64 KB of ffmpeg warnings blocked ffmpeg mid-write, stranding the streaming thread and its anyio threadpool token for good. Enough of those and every sync endpoint stops responding until restart. The trigger is how noisy the segments are, not how long the clip is.
Send stderr to a temp file instead, and guarantee ffmpeg teardown and playlist cleanup on every exit path, including client disconnect.
Also fixes two bugs the deadlock hid: the failure branch was unreachable because returncode is None mid-loop, so the playlist file leaked and ffmpeg's logs were never reported. Playlist files now get a unique name so concurrent requests for one range cannot delete each other's input.
Extracts the terminate helper motion search already had into frigate/util/ffmpeg.py, now shared by both streaming call sites.
* add import/export for camera group layouts and streaming settings
Camera group layouts and per-camera streaming settings are stored in the browser's IndexedDB, so they are tied to a single browser on a single device. Users with more than one device have to rebuild every group layout and re-pick every camera's stream settings by hand, and clearing browser data loses the work.
Add a Backup & Restore card to Settings > UI Settings that exports these settings to a JSON file and imports that file on another device. Import shows a confirmation dialog with per-section counts, switches for layouts, streaming settings, and UI preferences, and warnings about camera groups or cameras in the file that are not on this server.
Server-side storage is deliberately avoided. These are per-device presentation settings: a layout arranged for a desktop is wrong on a tablet, and continuous full-resolution streams that are free on a wired LAN are not on a phone. An explicit file moves settings only when the user chooses to move them.
Implementation notes:
- web/src/utils/uiSettingsTransfer.ts owns a registry of transferable IndexedDB keys. Each entry records whether the key is user-namespaced, matching which persistence hook wrote it, plus a zod schema for its value.
- Only registry-known keys are ever written, and only when their value passes that schema. The file format deliberately lets unknown keys survive parsing, so this filter is what prevents a hand-edited file from writing arbitrary storage keys or out-of-range values.
- Export falls back to the legacy un-namespaced key, because the username migration runs lazily on first mount of each owning hook.
- Streaming settings merge per group rather than replacing the whole map, so groups configured only on the receiving device survive.
- Import writes storage and then reloads, because useUserPersistence reads a key only on mount and StreamingSettingsProvider would otherwise write its stale in-memory state back over the import.
- playbackBandwidthEstimate, frigate-search-history, and live-layout are excluded: the first two are measurements and user data rather than preferences, and live-layout's default is derived from the device.
* merge imported streaming settings per camera instead of per group
Allow audio classes to be grouped under a shared configured label.
Keep audio overrides separate from object labels and retain only the highest-scoring grouped detection.
Refs #23967
* refactor mqtt so that Frigate owns the transport lifecycle instead of delegating it to paho
* release the shutdown barrier on worker crash and replay retained publishes the broker never acked
* collapse in-flight retained values by topic and release the shutdown barrier from a finally
* replay the outage buffer before the publish queue so newer values are not reverted
* serve a segment startup ladder so seeks begin playing sooner
nginx-vod was handed one 10s segment per recording file, so every playlist start had to download and decode a full segment before the first frame. Declare real keyframe data per clip and let nginx cut short leading segments from it.
- add vod_bootstrap_segment_durations 1000/2000/4000 so each playlist starts with 1s/2s/4s segments before settling at 10s
- emit real clip-relative keyFrameDurations (plus firstKeyFrameOffset when nonzero) from the recording keyframe index; rows without an index keep the whole-clip declaration, the only safe cut without keyframe knowledge
- drop the manifest's segment_duration field, which was always inert: nginx-vod parses only camelCase segmentDuration
- rebuild the player source at the seek target, quantized to a 10s grid, so the ladder applies to every seek and seek URLs stay repeatable for nginx's mapping and response caches
- route the seek model, in-range checks, and the stale-report guard through the source window rather than the chunk range
- bridge repositioning seeks (>2s from the last played timestamp) through the preview player and hold the release anchor one commit, so neither path paints a stale frame
- clear a pending loading timer before replacing it; an orphaned timer escaped onPlaying's clearTimeout and flashed loading mid-playback
* keep recordings queries on their indexes
Several recordings queries degraded into full scans or large sorts on big databases: the planner ignored index order, or the query shape gave it nothing tight to seek on. Reshape them into bounded seeks and add the composite index the per-stream lookups need.
- index recordings on (camera, stream_type, start_time DESC) and drop the (camera, stream_type) index it supersedes
- walk the recordings summary day by day with EXISTS probes and per-camera MIN/MAX seeks, skipping ahead over empty gaps instead of bucketing every row for the requested cameras
- run the summary endpoint on the event loop rather than the threadpool
- bound the unavailable-recordings query by start_time per camera and merge the results in Python
- bound the expire query's start_time so it seeks the retention window instead of scanning a camera's whole history
- enumerate deleted cameras with one index seek each rather than a camera NOT IN (...) scan
- compute bandwidth with segment_size filtered in a CASE projection; as a WHERE predicate it baited the planner into the (camera, segment_size) index plus a full sort of the camera's history
- fall back to a 1000-segment window when the recent 100 are all zero-size, so an ingest glitch doesn't report zero bandwidth
- limit the needs_refresh count instead of counting every segment
- cover sub-only and sparse calendar days, midnight-spanning day attribution, multi-camera gap merging, deleted-camera expiry, and zero-size segment runs
* fix mypy
* add ptz controls to camera via wizard when onvif has already been probed
* i18n
* add e2e test
* backend add and remove subscriber
* tweaks
* turn on switch by default if pan and/or tilt capability is available
* fix test
* add sub stream recording with adaptive quality playback
Optionally record a second, lower bitrate stream alongside the main
recording stream via a `record_sub` input role and `record.sub` config block, with its own retention windows.
Recordings rows now carry the stream type plus the media details needed to serve both streams from one manifest: video codec, audio presence, audio codec and rate, and a record-time keyframe index.
Playback resolves coverage across both streams and merges them into a single VOD sequence, falling back to a discontinuity manifest with per-clip init segments when the media signatures differ. The player exposes a quality selector, and an auto governor picks the stream from stall time, bandwidth, codec support, and the save-data hint.
* fix tests and i18n
* Guard object processor queue handlers against unknown cameras
* Skip embeddings post processing for removed cameras
* End review segments for removed cameras
* Drop queued autotracker moves for removed cameras
* Release tracked event thumbnails when skipping a removed camera
* Add locked accessors for camera states
* Read camera states through the processor accessors
* Guard output and recording paths against cameras not yet known
* Resolve camera state once in ONVIF, notification, and transcription paths
* Add combined motion and object Birdseye mode
Add a motion_objects mode that keeps Birdseye active when motion is detected or a confirmed tracked object is present, including stationary objects.
Wire the mode through configuration, runtime commands, API schemas, documentation, and UI labels. Exclude false-positive trackers and add regression coverage for Birdseye activation and MQTT validation.
* Refactor Birdseye activity types as booleans
Replace combination-specific Birdseye modes with composable boolean activity types for motion, active objects, stationary objects, and continuous display.
Preserve legacy single-mode configuration and MQTT inputs, support canonical comma-separated MQTT combinations, and allow scalar YAML values to be replaced by nested settings through the config API.
* Preserve OpenVINO config translations
Regenerate the configuration translations with the OpenVINO detector schema available so the unrelated production detector labels remain intact.
* Preserve partial Birdseye mode overrides
Allow an empty activity selection with a canonical NONE MQTT state so partial camera and profile overrides can disable inherited flags without failing validation.
Add regression coverage for camera and profile inheritance, document the NONE contract, and keep the generated schema fixture scoped to Birdseye.
* Address Birdseye activity review feedback
Move scalar mode compatibility into the 0.18-1 config migration and reject empty activity selections instead of publishing a NONE state.
Pass activity signals through a frozen dataclass, preserve existing active-object tracker behavior, and require confirmed stationary objects. Revert the generic YAML mutation and cover migration, inheritance, MQTT, and activation regressions.
* Move Birdseye migration to 0.19
Use the 0.19-0 configuration revision for converting scalar Birdseye modes to composable activity flags, and update the migration regression coverage accordingly.
* Remove Birdseye migration test
Drop the dedicated config migration test as requested during review while retaining the 0.19-0 migration implementation.
* Docs: fix Synaptics default model path in object detector docs
The model is installed at /synaptics/mobilenet.synap by docker/synaptics/Dockerfile,
matching the config examples in the same section.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Docs: warn about kernel Oops with v4l2m2m hwaccel on ASUS GT-BE19000AI
Enabling the recommended h264_v4l2m2m hwaccel args on the GT-BE19000AI AI
board (SL1680, firmware kernel 5.15.140) triggers a LIST_POISON dereference
Oops in the vendor vpu driver during decoder teardown, requiring a reboot.
Observed and captured on real hardware. Also fixes a Synaptics typo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Docs: drop GT-BE19000AI hwaccel warning per review
Remove the device-specific v4l2m2m kernel Oops warning as requested by
maintainer review; upstream/vendor gotchas are not documented here. The
Synaptics typo fix and model path correction remain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Currently translated at 100.0% (46 of 46 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (62 of 62 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (54 of 54 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (800 of 800 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (474 of 474 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (141 of 141 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (185 of 185 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (1295 of 1295 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (145 of 145 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (108 of 108 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (50 of 50 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (10 of 10 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (240 of 240 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (46 of 46 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (54 of 54 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (800 of 800 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (474 of 474 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (141 of 141 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (185 of 185 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (1295 of 1295 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (100 of 100 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (60 of 60 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (145 of 145 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (67 of 67 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (129 of 129 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (108 of 108 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (500 of 500 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (46 of 46 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (54 of 54 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (800 of 800 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (1295 of 1295 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (49 of 49 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (100 of 100 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (145 of 145 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (67 of 67 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (10 of 10 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (26 of 26 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (108 of 108 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (240 of 240 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (240 of 240 strings)
Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 100.0% (500 of 500 strings)
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Co-authored-by: windasd <me@windasd.tw>
Co-authored-by: 蘭蘭露 <flandrescarlettw@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/audio/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/common/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-auth/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-camera/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-dialog/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-player/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-cameras/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-global/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/objects/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-chat/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-classificationmodel/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-configeditor/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-events/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-explore/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-facelibrary/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-live/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-motionsearch/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-replay/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-search/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-settings/zh_Hant/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-system/zh_Hant/
Translation: Frigate NVR/Config - Cameras
Translation: Frigate NVR/Config - Global
Translation: Frigate NVR/audio
Translation: Frigate NVR/common
Translation: Frigate NVR/components-auth
Translation: Frigate NVR/components-camera
Translation: Frigate NVR/components-dialog
Translation: Frigate NVR/components-player
Translation: Frigate NVR/objects
Translation: Frigate NVR/views-chat
Translation: Frigate NVR/views-classificationmodel
Translation: Frigate NVR/views-configeditor
Translation: Frigate NVR/views-events
Translation: Frigate NVR/views-explore
Translation: Frigate NVR/views-facelibrary
Translation: Frigate NVR/views-live
Translation: Frigate NVR/views-motionSearch
Translation: Frigate NVR/views-replay
Translation: Frigate NVR/views-search
Translation: Frigate NVR/views-settings
Translation: Frigate NVR/views-system
* fix jumping timeline handles in debug replay range selection
The timeline mirrored the export range into local state that was never reset when the range cleared, so starting a second selection after cancelling one left the stale mirror disagreeing with the new range. The sync effect and the draggable position effect then rewrote each other's values every commit until React hit its update limit. The handles now write the range directly.
* add mobile test
* fix incorrect backchannel docs
`#backchannel=0` is a native rtsp source param, not an ffmpeg one, and the troubleshooting warning had it listed as ffmpeg-only. Any `#` modifier on a bare `rtsp://` source turns the backchannel off unless the URL explicitly contains `#backchannel=1`, so a dedicated two-way talk stream can't carry `#video=h264` or anything else.
* add rotation faq
description:Visible on the System Metrics page in the Web UI. Please include the full version including the build identifier (eg. 0.18.0-beta1, 0.18.0-8b72c7a, etc.)
description:Visible on the System Metrics page in the Web UI. Please include the full version including the build identifier (eg. 0.19.0-beta1, 0.19.0-8b72c7a, etc.)
Use this form to submit a reproducible bug in Frigate or Frigate's UI.
Use this form to submit a reproducible bug in Frigate or Frigate's UI.
**⚠️ If you are running a beta version (0.18.0-beta or similar), please use the [Beta Support template](https://github.com/blakeblackshear/frigate/discussions/new?category=beta-support) instead.**
If you are running on Proxmox, please see the [Proxmox FAQ](https://github.com/blakeblackshear/frigate/discussions/23916) and reproduce the issue on a standard Docker install first (bare metal, or a VM running plain Debian/Ubuntu) before submitting here.
**⚠️ If you are running a beta version (0.19.0-beta or similar), please use the [Beta Support template](https://github.com/blakeblackshear/frigate/discussions/new?category=beta-support) instead.**
Before submitting your bug report, please ask the AI with the "Ask AI" button on the [official documentation site][ai] about your issue, [search the discussions][discussions], look at recent open and closed [pull requests][prs], read the [official Frigate documentation][docs], and read the [Frigate FAQ][faq] pinned at the Discussion page to see if your bug has already been fixed by the developers or reported by the community.
Before submitting your bug report, please ask the AI with the "Ask AI" button on the [official documentation site][ai] about your issue, [search the discussions][discussions], look at recent open and closed [pull requests][prs], read the [official Frigate documentation][docs], and read the [Frigate FAQ][faq] pinned at the Discussion page to see if your bug has already been fixed by the developers or reported by the community.
download:If no custom model path or URL is provided, the Hailo detector automatically downloads the default model (YOLOv6n) from the Hailo Model Zoo on first startup based on the detected hardware. Once cached under `/config/model_cache/hailo`, the model works fully offline.
download:If no custom model path or URL is provided, the Hailo detector automatically downloads the default model (YOLOv6n) from the Hailo Model Zoo on first startup, choosing the build that matches the attached device. Once cached under `/config/model_cache`, the model works fully offline.
ui:|-
ui:|-
Navigate to **Settings > System > Detection models** and select **Hailo** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure the model settings:
Navigate to **Settings > System > Detection models** and select **Hailo** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure the model settings:
@@ -60,7 +60,7 @@ hailo8l:
yaml:|-
yaml:|-
models:
models:
- devices:
- devices:
- hailo8l:PCIe
- hailo:PCIe
width: 320
width: 320
height: 320
height: 320
input_tensor: nhwc
input_tensor: nhwc
@@ -101,7 +101,7 @@ hailo8l:
yaml:|-
yaml:|-
models:
models:
- devices:
- devices:
- hailo8l:PCIe
- hailo:PCIe
width: 300
width: 300
height: 300
height: 300
input_tensor: nhwc
input_tensor: nhwc
@@ -824,24 +824,6 @@ cpu:
models:
models:
- devices:
- devices:
- cpu:3
- cpu:3
deepstack:
title:DeepStack / CodeProject.AI
models:
- key:yolo
label:YOLO
recommended:true
download:This detector runs object detection over the network against a CodeProject.AI or DeepStack server, so no model is downloaded into Frigate itself. Visit the [CodeProject.AI official website](https://www.codeproject.com/Articles/5322557/CodeProject-AI-Server-AI-the-easy-way) to download and install the AI server on your preferred device (e.g. Raspberry Pi, Nvidia Jetson, or other compatible hardware) before configuring the detector.
ui:|-
Navigate to **Settings > System > Detection models** and add a model. The CodeProject.AI server is not reported by the hardware probe, so set `devices` to `deepstack:http://<your_codeproject_ai_server_ip>:<port>/v1/vision/detection` in YAML.
# ├── ssdlite_mobilenet.dfp (a file ending with .dfp)
# ├── ssdlite_mobilenet.dfp (a file ending with .dfp)
# └── ssdlite_mobilenet_post.onnx (optional; only if the model includes a cropped post-processing network)
# └── ssdlite_mobilenet_post.onnx (optional; only if the model includes a cropped post-processing network)
deepx:
title:DEEPX NPU
models:
- key:yolo
label:YOLO
recommended:true
download:Nomodel is bundled with Frigate. Download a pre-compiled YOLO `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo) or compile your own with DX-COM, then bind-mount it into the container and point the model's `path` at it. The recommended model is `yolox-s_640x640_ppu.dxnn`. Its Post-Processing Unit (PPU) compile moves candidate selection onto the NPU, which makes it the fastest ModelZoo model measured through Frigate (about 13 ms on a DX-M1). The output layout is read from the compiled model, so anchor-based, anchor-free, NMS-in-head and PPU models (anchor-based or anchor-free) all need no extra configuration; prefer a `PPU` variant whenever the ModelZoo offers one. PPU models must be compiled with DX-COM 2.4.0 or later, which writes the head layout Frigate reads into the file.
ui:|-
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
download:Nomodel is bundled with Frigate. Download a pre-compiled YOLOX `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo), then bind-mount it into the container and point the model's `path` at it. The `_ppu` variant is faster and also works with the `yolo-generic` model type; the plain export needs `yolox` so its raw head is decoded.
ui:|-
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
download:A synap model is provided in the container at `/mobilenet.synap` and is used by this detector type by default. The model comes from the [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
download:A synap model is provided in the container at `/synaptics/mobilenet.synap` and is used by this detector type by default. The model comes from the [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
ui:|-
ui:|-
Navigate to **Settings > System > Detection models** and select **Synaptics NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
Navigate to **Settings > System > Detection models** and select **Synaptics NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
import AnalyticsFields from "@site/src/components/AnalyticsFields";
import NavPath from "@site/src/components/NavPath";
Frigate can send one anonymous usage report a day. The reports show the maintainers which hardware to support, which features people use, and how releases perform. Sharing is off until you turn it on.
## Turning it on
Enable **Share anonymous analytics** at <NavPath path="Settings > System > Telemetry" />, or set it in your config:
```yaml
telemetry:
analytics:true
```
The same page has a **Preview the report** button that shows exactly what the next report contains.
## How it's sent
- Once a day, as a JSON POST to `https://analytics.frigate.video/report`
- The server looks up your country and region from your IP address and never stores the address
- Raw reports are kept for 60 days; only aggregate totals are published
- A random install ID, stored in `/config/.analytics.json`, keeps your install from being counted twice. Turning sharing off deletes it
## What's never sent
- Camera, zone, group, profile, or user names
- Object labels, face names, or license plate text
- IP addresses, hostnames, URLs, or stream paths
- Credentials or API keys
- Events, recordings, or anything from them
## Every field
Fields marked public appear in the published totals. The machine-readable schema is [frigate-analytics-schema.json](pathname:///frigate-analytics-schema.json).
@@ -204,7 +204,7 @@ Frequently-heard labels like `speech` can generate a lot of events, and each eve
### Audio Transcription
### Audio Transcription
Frigate supports fully local audio transcription using either `sherpa-onnx` or OpenAI's open-source Whisper models via `faster-whisper`. The goal of this feature is to support Semantic Search for `speech` audio events. Frigate is not intended to act as a continuous, fully-automatic speech transcription service. Automatically transcribing all speech (or queuing many audio events for transcription) requires substantial CPU (or GPU) resources and is impractical on most systems. For this reason, transcriptions for events are initiated manually from the UI or the API rather than being run continuously in the background.
Frigate supports fully local audio transcription using either `sherpa-onnx` or OpenAI's open-source Whisper models via `faster-whisper`, and can alternatively offload transcription to a [GenAI provider](#genai-provider). The goal of this feature is to support Semantic Search for `speech` audio events. Frigate is not intended to act as a continuous, fully-automatic speech transcription service. Automatically transcribing all speech (or queuing many audio events for transcription) requires substantial CPU (or GPU) resources and is impractical on most systems. For this reason, transcriptions for events are initiated manually from the UI or the API rather than being run continuously in the background.
:::info
:::info
@@ -224,6 +224,7 @@ To enable transcription, configure it globally and optionally disable for specif
- Set **Audio transcription model or GenAI provider name** to `whisper` for Frigate's built-in local models, or to the name of a GenAI provider
- Set **Transcription device** to the desired device
- Set **Transcription device** to the desired device
- Set **Model size** to the desired size
- Set **Model size** to the desired size
@@ -235,6 +236,7 @@ To enable transcription, configure it globally and optionally disable for specif
```yaml
```yaml
audio_transcription:
audio_transcription:
enabled:True
enabled:True
model:whisper
device:...
device:...
model_size:...
model_size:...
```
```
@@ -263,20 +265,88 @@ The optional config parameters that can be set at the global level include:
- **`enabled`**: Enable or disable the audio transcription feature.
- **`enabled`**: Enable or disable the audio transcription feature.
- Default: `False`
- Default: `False`
- It is recommended to only configure the features at the global level, and enable it at the individual camera level.
- It is recommended to only configure the features at the global level, and enable it at the individual camera level.
- **`model`**: The transcription backend.
- Default: `whisper`
-`whisper` uses Frigate's built-in local models, described by `device` and `model_size` below.
- Any other value must name a key in your `genai` config whose entry has `transcribe` in its `roles`. See [GenAI Provider](#genai-provider).
- **`device`**: Device to use to run transcription and translation models.
- **`device`**: Device to use to run transcription and translation models.
- Default: `CPU`
- Default: `CPU`
- This can be `CPU` or `GPU`. The `sherpa-onnx` models are lightweight and run on the CPU only. The `whisper` models can run on GPU but are only supported on CUDA hardware.
- This can be `CPU` or `GPU`. The `sherpa-onnx` models are lightweight and run on the CPU only. The `whisper` models can run on GPU but are only supported on CUDA hardware.
- Ignored when `model` names a GenAI provider.
- **`model_size`**: The size of the model used for live transcription.
- **`model_size`**: The size of the model used for live transcription.
- Default: `small`
- Default: `small`
- This can be `small` or `large`. The `small` setting uses `sherpa-onnx` models that are fast, lightweight, and always run on the CPU but are not as accurate as the `whisper` model.
- This can be `small` or `large`. The `small` setting uses `sherpa-onnx` models that are fast, lightweight, and always run on the CPU but are not as accurate as the `whisper` model.
- This config option applies to **live transcription only**. Recorded `speech` events will always use a different `whisper` model (and can be accelerated for CUDA hardware if available with `device: GPU`).
- This config option applies to **live transcription only**. With `model: whisper`, recorded `speech` events always use a different `whisper` model (and can be accelerated for CUDA hardware if available with `device: GPU`).
- **`language`**: Defines the language used by `whisper` to translate `speech` audio events (and live audio only if using the `large` model).
- Ignored when `model` names a GenAI provider.
- Default: `en`
- **`language`**: Defines the language used to transcribe and translate `speech` audio events (and live audio only if using the `large` model or a GenAI provider).
-You must use a valid [language code](https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10).
-Default: `auto`
-`auto` lets the model detect the language itself, which most models do well. Set an explicit language only if detection is picking the wrong one.
- Otherwise you must use a valid [language code](https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10).
- Transcriptions for `speech` events are translated.
- Transcriptions for `speech` events are translated.
- Live audio is translated only if you are using the `large` model. The `small``sherpa-onnx` model is English-only.
- Live audio is translated only if you are using the `large` model. The `small``sherpa-onnx` model is English-only.
The only field that is valid at the camera level is `enabled`.
The only field that is valid at the camera level is `enabled`. In particular `model` is global only: the transcription backend is a process-wide resource shared by every camera.
#### GenAI Provider
Frigate can send audio to a GenAI provider for transcription when that provider has the `transcribe` role. This is useful if you already run a GenAI provider, or if you do not have the CPU/GPU headroom for a local whisper model. Supported providers are **OpenAI**, **Azure OpenAI**, **Gemini**, and **llama.cpp** with an audio-capable model (a dedicated ASR model such as Qwen3-ASR, or a general multimodal model that accepts audio). Ollama is not supported as it has no audio input.
To use a GenAI provider for audio transcription:
1. Configure a GenAI provider with `transcribe` in its `roles`.
2. Set the audio transcription model to that GenAI config key (e.g. `whisper_cloud`).
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Enrichments > Audio transcription" />.
| **Audio transcription model or GenAI provider name** | Set to the GenAI config key (e.g. `whisper_cloud`) to use a configured GenAI provider for transcription |
The GenAI provider must also be configured with the `transcribe` role under <NavPath path="Settings > Enrichments > Generative AI" />.
</TabItem>
<TabItem value="yaml">
```yaml
genai:
whisper_cloud:
provider:openai
api_key:your-api-key
model:gpt-transcribe
roles:
- transcribe
audio_transcription:
enabled:True
model:whisper_cloud
language:en
```
</TabItem>
</ConfigTabs>
:::warning
**Give `transcribe` its own `genai` entry.** A `genai` entry has a single `model` string that is shared by every role it holds, so `roles: [descriptions, transcribe]` would send the same model name to both the chat endpoint and the transcription endpoint. Transcription models and chat models are almost never the same model, so define a dedicated entry as shown above.
:::
:::warning
**Live transcription against a metered provider is billed continuously.** In live mode Frigate uploads an overlapping ~2 second window of audio roughly once per second, per camera, for as long as audio stays above that camera's `audio.min_volume`. Windows below that threshold are never uploaded, which is what keeps a quiet camera near zero requests, but a camera pointed at a busy street will keep sending.
Three things keep this opt-in: `transcribe` is not one of the default roles, live transcription is off by default, and the volume gate suppresses silence. Transcription of recorded `speech` events is unaffected - it remains a manual, one-request-per-event action.
:::
`device` and `model_size` have no effect on this path and no local model is ever downloaded.
`language` defaults to `auto`, which sends no language hint and lets the model detect it. Most audio models detect language well, so leave it on `auto` unless detection is picking the wrong one.
When set explicitly, it is sent as the transcription endpoint's native `language` parameter for OpenAI, Azure, and llama.cpp, and as part of the prompt for Gemini. This matters for dedicated ASR models such as Qwen3-ASR: they read the prompt as contextual biasing rather than as an instruction, so a language named in the prompt is ignored, while the endpoint parameter is honored.
#### Live transcription
#### Live transcription
@@ -292,6 +362,8 @@ Results can be error-prone due to a number of factors, including:
For speech sources close to the camera with minimal background noise, use the `small` model.
For speech sources close to the camera with minimal background noise, use the `small` model.
A [GenAI provider](#genai-provider) is generally the most accurate option for live transcription, at the cost of a network round trip per window. That round trip has to stay under about a second to keep up with the audio; if it does not, Frigate drops the oldest buffered audio rather than letting the backlog grow.
If you have CUDA hardware, you can experiment with the `large``whisper` model on GPU. Performance is not quite as fast as the `sherpa-onnx``small` model, but live transcription is far more accurate. Using the `large` model with CPU will likely be too slow for real-time transcription.
If you have CUDA hardware, you can experiment with the `large``whisper` model on GPU. Performance is not quite as fast as the `sherpa-onnx``small` model, but live transcription is far more accurate. Using the `large` model with CPU will likely be too slow for real-time transcription.
#### Transcription and translation of `speech` audio events
#### Transcription and translation of `speech` audio events
@@ -308,7 +380,7 @@ Only one `speech` event may be transcribed at a time. Frigate does not automatic
:::
:::
Recorded `speech` events will always use a `whisper` model, regardless of the `model_size` config setting. Without a supported Nvidia GPU, generating transcriptions for longer `speech` events may take a fair amount of time, so be patient.
With `model: whisper`, recorded `speech` events always use a `whisper` model, regardless of the `model_size` config setting. Without a supported Nvidia GPU, generating transcriptions for longer `speech` events may take a fair amount of time, so be patient. With a [GenAI provider](#genai-provider), the recorded clip is sent to the provider instead and no local model is used.
The examples on this page all use `my_provider`, but the name is arbitrary and is only used to reference the provider elsewhere in the config (for example, `semantic_search.model`).
The examples on this page all use `my_provider`, but the name is arbitrary and is only used to reference the provider elsewhere in the config (for example, `semantic_search.model`).
Each provider handles one or more **roles**: `chat`, `descriptions`, and`embeddings`. A provider handles all three by default, and each role may be assigned to exactly one provider. Define a single provider if you want it to do everything, or split the roles across several providers using the `roles` option.
Each provider handles one or more **roles**: `chat`, `descriptions`, `embeddings`, and `transcribe`. A provider handles the first three by default; `transcribe` must always be listed explicitly, and is not available on Ollama, which has no audio input. Each role may be assigned to exactly one provider. Define a single provider if you want it to do everything, or split the roles across several providers using the `roles` option.
If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
@@ -63,11 +63,11 @@ Running Generative AI models on CPU is not recommended, as high inference times
You must use a vision-capable model with Frigate. The following models are recommended for local deployment of the `descriptions` and `chat` roles:
You must use a vision-capable model with Frigate. The following models are recommended for local deployment of the `descriptions` and `chat` roles:
| Model | Notes |
| Model | Review [frame mode](/configuration/genai/genai_review#frame-mode) | Notes |
| `qwen3-vl` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. |
| `qwen3-vl` |`frames` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. Follows a sequence of frames on its own. |
| `qwen3.6`/`qwen3.8` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `qwen3.6`/`qwen3.8` |`frames` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. |
| `gemma4` |`annotated_frames` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. Loses track of activity that repeats or reverses, so it benefits from annotated frames. |
#### Embedding models
#### Embedding models
@@ -77,6 +77,17 @@ The `embeddings` role needs a different kind of model. Text queries are matched
| `qwen3-vl-embedding` | Multimodal embeddings for [Semantic Search](/configuration/semantic_search#genai-provider). Must be served by llama.cpp started with `--embeddings` and `--mmproj`. |
| `qwen3-vl-embedding` | Multimodal embeddings for [Semantic Search](/configuration/semantic_search#genai-provider). Must be served by llama.cpp started with `--embeddings` and `--mmproj`. |
#### Transcription models
The `transcribe` role needs a model that accepts audio input. A text-only or vision-only model cannot serve this role. The following are recommended for local deployment of the `transcribe` role:
| `qwen3-asr` | Dedicated speech recognition model covering 30 languages, and the better choice for transcription quality. It only transcribes, so it cannot be shared with the `descriptions` or `chat` roles. |
| `gemma4` | General multimodal model that accepts audio as well as images, so one served model can cover `transcribe` alongside the other roles. Transcript quality is below `qwen3-asr`, particularly on noisy audio. |
Both must be served by llama.cpp started with the matching audio `--mmproj`. llama.cpp only reports audio support when an audio projector is loaded. Without it Frigate sees the model as text-only and the `transcribe` role is unavailable in the UI. Frigate transcribes through the server's `/v1/audio/transcriptions` route, which llama.cpp serves for any audio-capable model.
:::info
:::info
Each model is available in multiple parameter sizes (3b, 4b, 8b, etc.). Larger sizes are more capable of complex tasks and understanding of situations, but requires more memory and computational resources. It is recommended to try multiple models and experiment to see which performs best.
Each model is available in multiple parameter sizes (3b, 4b, 8b, etc.). Larger sizes are more capable of complex tasks and understanding of situations, but requires more memory and computational resources. It is recommended to try multiple models and experiment to see which performs best.
Review items are sent to the model as a sequence of still frames. Some models follow that sequence well on their own; others lose track of activity that repeats or reverses, and describe a single trip when the subject actually made several. The `frame_mode` option controls how those frames are presented.
- `frames` (default): the prompt followed by the frames, exactly as earlier versions of Frigate sent them.
- `annotated_frames`: each frame is preceded by its frame number and elapsed time, along with notes describing what the object tracker recorded at that moment, such as an object being first detected, starting to move, turning around, stopping, or no longer being detected.
The notes come from tracking data rather than from the images, so they describe activity the model may not have picked up on its own. In testing with a person carrying three waste bins to the curb one at a time, `gemma4` described a single trip on every attempt with `frames`, and consistently described multiple trips with `annotated_frames`. Models that already handle these sequences well, such as the `qwen3-vl` family, gain little and should stay on `frames`.
Annotated mode also caps the number of frames, since the notes already establish the order of events and extra near-duplicate frames tend to crowd out the middle of a clip. Longer review items are sampled more sparsely as a result, and typically use fewer tokens than `frames` mode for the same item.
:::note
Annotated mode needs tracking data for the review item. If none is available, Frigate falls back to sending plain frames for that item.
:::
<ConfigTabs>
<TabItem value="ui">
Navigate to <NavPath path="Settings > Global configuration > Review" />.
- Set **GenAI config > Frame mode** to the desired mode (e.g., `annotated_frames`)
</TabItem>
<TabItem value="yaml">
```yaml {4}
review:
genai:
enabled: true
frame_mode: annotated_frames
```
</TabItem>
</ConfigTabs>
### Response Style
### Response Style
Different models respond to the built-in prompt with very different writing styles: some produce natural narration while others sound short and mechanical. The `response_style` option selects a writing style preset that rewords the prompt's instructions for the user-facing fields (the title, short summary, and scene description). Presets replace those instructions rather than adding extra ones, so the model never receives competing style directions.
Different models respond to the built-in prompt with very different writing styles: some produce natural narration while others sound short and mechanical. The `response_style` option selects a writing style preset that rewords the prompt's instructions for the user-facing fields (the title, short summary, and scene description). Presets replace those instructions rather than adding extra ones, so the model never receives competing style directions.
@@ -28,6 +28,24 @@ WebRTC may use an external STUN server for NAT traversal. MSE and HLS streaming
:::
:::
### Selecting a streaming technology
Frigate [defaults to MSE](#why-does-frigate-prefer-mse-over-webrtc-for-live-view) for restreamed cameras by design. To use WebRTC, select it explicitly from a camera's single-camera Live view settings (the settings menu in the camera's Live view header on desktop, or the settings drawer on mobile). Three related controls work together:
- **Stream**: _what_ to play. This lists the [streams you've configured](#setting-streams-for-live-ui) (for example `Main Stream` and `Sub Stream`).
- **Force low-bandwidth mode**: a switch that always plays Frigate's built-in low-bandwidth feed (the stream assigned the `detect` role, using JSMpeg) instead of the selected stream. It works anywhere without go2rtc and is useful on slow or metered connections. While it is enabled, the stream and streaming technology selectors are disabled; your stream and technology choices are restored when you turn it off.
- **Streaming Technology**: _how_ to play the selected stream, listing **MSE** and **WebRTC**. It is only shown for a restreamed stream.
- The choices are saved **per device, per camera** in your browser's local storage.
- **WebRTC is only selectable when it can actually work for that stream.** When it can't, the option is shown disabled with the reason inline, and a more detailed reason (the failing codecs, or why the connectivity check failed) is logged to your browser's console. Common reasons:
- **Not configured**: no `candidates` or `ice_servers` are set under `go2rtc.webrtc` (see [WebRTC extra configuration](#webrtc-extra-configuration)).
- **Could not connect**: e.g. port `8555` isn't reachable, or a STUN/TURN server is misconfigured. Frigate runs a one-time WebRTC connectivity check when the Live view opens; the option may briefly show as "checking" while it runs.
- **Unsupported video codec**: the stream's video codec can't be played over WebRTC in your browser, most commonly H.265/HEVC in Firefox or Edge.
- **Unsupported audio codec**: WebRTC needs opus or G.711 audio, so a stream whose playback audio is only AAC (without an added opus/G.711 track) can't carry audio over WebRTC. See [Audio Support](#audio-support) for how to add one.
- **Unsupported browser**: the browser doesn't support WebRTC.
When WebRTC isn't available, Frigate automatically uses MSE (or falls back to JSMpeg), so live view keeps working regardless of the selection.
### Camera Settings Recommendations
### Camera Settings Recommendations
If you are using go2rtc, you should adjust the following settings in your camera's firmware for the best experience with Live view:
If you are using go2rtc, you should adjust the following settings in your camera's firmware for the best experience with Live view:
@@ -157,6 +175,17 @@ WebRTC works by creating a TCP or UDP connection on port `8555`. However, it req
- stun:8555
- stun:8555
```
```
- The web UI uses the STUN and TURN servers in `ice_servers` and falls back to Google's public STUN server when none are set:
```yaml title="config.yml"
go2rtc:
webrtc:
ice_servers:
- urls: [turn:turn.example.com:3478]
username: frigate
credential: password
```
- For access through Tailscale, the Frigate system's Tailscale IP must be added as a WebRTC candidate. Tailscale IPs all start with `100.`, and are reserved within the `100.64.0.0/10` CIDR block.
- For access through Tailscale, the Frigate system's Tailscale IP must be added as a WebRTC candidate. Tailscale IPs all start with `100.`, and are reserved within the `100.64.0.0/10` CIDR block.
- Note that some browsers may not support H.265 (HEVC). You can check your browser's current version for H.265 compatibility [here](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness).
- Note that some browsers may not support H.265 (HEVC). You can check your browser's current version for H.265 compatibility [here](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness).
@@ -206,6 +235,8 @@ For devices that support two way talk, Frigate can be configured to use the feat
- Ensure you access Frigate via https (may require [opening port 8971](/frigate/installation/#ports)).
- Ensure you access Frigate via https (may require [opening port 8971](/frigate/installation/#ports)).
- For the Home Assistant Frigate card, [follow the docs](http://card.camera/#/usage/2-way-audio) for the correct source.
- For the Home Assistant Frigate card, [follow the docs](http://card.camera/#/usage/2-way-audio) for the correct source.
The two-way talk control in the single-camera Live view is only enabled when WebRTC is available; if WebRTC isn't configured or can't connect, the control is shown disabled.
To use the Reolink Doorbell with two way talk, you should use the [recommended Reolink configuration](/configuration/camera_specific#reolink-cameras)
To use the Reolink Doorbell with two way talk, you should use the [recommended Reolink configuration](/configuration/camera_specific#reolink-cameras)
As a starting point to check compatibility for your camera, view the list of cameras supported for two-way talk on the [go2rtc repository](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#two-way-audio). For cameras in the category `ONVIF Profile T`, you can use the [ONVIF Conformant Products Database](https://www.onvif.org/conformant-products/)'s FeatureList to check for the presence of `AudioOutput`. A camera that supports `ONVIF Profile T` _usually_ supports this, but due to inconsistent support, a camera that explicitly lists this feature may still not work. If no entry for your camera exists on the database, it is recommended not to buy it or to consult with the manufacturer's support on the feature availability.
As a starting point to check compatibility for your camera, view the list of cameras supported for two-way talk on the [go2rtc repository](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#two-way-audio). For cameras in the category `ONVIF Profile T`, you can use the [ONVIF Conformant Products Database](https://www.onvif.org/conformant-products/)'s FeatureList to check for the presence of `AudioOutput`. A camera that supports `ONVIF Profile T` _usually_ supports this, but due to inconsistent support, a camera that explicitly lists this feature may still not work. If no entry for your camera exists on the database, it is recommended not to buy it or to consult with the manufacturer's support on the feature availability.
@@ -313,8 +313,16 @@ To remove root from the container entirely, add Docker's `user:`:
```yaml
```yaml
user:"1000:1000"# NOT compatible with PUID/PGID, see the run modes table
user:"1000:1000"# NOT compatible with PUID/PGID, see the run modes table
tmpfs:
- /tmp:size=256m
- /tmp/cache:size=1000000000
- /run:exec,nosuid,nodev,mode=0755,uid=1000,gid=1000,size=16m# uid must match user:
```
```
`/run` has to be owned by that uid as well. s6 writes its runtime state there before anything else starts, and with no root in the container a root-owned `/run` stops it during init with `cannot create /run/test of writability`. Keep `uid` and `gid` in the tmpfs options matching `user:`, and don't carry that pair back into the default mode, where a root-owned `/run` is what keeps the unprivileged services out of s6's runtime state.
This only bites once root is genuinely gone. s6's init helper is setuid, so `user:` on its own still lets init regain root and correct `/run` itself. The `no-new-privileges:true` above is what blocks that, which is also what makes the `/run` ownership mandatory. Dropping it would hide the problem by handing init root again.
Two things change, and the first one will break a working install if you skip it. The startup device grants can't run, because there is no root left to run them, so every device you pass stops working until you grant that uid access yourself with `group_add:` or a udev rule; see [Manual setup](#manual-setup). Expect this to surface as a driver error rather than a permission error, like `No VA display found` from VAAPI. And every service then runs as that one uid, so go2rtc no longer gets its own restricted user. `/config` and `/media/frigate` have to be owned by that uid already, since Frigate never adjusts ownership in this mode. Switching an existing install over also leaves `/config/go2rtc_homekit.yml` owned by the go2rtc user, which this mode can't write; `chown` it to your uid or HomeKit pairing changes stop persisting. Frigate warns and starts either way.
Two things change, and the first one will break a working install if you skip it. The startup device grants can't run, because there is no root left to run them, so every device you pass stops working until you grant that uid access yourself with `group_add:` or a udev rule; see [Manual setup](#manual-setup). Expect this to surface as a driver error rather than a permission error, like `No VA display found` from VAAPI. And every service then runs as that one uid, so go2rtc no longer gets its own restricted user. `/config` and `/media/frigate` have to be owned by that uid already, since Frigate never adjusts ownership in this mode. Switching an existing install over also leaves `/config/go2rtc_homekit.yml` owned by the go2rtc user, which this mode can't write; `chown` it to your uid or HomeKit pairing changes stop persisting. Frigate warns and starts either way.
This mode can also take `cap_drop: [ALL]`, which the default mode cannot: starting as root needs `CAP_CHOWN` for the ownership sweep, `CAP_SETUID` and `CAP_SETGID` to drop to the runtime user, and `CAP_FOWNER` for the device grants.
This mode can also take `cap_drop: [ALL]`, which the default mode cannot: starting as root needs `CAP_CHOWN` for the ownership sweep, `CAP_SETUID` and `CAP_SETGID` to drop to the runtime user, and `CAP_FOWNER` for the device grants.
@@ -22,8 +22,9 @@ Frigate supports multiple different detectors that work on different types of ha
**Most Hardware**
**Most Hardware**
- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB, Mini PCIe, and m.2 formats allowing for a wide range of compatibility with devices.
- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB, Mini PCIe, and m.2 formats allowing for a wide range of compatibility with devices.
- [Hailo](#hailo-8): The Hailo8 and Hailo8L AI Acceleration module is available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
- [Hailo](#hailo): The Hailo-8, Hailo-8L and Hailo-8R AI Acceleration modules are available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
- <CommunityBadge /> [MemryX](#memryx-mx3): The MX3 Acceleration module is available in m.2 format, offering broad compatibility across various platforms.
- <CommunityBadge /> [MemryX](#memryx-mx3): The MX3 Acceleration module is available in m.2 format, offering broad compatibility across various platforms.
- <CommunityBadge /> [DEEPX](#deepx-npu): The DEEPX NPU is available in m.2 format and as a HAT+ for the Raspberry Pi 5, offering broad compatibility across various platforms.
**AMD**
**AMD**
@@ -285,9 +286,9 @@ models:
---
---
## Hailo-8
## Hailo
This detector is available for use with both Hailo-8 and Hailo-8L AI Acceleration Modules. The integration automatically detects your hardware architecture via the Hailo CLI and selects the appropriate default model if no custom model is specified.
This detector is available for use with the Hailo-8, Hailo-8L and Hailo-8R AI Acceleration Modules. The integration identifies which of them is attached and selects the matching default model if no custom model is specified.
See the [installation docs](../frigate/installation.md#hailo-8) for information on configuring the Hailo hardware.
See the [installation docs](../frigate/installation.md#hailo-8) for information on configuring the Hailo hardware.
@@ -308,11 +309,11 @@ The HailoRT runtime is not part of the Frigate image. It is downloaded and insta
When configuring the Hailo detector, you have two options to specify the model: a local **path** or a **URL**.
When configuring the Hailo detector, you have two options to specify the model: a local **path** or a **URL**.
If both are provided, the detector will first check for the model at the given local path. If the file is not found, it will download the model from the specified URL. The model file is cached under `/config/model_cache/hailo`.
If both are provided, the detector will first check for the model at the given local path. If the file is not found, it will download the model from the specified URL. The model file is cached under `/config/model_cache/hailo`.
For additional ready-to-use models, please visit: https://github.com/hailo-ai/hailo_model_zoo
For additional ready-to-use models, please visit: https://github.com/hailo-ai/hailo_model_zoo
Hailo8 supports all models in the Hailo Model Zoo that include HailoRT post-processing. You're welcome to choose any of these pre-configured models for your implementation.
Hailo supports all models in the Hailo Model Zoo that include HailoRT post-processing. You're welcome to choose any of these pre-configured models for your implementation.
> **Note:**
> **Note:**
> The config.path parameter can accept either a local file path or a URL ending with .hef. When provided, the detector will first check if the path is a local file path. If the file exists locally, it will use it directly. If the file is not found locally or if a URL was provided, it will attempt to download the model from the specified URL.
> The config.path parameter can accept either a local file path or a URL ending with .hef. When provided, the detector will first check if the path is a local file path. If the file exists locally, it will use it directly. If the file is not found locally or if a URL was provided, it will attempt to download the model from the specified URL.
@@ -362,7 +363,7 @@ Intel NPUs cannot be used under Home Assistant OS, which does not include the NP
:::warning
:::warning
The network-based detectors (Deepstack and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, and the Apple Silicon client ignores`request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
The Apple Silicon detector client is being reworked. Its extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now, and`request_timeout_ms` and `linger_ms` are ignored. Anything else is dropped when your config is migrated.
:::
:::
@@ -540,30 +541,6 @@ A TensorFlow Lite model is provided in the container at `/cpu_model.tflite` and
When using CPU detectors, you can add one CPU detector per camera. Adding more detectors than the number of cameras should not improve performance.
When using CPU detectors, you can add one CPU detector per camera. Adding more detectors than the number of cameras should not improve performance.
## Deepstack / CodeProject.AI Server Detector
:::warning
The network-based detectors (Deepstack and the Apple Silicon client) are being reworked. Their extra options no longer have a place in the config, so only the endpoint carried in the device string is honored right now: Deepstack ignores `api_key` and `api_timeout`, and the Apple Silicon client ignores `request_timeout_ms` and `linger_ms`. Anything else is dropped when your config is migrated.
:::
The Deepstack / CodeProject.AI Server detector for Frigate allows you to integrate Deepstack and CodeProject.AI object detection capabilities into Frigate. CodeProject.AI and DeepStack are open-source AI platforms that can be run on various devices such as the Raspberry Pi, Nvidia Jetson, and other compatible hardware. It is important to note that the integration is performed over the network, so the inference times may not be as fast as native Frigate detectors, but it still provides an efficient and reliable solution for object detection and tracking.
### Setup {#setup-deepstack}
To get started with CodeProject.AI, visit their [official website](https://www.codeproject.com/Articles/5322557/CodeProject-AI-Server-AI-the-easy-way) to follow the instructions to download and install the AI server on your preferred device. Detailed setup instructions for CodeProject.AI are outside the scope of the Frigate documentation.
To integrate CodeProject.AI into Frigate, configure the detector as follows:
Replace `<your_codeproject_ai_server_ip>` and `<port>` with the IP address and port of your CodeProject.AI server.
To verify that the integration is working correctly, start Frigate and observe the logs for any error messages related to CodeProject.AI. Additionally, you can check the Frigate web interface to see if the objects detected by CodeProject.AI are being displayed and tracked properly.
# Community Supported Detectors
# Community Supported Detectors
## MemryX MX3
## MemryX MX3
@@ -636,6 +613,63 @@ For detailed instructions on compiling models, refer to the [MemryX Compiler](ht
---
---
## DEEPX NPU
This detector is available for use with the DEEPX NPU, both the DX-M1 M.2 module and the DX-M1M on the Sixfab AI HAT+ for the Raspberry Pi 5. The configuration below applies unchanged to either form factor. DEEPX NPU support in Frigate is developed and maintained by [Sixfab](https://sixfab.com).
See the [installation docs](../frigate/installation.md#deepx-npu) for information on installing the DEEPX kernel driver and runtime on the host and passing the NPU through to the container.
To run a model on a DEEPX NPU, list a `deepx` device on that model.
:::info
The DX-RT Python bindings are not part of the Frigate image. They are downloaded and installed into `/config/.local` the first time a DEEPX device is configured, verified against pinned checksums, and updated automatically when a Frigate release pins a new version. If the container has no internet access, see [Detector runtimes](/frigate/network_requirements#detector-runtimes) for how to provide the files yourself.
Frigate does not bundle a model for this detector. Models must be compiled to DEEPX's `.dxnn` format. Two model types are supported:
- `yolo-generic` for YOLO object detection models, the recommended default. The detector reads the model's output layout from the compiled file, so anchor-based, anchor-free and NMS-in-head models all work with the same configuration, as do models compiled with DEEPX's Post-Processing Unit (PPU) support.
- `yolox` for YOLOX models compiled without PPU support, whose raw head needs Frigate's YOLOX decoder. A YOLOX model compiled with PPU support works under either `yolox` or `yolo-generic`.
The quickest way to get one is the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo), which publishes pre-compiled `.dxnn` files for a range of YOLO object detection models. Download the `.dxnn`, bind-mount it into the container, and point the model's `path` at it. Alternatively, compile your own model with the DX-COM compiler. The recommended starting point is `yolox-s_640x640_ppu.dxnn`, the fastest ModelZoo model measured through Frigate:
For PPU models, use a `.dxnn` compiled with DX-COM 2.4.0 or later. Frigate reads the PPU head layout the compiler writes into the file and refuses to load a PPU model without it.
`model_type` must be set to `yolo-generic` or `yolox` to match the model; `yolo-generic` is the recommended default unless the model is a raw YOLOX export. Frigate defaults it to `ssd`, which this detector does not support, so the detector refuses to start on a model that leaves it unset.
`width` and `height` must match the resolution the model was compiled for. Quantization parameters are baked into the `.dxnn` file at compile time, so no normalization is applied on the host and Frigate's default `input_tensor`, `input_pixel_format`, and `input_dtype` values do not need to be overridden.
A DEEPX device is `PCIe:<index>`, as reported on the detector settings page. The NPU daemon multiplexes across processes, so the same device may be listed more than once to run additional inference processes against it:
```yaml
models:
- devices:
- deepx:PCIe:0
- deepx:PCIe:0
```
#### Label maps
The object detection models in the DEEPX ModelZoo are trained on the standard 80-class COCO label set, so `labelmap_path` must be set to `/labelmap/coco-80.txt`. Frigate's default label map uses an extended 91-class COCO scheme, and leaving it in place will cause detections to be reported as the wrong object type. For `yolo-generic` models the label map is also what the detector uses to tell the output layout, so a label map with the wrong number of classes is reported as an error at startup.
---
## NVidia TensorRT Detector
## NVidia TensorRT Detector
Nvidia Jetson devices may be used for object detection using the TensorRT libraries. Due to the size of the additional libraries, this detector is only provided in images with the `-tensorrt-jp6` tag suffix, e.g. `ghcr.io/blakeblackshear/frigate:stable-tensorrt-jp6`. This detector is designed to work with Yolo models for object detection.
Nvidia Jetson devices may be used for object detection using the TensorRT libraries. Due to the size of the additional libraries, this detector is only provided in images with the `-tensorrt-jp6` tag suffix, e.g. `ghcr.io/blakeblackshear/frigate:stable-tensorrt-jp6`. This detector is designed to work with Yolo models for object detection.
@@ -450,7 +450,7 @@ For advanced use cases, the [custom export HTTP API](../integrations/api/export-
POST /export/custom/{camera_name}/start/{start_time}/end/{end_time}
POST /export/custom/{camera_name}/start/{start_time}/end/{end_time}
```
```
The request body accepts `ffmpeg_input_args` and `ffmpeg_output_args` to control encoding, frame rate, filters, and other FFmpeg options. If neither is provided, Frigate defaults to time-lapse output settings (25x speed, 30 FPS).
The request body accepts `ffmpeg_input_args` and `ffmpeg_output_args` to control encoding, frame rate, filters, and other FFmpeg options. If neither is provided, Frigate defaults to time-lapse output settings (25x speed, 30 FPS) with audio removed (`-an`). When providing your own `ffmpeg_input_args`, include `-an` if you want audio stripped from the export.
The following example exports a time-lapse at 60x speed with 25 FPS:
The following example exports a time-lapse at 60x speed with 25 FPS:
@@ -197,7 +197,7 @@ For cameras that support two-way talk, go2rtc will automatically establish an au
To prevent this, you must configure two separate stream instances:
To prevent this, you must configure two separate stream instances:
1. One stream instance with `#backchannel=0` for Frigate's viewing, recording, and detection (prevents go2rtc from establishing the blocking backchannel)
1. One stream instance with `#backchannel=0` for Frigate's viewing, recording, and detection (prevents go2rtc from establishing the blocking backchannel)
2. A second stream instance without `#backchannel=0` for two-way talk functionality (can be used by Frigate's WebRTC viewer or other applications)
2. A second stream instance with no `#` parameters at all for two-way talk functionality (can be used by Frigate's WebRTC viewer or other applications)
Configuration example:
Configuration example:
@@ -215,6 +215,8 @@ In this configuration:
- `front_door` stream is used by Frigate for viewing, recording, and detection. The `#backchannel=0` parameter prevents go2rtc from establishing the audio output backchannel, so it won't block two-way talk access.
- `front_door` stream is used by Frigate for viewing, recording, and detection. The `#backchannel=0` parameter prevents go2rtc from establishing the audio output backchannel, so it won't block two-way talk access.
- `front_door_twoway` stream is used for two-way talk functionality. This stream can be used by Frigate's WebRTC viewer when two-way talk is enabled, or by other applications (like Home Assistant Advanced Camera Card) that need access to the camera's audio output channel.
- `front_door_twoway` stream is used for two-way talk functionality. This stream can be used by Frigate's WebRTC viewer when two-way talk is enabled, or by other applications (like Home Assistant Advanced Camera Card) that need access to the camera's audio output channel.
Any `#` parameter on a bare `rtsp://` source disables the backchannel unless the URL explicitly contains `#backchannel=1`. A two-way talk stream with something like `#video=h264` on it silently loses two-way audio, and Frigate will report that two-way talk is unavailable for that stream.
## Security: Restricted Stream Sources
## Security: Restricted Stream Sources
For security reasons, the `echo:`, `expr:`, and `exec:` stream sources are disabled by default in go2rtc. These sources allow arbitrary command execution and can pose security risks if misconfigured.
For security reasons, the `echo:`, `expr:`, and `exec:` stream sources are disabled by default in go2rtc. These sources allow arbitrary command execution and can pose security risks if misconfigured.
@@ -204,11 +204,20 @@ Light guidelines and advice:
npm run lint
npm run lint
```
```
-Add to unit tests and ensure they pass. As much as possible, you should strive to _increase_ test coverage whenever making changes. This will help ensure features do not accidentally become broken in the future.
-Ensure the backend [unit tests](#unit-tests) pass. Your PR cannot be merged unless tests pass.
- If you run into error messages like "TypeError: Cannot read properties of undefined (reading 'context')" when running tests, this may be due to these issues (https://github.com/vitest-dev/vitest/issues/1910, https://github.com/vitest-dev/vitest/issues/1652) in vitest, but I haven't been able to resolve them.
```shell
python3 -u -m unittest
```
- Ensure the end-to-end tests pass. They run in Playwright against a production build with mocked API data, so they don't need a running Frigate instance. Add or update tests in `web/e2e/specs/` when you change UI behavior.
```console
```console
npm run test
# First-time setup
npx playwright install chromium
# Build the app and run all tests
npm run e2e:build && npm run e2e
```
```
- Test in different browsers. Firefox, Chrome, and Safari all have different quirks that make them unique targets to interact with.
- Test in different browsers. Firefox, Chrome, and Safari all have different quirks that make them unique targets to interact with.
@@ -54,7 +54,7 @@ Frigate supports multiple different detectors that work on different types of ha
**Most Hardware**
**Most Hardware**
- [Hailo](#hailo-8): The Hailo8 and Hailo8L AI Acceleration module is available in m.2 format with a HAT for RPi devices offering a wide range of compatibility with devices.
- [Hailo](#hailo-8): The Hailo-8, Hailo-8L and Hailo-8R AI Acceleration modules are available in m.2 format with a HAT for RPi devices offering a wide range of compatibility with devices.
- [Supports many model architectures](../../configuration/object_detectors#configuration-hailo)
- [Supports many model architectures](../../configuration/object_detectors#configuration-hailo)
- Runs best with tiny or small size models
- Runs best with tiny or small size models
@@ -65,6 +65,11 @@ Frigate supports multiple different detectors that work on different types of ha
- [Supports many model architectures](../../configuration/object_detectors#memryx-mx3)
- [Supports many model architectures](../../configuration/object_detectors#memryx-mx3)
- Runs best with tiny, small, or medium-size models
- Runs best with tiny, small, or medium-size models
- <CommunityBadge /> [DEEPX](#deepx-npu): The DEEPX NPU is available in m.2 format and as a HAT+ for the Raspberry Pi 5, allowing for a wide range of compatibility with devices.
- [Supports YOLO model architectures](../../configuration/object_detectors#deepx-npu)
- Runs best with tiny or small size models
- Runs efficiently on low power hardware
**AMD**
**AMD**
- [ROCm](#rocm---amd-gpu): ROCm can run on AMD Discrete GPUs to provide efficient object detection
- [ROCm](#rocm---amd-gpu): ROCm can run on AMD Discrete GPUs to provide efficient object detection
@@ -111,12 +116,13 @@ Frigate supports multiple different detectors that work on different types of ha
### Hailo-8
### Hailo-8
Frigate supports both the Hailo-8 and Hailo-8L AI Acceleration Modules on compatible hardware platforms, including the Raspberry Pi 5 with the PCIe hat from the AI kit. The Hailo detector integration in Frigate automatically identifies your hardware type and selects the appropriate default model when a custom model isn’t provided.
Frigate supports the Hailo-8, Hailo-8L and Hailo-8R AI Acceleration Modules on compatible hardware platforms, including the Raspberry Pi 5 with the PCIe hat from the AI kit. The Hailo detector integration in Frigate identifies which of them is attached and selects the matching default model when a custom model isn’t provided.
**Default Model Configuration:**
**Default Model Configuration:**
- **Hailo-8L:** Default model is **YOLOv6n**.
- **Hailo-8L:** Default model is **YOLOv6n**, compiled for the Hailo-8L.
- **Hailo-8:** Default model is **YOLOv6n**.
- **Hailo-8:** Default model is **YOLOv6n**, compiled for the Hailo-8.
- **Hailo-8R:** Default model is the **Hailo-8** build of **YOLOv6n**, since the Hailo Model Zoo publishes no Hailo-8R build.
In real-world deployments, even with multiple cameras running concurrently, Frigate has demonstrated consistent performance. Testing on x86 platforms, with dual PCIe lanes, yields further improvements in FPS, throughput, and latency compared to the Raspberry Pi setup.
In real-world deployments, even with multiple cameras running concurrently, Frigate has demonstrated consistent performance. Testing on x86 platforms, with dual PCIe lanes, yields further improvements in FPS, throughput, and latency compared to the Raspberry Pi setup.
@@ -256,6 +262,32 @@ The MX3 is a pipelined architecture, where the maximum frames per second support
Inference speeds may vary depending on the host platform. The above data was measured on an **Intel 13700 CPU**. Platforms like Raspberry Pi, Orange Pi, and other ARM-based SBCs have different levels of processing capability, which may limit total FPS.
Inference speeds may vary depending on the host platform. The above data was measured on an **Intel 13700 CPU**. Platforms like Raspberry Pi, Orange Pi, and other ARM-based SBCs have different levels of processing capability, which may limit total FPS.
### DEEPX NPU
Frigate supports the DEEPX NPU in both of its form factors: the **DX-M1** M.2 module, which works on x86 (Intel/AMD) and ARM-based SBCs such as the Raspberry Pi 5, and the **DX-M1M** on the [Sixfab AI HAT+](https://docs.sixfab.com/docs/ai-hat-plus-raspberry-pi-5-quickstart) for the Raspberry Pi 5. Both use the same driver and runtime, so the configuration is identical for either one. DEEPX NPU support in Frigate is developed and maintained by [Sixfab](https://sixfab.com).
The DEEPX driver and runtime run on the Docker host rather than inside the Frigate container and must be installed before the NPU can be used. See the [installation docs](installation.md#deepx-npu) for the setup steps and [the detector docs](/configuration/object_detectors#deepx-npu) for the configuration.
Frigate does not bundle a model for this detector. Models use DEEPX's `.dxnn` format, and pre-compiled YOLO models can be downloaded from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo). Prefer a model with a `_ppu` suffix whenever one is available for the architecture you want: these run part of the post-processing on the NPU itself and are considerably faster, roughly 2.5x for the same architecture and input size. **YOLOX-S with PPU is the recommended starting point.**
Inference times for a few recommended models, measured through Frigate's own stats on a DX-M1:
Other ModelZoo YOLO variants are also supported but have not been measured. Inference speeds vary with the host platform, so a slower host such as a Raspberry Pi 5 will report higher times than those above.
:::note
A few ModelZoo models can not be used with Frigate: SSD models (they are trained on Pascal VOC, so their labels do not match Frigate's), DAMO-YOLO models, face and pose models, and the PPU builds of YOLOv7.
:::
### Nvidia Jetson
### Nvidia Jetson
Jetson devices are supported via the TensorRT or ONNX detectors when running Jetpack 6. It will [make use of the Jetson's hardware media engine](/configuration/hardware_acceleration_video#nvidia-jetson) when configured with the [appropriate presets](/configuration/ffmpeg_presets#hwaccel-presets), and will make use of the Jetson's GPU and DLA for object detection when configured with the [TensorRT detector](/configuration/object_detectors#nvidia-tensorrt-detector).
Jetson devices are supported via the TensorRT or ONNX detectors when running Jetpack 6. It will [make use of the Jetson's hardware media engine](/configuration/hardware_acceleration_video#nvidia-jetson) when configured with the [appropriate presets](/configuration/ffmpeg_presets#hwaccel-presets), and will make use of the Jetson's GPU and DLA for object detection when configured with the [TensorRT detector](/configuration/object_detectors#nvidia-tensorrt-detector).
@@ -122,7 +122,7 @@ Additionally, the USB Coral draws a considerable amount of power. If using any o
### Hailo-8
### Hailo-8
The Hailo-8 and Hailo-8L AI accelerators are available in both M.2 and HAT form factors for the Raspberry Pi. The M.2 version typically connects to a carrier board for PCIe, which then interfaces with the Raspberry Pi 5 as part of the AI Kit. The HAT version can be mounted directly onto compatible Raspberry Pi models. Both form factors have been successfully tested on x86 platforms as well, making them versatile options for various computing environments.
The Hailo-8, Hailo-8L and Hailo-8R AI accelerators are available in both M.2 and HAT form factors for the Raspberry Pi. The M.2 version typically connects to a carrier board for PCIe, which then interfaces with the Raspberry Pi 5 as part of the AI Kit. The HAT version can be mounted directly onto compatible Raspberry Pi models. Both form factors have been successfully tested on x86 platforms as well, making them versatile options for various computing environments.
The HailoRT runtime is not part of the Frigate image; Frigate downloads and installs it at first start once a Hailo detector is configured. Containers without internet access can provide the files themselves, see [Detector runtimes](/frigate/network_requirements#detector-runtimes).
The HailoRT runtime is not part of the Frigate image; Frigate downloads and installs it at first start once a Hailo detector is configured. Containers without internet access can provide the files themselves, see [Detector runtimes](/frigate/network_requirements#detector-runtimes).
@@ -300,7 +300,7 @@ If you are using `docker run`, add this option to your command `--device /dev/ha
#### Configuration
#### Configuration
Finally, configure [hardware object detection](/configuration/object_detectors#hailo-8) to complete the setup.
Finally, configure [hardware object detection](/configuration/object_detectors#hailo) to complete the setup.
### MemryX MX3
### MemryX MX3
@@ -381,6 +381,99 @@ If you can't use Docker Compose, you can run the container with something simila
Finally, configure [hardware object detection](/configuration/object_detectors#memryx-mx3) to complete the setup.
Finally, configure [hardware object detection](/configuration/object_detectors#memryx-mx3) to complete the setup.
### DEEPX NPU
The DEEPX NPU is available in two form factors, and Frigate supports both:
- **DX-M1** in the M.2 2280 form factor (like an NVMe SSD), for x86 (Intel/AMD) PCs, the Raspberry Pi 5, and other ARM SBCs with an exposed PCIe M.2 slot.
- **DX-M1M** on the [Sixfab AI HAT+](https://docs.sixfab.com/docs/ai-hat-plus-raspberry-pi-5-quickstart), a HAT+ board that connects to the Raspberry Pi 5 over PCIe Gen 3 x1.
Both present the NPU through the same PCIe driver and DX-RT runtime, so the setup below and the detector configuration are identical for either one. Nothing needs to change when moving between them.
DEEPX NPU support in Frigate is developed and maintained by [Sixfab](https://sixfab.com).
#### Versions
A DEEPX install has several separately versioned pieces, and they all have to agree. The driver, the runtime, and the daemon live on the Docker host; Frigate itself carries only the Python bindings, which it downloads on first start:
| Component | Version | Installed on | Installed by |
| NPU firmware | `v2.7.4` | The module | Flashed from the host |
| DX-RT bindings | `v3.4.0` | Frigate | Downloaded at first start |
:::warning
A version mismatch does not produce a startup error. It typically shows up as inference requests that are accepted but never return a result, so detections simply stop appearing while Frigate looks healthy. If that happens after a Frigate upgrade, check every version in the table before anything else.
:::
The installation script installs the DX-RT runtime on the host and enables `dxrt.service`, so the daemon starts at boot and any other program on the host can share the NPU with Frigate. Check the firmware version with `dxrt-cli --status` and update the module if it does not match the table above.
#### Installation
The DEEPX kernel driver must be installed on the host rather than in the container, because containers share the host kernel and cannot load kernel modules. Installing it creates the `/dev/dxrt*` device nodes that are passed through to Frigate. The same script installs the DX-RT runtime and enables `dxrt.service`, the daemon that owns the NPU and hands work to it on behalf of Frigate and anything else on the host.
1. Copy or download [this script](https://github.com/blakeblackshear/frigate/blob/dev/docker/deepx/user_installation.sh).
2. Ensure it has execution permissions with `sudo chmod +x user_installation.sh`
3. Run the script with `./user_installation.sh`
4. **Restart your computer** to complete driver installation.
Confirm the NPU is visible before continuing:
```bash
ls /dev/dxrt*
```
Then confirm the daemon is running and listening in `/run/dxrt`:
```bash
systemctl is-active dxrt.service
ls /run/dxrt/
```
#### Setup
To set up Frigate, follow the default installation instructions, for example: `ghcr.io/blakeblackshear/frigate:stable`
#### Docker configuration
Frigate needs the NPU device node and the directory holding the daemon's socket:
```yaml
services:
frigate:
devices:
- /dev/dxrt0:/dev/dxrt0
volumes:
- /run/dxrt:/run/dxrt
```
If you can't use Docker Compose, add `--device /dev/dxrt0:/dev/dxrt0 -v /run/dxrt:/run/dxrt` to your `docker run` command.
Add one `--device` per NPU, contiguously from `/dev/dxrt0`, since the client stops enumerating at the first gap.
The installation script configures `dxrt.service` to place its socket in `/run/dxrt` through a systemd drop-in. Mounting the directory rather than the socket file means the container sees the new socket after `dxrt.service` is restarted, rather than holding on to a deleted one.
`dxrtd` listens on an abstract socket as well, but that one does not cross into a container, so Frigate names the filesystem socket through `DXRT_DYNAMIC_IPC_ENDPOINT` on your behalf. Set that variable on the container yourself only if the daemon listens somewhere else, which means you also set it for `dxrtd` through its own systemd drop-in. The script writes `/etc/systemd/system/dxrt.service.d/frigate.conf` for exactly that, and has `dxrt.service` link the socket to `/tmp/dxrt_dynamic_ipc.sock` when it starts, so the host's own `dxrt-cli` and `dxtop` keep finding it at the default path they fall back to.
:::note
The DX-RT client exits when `dxrt.service` stops, so restart the Frigate container after restarting `dxrt.service`.
:::
The device node is needed as well as the socket, because the client opens the NPU directly even though the daemon arbitrates access. Without it, inference fails with `Device not found`.
`/dev/shm` does not need sharing.
The DX-RT python bindings are not shipped in the Frigate image. Frigate downloads them on first start when a DEEPX detector is configured, and caches them under `/config`.
#### Configuration
Finally, configure [hardware object detection](/configuration/object_detectors#deepx-npu) to complete the setup.
### Rockchip platform
### Rockchip platform
Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and necessary drivers (especially rkvdec2 and rknpu). To check, enter the following commands:
Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and necessary drivers (especially rkvdec2 and rknpu). To check, enter the following commands:
| [AXERA AXEngine](/configuration/object_detectors) | Detection model | HuggingFace |
| [AXERA AXEngine](/configuration/object_detectors) | Detection model | HuggingFace |
:::note
:::note
@@ -60,16 +60,16 @@ The default CPU, EdgeTPU, and OpenVINO object detection models are bundled into
The SDKs for a few hardware detectors are not shipped in the Frigate image. They are downloaded the first time that detector is configured, verified against checksums pinned in the Frigate release, and installed into the Frigate user's home directory (`/config/.local` by default). Once installed they are not downloaded again until a Frigate release pins a new version.
The SDKs for a few hardware detectors are not shipped in the Frigate image. They are downloaded the first time that detector is configured, verified against checksums pinned in the Frigate release, and installed into the Frigate user's home directory (`/config/.local` by default). Once installed they are not downloaded again until a Frigate release pins a new version.
If the container cannot reach GitHub, provide the files yourself:
If the container cannot reach GitHub, provide the files yourself:
1. Download the files for your architecture on a machine with internet access.
1. Download the files for your architecture on a machine with internet access.
2. Place them, with exactly the file names listed above, in `/config/model_cache/runtimes/<detector>/`, where `<detector>` is the detector `type` from your config (`hailo8l`, `memryx`, or `axengine`).
2. Place them, with exactly the file names listed above, in `/config/model_cache/runtimes/<detector>/`, where `<detector>` is the detector named in your config's `devices` (`hailo`, `memryx`, or `axengine`).
3. Start Frigate. Files whose checksum matches are installed without any download; a file with the wrong checksum is discarded and downloaded again, so a failed startup log names the file to replace.
3. Start Frigate. Files whose checksum matches are installed without any download; a file with the wrong checksum is discarded and downloaded again, so a failed startup log names the file to replace.
The `GITHUB_ENDPOINT` mirror variable below applies to these downloads as well.
The `GITHUB_ENDPOINT` mirror variable below applies to these downloads as well.
@@ -134,6 +134,15 @@ telemetry:
version_check:false
version_check:false
```
```
### Anonymous Analytics
If [anonymous analytics](/configuration/advanced/analytics) sharing is turned on, Frigate sends one report a day to `https://analytics.frigate.video`. It's off by default, so no outbound connection happens unless you enable it:
```yaml
telemetry:
analytics:true
```
### Push Notifications
### Push Notifications
When [notifications](/configuration/notifications) are enabled and users have registered for push notifications in the web UI, Frigate sends push messages through the browser vendor's push service (e.g., Google FCM, Mozilla autopush). This requires internet access from the Frigate server to these push endpoints.
When [notifications](/configuration/notifications) are enabled and users have registered for push notifications in the web UI, Frigate sends push messages through the browser vendor's push service (e.g., Google FCM, Mozilla autopush). This requires internet access from the Frigate server to these push endpoints.
@@ -142,16 +151,12 @@ When [notifications](/configuration/notifications) are enabled and users have re
If an [MQTT broker](/integrations/mqtt) is configured, Frigate maintains a connection to the broker's host and port. This is typically a local network connection, but will require internet if you use a cloud-hosted MQTT broker.
If an [MQTT broker](/integrations/mqtt) is configured, Frigate maintains a connection to the broker's host and port. This is typically a local network connection, but will require internet if you use a cloud-hosted MQTT broker.
### DeepStack / CodeProject.AI
When using the [DeepStack detector plugin](/configuration/object_detectors), Frigate sends images to the configured API endpoint for inference. This is typically local but depends on where the service is hosted.
## WebRTC (STUN)
## WebRTC (STUN)
For [WebRTC live streaming](/configuration/live), Frigate uses STUN for NAT traversal:
For [WebRTC live streaming](/configuration/live), Frigate uses STUN for NAT traversal:
- **go2rtc** defaults to a local STUN listener (`stun:8555`), no internet required.
- **go2rtc** defaults to a local STUN listener (`stun:8555`), no internet required.
- **The web UI's WebRTC player** includes a fallback to Google's public STUN server (`stun:stun.l.google.com:19302`), which requires internet.
- **The web UI** uses the servers in `go2rtc.webrtc.ice_servers` for its WebRTC player and for the WebRTC connectivity check it runs when the Live view loads. If none are set, it uses Google's public STUN server (`stun:stun.l.google.com:19302`), which requires internet access from the browser. Set `ice_servers` to a STUN or TURN server on your network to avoid this.
## Home Assistant Supervisor
## Home Assistant Supervisor
@@ -174,7 +179,7 @@ To run Frigate in an air-gapped or offline environment:
2.**Pre-download the training base weights**: If you plan to train custom classification models, set `TF_KERAS_MOBILENET_V2_WEIGHTS_URL` before training, then run one training job while online. Without this variable the base weights are cached outside `/config/` and are lost whenever the container is recreated, so a later training run will fail offline. If the machine never has internet access, copy the weights in manually as described below.
2.**Pre-download the training base weights**: If you plan to train custom classification models, set `TF_KERAS_MOBILENET_V2_WEIGHTS_URL` before training, then run one training job while online. Without this variable the base weights are cached outside `/config/` and are lost whenever the container is recreated, so a later training run will fail offline. If the machine never has internet access, copy the weights in manually as described below.
3.**Disable version check**: Set `telemetry.version_check: false` in your configuration.
3.**Disable version check**: Set `telemetry.version_check: false` in your configuration.
4.**Block outbound model requests**: Set the `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1` environment variables to prevent HuggingFace and Transformers from attempting any network requests.
4.**Block outbound model requests**: Set the `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1` environment variables to prevent HuggingFace and Transformers from attempting any network requests.
5.**Avoid cloud features**: Do not configure Frigate+, Generative AI providers that require internet, or cloud MQTT brokers.
5.**Avoid cloud features**: Do not configure Frigate+, Generative AI providers that require internet, or cloud MQTT brokers, and leave anonymous analytics off (its default).
6.**Use local model mirrors**: If limited internet is available, set the `HF_ENDPOINT`, `GITHUB_ENDPOINT`, `GITHUB_RAW_ENDPOINT`, and `TF_KERAS_MOBILENET_V2_WEIGHTS_URL` environment variables to point to local mirrors.
6.**Use local model mirrors**: If limited internet is available, set the `HF_ENDPOINT`, `GITHUB_ENDPOINT`, `GITHUB_RAW_ENDPOINT`, and `TF_KERAS_MOBILENET_V2_WEIGHTS_URL` environment variables to point to local mirrors.
After these steps, Frigate will operate with no outbound internet connections.
After these steps, Frigate will operate with no outbound internet connections.
@@ -41,7 +41,7 @@ Rockchip models are automatically converted as of 0.17. For 0.16, YOLOv9 onnx mo
## Supported detector types
## Supported detector types
Currently, Frigate+ models support CPU (`cpu`), Google Coral (`edgetpu`), OpenVino (`openvino`), ONNX (`onnx`), Hailo (`hailo8l`), and Rockchip (`rknn`) detectors.
Currently, Frigate+ models support CPU (`cpu`), Google Coral (`edgetpu`), OpenVino (`openvino`), ONNX (`onnx`), Hailo (`hailo`), and Rockchip (`rknn`) detectors.
| Hardware | Recommended Detector Type | Recommended Model Type |
| Hardware | Recommended Detector Type | Recommended Model Type |
@@ -39,6 +39,20 @@ To do this efficiently the following setup is required:
When this is done correctly, the GPU will do the decoding and scaling which will result in a small increase in CPU usage but with better results.
When this is done correctly, the GPU will do the decoding and scaling which will result in a small increase in CPU usage but with better results.
### How can I rotate my camera's video feed?
Rotation is best done in the camera's firmware settings (usually called rotate, flip, or corridor mode) so the video arrives already rotated and no extra processing is needed. Check there first.
If your camera does not support rotation, go2rtc's ffmpeg module can rotate the stream with the `#rotate` parameter (`90`, `180`, `270`, or `-90`), but this is not recommended: rotation requires transcoding (re-encoding) the video, which significantly increases CPU usage, especially for high resolution streams.
Point the camera's inputs at the restream as described in the [restream docs](/configuration/restream.md), and swap `detect -> width` and `detect -> height` to match the rotated resolution.
### My mjpeg stream or snapshots look green and crazy
### My mjpeg stream or snapshots look green and crazy
This almost always means that the width/height defined for your camera are not correct. Double check the resolution with VLC or another player. Also make sure you don't have the width and height values backwards.
This almost always means that the width/height defined for your camera are not correct. Double check the resolution with VLC or another player. Also make sure you don't have the width and height values backwards.
@@ -170,6 +184,6 @@ Filters and masks only hide the incorrect result - they don't teach Frigate what
### Where do I see problems Frigate has detected?
### Where do I see problems Frigate has detected?
Open System > Health. The Notices list shows problems the backend has noticed on its own, such as a camera whose ffmpeg keeps crashing, a detector that had to be restarted, a model download that failed, or recordings being deleted before their retention period. Entries that describe a one-time event can be dismissed; entries that describe an ongoing condition clear themselves once it is fixed.
Open System > Health. The Notices list keeps a record of problems Frigate has found, and you can dismiss any entry to acknowledge it. Ongoing conditions, such as an offline camera or recordings deleted before their retention period, appear in the status bar for admins until they clear, and the status bar links to the Notices list while it has undismissed entries. On mobile, tap the warning icon in the bottom navigation bar to see them.
The Hardware section below the notices shows whether the detection hardware, hardware acceleration, and enrichment devices in your config were found and are being used, so a GPU that silently fell back to the CPU shows up as a warning. Run stream checks to probe every camera's streams for the same problems the camera wizard reports.
The Hardware section below the notices shows whether the detection hardware, hardware acceleration, and enrichment devices in your config were found and are being used, so a GPU that silently fell back to the CPU shows up as a warning. Run stream checks to probe every camera's streams for the same problems the camera wizard reports.
The `#`-modifiers (`#video=`, `#audio=`, `#hardware`,`#backchannel=0`, …) **only take effect on a source that is prefixed with `ffmpeg:`**. Adding them to a bare `rtsp://…#audio=opus` source does nothing: go2rtc ignores them. Likewise, when a source references another stream by name (e.g. `ffmpeg:back#audio=aac`), the name must match the stream key **exactly** (it is case sensitive), or the transcode is silently never produced. This is the single most common configuration mistake. In the Frigate UI, the **Use compatibility mode (ffmpeg)** toggle adds the `ffmpeg:` prefix for you.
The transcoding modifiers (`#video=`, `#audio=`, `#hardware`, …) **only take effect on a source that is prefixed with `ffmpeg:`**. Adding them to a bare `rtsp://…#audio=opus` source does nothing: go2rtc ignores them. Likewise, when a source references another stream by name (e.g. `ffmpeg:back#audio=aac`), the name must match the stream key **exactly** (it is case sensitive), or the transcode is silently never produced. This is the single most common configuration mistake. In the Frigate UI, the **Use compatibility mode (ffmpeg)** toggle adds the `ffmpeg:` prefix for you.
A bare `rtsp://` source reads a different set of modifiers: `#backchannel=`, `#media=`, `#timeout=`, and `#transport=`. These do nothing on an `ffmpeg:` source. Adding **any** modifier to a bare `rtsp://` source also disables the camera's backchannel unless the URL explicitly contains `#backchannel=1`, so a stream dedicated to two-way talk should carry no modifiers at all.
:::
:::
@@ -153,7 +155,7 @@ WebRTC is only attempted when MSE fails or when using a camera's two-way talk fe
- **Codec mismatch**: WebRTC cannot carry H.265 or AAC. The stream backing the WebRTC view must provide Opus (or PCMA/PCMU) audio and H.264 video. Add an `ffmpeg:back#audio=opus` source as shown above.
- **Codec mismatch**: WebRTC cannot carry H.265 or AAC. The stream backing the WebRTC view must provide Opus (or PCMA/PCMU) audio and H.264 video. Add an `ffmpeg:back#audio=opus` source as shown above.
- **Port `8555` not reachable, or no candidates set**: WebRTC needs port `8555` (both TCP and UDP) open and a reachable candidate advertised. On Docker installs running on a custom/overlay network, go2rtc may advertise unreachable container IPs as ICE candidates; setting `webrtc.filters.candidates: []` and supplying only your host's LAN IP resolves this. See [WebRTC extra configuration](/configuration/live#webrtc-extra-configuration).
- **Port `8555` not reachable, or no candidates set**: WebRTC needs port `8555` (both TCP and UDP) open and a reachable candidate advertised. On Docker installs running on a custom/overlay network, go2rtc may advertise unreachable container IPs as ICE candidates; setting `webrtc.filters.candidates: []` and supplying only your host's LAN IP resolves this. See [WebRTC extra configuration](/configuration/live#webrtc-extra-configuration).
- **Two-way talk** additionally requires a secure context (HTTPS or the authenticated port `8971`, because browsers block microphone access on plain HTTP). The camera's RTSP backchannel must also be handled correctly: go2rtc seizes the backchannel by default, which blocks two-way audio for other consumers and can inject static. Disable it on the primary stream with `#backchannel=0` and use a separate dedicated stream for talk, as documented in [preventing go2rtc from blocking two-way audio](/configuration/restream#two-way-talk-restream).
- **Two-way talk** additionally requires a secure context (HTTPS or the authenticated port `8971`, because browsers block microphone access on plain HTTP). The camera's RTSP backchannel must also be handled correctly: go2rtc seizes the backchannel by default, which blocks two-way audio for other consumers and can inject static. Disable it on the primary stream with `#backchannel=0` and use a separate dedicated stream for talk, carrying no `#` modifiers of any kind, as documented in [preventing go2rtc from blocking two-way audio](/configuration/restream#two-way-talk-restream).
@@ -21,6 +21,17 @@ class GenAIRoleEnum(str, Enum):
chat="chat"
chat="chat"
descriptions="descriptions"
descriptions="descriptions"
embeddings="embeddings"
embeddings="embeddings"
transcribe="transcribe"
# Providers that can accept audio input for the transcribe role. Ollama has no
# audio input support, so claiming the role there would fail at request time.
TRANSCRIBE_CAPABLE_PROVIDERS={
GenAIProviderEnum.openai,
GenAIProviderEnum.azure_openai,
GenAIProviderEnum.gemini,
GenAIProviderEnum.llamacpp,
}
classGenAIConfig(FrigateBaseModel):
classGenAIConfig(FrigateBaseModel):
@@ -52,7 +63,7 @@ class GenAIConfig(FrigateBaseModel):
GenAIRoleEnum.chat,
GenAIRoleEnum.chat,
],
],
title="Roles",
title="Roles",
description="GenAI roles (chat, descriptions, embeddings); one provider per role.",
description="GenAI roles (chat, descriptions, embeddings, transcribe); one provider per role. Only chat, descriptions, and embeddings are granted by default; transcribe must be listed explicitly.",
)
)
provider_options:dict[str,Any]=Field(
provider_options:dict[str,Any]=Field(
default={},
default={},
@@ -66,3 +77,17 @@ class GenAIConfig(FrigateBaseModel):
description="Runtime options passed to the provider for each inference call.",
description="Runtime options passed to the provider for each inference call.",
json_schema_extra={"additionalProperties":{}},
json_schema_extra={"additionalProperties":{}},
)
)
@model_validator(mode="after")
defvalidate_transcribe_provider(self)->Self:
"""Reject the transcribe role on providers that cannot accept audio input."""
if(
GenAIRoleEnum.transcribeinself.roles
andself.providernotinTRANSCRIBE_CAPABLE_PROVIDERS
):
raiseValueError(
f"GenAI provider '{self.provider.value}' does not support audio input "
@@ -20,6 +21,13 @@ class ImageSourceEnum(str, Enum):
recordings="recordings"
recordings="recordings"
classReviewFrameModeEnum(str,Enum):
"""How review frames are presented to the GenAI provider."""
frames="frames"
annotated_frames="annotated_frames"
classReviewResponseStyleEnum(str,Enum):
classReviewResponseStyleEnum(str,Enum):
"""Writing style presets for GenAI review descriptions."""
"""Writing style presets for GenAI review descriptions."""
@@ -153,6 +161,11 @@ class GenAIReviewConfig(FrigateBaseModel):
description="Preferred language to request from the GenAI provider for generated responses.",
description="Preferred language to request from the GenAI provider for generated responses.",
default=None,
default=None,
)
)
frame_mode:ReviewFrameModeEnum=Field(
default=ReviewFrameModeEnum.frames,
title="Frame mode",
description="How frames are presented to the model. 'frames' sends the prompt followed by the frames, which suits models that track a sequence well on their own. 'annotated_frames' labels each frame and interleaves notes derived from object tracking, which helps models that lose track of activity that repeats or reverses.",
@@ -5,6 +5,7 @@ from pydantic import ConfigDict, Field, field_validator
from.baseimportFrigateBaseModel
from.baseimportFrigateBaseModel
__all__=[
__all__=[
"AudioTranscriptionModelEnum",
"CameraFaceRecognitionConfig",
"CameraFaceRecognitionConfig",
"CameraLicensePlateRecognitionConfig",
"CameraLicensePlateRecognitionConfig",
"CameraAudioTranscriptionConfig",
"CameraAudioTranscriptionConfig",
@@ -20,6 +21,10 @@ class SemanticSearchModelEnum(str, Enum):
jinav2="jinav2"
jinav2="jinav2"
classAudioTranscriptionModelEnum(str,Enum):
whisper="whisper"
classEnrichmentsDeviceEnum(str,Enum):
classEnrichmentsDeviceEnum(str,Enum):
GPU="GPU"
GPU="GPU"
CPU="CPU"
CPU="CPU"
@@ -53,10 +58,35 @@ class AudioTranscriptionConfig(FrigateBaseModel):
description="Enable or disable automatic audio transcription for all cameras; can be overridden per-camera.",
description="Enable or disable automatic audio transcription for all cameras; can be overridden per-camera.",
)
)
language:str=Field(
language:str=Field(
default="en",
default="auto",
title="Transcription language",
title="Transcription language",
description="Language code used for transcription/translation (for example 'en' for English). See https://whisper-api.com/docs/languages/ for supported language codes.",
description="Language code used for transcription/translation (for example 'en' for English), or 'auto' to let the model detect it. See https://whisper-api.com/docs/languages/ for supported language codes.",
)
)
model:AudioTranscriptionModelEnum|str|None=Field(
default=AudioTranscriptionModelEnum.whisper,
title="Audio transcription model or GenAI provider name",
description="The transcription backend: 'whisper' for Frigate's built-in local models, or the name of a GenAI provider with the transcribe role.",
)
@field_validator("model",mode="before")
@classmethod
defcoerce_model_enum(cls,v):
# An absent value ("model:" with nothing after it, or an explicit null)
# means unspecified, so fall back to the built-in backend. Left as None
# it would pass the GenAI-provider validation, which only inspects
# strings, and then be treated as a provider name that resolves to no
# client, turning transcription into a silent no-op.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Josh Hawkins