* Resize classification training images the same way as inference
Training let Keras resize dataset images to 224x224, which uses nearest
neighbor by default. Inference resizes the crop with cv2.resize
(bilinear). For crops much smaller than 224 px the two results look very
different, so a model could fit its training images perfectly and still
report the same state for every live frame.
Add resize_classification_image() and use it everywhere an image is
resized for a classification model. Before training, the dataset is
copied to a temporary directory through that same function, so Keras is
given images that are already the model input size and does no resizing
of its own.
The new test runs a frame through the state processor and the crop it
saves through the training process (stopped at model.fit), and checks
the model receives the same pixels from both, for random frame and crop
sizes. Adds hypothesis as a dev dependency.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Fail training when a prepared image cannot be written
cv2.imwrite returns False instead of raising, so a full temporary
directory would have dropped images from training without any error.
Raise with the file path instead; the training process already logs
the exception and fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Trim classification parity tests and drop hypothesis
Per review: the parity test is now a plain loop over fixed crop sizes
(64, 79, 224 and 600 px) on a fixed 1280x720 frame, which covers both
upscaled and downscaled crops. The 224x224 check and the failed-write
test are removed; the parity test already covers the prepared image
size. hypothesis is no longer a dev dependency.
The 64, 79 and 600 px cases still fail on dev and pass with the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Victor Bajanov <victorb@spunmonkey.org>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
* bound recording snapshot lookups by segment length
Ordering the recording snapshot and Frigate+ lookups by stream_type ahead of start_time broke index order, so SQLite fetched and sorted every older recording for the camera on each request (about 100 ms on a 30 day camera, run on the event loop). A start_time lower bound of the frame time minus MAX_SEGMENT_DURATION keeps the lookup to a few rows.
* bound recording overlap queries by segment length
Clip, export, review and recording deletion, debug replay, motion search, and audio transcription found overlapping recordings with an OR clause that gave SQLite no start_time range, so every call read the camera's whole recording history (30 to 300 ms on a 30 day database). The classification and review description point lookups did the same whenever the timestamp had no recording. A shared recording_overlap helper now adds a start_time lower bound of MAX_SEGMENT_DURATION before the range, which drops no rows because segments are always shorter than that.
* probe webrtc before reporting it as not configured
The WebRTC gate read only the Frigate config, so an install with no go2rtc candidates showed WebRTC as not configured, even though go2rtc fills in default candidates at startup (including the internal one under the HA app) and WebRTC often worked. Unset candidates now run the connectivity probe first, and only a failed probe reports not configured. The not-configured reason shows as a neutral note that names the setting and says WebRTC is optional, rather than as a red error.
* validate trusted_proxies at config load
Malformed trusted_proxies entries were only parsed per request in get_remote_addr, so a typo logged a warning on every proxied request and the entry was skipped, collapsing all clients behind that proxy into one rate-limit bucket. The config now rejects unparseable entries at startup and get_remote_addr no longer needs the fallback. Host bits are accepted (192.168.1.1/24 reads as 192.168.1.0/24) so a common typo doesn't take down safe mode, which copies the auth section verbatim.
* remove path data from thumbnail payloads
The thumbnail data carried a full copy of the object's path, which went out as snapshot.path_data in every events message alongside the top level path_data. Nothing reads the snapshot copy, and the path has no length limit, so it only made events payloads larger.
* hide incompatible frigate+ models behind a toggle
* keep the probe failure detail in the not-configured console message
* use a unique id for each show incompatible models toggle
* pad the bottom of the desktop timeline so zoom buttons don't cover segments
* create the admin row when auth is disabled so notification registrations persist
Installs that never enabled auth had no admin row, so registering for notifications returned 200 but saved nothing. Frigate now creates an admin row with an empty password hash when auth is disabled, and treats that empty hash as first-time setup when auth is enabled later so a password is still generated and printed. Registration returns 404 when no user row was updated, and the success toast only shows after the request succeeds. The Users settings page hides Add User and Reset Password when auth is disabled and notes that accounts only apply once built-in auth is enabled.
* fix classification wizard finding no sample images on large databases
The wizard grouped tracked objects by camera and 6 hour block, oldest first, then kept the first 100. With more than 100 groups that was always the oldest objects, which often have no snapshot or thumbnail left on disk, so no examples were generated. Shuffle the selection before truncating, and keep extracting from the remaining tracked objects until 100 usable images are found.
* add notice and system ui message when a camera isn't using go2rtc
* add docs note about privacy masks
* match index-less device strings to hardware units in the detection models picker
A config with `edgetpu:usb` was reported as hardware that wasn't found, because the picker compared device strings exactly and the probe reports the unit as `edgetpu:usb:0`. A device without an index now resolves to the first unit of that kind, so the hardware dropdown, the unit checkboxes, and the model summary all recognize it.
* don't treat unknown audio as an audio-bearing stream
A recording row with NULL has_audio (ffprobe failed and the cv2 fallback can't report audio) marked the whole stream as audio-bearing, so every confirmed video-only row on it was dropped as a glitch. A stream now counts as audio-bearing only when a row is known to carry audio.
* stage main+sub exports on disk instead of /tmp/cache
A main+sub export stages a full copy of itself before the final file is written. That copy went to /tmp/cache, so a long export outgrew the tmpfs and failed with no space left on device. Staged runs are now written to the exports directory, and startup removes any left behind by a killed export.
* address review feedback
- stage main+sub export runs in a staging subfolder of the exports directory, so media sync can't delete them mid-export and startup cleanup can't remove a finished export with a matching name or fail on an unremovable file
- run object classification example collection off the event loop
- prefer an exact hardware unit match over an index-less one, so an AMD GPU's "onnx" no longer selects the NVIDIA entry
- add tests for the classification fallback and index-less hardware matching
* serialize example collection and clean up exports when staging dir creation fails
- overlapping object example requests could delete each other's images in the shared temp and train directories, so collection now runs under a lock
- a failed makedirs for the staging directory raised past the failed-export cleanup and left the export spinning until restart, so it now fails the run through the normal cleanup path
* revert
* add sub stream recording with adaptive quality playback
Optionally record a second, lower bitrate stream alongside the main
recording stream via a `record_sub` input role and `record.sub` config block, with its own retention windows.
Recordings rows now carry the stream type plus the media details needed to serve both streams from one manifest: video codec, audio presence, audio codec and rate, and a record-time keyframe index.
Playback resolves coverage across both streams and merges them into a single VOD sequence, falling back to a discontinuity manifest with per-clip init segments when the media signatures differ. The player exposes a quality selector, and an auto governor picks the stream from stall time, bandwidth, codec support, and the save-data hint.
* fix tests and i18n
* Pin ruff
* Add python upgrade fixes
This enables python upgrade checks in ruff to look for deprecated types and patterns. This namely fixes:
- usage of deprecated `Typing` which is now built in
- some specific exceptions which are caught and have new aliases
Some specific UP checks were also ignored as they are stylistic / unimportant and likely to cause bugs
* Remove async blocking calls
Use asyncio.to_thread on two remaining blocking calls to fix hanging event thread loop. Enable this specific rule to block it in the future.
* Use proper logging mechanism
* Correctly format logs
* Raise with context
When raising an exception include the from context to improve debugging
* Cleanup
* add randomness to object classification
also ensure train_dir is fresh if user has regenerated examples
* frontend refresh button
* fix radix dropdown issue
* i18n
* Strip model name before training
* Handle options file for go2rtc option
* Make reviewed optional and add null to API call
* Send reviewed for dashboard
* Allow setting context size for openai compatible endpoints
* push empty go2rtc config to avoid homekit error in log
* Add option to set runtime options for LLM providers
* Docs
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix genai callbacks in MQTT
* Cleanup cursor pointer for classification cards
* Cleanup
* Handle unknown SOCs for RKNN converter by only using known SOCs
* don't allow "none" as a classification class name
* change internal port user to admin and default unspecified username to viewer
* keep 5000 as anonymous user
* suppress tensorflow logging during classification training
* Always apply base log level suppressions for noisy third-party libraries even if no specific logConfig is provided
* remove decorator and specifically suppress TFLite delegate creation messages
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Wait for config to load before evaluating route access
Fix race condition where custom role users are temporarily denied access after login while config is still loading. Defer route rendering in DefaultAppView until config is available so the complete role list is known before ProtectedRoute evaluates permissions
* Use batching for state classification generation
* Ignore incorrect scoring images if they make it through the deletion
* Delete unclassified images
* mitigate tensorflow atexit crash by pre-importing tflite/tensorflow on main thread
Pre-import Interpreter in embeddings maintainer and add defensive lazy imports in classification processors to avoid worker-thread tensorflow imports causing "can't register atexit after shutdown"
* don't require old password for users with admin role when changing passwords
* don't render actions menu if no options are available
* Remove hwaccel arg as it is not used for encoding
* change password button text
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* ensure audio events display timeline entries in tracking details
* tweak tracking details layout for small desktop sizes
* update transcription docs
* Update classification docs for training recommendations
* Make number of classification images to be kept configurable
* Add bird to classification reference
* Fix incorrect averaging of the segments so it correctly only uses the most recent segments
* fix trigger logic
* add ability to download clean snapshot
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Implement extraction of images for classification state models
* Add object classification dataset preparation
* Add first step wizard
* Update i18n
* Add state classification image selection step
* Improve box handling
* Add object selector
* Improve object cropping implementation
* Fix state classification selection
* Finalize training and image selection step
* Cleanup
* Design optimizations
* Cleanup mobile styling
* Update no models screen
* Cleanups and fixes
* Fix bugs
* Improve model training and creation process
* Cleanup
* Dynamically add metrics for new model
* Add loading when hitting continue
* Improve image selection mechanism
* Remove unused translation keys
* Adjust wording
* Add retry button for image generation
* Make no models view more specific
* Adjust plus icon
* Adjust form label
* Start with correct type selected
* Cleanup sizing and more font colors
* Small tweaks
* Add tips and more info
* Cleanup dialog sizing
* Add cursor rule for frontend
* Cleanup
* remove underline
* Lazy loading
* Ui improvements
* Improve image cropping and model saving
* Improve naming
* Add logs for training
* Improve model labeling
* Don't set sub label for none object classification
* Cleanup
* Ignore numpy get limits warning
* Add function wrapper to redirect stdout and stderr to logpipe
* Save stderr too
* Add more to catch
* run logpipe
* Use other logging redirect class
* Use other logging redirect class
* add decorator for redirecting c/c++ level output to logger
* fix typing
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Combine base and arm trt detectors
* Remove unused deps for amd64 build
* Add missing packages and cleanup ldconfig
* Expand packages for tensorflow model training
* Cleanup
* Refactor training to not reserve memory
* Implement model training via ZMQ and add model states to represent training
* Get model updates working
* Improve toasts and model state
* Clean up logging
* Add back in
* Setup basic training structure
* Build out route
* Handle model configs
* Add image fetch APIs
* Implement model training screen with dataset selection
* Implement viewing of training images
* Adjust directories
* Implement viewing of images
* Add support for deleting images
* Implement full deletion
* Implement classification model training
* Improve naming
* More renaming
* Improve layout
* Reduce logging
* Cleanup