Compare commits

...
Author SHA1 Message Date
Josh Hawkins d149de038a cover events/search in sub label filter test 2026-11-12 07:41:00 -06:00
Josh Hawkins a3e1966013 fix sub label filter for non-ascii names
The sub_labels filter lowercased the requested name in Python but the stored value with SQLite's LOWER(), which only folds ASCII. Cyrillic and other non-ASCII names never matched, so filtering by a face name returned no events. Both sides are now lowered in SQLite, which keeps ASCII matching case-insensitive and non-ASCII matching exact.
2026-11-12 07:34:25 -06:00
Josh HawkinsandGitHub 06d1365056 Report ownership scan progress (#24940)
CI / AMD64 Build (push) Waiting to run
CI / AMD64 Smoke Test (push) Blocked by required conditions
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
* report ownership scan progress

The ownership scan printed nothing until it finished, which took 30+ minutes on large recordings volumes and looked stuck. It now logs a running count of checked and mismatched entries every 30 seconds.

* check the scan clock on every entry
2026-11-11 08:52:58 -07:00
Josh HawkinsandGitHub 8a88cefc9a Autotracking tweaks (#24935)
* fix autotracking movement time prediction and recorded move times

Movement time prediction multiplied the fitted intercept by the saved zero-step calibration time instead of 1, so predictions didn't match the regression. In absolute zoom mode, a pan/tilt followed by a zoom recorded the zoom's timestamps against the pan/tilt distance, because _zoom_absolute resets the PTZ start and stop times. The pan/tilt times are now captured before the zoom runs.

* predict velocity and zoom size for objects approaching the autotracked camera

The velocity check required both box corners to move in the same direction, but the corners of an approaching object diverge, so its velocity was rejected and zeroed and the camera aimed where the object had been. That check is removed, and the existing magnitude, delta, and variance checks still reject noise. The zoom size prediction fit area against epoch time with no intercept, which returned roughly the mean area instead of a forecast. It now scales the target box by the growth the corner velocities predict over the movement time.

* keep predicted target box positive and add regression tests

A shrinking object's predicted size could reach zero, so the relative zoom ratio divided by zero and sent a NaN zoom to the camera. The predicted size is now floored at 1 px. Tests also cover the movement time intercept and the pan/tilt timestamps that absolute zoom overwrites.
2026-11-11 08:38:21 -07:00
Josh HawkinsandGitHub fba967c74b fix annotation offset seeking in tracking details (#24936)
CI / AMD64 Build (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
The VOD manifest used to find the keyframe-snapped clip start was built from the live annotation offset while the playing clip used the offset from when the event opened. Each offset step fetched a manifest for a different clip, so the seek mapping drifted: the video jumped forward about one GOP every second of offset and flickered while each new manifest loaded. The manifest now uses the same stable source offset as the playlist
2026-11-10 07:57:21 -06:00
7 changed files with 227 additions and 82 deletions
+14 -1
View File
@@ -105,6 +105,8 @@ if [[ -n "$sentinel" ]]; then
mismatch_expr+=(-a ! -path "$sentinel")
fi
exec 3>&1
for path in "$@"; do
# An absent root is an incomplete sweep, not a finished one: /media/frigate
# is not in the image, so a boot before the volume is mounted would
@@ -121,7 +123,18 @@ for path in "$@"; do
# stale mount. Tolerate it rather than aborting under errexit, but never
# read a failed scan as "nothing to do": that would record the sweep as
# complete without having looked.
if ! count=$(find "$path" -name lost+found -prune -o "${mismatch_expr[@]}" -printf '.' 2>/dev/null | wc -c); then
# The walk prints nothing until it ends, so report every 30s through fd 3
# (stdout) while the count itself comes back through the substitution.
if ! count=$(find "$path" -name lost+found -prune -o "(" "${mismatch_expr[@]}" ")" -printf 'm\n' -o -printf '.\n' 2>/dev/null \
| awk -v path="$path" '
BEGIN { next_report = systime() + 30 }
$0 == "m" { mismatched++ }
systime() >= next_report {
printf "[INFO] fix-ownership: still scanning %s, %d entries checked, %d need changes\n", path, NR, mismatched > "/dev/fd/3"
fflush("/dev/fd/3")
next_report = systime() + 30
}
END { print mismatched + 0 }'); then
swept_clean=0
echo "[WARN] fix-ownership: could not scan ${path}; will retry on next boot"
continue
+2 -1
View File
@@ -71,13 +71,14 @@ Both the script and the boot sweep report progress, so you can tell a slow sweep
```
[INFO] fix-ownership: scanning /media/frigate for ownership mismatches; this may take a while on large filesystems
[INFO] fix-ownership: still scanning /media/frigate, 2481000 entries checked, 2390112 need changes
[WARN] fix-ownership: adjusting ownership of 4823941 entries under /media/frigate
[INFO] fix-ownership: /media/frigate 5% (241197/4823941 entries)
[INFO] fix-ownership: /media/frigate 10% (482394/4823941 entries)
[INFO] fix-ownership: finished /media/frigate in 12m 4s
```
The scan has no percentage because the total isn't known until it finishes. Watch the boot sweep with `docker logs -f frigate`.
The scan has no percentage because the total isn't known until it finishes, so it reports a running count every 30 seconds instead. Watch the boot sweep with `docker logs -f frigate`.
Once the volumes are aligned, start Frigate normally. A file at `/config/.permissions_version` records what was done, so later boots skip the sweep unless you change `PUID`/`PGID`.
+14 -22
View File
@@ -199,18 +199,14 @@ def events(
sub_label_clauses.append(Event.sub_label.is_null())
for label in filtered_sub_labels:
lowered = label.lower()
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) == lowered
) # include exact matches (case-insensitive)
# SQLite LOWER() only folds ASCII, so lower both sides in SQL to
# keep non-ASCII names matching exactly
stored = fn.LOWER(Event.sub_label.cast("text"))
sub_label_clauses.append(stored == fn.LOWER(label))
# include this label when part of a list (LIKE is case-insensitive in sqlite for ASCII)
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) % f"*{lowered},*"
)
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) % f"*, {lowered}*"
)
# include this label when part of a list
sub_label_clauses.append(stored % fn.LOWER(f"*{label},*"))
sub_label_clauses.append(stored % fn.LOWER(f"*, {label}*"))
sub_label_clause = reduce(operator.or_, sub_label_clauses)
clauses.append(sub_label_clause)
@@ -633,18 +629,14 @@ def events_search(
sub_label_clauses.append(Event.sub_label.is_null())
for label in filtered_sub_labels:
lowered = label.lower()
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) == lowered
) # include exact matches (case-insensitive)
# SQLite LOWER() only folds ASCII, so lower both sides in SQL to
# keep non-ASCII names matching exactly
stored = fn.LOWER(Event.sub_label.cast("text"))
sub_label_clauses.append(stored == fn.LOWER(label))
# include this label when part of a list (LIKE is case-insensitive in sqlite for ASCII)
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) % f"*{lowered},*"
)
sub_label_clauses.append(
fn.LOWER(Event.sub_label.cast("text")) % f"*, {lowered}*"
)
# include this label when part of a list
sub_label_clauses.append(stored % fn.LOWER(f"*{label},*"))
sub_label_clauses.append(stored % fn.LOWER(f"*, {label}*"))
event_filters.append(reduce(operator.or_, sub_label_clauses))
+40 -51
View File
@@ -585,26 +585,36 @@ class PtzAutoTracker(threading.Thread):
def _predict_movement_time(self, camera, pan, tilt):
combined_movement = abs(pan) + abs(tilt)
input_data = np.array([self.intercept[camera], combined_movement])
input_data = np.array([1, combined_movement])
return np.dot(self.move_coefficients[camera], input_data)
def _predict_area_after_time(self, camera, time):
return np.dot(
self.tracked_object_metrics[camera]["area_coefficients"],
[self.tracked_object_history[camera][-1]["frame_time"] + time],
)
def _predict_target_box(self, camera, predicted_time):
target_box = self.tracked_object_metrics[camera]["target_box"]
tom = self.tracked_object_metrics[camera]
target_box = tom["target_box"]
if not predicted_time:
return target_box
frame_shape = self.config.cameras[camera].frame_shape
camera_config = self.config.cameras[camera]
left, top, right, bottom = self.tracked_object_history[camera][-1]["box"]
size = max(right - left, bottom - top)
return target_box + self._predict_area_after_time(camera, predicted_time) / (
frame_shape[0] * frame_shape[1]
if size <= 0:
return target_box
# the corner velocities diverge as the object approaches the camera
frames = camera_config.detect.fps * predicted_time
velocity = tom["velocity"]
predicted_size = max(
right - left + frames * (velocity[2] - velocity[0]),
bottom - top + frames * (velocity[3] - velocity[1]),
1,
)
# target_box is the squared size raised to the zoom factor
return target_box * (predicted_size / size) ** (
2 * camera_config.onvif.autotracking.zoom_factor
)
def _calculate_tracked_object_metrics(self, camera, obj):
@@ -660,26 +670,6 @@ class PtzAutoTracker(threading.Thread):
filtered_areas = remove_outliers(areas) if len(areas) > 3 else areas
# Filter entries that are not touching the frame edge
filtered_areas_not_touching_edge = [
entry
for entry in filtered_areas
if self._touching_frame_edges(camera, entry["box"]) == 0
]
# Calculate regression for area change predictions
if len(filtered_areas_not_touching_edge):
X = np.array(
[item["frame_time"] for item in filtered_areas_not_touching_edge]
)
y = np.array([item["area"] for item in filtered_areas_not_touching_edge])
tom["area_coefficients"] = np.linalg.lstsq(X.reshape(-1, 1), y, rcond=None)[
0
]
else:
tom["area_coefficients"] = np.array([0])
weights = np.arange(1, len(filtered_areas) + 1)
weighted_area = np.average(
[item["area"] for item in filtered_areas], weights=weights
@@ -744,6 +734,8 @@ class PtzAutoTracker(threading.Thread):
continue
else:
pan_tilt_times = None
if (
camera_config.onvif.autotracking.zooming
== ZoomingModeEnum.relative
@@ -755,17 +747,28 @@ class PtzAutoTracker(threading.Thread):
await self._wait_until_stopped(camera, metrics)
# _zoom_absolute resets the metrics timestamps
pan_tilt_times = (
metrics.start_time.value,
metrics.stop_time.value,
)
if zoom > 0 and metrics.zoom_level.value != zoom:
await self.onvif._zoom_absolute(camera, zoom, 1)
await self._wait_until_stopped(camera, metrics)
start_time, stop_time = pan_tilt_times or (
metrics.start_time.value,
metrics.stop_time.value,
)
if camera_config.onvif.autotracking.movement_weights:
logger.debug(
f"{camera}: Predicted movement time: {self._predict_movement_time(camera, pan, tilt)}"
)
logger.debug(
f"{camera}: Actual movement time: {metrics.stop_time.value - metrics.start_time.value}"
f"{camera}: Actual movement time: {stop_time - start_time}"
)
# save metrics for better estimate calculations
@@ -781,8 +784,8 @@ class PtzAutoTracker(threading.Thread):
{
"pan": pan,
"tilt": tilt,
"start_timestamp": metrics.start_time.value,
"end_timestamp": metrics.stop_time.value,
"start_timestamp": start_time,
"end_timestamp": stop_time,
}
)
@@ -845,7 +848,6 @@ class PtzAutoTracker(threading.Thread):
# Thresholds
x_mags_thresh = camera_width / camera_fps / 2
y_mags_thresh = camera_height / camera_fps / 2
dir_thresh = 0.93
delta_thresh = 20
var_thresh = 10
@@ -863,24 +865,12 @@ class PtzAutoTracker(threading.Thread):
stdev_list = np.std(velocities, axis=0)
high_variances = np.any(stdev_list > var_thresh)
# Check direction difference
# the corners of an object moving toward or away from the camera move
# in different directions, so the delta bounds how far they may differ
velocities = np.round(velocities)
invalid_dirs = False
if np.all(np.linalg.norm(velocities, axis=1)):
cosine_sim = np.dot(velocities[0], velocities[1]) / (
np.linalg.norm(velocities[0]) * np.linalg.norm(velocities[1])
)
dir_thresh = 0.6 if np.all(delta < delta_thresh / 2) else dir_thresh
invalid_dirs = cosine_sim < dir_thresh
# Combine
invalid = (
invalid_x_mags
or invalid_y_mags
or invalid_dirs
or invalid_delta
or high_variances
)
invalid = invalid_x_mags or invalid_y_mags or invalid_delta or high_variances
if invalid:
logger.debug(
@@ -891,7 +881,6 @@ class PtzAutoTracker(threading.Thread):
for var_name, is_invalid in [
("invalid_x_mags", invalid_x_mags),
("invalid_y_mags", invalid_y_mags),
("invalid_dirs", invalid_dirs),
("invalid_delta", invalid_delta),
("high_variances", high_variances),
]
+34
View File
@@ -485,6 +485,40 @@ class TestHttpApp(BaseTestHttp):
assert sub_labels
assert sub_labels == [sub_label]
def test_get_event_list_filter_sub_labels(self):
sub_labels = {
"1.cyrillic": "Кузнецова Ира",
"2.joined": "Ира, Bob",
"3.ascii": "Alice",
"4.none": None,
}
mock_embeddings = Mock()
mock_embeddings.search_thumbnail.return_value = [(id, 0.1) for id in sub_labels]
self.app.frigate_config.semantic_search.enabled = True
self.app.embeddings = mock_embeddings
with AuthTestClient(self.app) as client:
for id, sub_label in sub_labels.items():
super().insert_mock_event(id)
Event.update(sub_label=sub_label).where(Event.id == id).execute()
search_params = {"search_type": "similarity", "event_id": "1.cyrillic"}
for path, params in (("/events", {}), ("/events/search", search_params)):
def ids(query: str) -> set[str]:
events = client.get(
path, params={**params, "sub_labels": query}
).json()
return {event["id"] for event in events}
assert ids("Кузнецова Ира") == {"1.cyrillic"}
assert ids("Ира") == {"2.joined"}
assert ids("bob") == {"2.joined"}
assert ids("ALICE") == {"3.ascii"}
assert ids("None") == {"4.none"}
####################################################################################################################
################################### GET /metrics Endpoint #########################################################
####################################################################################################################
+117 -2
View File
@@ -11,11 +11,14 @@ KeyError on the autotracker thread or silently keep the wrong state:
running its motion estimator for a camera that could not autotrack.
"""
import asyncio
import unittest
from unittest.mock import MagicMock
from unittest.mock import AsyncMock, MagicMock
import numpy as np
from frigate.camera import PTZMetrics
from frigate.config import FrigateConfig
from frigate.config import FrigateConfig, ZoomingModeEnum
from frigate.config.camera.updater import CameraConfigUpdateEnum
from frigate.ptz.autotrack import PtzAutoTracker, calculate_max_target_box
@@ -123,6 +126,118 @@ class TestAutotrackerDisable(unittest.TestCase):
self.assertIs(payload, autotracking)
class TestAutotrackerPrediction(unittest.TestCase):
def test_movement_time_uses_the_fitted_intercept_unchanged(self) -> None:
tracker = _make_tracker()
tracker.intercept = {CAMERA: 0.5}
tracker.move_coefficients = {CAMERA: [0.3, 0.8]}
self.assertAlmostEqual(tracker._predict_movement_time(CAMERA, 0.25, 0.25), 0.7)
def test_shrinking_box_keeps_a_positive_target_box(self) -> None:
# 60 px shrinking 6 px/frame reaches zero after 2 s at 5 fps
target_box = self._target_box_for((0, 0, 60, 60), [3, 3, -3, -3], 2.0)
self.assertGreater(target_box, 0)
self.assertTrue(np.isfinite(target_box))
def test_approaching_object_keeps_its_velocity(self) -> None:
# the box grows, so the corners move in opposite directions
tracker = _make_tracker()
obj = MagicMock()
obj.obj_data = {"estimate_velocity": np.array([[-3.0, -4.0], [3.0, 1.0]])}
valid, velocity = tracker._get_valid_velocity(CAMERA, obj)
self.assertTrue(valid)
self.assertEqual(list(velocity), [-3, -4, 3, 1])
def test_corners_that_disagree_too_much_are_rejected(self) -> None:
tracker = _make_tracker()
obj = MagicMock()
obj.obj_data = {"estimate_velocity": np.array([[-15.0, 0.0], [15.0, 0.0]])}
valid, velocity = tracker._get_valid_velocity(CAMERA, obj)
self.assertFalse(valid)
self.assertFalse(np.any(velocity))
def _target_box(self, velocity: list[float], predicted_time: float) -> float:
return self._target_box_for((900, 400, 1000, 600), velocity, predicted_time)
def _target_box_for(
self,
box: tuple[int, int, int, int],
velocity: list[float],
predicted_time: float,
) -> float:
tracker = _make_tracker()
tracker.tracked_object_history = {CAMERA: [{"box": box}]}
tracker.tracked_object_metrics = {
CAMERA: {"target_box": 0.2, "velocity": np.array(velocity)}
}
return tracker._predict_target_box(CAMERA, predicted_time)
def test_target_box_grows_with_an_approaching_object(self) -> None:
# the 200px height gains 4px per frame at 5 fps for 1 second
zoom_factor = 0.3
self.assertAlmostEqual(
self._target_box([-1, -2, 1, 2], 1.0),
0.2 * (220 / 200) ** (2 * zoom_factor),
)
def test_target_box_is_unchanged_without_growth_or_time(self) -> None:
self.assertAlmostEqual(self._target_box([5, 0, 5, 0], 1.0), 0.2)
self.assertEqual(self._target_box([-1, -2, 1, 2], 0), 0.2)
class TestAutotrackerMoveQueue(unittest.IsolatedAsyncioTestCase):
async def test_absolute_zoom_does_not_overwrite_the_pan_tilt_sample(self) -> None:
tracker = _make_tracker()
autotracking = tracker.config.cameras[CAMERA].onvif.autotracking
autotracking.zooming = ZoomingModeEnum.absolute
autotracking.calibrate_on_startup = True
metrics = tracker.ptz_metrics[CAMERA]
tracker.stop_event = MagicMock()
tracker.stop_event.is_set.side_effect = [False, True]
tracker.move_queues = {CAMERA: asyncio.Queue()}
tracker.move_queue_locks = {CAMERA: asyncio.Lock()}
tracker.intercept = {CAMERA: 0.5}
tracker.move_metrics = {CAMERA: []}
tracker._wait_until_stopped = AsyncMock()
tracker._calculate_move_coefficients = MagicMock()
async def move_relative(*_args) -> None:
metrics.start_time.value = 100.0
metrics.stop_time.value = 102.0
async def zoom_absolute(*_args) -> None:
metrics.start_time.value = 103.0
metrics.stop_time.value = 0
tracker.onvif._move_relative = AsyncMock(side_effect=move_relative)
tracker.onvif._zoom_absolute = AsyncMock(side_effect=zoom_absolute)
tracker.move_queues[CAMERA].put_nowait((50.0, 0.5, 0.5, 0.5))
await tracker._process_move_queue(CAMERA)
tracker.onvif._zoom_absolute.assert_awaited_once()
self.assertEqual(
tracker.move_metrics[CAMERA],
[
{
"pan": 0.5,
"tilt": 0.5,
"start_timestamp": 100.0,
"end_timestamp": 102.0,
}
],
)
class TestMaxTargetBox(unittest.TestCase):
def test_follows_zoom_factor(self) -> None:
self.assertAlmostEqual(calculate_max_target_box(0.5), 0.6**2)
@@ -150,17 +150,18 @@ export function TrackingDetails({
// Fetch the VOD manifest JSON to get the actual clipFrom after keyframe
// snapping. The backend may snap clipFrom backwards to a keyframe, making
// the video start earlier than the requested time.
// the video start earlier than the requested time. Uses sourceOffsetRef so
// the manifest describes the same clip as videoSource.
const vodManifestUrl = useMemo(() => {
if (!event.camera) return null;
const startTime =
event.start_time + annotationOffset / 1000 - REVIEW_PADDING;
const sourceOffset = sourceOffsetRef.current;
const startTime = event.start_time + sourceOffset / 1000 - REVIEW_PADDING;
const endTime =
(event.end_time ?? Date.now() / 1000) +
annotationOffset / 1000 +
sourceOffset / 1000 +
REVIEW_PADDING;
return `vod/clip/${event.camera}/start/${startTime}/end/${endTime}`;
}, [event, annotationOffset]);
}, [event]);
const { data: vodManifest } = useSWR<VodManifest>(vodManifestUrl, null, {
revalidateOnFocus: false,