Compare commits

...
5 Commits
Author SHA1 Message Date
teajayseeandGitHub 7376d4303b Merge 36c61a3607 into ec3fb00494 2026-06-23 22:44:44 +00:00
ec3fb00494 perf(track): use sum()/len() instead of np.mean in average_boxes (#23521)
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* perf(track): avoid numpy reductions on tiny box lists in position smoothing

update_position runs per tracked object per frame. While a position has
fewer than 10 samples it calls np.percentile four times, and average_boxes
(per stationary object per frame) calls np.mean four times - all on lists of
at most 10 ints, where numpy's per-call dispatch/validation overhead
dominates the actual work.

Replace them with pure-Python equivalents:
- average_boxes: sum()/len() instead of np.mean (bit-identical output)
- interpolated_percentile(): linear-interpolated percentile matching
  numpy.percentile (including its lerp branch at frac>=0.5) for the small
  lists used here, in place of np.percentile

Measured in the release image (numpy 1.26.4) on a 10-element list:
np.percentile 18735 ns -> 191 ns/call (98x); np.mean-based average_boxes
7480 ns -> 591 ns (12.7x); ~74 us saved per object-frame in update_position.
A live py-spy --gil profile of a camera process_frames worker showed
np.percentile (update_position) and np.mean (average_boxes) among the top
Frigate-owned on-CPU frames.

Output is unchanged: added tests assert both helpers are bit-identical to
numpy over randomized small inputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Drop interpolated_percentile, keep only average_boxes

Per review: reimplementing np.percentile hurts readability and risks
divergence from numpy (e.g. numpy 2.x). Revert update_position to
np.percentile and remove the helper; keep only the average_boxes change
(sum()/len() instead of np.mean), which stays bit-identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:47:04 -06:00
081d6f95ef perf(track): avoid per-frame allocations and list lookups in tracker (#23523)
Two small per-object-per-frame improvements in the tracker hot path
(match_and_update), both bit-identical:

- get_stationary_threshold returned a freshly constructed StationaryThresholds
  (a dataclass plus a list) on every call for any label not in the three
  known lists - i.e. for common labels like person/dog. The default thresholds
  are constant and never mutated, so return a shared module-level singleton,
  as the other three cases already do.
- untracked_object_boxes membership used `box not in [list of boxes]` (O(n));
  build a set of box tuples for O(1) membership. Boxes are hashable as tuples
  and output is unchanged.

get_stationary_threshold appeared in a live py-spy --gil profile of a camera
process_frames worker. Adds tests for the threshold lookups.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:24:05 -06:00
a2b46f5d84 perf(util): cut redundant work in per-frame detection consolidation (#23522)
video/detect.py runs these for every frame:

- get_cluster_candidates: used_boxes was a list with `in` membership tests
  inside the nested loop (O(n) per check). It is only ever membership-tested,
  so switching it to a set (O(1)) leaves output unchanged.
- get_consolidated_object_detections: area(current_box) was recomputed on
  every inner-loop iteration though it is loop-invariant; hoist it to one
  call per outer detection.

Both are bit-identical (verified against the previous implementations over
randomized inputs). Measured in the release image, get_cluster_candidates on
a frame of 30 detection boxes: 59.2 us -> 42.1 us (1.4x); the gain scales
with the number of boxes per frame.

Adds a partition-invariant test (every box index lands in exactly one
cluster).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:23:18 -06:00
TomandClaude Fable 5 36c61a3607 Corroborate record watchdog staleness against the cache dir before restarting
The record watchdog treats a stale maintainer heartbeat as a dead
recorder. But the heartbeat is published by the recording maintainer,
so whenever the maintainer lags (e.g. "Unable to keep up with recording
segments in cache", #9661) every camera looks stale at once and all
record processes restart together - while recording was actually
healthy. The restart churn then produces more, shorter segments,
making the maintainer fall further behind.

Before restarting on staleness, check the camera's newest cache
segment on disk: if a segment is fresher than the staleness threshold,
the recorder is demonstrably writing - log a warning, adopt the disk
mtime as the heartbeat, and skip the restart. The invalid-segment path
is untouched.

Validated on a 26-camera production deployment (0.17.1 backport of
this change): synchronized mass restarts went from 52/hour to zero,
with heartbeat-stale events still occurring ~2/hour but now correctly
identified as maintainer lag instead of recording failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 16:41:33 +01:00
7 changed files with 130 additions and 23 deletions
+16
View File
@@ -1,6 +1,22 @@
import random
import unittest
import numpy as np
from frigate.track.tracked_object import TrackedObjectAttribute
from frigate.util.object import average_boxes
class TestBoxStatistics(unittest.TestCase):
def test_average_boxes_matches_numpy(self) -> None:
rng = random.Random(0)
for _ in range(5000):
boxes = [
[rng.randint(0, 4000) for _ in range(4)]
for _ in range(rng.randint(1, 10))
]
expected = [float(np.mean([b[i] for b in boxes])) for i in range(4)]
self.assertEqual(average_boxes(boxes), expected)
class TestAttribute(unittest.TestCase):
@@ -0,0 +1,39 @@
"""Tests for stationary object classification thresholds."""
import unittest
from frigate.track.stationary_classifier import (
DEFAULT_OBJECT_THRESHOLDS,
DYNAMIC_OBJECT_THRESHOLDS,
NON_STATIONARY_OBJECT_THRESHOLDS,
STATIONARY_OBJECT_THRESHOLDS,
StationaryThresholds,
get_stationary_threshold,
)
class TestStationaryThresholds(unittest.TestCase):
def test_known_labels_return_expected_singletons(self) -> None:
self.assertIs(get_stationary_threshold("package"), STATIONARY_OBJECT_THRESHOLDS)
self.assertIs(get_stationary_threshold("car"), DYNAMIC_OBJECT_THRESHOLDS)
self.assertIs(
get_stationary_threshold("license_plate"),
NON_STATIONARY_OBJECT_THRESHOLDS,
)
def test_unknown_label_returns_shared_default(self) -> None:
# an unknown label must reuse the shared default instance, not allocate
# a fresh one on every call (this runs per object per frame)
first = get_stationary_threshold("person")
second = get_stationary_threshold("dog")
self.assertIs(first, DEFAULT_OBJECT_THRESHOLDS)
self.assertIs(second, DEFAULT_OBJECT_THRESHOLDS)
def test_default_matches_a_fresh_instance(self) -> None:
# the shared default must be value-equivalent to the previous
# per-call StationaryThresholds()
self.assertEqual(DEFAULT_OBJECT_THRESHOLDS, StationaryThresholds())
if __name__ == "__main__":
unittest.main()
+18
View File
@@ -82,6 +82,24 @@ class TestRegion(unittest.TestCase):
assert len(cluster_candidates) == 2
def test_cluster_candidates_partition_boxes(self):
# every box index must appear in exactly one cluster (no box used twice,
# none dropped) - the invariant the used-box tracking enforces
boxes = [
(100, 100, 200, 200),
(202, 150, 252, 200),
(210, 160, 260, 210),
(900, 900, 950, 950),
(905, 905, 955, 955),
]
cluster_candidates = get_cluster_candidates(
self.frame_shape, self.min_region_size, boxes
)
assigned = [idx for cluster in cluster_candidates for idx in cluster]
self.assertEqual(sorted(assigned), list(range(len(boxes))))
def test_transliterate_to_latin(self):
self.assertEqual(transliterate_to_latin("frégate"), "fregate")
self.assertEqual(transliterate_to_latin("utilité"), "utilite")
+4 -2
View File
@@ -641,9 +641,11 @@ class NorfairTracker(ObjectTracker):
self.deregister(self.track_id_map[e_id], e_id)
# update list of object boxes that don't have a tracked object yet
tracked_object_boxes = [obj["box"] for obj in self.tracked_objects.values()]
tracked_object_boxes = {
tuple(obj["box"]) for obj in self.tracked_objects.values()
}
self.untracked_object_boxes = [
o[2] for o in detections if o[2] not in tracked_object_boxes
o[2] for o in detections if tuple(o[2]) not in tracked_object_boxes
]
def print_objects_as_table(self, tracked_objects: Sequence) -> None:
+4 -1
View File
@@ -63,6 +63,9 @@ NON_STATIONARY_OBJECT_THRESHOLDS = StationaryThresholds(
max_stationary_history=4,
)
# Default thresholds for any other object label
DEFAULT_OBJECT_THRESHOLDS = StationaryThresholds()
def get_stationary_threshold(label: str) -> StationaryThresholds:
"""Get the stationary thresholds for a given object label."""
@@ -76,7 +79,7 @@ def get_stationary_threshold(label: str) -> StationaryThresholds:
if label in NON_STATIONARY_OBJECT_THRESHOLDS.objects:
return NON_STATIONARY_OBJECT_THRESHOLDS
return StationaryThresholds()
return DEFAULT_OBJECT_THRESHOLDS
class StationaryMotionClassifier:
+15 -19
View File
@@ -339,18 +339,13 @@ def reduce_boxes(boxes, iou_threshold=0.0):
def average_boxes(boxes: list[list[int, int, int, int]]) -> list[int, int, int, int]:
"""Return a box that is the average of a list of boxes."""
x_mins = []
y_mins = []
x_max = []
y_max = []
for box in boxes:
x_mins.append(box[0])
y_mins.append(box[1])
x_max.append(box[2])
y_max.append(box[3])
return [np.mean(x_mins), np.mean(y_mins), np.mean(x_max), np.mean(y_max)]
n = len(boxes)
return [
sum(box[0] for box in boxes) / n,
sum(box[1] for box in boxes) / n,
sum(box[2] for box in boxes) / n,
sum(box[3] for box in boxes) / n,
]
def median_of_boxes(boxes: list[list[int, int, int, int]]) -> list[int, int, int, int]:
@@ -401,13 +396,13 @@ def get_cluster_candidates(frame_shape, min_region, boxes):
# determined by the max_region size minus half the box + 20%
# TODO: see if we can do this with numpy
cluster_candidates = []
used_boxes = []
used_boxes = set()
# loop over each box
for current_index, b in enumerate(boxes):
if current_index in used_boxes:
continue
cluster = [current_index]
used_boxes.append(current_index)
used_boxes.add(current_index)
cluster_boundary = get_cluster_boundary(b, min_region)
# find all other boxes that fit inside the boundary
for compare_index, compare_box in enumerate(boxes):
@@ -436,7 +431,7 @@ def get_cluster_candidates(frame_shape, min_region, boxes):
if should_cluster:
cluster.append(compare_index)
used_boxes.append(compare_index)
used_boxes.add(compare_index)
cluster_candidates.append(cluster)
# return the unique clusters only
@@ -558,6 +553,7 @@ def reduce_detections(
current_detection = sorted_by_area[current_detection_idx]
current_label = current_detection[0]
current_box = current_detection[2]
current_area = area(current_box)
overlap = 0
for to_check_idx in range(
min(current_detection_idx + 1, len(sorted_by_area)),
@@ -568,14 +564,14 @@ def reduce_detections(
# if area of current detection / area of check < 5% they should not be compared
# this covers cases where a large car parked in a driveway doesn't block detections
# of cars in the street behind it
if area(current_box) / area(to_check) < 0.05:
if current_area / area(to_check) < 0.05:
continue
intersect_box = intersection(current_box, to_check)
# if % of smaller detection is inside of another detection, consolidate
if intersect_box is not None and area(intersect_box) / area(
current_box
) > LABEL_CONSOLIDATION_MAP.get(
if intersect_box is not None and area(
intersect_box
) / current_area > LABEL_CONSOLIDATION_MAP.get(
current_label, LABEL_CONSOLIDATION_DEFAULT
):
overlap = 1
+34 -1
View File
@@ -9,6 +9,7 @@ from collections import deque
from datetime import datetime, timedelta, timezone
from multiprocessing import Queue, Value
from multiprocessing.synchronize import Event as MpEvent
from pathlib import Path
from typing import Any
from frigate.camera import CameraMetrics
@@ -22,7 +23,7 @@ from frigate.config.camera.updater import (
CameraConfigUpdateEnum,
CameraConfigUpdateSubscriber,
)
from frigate.const import PROCESS_PRIORITY_HIGH
from frigate.const import CACHE_DIR, PROCESS_PRIORITY_HIGH
from frigate.log import LogPipe
from frigate.util.builtin import EventsPerSecond, get_record_segment_time
from frigate.util.ffmpeg import start_or_restart_ffmpeg, stop_ffmpeg
@@ -453,6 +454,38 @@ class CameraWatchdog(threading.Thread):
invalid_stale = invalid_stale_condition
if cache_stale or valid_stale or invalid_stale:
# The staleness above is measured from the recording
# maintainer's IPC heartbeat, which lags whenever the
# maintainer falls behind (e.g. "Unable to keep up with
# recording segments in cache"). A late message is not
# a dead recorder: corroborate against the cache dir
# before restarting, otherwise every camera restarts
# together on maintainer lag and the resulting segment
# churn makes the overload worse.
if not invalid_stale:
newest_on_disk = max(
(
f.stat().st_mtime
for f in Path(CACHE_DIR).glob(
f"{self.config.name}@*"
)
),
default=0.0,
)
if (
newest_on_disk > 0
and now_utc.timestamp() - newest_on_disk
< self.record_stale_threshold
):
self.logger.warning(
f"Recording heartbeat for {self.config.name} is stale but a cache "
f"segment is only {now_utc.timestamp() - newest_on_disk:.0f}s old — "
"skipping the record process restart (maintainer heartbeat lag, "
"not a recording failure)."
)
self.latest_cache_segment_time = newest_on_disk
continue
if cache_stale:
reason = "No new recording segments were created"
elif valid_stale: