frigate/frigate/test/test_shared_memory_frame_manager.py
Josh Hawkins 3a09d01bbe
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
Debug replay resolution (#23287)
* unlink shm frames when camera is removed

* drop stale shm cache refs when cached segment is too small for requested shape

* skip new-object frame cache write when current_frame is unavailable

* add tests

* use setdefault when adding a new camera

Multiple subscribers in the same process each unpickle the ZMQ payload independently and would otherwise write divergent Python objects to the shared cameras dict — leaving long-lived references (e.g. CameraState.camera_config) pointing at a copy that subsequent in-place mutations like apply_section_update can never reach. setdefault collapses everyone onto the first writer's object so attribute mutations propagate to every consumer in this process.

* rebuild ffmpeg commands on detect update

Rebuild the cached ffmpeg cmd so the next process spawn picks up new resolution/fps. Running cameras keep their existing cmd (ffmpeg_cmds is only read at process startup); replay cameras are recycled by CameraMaintainer to pick up the rebuilt cmd

* drop stale shm cache refs when cached segment size doesn't match requested shape

The cached SharedMemoryFrameManager reference can point at a segment whose
size no longer matches the requested shape — the segment was unlinked and
recreated at a different size in a camera add/remove cycle. This catches
both a resolution increase (cached too small) and a decrease (cached too
large, pointing at an orphaned inode whose stale bytes would otherwise be
misinterpreted at the new shape, producing distorted/miscolored YUV frames).

After reopening, if the OS-level segment still doesn't match the requested
shape we're in a transient mid-recreate state — either the maintainer
hasn't allocated the new segment yet (size too small) or we opened a
pre-recycle segment (size too big). Either way, skip the frame and don't
cache the mismatched ref.

* recycle replay camera on detect update

* discard tracked-object state when detect resolution changes mid-session

When detect resolution changes mid-session every tracked object we hold
was localized against the old pixel grid. Their boxes no longer
correspond to anything in the new frame, and the `end` callback that
fires when their IDs disappear from the new detect process's detections
publishes those stale boxes to consumers (LPR, snapshot crop) that slice
the new frame and crash on empty arrays. Drop the tracked-object state
on a shape change so no stale boxes ever cross the CameraState boundary.

Belt-and-suspenders: also drop any incoming batch whose boxes exceed the
current detect resolution. These are in-flight queue entries from the
pre-recycle detect process that beat the new detect process to the
queue; processing them would re-introduce stale-resolution tracked
objects we just dropped above. The per-camera detect process clamps
legitimate boxes to detect.width-1 / detect.height-1, so any coord
beyond that is unambiguously stale.

* rebuild motion and object filter masks on detect resolution change

Apply the detect update first so frame_shape reflects the new resolution
before we rebuild dependents.

Motion's rasterized_mask is sized to frame_shape at construction. When
detect resolution changes we must rebuild RuntimeMotionConfig so the
mask matches the new frame size; otherwise consumers like the LPR
processor and motion detector hit a shape mismatch when they index
frames with the stale mask.

Same story for per-object filter masks — rebuild RuntimeFilterConfig at
the new frame_shape so the merged global+per-object masks they hold
match what they'll be indexed against.

* republish motion and objects on in-memory detect resize

A detect resolution change also invalidates the rasterized masks on
motion and per-object filters. apply_section_update has rebuilt them at
the new frame_shape; publish them too so other processes replace their
old values.

* add test

* frontend

* add refresh topic for camera maintainer recycle action

The maintainer's recycle branch is doing an action (recycle the camera)
in response to a section-level signal. Introduce a
CameraConfigUpdateEnum.refresh case as an explicit action signal — the
maintainer subscribes to refresh instead of detect, parallel with add
and remove. Publishers fire refresh alongside detect when a recycle is
needed; section-level subscribers keep their existing topic.

Since no main-process subscriber listens for detect anymore, the
refresh handler calls recreate_ffmpeg_cmds() explicitly so the shared
CameraConfig's ffmpeg_cmds is rebuilt before the new subprocesses
spawn.

* factor stale-resolution state drop into a CameraState method
2026-05-22 08:39:52 -06:00

157 lines
6.3 KiB
Python

"""Tests for SharedMemoryFrameManager cache invalidation.
Covers the case where a SHM segment is unlinked and recreated at a
different size across a camera add/remove cycle while a long-lived
in-process cache (e.g. TrackedObjectProcessor) still holds a ref to
the old, smaller segment.
"""
import unittest
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
from frigate.util.image import SharedMemoryFrameManager
def _fake_shm(size: int) -> SimpleNamespace:
"""A minimal stand-in for UntrackedSharedMemory with .size and .buf."""
return SimpleNamespace(size=size, buf=bytearray(size), close=lambda: None)
class TestSharedMemoryFrameManagerGet(unittest.TestCase):
def test_get_reopens_when_cached_segment_is_smaller_than_shape(self) -> None:
"""A cached ref to an older smaller segment must be dropped and the
current (correctly sized) segment reopened. Without this, np.ndarray
would raise "buffer is too small for requested array" when the
in-memory cache pointed at an old SHM after a same-name resize."""
manager = SharedMemoryFrameManager()
small = _fake_shm(size=100)
current = _fake_shm(size=2_500)
manager.shm_store["cam_frame0"] = small
with patch("frigate.util.image.UntrackedSharedMemory", return_value=current):
arr = manager.get("cam_frame0", (50, 50))
self.assertIsNotNone(arr)
self.assertEqual(arr.shape, (50, 50))
self.assertIs(manager.shm_store["cam_frame0"], current)
def test_get_reopens_when_cached_segment_is_larger_than_shape(self) -> None:
"""Symmetric to the smaller-cache case: when detect resolution drops,
the SHM is unlinked and recreated at a smaller size. A cached ref to
the old, larger segment still satisfies any size check but points at
an orphaned inode whose stale bytes get reinterpreted at the new
shape — producing miscolored, distorted YUV frames downstream. Drop
the cache so we reopen by name and bind to the current segment."""
manager = SharedMemoryFrameManager()
old_large = _fake_shm(size=10_000)
current = _fake_shm(size=2_500)
manager.shm_store["cam_frame0"] = old_large
with patch("frigate.util.image.UntrackedSharedMemory", return_value=current):
arr = manager.get("cam_frame0", (50, 50))
self.assertIsNotNone(arr)
self.assertEqual(arr.shape, (50, 50))
self.assertIs(manager.shm_store["cam_frame0"], current)
def test_get_keeps_cached_segment_when_size_matches(self) -> None:
"""Don't pay the reopen cost when the cached ref is the right size."""
manager = SharedMemoryFrameManager()
cached = _fake_shm(size=2_500)
manager.shm_store["cam_frame0"] = cached
with patch("frigate.util.image.UntrackedSharedMemory") as untracked_shm_cls:
arr = manager.get("cam_frame0", (50, 50))
untracked_shm_cls.assert_not_called()
self.assertIsNotNone(arr)
self.assertIs(manager.shm_store["cam_frame0"], cached)
def test_get_opens_fresh_when_no_cache_entry(self) -> None:
manager = SharedMemoryFrameManager()
fresh = _fake_shm(size=2_500)
with patch("frigate.util.image.UntrackedSharedMemory", return_value=fresh):
arr = manager.get("cam_frame0", (50, 50))
self.assertIsNotNone(arr)
self.assertIs(manager.shm_store["cam_frame0"], fresh)
def test_get_returns_none_when_segment_missing(self) -> None:
manager = SharedMemoryFrameManager()
with patch(
"frigate.util.image.UntrackedSharedMemory",
side_effect=FileNotFoundError,
):
arr = manager.get("cam_frame0", (50, 50))
self.assertIsNone(arr)
def test_get_returns_none_when_reopened_segment_is_still_too_small(self) -> None:
"""Race during a same-name SHM recreate: cache is stale, we reopen
by name, but the maintainer hasn't allocated the new segment yet —
the reopened ref is also too small. Skip the frame (return None)
rather than crash on np.ndarray."""
manager = SharedMemoryFrameManager()
small_cached = _fake_shm(size=100)
still_small_after_reopen = _fake_shm(size=100)
manager.shm_store["cam_frame0"] = small_cached
with patch(
"frigate.util.image.UntrackedSharedMemory",
return_value=still_small_after_reopen,
):
arr = manager.get("cam_frame0", (50, 50))
self.assertIsNone(arr)
# Don't cache the too-small reopened ref — next call will re-open
# once the maintainer has finished recreating the segment.
self.assertNotIn("cam_frame0", manager.shm_store)
def test_get_handles_n_dimensional_shape(self) -> None:
"""np.prod must be used (not raw multiplication) for tuple shapes."""
manager = SharedMemoryFrameManager()
# YUV-shaped frame: (height * 3/2, width) for 1920x1080 = 3,110,400
big_enough = _fake_shm(size=3_110_400)
manager.shm_store["cam_frame0"] = big_enough
with patch("frigate.util.image.UntrackedSharedMemory") as untracked_shm_cls:
arr = manager.get("cam_frame0", (1620, 1920))
untracked_shm_cls.assert_not_called()
self.assertIsNotNone(arr)
self.assertEqual(arr.shape, (1620, 1920))
class TestSharedMemoryFrameManagerGetRecreatesLargerSegment(unittest.TestCase):
"""End-to-end-style: simulates the full unlink-and-recreate cycle."""
def test_segment_grows_then_get_succeeds(self) -> None:
manager = SharedMemoryFrameManager()
# Phase 1: existing camera at 320x240 YUV — 320 * 240 * 1.5 = 115_200
small = _fake_shm(size=115_200)
manager.shm_store["cam_frame0"] = small
arr_small = np.ndarray((360, 320), dtype=np.uint8, buffer=small.buf)
self.assertEqual(arr_small.shape, (360, 320))
# Phase 2: restart at 1920x1080 — new SHM segment, larger size.
large = _fake_shm(size=3_110_400)
with patch("frigate.util.image.UntrackedSharedMemory", return_value=large):
arr_large = manager.get("cam_frame0", (1620, 1920))
self.assertIsNotNone(arr_large)
self.assertEqual(arr_large.shape, (1620, 1920))
if __name__ == "__main__":
unittest.main()