Keep frame shared memory mapped (#24644)

* keep frame shm open and skip overwritten frames

Every reader closed and reopened a frame's shared memory segment on each frame, so each process paid a page fault on every page of a fresh mapping (about 760 for a 1080p frame) per frame. Readers now keep segments mapped. Capture stamps each slot's frame time in a per-camera `{camera}_frame_times` segment, and `get()` returns None when the slot no longer holds the requested frame, reopening once by name so a mapping left over from a same-size camera recreate heals itself. Capture also reuses a slot when the frame queue is full, since no reader was told about it.

* verify the frame for embeddings object updates

Tracked object events only carried the object's own frame_time, which stays at its last detection when the object isn't detected in a frame, so embeddings couldn't check the slot it read and could crop face or plate data from a frame that had already been overwritten. Events now carry the time of the frame they name, and embeddings passes it to `get()`.

* log when a frame slot was overwritten before it was read

A lapped read and a missing segment ended up as the same debug line at every caller, so there was no way to tell a slow reader from a missing frame.

* bring the FrameManager interface in line with its implementation

The abstract get() still had a timeout_ms signature from an older implementation and frame_times() wasn't declared, though capture and detect are typed against the interface. get() also didn't say how long the array it returns stays valid.

* drop a camera's cached frame mappings when it is removed or recreated

A reader that kept its mappings open stayed on a removed camera's segments, which pinned their shared memory until restart, and after a same-size recreate the ledger reopen only refreshed the slot being read while the other cached slots still returned the old camera's pixels. Readers now drop every cached slot for a camera on its remove update, and get() does the same when a ledger mismatch turns out to be a replaced ledger rather than a lapped slot.

* match a camera's frame names exactly in close_camera

The prefix match on `{camera}_frame` also caught another camera whose name starts with it, so removing `cam` dropped the cached ring of a camera named `cam_frame`.

frigate/util/image.py

@@ -990,13 +990,23 @@ class FrameManager(ABC):
         pass
 
     @abstractmethod
-    def get(self, name: str, timeout_ms: int = 0):
+    def get(
+        self, name: str, shape: tuple[int, ...], frame_time: float | None = None
+    ) -> np.ndarray | None:
+        pass
+
+    @abstractmethod
+    def frame_times(self, camera: str) -> np.ndarray | None:
         pass
 
     @abstractmethod
     def close(self, name: str):
         pass
 
+    @abstractmethod
+    def close_camera(self, camera: str) -> None:
+        pass
+
     @abstractmethod
     def delete(self, name: str):
         pass
@@ -1079,7 +1089,56 @@ class SharedMemoryFrameManager(FrameManager):
             logger.info(f"the file {name} not found")
             return None
 
-    def get(self, name: str, shape) -> np.ndarray | None:
+    def frame_times(self, camera: str) -> np.ndarray | None:
+        """Per-slot write times for a camera's frame ring, stamped by capture."""
+        buf = self.write(f"{camera}_frame_times")
+
+        if buf is None:
+            return None
+
+        return np.ndarray((len(buf) // 8,), dtype=np.float64, buffer=buf)
+
+    def _holds(self, name: str, frame_time: float) -> bool:
+        camera, _, index = name.rpartition("_frame")
+        times = self.frame_times(camera)
+        return (
+            times is not None
+            and index.isdigit()
+            and int(index) < len(times)
+            and times[int(index)] == frame_time
+        )
+
+    def get(
+        self, name: str, shape: tuple[int, ...], frame_time: float | None = None
+    ) -> np.ndarray | None:
+        """Map a frame slot as an array without copying it.
+
+        The array points into shared memory, so it is only valid until the next
+        close(), delete(), or get() of the same name, and capture overwrites its
+        pixels once the ring wraps back to this slot. Copy it to keep it longer.
+
+        Args:
+            name: Slot name, such as front_door_frame3
+            shape: Array shape, which must match the slot's size
+            frame_time: Time of the frame the caller was told is in the slot
+
+        Returns:
+            The frame, or None when the slot is missing, the wrong size, or holds
+            a different frame than frame_time
+        """
+        if frame_time is not None and not self._holds(name, frame_time):
+            # a ledger left over from a same-size recreate of the camera looks
+            # the same as a lapped slot, so reopen it by name once
+            camera = name.rpartition("_frame")[0]
+            self.close(f"{camera}_frame_times")
+
+            if not self._holds(name, frame_time):
+                logger.debug("%s no longer holds the frame from %s", name, frame_time)
+                return None
+
+            # the ledger was replaced, so every cached slot maps the old segments
+            self.close_camera(camera)
+
         try:
             required = int(np.prod(shape))
             shm = self.shm_store.get(name)
@@ -1110,6 +1169,14 @@ class SharedMemoryFrameManager(FrameManager):
             self.shm_store[name].close()
             del self.shm_store[name]
 
+    def close_camera(self, camera: str) -> None:
+        """Drop every cached mapping for a camera's frame ring and ledger."""
+        for name in list(self.shm_store):
+            owner, _, suffix = name.rpartition("_frame")
+
+            if owner == camera and (suffix.isdigit() or suffix == "_times"):
+                self.close(name)
+
     def delete(self, name: str):
         if name in self.shm_store:
             self.shm_store[name].close()