Miscellaneous Fixes (0.17 beta) (#21558)

* mse player improvements

- fix WebSocket race condition by registering message handlers before sending and avoid closing CONNECTING sockets to eliminate "Socket is not connected" errors.
- attempt to resolve Safari MSE timeout and handler issues by wrapping temporary handlers in try/catch and stabilizing the permanent mse handler so SourceBuffer setup completes reliably.
- add intentional disconnect tracking to prevent unwanted reconnects during navigation/StrictMode cycles

* Update Ollama

* additional MSE tweaks

* Turn activity context prompt into a yaml example

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

docker/main/requirements-wheels.txt

@@ -48,7 +48,7 @@ onnxruntime == 1.22.*
 transformers == 4.45.*
 # Generative AI
 google-generativeai == 0.8.*
-ollama == 0.5.*
+ollama == 0.6.*
 openai == 1.65.*
 # push notifications
 py-vapid == 1.9.*

docs/docs/configuration/genai/review_summaries.md

@@ -31,40 +31,43 @@ Each installation and even camera can have different parameters for what is cons
 <details>
   <summary>Default Activity Context Prompt</summary>
 
-```
+```yaml
-### Normal Activity Indicators (Level 0)
+review:
-- Known/verified people in any zone at any time
+  genai:
-- People with pets in residential areas
+    activity_context_prompt: |
-- Deliveries or services during daytime/evening (6 AM - 10 PM): carrying packages to doors/porches, placing items, leaving
+      ### Normal Activity Indicators (Level 0)
-- Services/maintenance workers with visible tools, uniforms, or service vehicles during daytime
+      - Known/verified people in any zone at any time
-- Activity confined to public areas only (sidewalks, streets) without entering property at any time
+      - People with pets in residential areas
+      - Deliveries or services during daytime/evening (6 AM - 10 PM): carrying packages to doors/porches, placing items, leaving
+      - Services/maintenance workers with visible tools, uniforms, or service vehicles during daytime
+      - Activity confined to public areas only (sidewalks, streets) without entering property at any time
 
-### Suspicious Activity Indicators (Level 1)
+      ### Suspicious Activity Indicators (Level 1)
-- **Testing or attempting to open doors/windows/handles on vehicles or buildings** — ALWAYS Level 1 regardless of time or duration
+      - **Testing or attempting to open doors/windows/handles on vehicles or buildings** — ALWAYS Level 1 regardless of time or duration
-- **Unidentified person in private areas (driveways, near vehicles/buildings) during late night/early morning (11 PM - 5 AM)** — ALWAYS Level 1 regardless of activity or duration
+      - **Unidentified person in private areas (driveways, near vehicles/buildings) during late night/early morning (11 PM - 5 AM)** — ALWAYS Level 1 regardless of activity or duration
-- Taking items that don't belong to them (packages, objects from porches/driveways)
+      - Taking items that don't belong to them (packages, objects from porches/driveways)
-- Climbing or jumping fences/barriers to access property
+      - Climbing or jumping fences/barriers to access property
-- Attempting to conceal actions or items from view
+      - Attempting to conceal actions or items from view
-- Prolonged loitering: remaining in same area without visible purpose throughout most of the sequence
+      - Prolonged loitering: remaining in same area without visible purpose throughout most of the sequence
 
-### Critical Threat Indicators (Level 2)
+      ### Critical Threat Indicators (Level 2)
-- Holding break-in tools (crowbars, pry bars, bolt cutters)
+      - Holding break-in tools (crowbars, pry bars, bolt cutters)
-- Weapons visible (guns, knives, bats used aggressively)
+      - Weapons visible (guns, knives, bats used aggressively)
-- Forced entry in progress
+      - Forced entry in progress
-- Physical aggression or violence
+      - Physical aggression or violence
-- Active property damage or theft in progress
+      - Active property damage or theft in progress
 
-### Assessment Guidance
+      ### Assessment Guidance
-Evaluate in this order:
+      Evaluate in this order:
 
-1. **If person is verified/known** → Level 0 regardless of time or activity
+      1. **If person is verified/known** → Level 0 regardless of time or activity
-2. **If person is unidentified:**
+      2. **If person is unidentified:**
-   - Check time: If late night/early morning (11 PM - 5 AM) AND in private areas (driveways, near vehicles/buildings) → Level 1
+        - Check time: If late night/early morning (11 PM - 5 AM) AND in private areas (driveways, near vehicles/buildings) → Level 1
-   - Check actions: If testing doors/handles, taking items, climbing → Level 1
+        - Check actions: If testing doors/handles, taking items, climbing → Level 1
-   - Otherwise, if daytime/evening (6 AM - 10 PM) with clear legitimate purpose (delivery, service worker) → Level 0
+        - Otherwise, if daytime/evening (6 AM - 10 PM) with clear legitimate purpose (delivery, service worker) → Level 0
-3. **Escalate to Level 2 if:** Weapons, break-in tools, forced entry in progress, violence, or active property damage visible (escalates from Level 0 or 1)
+      3. **Escalate to Level 2 if:** Weapons, break-in tools, forced entry in progress, violence, or active property damage visible (escalates from Level 0 or 1)
 
-The mere presence of an unidentified person in private areas during late night hours is inherently suspicious and warrants human review, regardless of what activity they appear to be doing or how brief the sequence is.
+      The mere presence of an unidentified person in private areas during late night hours is inherently suspicious and warrants human review, regardless of what activity they appear to be doing or how brief the sequence is.
 ```
 
 </details>

web/src/components/player/MsePlayer.tsx

@@ -80,12 +80,15 @@ function MSEPlayer({
   const videoRef = useRef<HTMLVideoElement>(null);
   const wsRef = useRef<WebSocket | null>(null);
   const reconnectTIDRef = useRef<number | null>(null);
+  const intentionalDisconnectRef = useRef<boolean>(false);
   const ondataRef = useRef<((data: ArrayBufferLike) => void) | null>(null);
   const onmessageRef = useRef<{
     [key: string]: (msg: { value: string; type: string }) => void;
   }>({});
   const msRef = useRef<MediaSource | null>(null);
   const mseCodecRef = useRef<string | null>(null);
+  const mseTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
+  const mseResponseReceivedRef = useRef<boolean>(false);
 
   const wsURL = useMemo(() => {
     return `${baseUrl.replace(/^http/, "ws")}live/mse/api/ws?src=${camera}`;
@@ -152,8 +155,11 @@ function MSEPlayer({
   }, []);
 
   const onConnect = useCallback(() => {
-    if (!videoRef.current?.isConnected || !wsURL || wsRef.current) return false;
+    if (!videoRef.current?.isConnected || !wsURL || wsRef.current) {
+      return false;
+    }
 
+    intentionalDisconnectRef.current = false;
     setWsState(WebSocket.CONNECTING);
 
     setConnectTS(Date.now());
@@ -172,13 +178,50 @@ function MSEPlayer({
       setBufferTimeout(undefined);
     }
 
+    // Clear any pending MSE timeout
+    if (mseTimeoutRef.current !== null) {
+      clearTimeout(mseTimeoutRef.current);
+      mseTimeoutRef.current = null;
+    }
+
+    // Clear any pending reconnect attempts
+    if (reconnectTIDRef.current !== null) {
+      clearTimeout(reconnectTIDRef.current);
+      reconnectTIDRef.current = null;
+    }
+
     setIsPlaying(false);
 
     if (wsRef.current) {
-      setWsState(WebSocket.CLOSED);
+      const ws = wsRef.current;
-      wsRef.current.close();
       wsRef.current = null;
+      const currentReadyState = ws.readyState;
+
+      intentionalDisconnectRef.current = true;
+      setWsState(WebSocket.CLOSED);
+
+      // Remove event listeners to prevent them firing during close
+      try {
+        ws.removeEventListener("open", onOpen);
+        ws.removeEventListener("close", onClose);
+      } catch {
+        // Ignore errors removing listeners
+      }
+
+      // Only call close() if the socket is OPEN or CLOSING
+      // For CONNECTING or CLOSED sockets, just let it die
+      if (
+        currentReadyState === WebSocket.OPEN ||
+        currentReadyState === WebSocket.CLOSING
+      ) {
+        try {
+          ws.close();
+        } catch {
+          // Ignore close errors
+        }
+      }
     }
+    // eslint-disable-next-line react-hooks/exhaustive-deps
   }, [bufferTimeout]);
 
   const handlePause = useCallback(() => {
@@ -188,7 +231,14 @@ function MSEPlayer({
     }
   }, [isPlaying, playbackEnabled]);
 
-  const onOpen = () => {
+  const onOpen = useCallback(() => {
+    // If we were marked for intentional disconnect while connecting, close immediately
+    if (intentionalDisconnectRef.current) {
+      wsRef.current?.close();
+      wsRef.current = null;
+      return;
+    }
+
     setWsState(WebSocket.OPEN);
 
     wsRef.current?.addEventListener("message", (ev) => {
@@ -205,10 +255,27 @@ function MSEPlayer({
     ondataRef.current = null;
     onmessageRef.current = {};
 
+    // Reset the MSE response flag for this new connection
+    mseResponseReceivedRef.current = false;
+
+    // Create a fresh MediaSource for this connection to avoid stale sourceopen events
+    // from previous connections interfering with this one
+    const MediaSourceConstructor =
+      "ManagedMediaSource" in window ? window.ManagedMediaSource : MediaSource;
+    // @ts-expect-error for typing
+    msRef.current = new MediaSourceConstructor();
+
     onMse();
-  };
+    // onMse is defined below and stable
+    // eslint-disable-next-line react-hooks/exhaustive-deps
+  }, []);
 
   const reconnect = (timeout?: number) => {
+    // Don't reconnect if intentional disconnect was flagged
+    if (intentionalDisconnectRef.current) {
+      return;
+    }
+
     setWsState(WebSocket.CONNECTING);
     wsRef.current = null;
 
@@ -221,28 +288,79 @@ function MSEPlayer({
     }, delay);
   };
 
-  const onClose = () => {
+  const onClose = useCallback(() => {
+    // Don't reconnect if this was an intentional disconnect
+    if (intentionalDisconnectRef.current) {
+      // Reset the flag so future connects are allowed
+      intentionalDisconnectRef.current = false;
+      return;
+    }
+
     if (wsState === WebSocket.CLOSED) return;
     reconnect();
-  };
+    // reconnect is defined below and stable
+    // eslint-disable-next-line react-hooks/exhaustive-deps
+  }, [wsState]);
 
   const sendWithTimeout = (value: object, timeout: number) => {
     return new Promise<void>((resolve, reject) => {
+      // Don't start timeout if WS isn't connected - this can happen when
+      // sourceopen fires from a previous connection after we've already disconnected
+      if (!wsRef.current || wsRef.current.readyState !== WebSocket.OPEN) {
+        // Reject so caller knows this didn't work
+        reject(new Error("WebSocket not connected"));
+        return;
+      }
+
+      // If we've already received an MSE response for this connection, don't start another timeout
+      if (mseResponseReceivedRef.current) {
+        resolve();
+        return;
+      }
+
+      // Clear any existing MSE timeout from a previous attempt
+      if (mseTimeoutRef.current !== null) {
+        clearTimeout(mseTimeoutRef.current);
+        mseTimeoutRef.current = null;
+      }
+
       const timeoutId = setTimeout(() => {
-        reject(new Error("Timeout waiting for response"));
+        // Only reject if we haven't received a response yet
+        if (!mseResponseReceivedRef.current) {
+          mseTimeoutRef.current = null;
+          reject(new Error("Timeout waiting for response"));
+        }
       }, timeout);
 
-      send(value);
+      mseTimeoutRef.current = timeoutId;
 
       // Override the onmessageRef handler for mse type to resolve the promise on response
       const originalHandler = onmessageRef.current["mse"];
       onmessageRef.current["mse"] = (msg) => {
         if (msg.type === "mse") {
-          clearTimeout(timeoutId);
+          // Mark that we've received the response
-          if (originalHandler) originalHandler(msg);
+          mseResponseReceivedRef.current = true;
+
+          // Clear the timeout (use ref to clear the current one, not closure)
+          if (mseTimeoutRef.current !== null) {
+            clearTimeout(mseTimeoutRef.current);
+            mseTimeoutRef.current = null;
+          }
+
+          // Call original handler in try-catch so errors don't prevent promise resolution
+          if (originalHandler) {
+            try {
+              originalHandler(msg);
+            } catch (e) {
+              // Don't reject - we got the response, just let the error bubble
+            }
+          }
+
           resolve();
         }
       };
+
+      send(value);
     });
   };
 
@@ -292,13 +410,15 @@ function MSEPlayer({
             },
             (fallbackTimeout ?? 3) * 1000,
           ).catch(() => {
+            // Only report errors if we actually had a connection that failed
+            // If WS wasn't connected, this is a stale sourceopen event from a previous connection
             if (wsRef.current) {
               onDisconnect();
-            }
+              if (isIOS || isSafari) {
-            if (isIOS || isSafari) {
+                handleError("mse-decode", "Safari cannot open MediaSource.");
-              handleError("mse-decode", "Safari cannot open MediaSource.");
+              } else {
-            } else {
+                handleError("startup", "Error opening MediaSource.");
-              handleError("startup", "Error opening MediaSource.");
+              }
             }
           });
         },
@@ -532,13 +652,6 @@ function MSEPlayer({
       return;
     }
 
-    // iOS 17.1+ uses ManagedMediaSource
-    const MediaSourceConstructor =
-      "ManagedMediaSource" in window ? window.ManagedMediaSource : MediaSource;
-
-    // @ts-expect-error for typing
-    msRef.current = new MediaSourceConstructor();
-
     onConnect();
 
     return () => {