Miscellaneous fixes (#24498)

* fix auto quality recovery after a downswitch

The downswitch callback armed the upswitch probe, but `triggerDownswitch` reset the stall history right after the callback returned, which disarmed it again. Auto quality stayed on the sub stream until the next chunk boundary no matter how much the connection recovered. The governor now arms the probe itself after the reset.

The chunk boundary effect also ran on mount, so when coverage was already cached it immediately undid the low quality cold start the seed effect had just picked. It now only runs when the chunk actually changes.

* fix recording playback quality switches and silent codec failures

A quality switch changes `bufferLength` a commit before the new source arrives, and since it was a dependency of the hls.js setup effect, the player rebuilt once on the outgoing playlist (jumping back to its original `startPosition`) and again on the new one. The buffer length now updates the running instance's config instead.

A fatal codec error with no lower quality stream to fall back to did nothing at all, so playback just sat there with no message. It now shows the playback failure toast, which is also limited to once per source since hls.js and the video element can both report the same failure.

* retry failed WebRTC probes and skip offline streams

The connectivity probe only ever tried the first go2rtc stream and cached its result for the whole page session, so a single offline camera at the top of the go2rtc config marked WebRTC unreachable for every camera until a reload, as did any brief network hiccup. The probe now starts with the stream being viewed and moves on to the next one when go2rtc reports that it can't open the stream's source. A failed result is only reused for 30 seconds, and it's retried on the next mount or when the page becomes visible again.

* fix two-way talk on cameras with AAC audio

The mic button was enabled whenever WebRTC was globally available, but the live view only switched to the WebRTC player when the stream itself qualified for WebRTC, and AAC playback audio disqualifies it. On most cameras the mic showed as on while nothing was sent. Two-way talk only needs the stream's video to connect since the backchannel is sent, not received, so AAC playback audio no longer blocks it. The mic is also turned off when a stream switch makes talk unavailable.

* keep Frigate+ model references when saving the models section

`/api/config` served a Frigate+ model's path as the resolved `/config/model_cache/<id>` file, and since the models list is saved whole, editing any model in the settings UI wrote that cache path back to the config in place of `plus://<id>`. After a restart the model loaded as a custom model with the default labelmap. The config API now reports the `plus://<id>` reference the model was configured with, and the models section drops the fields the Frigate+ model info supplies (size, tensor, pixel format, dtype, and type) instead of pinning them in the config.

* allow models to share shareable detection hardware

The hardware picker treated every device another model listed as taken, so a second model couldn't pick an Intel GPU or the CPU that the first one already used. The backend only rejects reuse of devices that can't be shared (Coral, MemryX), so the picker now matches that and only marks exclusive units as claimed.

* fix model card state and camera counts in the models editor

Model cards were keyed by index, so deleting one handed its state (such as the selected model source tab) to the card after it. Cards are keyed by scene now, which is unique per model.

The camera count on each card also ignored the backend's fallback to the `all` model, so a camera whose detect scene had no model of its own wasn't counted anywhere. It's counted under `all` now, which also feeds the recommended detector count.

* share a unit's temperature across repeated detector devices

Detector temperatures were matched to units by counting detectors of each type, so a device listed twice to run a second inference process (`hailo:PCIe` and `hailo:PCIe#2`) showed the next unit's temperature, or none at all. Distinct devices are numbered now and repeats share their unit's reading.

* update monitored hardware after a runtime config swap

`swap_runtime_config` rebound the stats emitter to the new config but not its `HardwareStats`, which kept polling hardware for the old config and applied camera updates to the discarded object. It now follows the swap along with its camera update subscriber.

* time out model downloads that never respond

`download_from_url` had no timeout, so a proxy or server that accepted the connection and never answered hung the download forever, including runtime downloads during startup. Connect and read timeouts now fail it like any other download error. The read timeout applies per socket read, so large models still finish.

* resolve segment start times in segment order

A camera stream's cached segments are probed concurrently, and each one chained its start off `last_segment_end` as soon as its own probe finished. When segments backed up in the cache and a later probe finished first, it chained off the wrong segment and the earlier one then moved `last_segment_end` backwards, so rows lost their exact adjacency. Probes still run concurrently, but each segment now waits for the one before it to settle its start before resolving its own.

* plan exports from the same coverage the vod route serves

The vod manifest nulls video-only glitch rows on audio-bearing streams, but exports planned their stream runs from the raw coverage, so a glitch row could produce a mixed-stream file or a 404 that failed the export. `null_audio_glitches` now works out each stream's audio composition itself, and exports go through it like the manifest and its realized timelines do.

An unstaged auto export also paged its playlist and chapters over main whenever main had any rows in range, even when the manifest served the range from sub and main only contributed glitches or slivers at the edges. It now reads the rows of the stream its single run actually uses.

* keep staged export chapters aligned across stream hand-offs

Each staged run of a mixed-stream export is rendered from its own pinned vod playlist, and that playlist's first clip snaps back to the preceding keyframe, so every staged file runs up to a GOP longer than its slice of the merged timeline. Chapters were placed on the merged timeline, so they drifted further from the video at every hand-off. Chapter windows for staged exports are now planned the same way each run's playlist is, carrying that keyframe lead-in into the offsets.

* clarify which hardware units only one model can use

* add e2e tests for shareable hardware and Frigate+ model saves

* only show the path field for a Frigate+ model without an API key

Without `PLUS_API_KEY` the models editor has no Frigate+ tab, so a `plus://` model showed every custom model field. The size, format, type, and labelmap fields are all supplied by the Frigate+ model info and ignored for a Frigate+ model, so editing them did nothing. Only the path is shown now, which still lets the model be switched to a custom one.

* keep a configured input_dtype when saving a Frigate+ model

The backend only overwrites `input_dtype` when the Frigate+ model info supplies `inputDataType`, which older models don't, so a configured dtype still matters for them. Saving the models section was dropping it along with the fields the backend always overwrites.

* probe every go2rtc stream until one isn't refused

The probe stopped after three streams, so with three offline cameras ahead of a working one, WebRTC was marked unreachable everywhere. It only moves past a stream when go2rtc refuses it, which is quick, and a stream that hangs still ends the probe at its timeout, so the cap bought nothing.

* only reuse a failed WebRTC probe for the stream it started from

A failed probe was reused for 30 seconds by every caller, so a camera whose stream timed out kept WebRTC unavailable for the next camera viewed, including one opened while that probe was still running. A pass still counts for every stream since it proves the connection, but a failure is only reused by probes that start from the same stream.

* strip input_dtype from Frigate+ models again

A Frigate+ model's config comes entirely from its model info, and a missing `inputDataType` means the `int` default. Keeping `input_dtype` meant switching from a custom model with `input_dtype: float` to a Frigate+ model carried the stale dtype over, with the field hidden so it couldn't be corrected.

web/e2e/specs/settings/detection-models.spec.ts

@@ -217,6 +217,23 @@ test.describe("Detection models settings @high", () => {
     ).toBeDisabled();
   });
 
+  test("shareable hardware another model uses can still be picked", async ({
+    frigateApp,
+  }) => {
+    await installRoutes(frigateApp.page, [
+      { scene: "all", devices: ["openvino:GPU.0"] },
+      { scene: "outdoor", devices: ["openvino:GPU.1"] },
+    ]);
+    await openPage(frigateApp);
+
+    await expect(
+      frigateApp.page.locator("#models-0-openvino\\:GPU\\.1").first(),
+    ).toBeEnabled();
+    await expect(frigateApp.page.locator("#pageRoot")).not.toContainText(
+      "used by outdoor",
+    );
+  });
+
   test("adding a model appends a card with an unused scene", async ({
     frigateApp,
   }) => {
@@ -248,15 +265,13 @@ test.describe("Detection models settings @high", () => {
   test("a saved Frigate+ model opens on the Frigate+ tab", async ({
     frigateApp,
   }) => {
-    // the backend resolves plus:// to a cache path before serving the config
-    // back, so the plus metadata is the only signal the model is a Plus one
     await installRoutes(
       frigateApp.page,
       [
         {
           scene: "all",
           devices: ["openvino:GPU.0"],
-          path: "/config/model_cache/abc123",
+          path: "plus://abc123",
           plus: PLUS_MODEL,
         },
       ],
@@ -302,6 +317,65 @@ test.describe("Detection models settings @high", () => {
     expect(saves.at(-1)?.config_data?.models?.[0].path).toBe("plus://abc123");
   });
 
+  test("saving a Frigate+ model keeps its reference without the Frigate+ fields", async ({
+    frigateApp,
+  }) => {
+    // the backend fills these in from the Frigate+ model info when it loads
+    const saves = await installRoutes(
+      frigateApp.page,
+      [
+        {
+          scene: "all",
+          devices: ["openvino:GPU.0"],
+          path: "plus://abc123",
+          plus: PLUS_MODEL,
+          width: 320,
+          height: 320,
+          input_tensor: "nchw",
+          input_dtype: "float",
+          model_type: "yolo-generic",
+        },
+      ],
+      true,
+    );
+    await openPage(frigateApp);
+
+    await frigateApp.page.locator("#models-0-openvino\\:GPU\\.1").click();
+    await frigateApp.page.getByRole("button", { name: /^Save$/ }).click();
+    await expect.poll(() => saves.length).toBeGreaterThan(0);
+
+    const model = saves.at(-1)?.config_data?.models?.[0];
+    expect(model?.path).toBe("plus://abc123");
+    expect(model?.devices).toEqual(["openvino:GPU.0", "openvino:GPU.1"]);
+    expect(model).not.toHaveProperty("width");
+    expect(model).not.toHaveProperty("input_tensor");
+    expect(model).not.toHaveProperty("model_type");
+    // a leftover dtype from a custom model must not override the int default
+    expect(model).not.toHaveProperty("input_dtype");
+  });
+
+  test("a Frigate+ model only shows its path without a Frigate+ API key", async ({
+    frigateApp,
+  }) => {
+    await installRoutes(frigateApp.page, [
+      {
+        scene: "all",
+        devices: ["openvino:GPU.0"],
+        path: "plus://abc123",
+        width: 320,
+        height: 320,
+      },
+    ]);
+    await openPage(frigateApp);
+
+    const root = frigateApp.page.locator("#pageRoot");
+    await expect(root).toContainText("Custom object detector model path");
+    await expect(root).not.toContainText("Object detection model input width");
+    await expect(root).not.toContainText(
+      "Label map for custom object detector",
+    );
+  });
+
   test("a Frigate+ Hailo model is listed by the device it was built for", async ({
     frigateApp,
   }) => {

web/public/locales/en/views/settings.json

@@ -2063,7 +2063,7 @@
       "unrecognized": "This model is configured for hardware that was not found on this system: {{devices}}",
       "description": "The hardware this model runs its detection on.",
       "detectorCountDescription": "How many detection processes to run on this hardware. More detectors keep up with more cameras, at the cost of extra device memory.",
-      "unitsDescription": "Each unit runs its own detection process. A unit already used by another model can not be selected."
+      "unitsDescription": "Each unit runs its own detection process. A unit that can't be shared, such as a Coral, can only be used by one model."
     },
     "tabs": {
       "plus": "Frigate+",

web/src/components/config-form/sections/section-special-cases.ts

@@ -7,6 +7,7 @@
  */
 
 import { RJSFSchema } from "@rjsf/utils";
+import { omit } from "lodash";
 import { applySchemaDefaults } from "@/lib/config-schema";
 import { isJsonObject } from "@/lib/utils";
 import { HiddenFieldContext, JsonObject, JsonValue } from "@/types/configForm";
@@ -352,6 +353,18 @@ export function synthesizeMissingFilters(
   return { ...(data as JsonObject), filters: newFilters };
 }
 
+// a Frigate+ model's config comes from its model info, so these are either
+// redundant or stale leftovers from a custom model. A missing input_dtype
+// in the model info means the int default.
+const PLUS_SUPPLIED_MODEL_FIELDS = [
+  "width",
+  "height",
+  "input_tensor",
+  "input_pixel_format",
+  "input_dtype",
+  "model_type",
+];
+
 /**
  * Sanitize overrides payloads for section-specific quirks.
  */
@@ -360,6 +373,21 @@ export function sanitizeOverridesForSection(
   level: string,
   overrides: unknown,
 ): unknown {
+  // the models list is saved whole
+  if (sectionPath === "models" && Array.isArray(overrides)) {
+    return overrides.map((model) => {
+      if (
+        !isJsonObject(model) ||
+        typeof model.path !== "string" ||
+        !model.path.startsWith("plus://")
+      ) {
+        return model;
+      }
+
+      return omit(model, PLUS_SUPPLIED_MODEL_FIELDS);
+    });
+  }
+
   if (!overrides || !isJsonObject(overrides)) {
     return overrides;
   }

web/src/components/config-form/theme/fields/HardwarePicker.tsx

@@ -21,7 +21,8 @@ type HardwarePickerProps = {
   // scopes the unit checkbox ids, since several models can list the same unit
   idPrefix: string;
   devices: string[];
-  // device strings already taken by another model, mapped to that model's scene
+  // device strings already taken by another model, mapped to that model's
+  // scene. Only binding for hardware that can't be shared.
   claimedElsewhere: Record<string, string>;
   cameraCount: number;
   disabled?: boolean;
@@ -84,8 +85,11 @@ export function HardwarePicker({
         return;
       }
 
-      // start with the first unit no other model has taken
-      const free = entry.units.find((unit) => !claimedElsewhere[unit.device]);
+      // start with the first unit no other model has taken, falling back to
+      // a taken one when the hardware can be shared
+      const free =
+        entry.units.find((unit) => !claimedElsewhere[unit.device]) ??
+        (entry.unlimited ? entry.units[0] : undefined);
 
       if (!free) {
         onChange([]);
@@ -184,7 +188,9 @@ export function HardwarePicker({
             {t("detectionModels.hardware.unitsDescription")}
           </p>
           {selected.units.map((unit) => {
-            const claimedBy = claimedElsewhere[unit.device];
+            const claimedBy = selected.unlimited
+              ? undefined
+              : claimedElsewhere[unit.device];
 
             return (
               <div

web/src/components/config-form/theme/fields/ModelSourcePicker.tsx

@@ -38,9 +38,7 @@ function plusModelId(path: unknown): string | undefined {
 
 type ModelSourcePickerProps = {
   path: unknown;
-  // Frigate+ metadata the backend attaches to a saved model, and the only
-  // reliable signal that one is active: it resolves `plus://<id>` to a local
-  // cache path before serving the config back
+  // Frigate+ metadata the backend attaches to a saved model
   plus?: { id: string } | null;
   // the detector this model runs on, used to filter incompatible Plus models
   detector?: string;

web/src/components/config-form/theme/fields/ModelsField.tsx

@@ -60,6 +60,15 @@ const CUSTOM_MODEL_FIELDS = [
   "model_type",
 ];
 
+/**
+ * The fields a model can edit. A Frigate+ model's size, format, type, and
+ * labels come from its model info, so only its path stays editable.
+ */
+const editableFields = (model: DetectionModel): string[] =>
+  typeof model.path === "string" && model.path.startsWith("plus://")
+    ? ["path"]
+    : CUSTOM_MODEL_FIELDS;
+
 /** The detector a model runs on, which is the prefix of its device strings. */
 const detectorForModel = (model: DetectionModel): string | undefined =>
   model.devices?.[0]?.split(":")[0];
@@ -169,20 +178,23 @@ export function ModelsField(props: FieldProps) {
     [savedModels],
   );
 
-  // a model serves the cameras naming its scene, plus every camera that names
-  // no scene at all when it is the "all" model
+  // a model serves the cameras naming its scene, and like the backend, the
+  // "all" model also serves every camera whose scene has no model of its own
   const cameraCountForScene = useCallback(
     (scene: string | undefined): number => {
       if (!cameras) {
         return 0;
       }
 
+      const modelScenes = new Set(models.map((model) => model.scene ?? "all"));
+
       return Object.values(cameras).filter((camera) => {
-        const cameraScene = camera?.detect?.scene;
-        return cameraScene ? cameraScene === scene : scene === "all";
+        const cameraScene = camera?.detect?.scene ?? "all";
+        const servedBy = modelScenes.has(cameraScene) ? cameraScene : "all";
+        return servedBy === (scene ?? "all");
       }).length;
     },
-    [cameras],
+    [cameras, models],
   );
 
   const claimedByOtherModels = useCallback(
@@ -314,7 +326,9 @@ export function ModelsField(props: FieldProps) {
         );
 
         return (
-          <Card key={`${baseId}-${index}`} className="w-full">
+          // keyed by scene, which is unique per model, so deleting a card
+          // doesn't hand its state (such as the model source tab) to the next
+          <Card key={`${baseId}-${model.scene ?? index}`} className="w-full">
             <Collapsible
               open={open}
               onOpenChange={(nextOpen) =>
@@ -428,7 +442,7 @@ export function ModelsField(props: FieldProps) {
                     detector={detectorForModel(model)}
                     disabled={disabled || readonly}
                     onPathChange={(path) => updateModel(index, { path })}
-                    customFields={CUSTOM_MODEL_FIELDS.map((fieldName) =>
+                    customFields={editableFields(model).map((fieldName) =>
                       renderField(index, fieldName),
                     )}
                   />

web/src/components/player/HlsVideoPlayer.tsx

@@ -26,6 +26,7 @@ import { useIsAdmin } from "@/hooks/use-is-admin";
 // Android native hls does not seek correctly
 const USE_NATIVE_HLS = false;
 const HLS_MIME_TYPE = "application/vnd.apple.mpegurl" as const;
+const DEFAULT_MAX_BUFFER_LENGTH_S = 10;
 const unsupportedErrorCodes: number[] = [
   MediaError.MEDIA_ERR_SRC_NOT_SUPPORTED,
   MediaError.MEDIA_ERR_DECODE,
@@ -148,6 +149,42 @@ export default function HlsVideoPlayer({
   // a ref rather than an effect-scoped counter so the element error
   // handler can hold its toast while a recovery is still possible
   const mediaRecoveryBudgetRef = useRef(0);
+  // hls.js and the element can both report one failure, so each source
+  // toasts at most once
+  const failureReportedRef = useRef(false);
+
+  const reportPlaybackFailure = useCallback(
+    (code: number, message: string) => {
+      if (failureReportedRef.current) {
+        return;
+      }
+
+      failureReportedRef.current = true;
+      toast.error(t("toast.error.playRecordingsFailed", { code, message }), {
+        position: "top-center",
+      });
+    },
+    [t],
+  );
+  const reportPlaybackFailureRef = useRef(reportPlaybackFailure);
+
+  useEffect(() => {
+    reportPlaybackFailureRef.current = reportPlaybackFailure;
+  }, [reportPlaybackFailure]);
+
+  // a quality switch changes the buffer length a commit before the new
+  // source arrives, so it updates the live instance instead of rebuilding
+  // it on the outgoing playlist
+  const bufferLengthRef = useRef(bufferLength);
+
+  useEffect(() => {
+    bufferLengthRef.current = bufferLength;
+
+    if (hlsRef.current) {
+      hlsRef.current.config.maxBufferLength =
+        bufferLength ?? DEFAULT_MAX_BUFFER_LENGTH_S;
+    }
+  }, [bufferLength]);
 
   const applyVideoDimensions = useCallback(
     (width: number, height: number) => {
@@ -225,6 +262,7 @@ export default function HlsVideoPlayer({
     // the element already holds a decoded frame, and keeping it visible
     // bridges the gap while the new source loads
     const currentPlaybackRate = videoRef.current.playbackRate;
+    failureReportedRef.current = false;
 
     if (!useHlsCompat) {
       nativeRetryRef.current = 0;
@@ -236,7 +274,7 @@ export default function HlsVideoPlayer({
 
     // Base HLS configuration
     const hlsConfig: Partial<HlsConfig> = {
-      maxBufferLength: bufferLength ?? 10,
+      maxBufferLength: bufferLengthRef.current ?? DEFAULT_MAX_BUFFER_LENGTH_S,
       maxBufferSize: 20 * 1000 * 1000,
       startPosition: currentSource.startPosition,
     };
@@ -269,10 +307,18 @@ export default function HlsVideoPlayer({
             data.details ===
               Hls.ErrorDetails.BUFFER_INCOMPATIBLE_CODECS_ERROR ||
             data.details === Hls.ErrorDetails.BUFFER_ADD_CODEC_ERROR;
-          if (isCodecError && qualitySignalsRef.current.onFatalCodecError?.()) {
+          if (isCodecError) {
+            // with no stream to fall back to, the element may never raise
+            // an error of its own, so report the failure here
+            if (!qualitySignalsRef.current.onFatalCodecError?.()) {
+              reportPlaybackFailureRef.current(
+                MediaError.MEDIA_ERR_SRC_NOT_SUPPORTED,
+                data.details,
+              );
+            }
             return;
           }
-          if (!isCodecError && mediaRecoveryBudgetRef.current > 0) {
+          if (mediaRecoveryBudgetRef.current > 0) {
             mediaRecoveryBudgetRef.current -= 1;
             hls.recoverMediaError();
           }
@@ -308,7 +354,7 @@ export default function HlsVideoPlayer({
         hlsRef.current.destroy();
       }
     };
-  }, [videoRef, hlsRef, useHlsCompat, currentSource, bufferLength]);
+  }, [videoRef, hlsRef, useHlsCompat, currentSource]);
 
   // state handling
 
@@ -714,15 +760,7 @@ export default function HlsVideoPlayer({
                 }
               }
 
-              toast.error(
-                t("toast.error.playRecordingsFailed", {
-                  code: mediaError.code,
-                  message: mediaError.message,
-                }),
-                {
-                  position: "top-center",
-                },
-              );
+              reportPlaybackFailure(mediaError.code, mediaError.message);
             }}
           />
         </div>

web/src/components/player/dynamic/AutoQualityGovernor.ts

@@ -340,8 +340,11 @@ export class AutoQualityGovernor {
   private triggerDownswitch(reason: DownswitchReason): boolean {
     const handled = this.requestDownswitch(reason);
     if (handled) {
-      // the low stream starts with a clean record
+      // the low stream starts with a clean record, and the probe lets a
+      // recovered connection (or a wrong downswitch) return to full
+      // quality mid-chunk rather than at the next boundary
       this.resetStallHistory();
+      this.armUpswitchProbe();
     }
     return handled;
   }

web/src/components/player/dynamic/DynamicVideoPlayer.tsx

@@ -484,9 +484,6 @@ export default function DynamicVideoPlayer({
       }
       setAutoLowQuality(true);
       setAutoLowReason(reason === "codec" ? "codec" : "bandwidth");
-      // so a recovered connection (or a wrong downswitch) returns to
-      // full quality mid-chunk rather than at the next boundary
-      governor.armUpswitchProbe();
       return true;
     };
     tryUpswitchRef.current = () => {
@@ -495,7 +492,7 @@ export default function DynamicVideoPlayer({
         setAutoLowReason(undefined);
       }
     };
-  }, [resolvedQuality, subAvailable, governor]);
+  }, [resolvedQuality, subAvailable]);
 
   // persisted across sessions so a device on a known-slow connection
   // starts low instead of paying the first stall to find out
@@ -616,7 +613,19 @@ export default function DynamicVideoPlayer({
     governor.sourceLoadStarted();
   }, [source, isScrubbing, governor]);
 
+  const lastChunkRef = useRef(timeRange);
   useEffect(() => {
+    // the seed effect picks the starting quality, so only a later chunk
+    // boundary reconsiders it
+    const lastChunk = lastChunkRef.current;
+    lastChunkRef.current = timeRange;
+    if (
+      lastChunk.after === timeRange.after &&
+      lastChunk.before === timeRange.before
+    ) {
+      return;
+    }
+
     // a chunk boundary is where full quality may be retried, and a
     // natural point to persist what the governor has learned
     setAutoLowQuality((prev) => prev && !governor.shouldRetryMain());

web/src/hooks/use-webrtc-availability.ts

@@ -79,7 +79,6 @@ export function evaluateStreamWebRTCAvailability(args: {
 /** Console detail for a reason decided by the global availability check. */
 function describeGlobalReason(
   reason: WebRTCUnavailableReason,
-  testStream: string | undefined,
   probeDetail: string | undefined,
 ): string {
   switch (reason) {
@@ -88,7 +87,7 @@ function describeGlobalReason(
     case "not-configured":
       return "No candidates or ice_servers are set under go2rtc.webrtc.";
     case "unreachable":
-      return `The connectivity probe against stream '${testStream}' failed: ${probeDetail ?? "unknown cause"}.`;
+      return `The connectivity probe failed against ${probeDetail ?? "an unknown stream"}.`;
     default:
       return "";
   }
@@ -98,9 +97,13 @@ function describeGlobalReason(
 let lastProbeSignature: string | null = null;
 
 /**
- * Once-per-session: browser support, go2rtc config, and a live handshake probe.
+ * Browser support, go2rtc config, and a live handshake probe shared by the
+ * page. A failed probe is retried on a later mount or when the page becomes
+ * visible again once its cached result has expired.
  */
-export function useWebRTCGloballyAvailable(): GlobalAvailability {
+export function useWebRTCGloballyAvailable(
+  preferredStream?: string,
+): GlobalAvailability {
   const { data: config } = useSWR<FrigateConfig>("config");
   const [probe, setProbe] = useState<{
     state: "pending" | "pass" | "fail";
@@ -118,11 +121,19 @@ export function useWebRTCGloballyAvailable(): GlobalAvailability {
     );
   }, [config]);
 
-  // Representative restreamed stream to probe against.
-  const testStream = useMemo(() => {
-    const streams = config?.go2rtc?.streams ?? {};
-    return Object.keys(streams)[0];
-  }, [config]);
+  // the stream being viewed is the one most likely to be online
+  const testStreams = useMemo(() => {
+    const streams = Object.keys(config?.go2rtc?.streams ?? {});
+
+    if (!preferredStream || !streams.includes(preferredStream)) {
+      return streams;
+    }
+
+    return [preferredStream, ...streams.filter((s) => s !== preferredStream)];
+  }, [config, preferredStream]);
+  const testStreamsKey = testStreams.join("\n");
+
+  const [retryToken, setRetryToken] = useState(0);
 
   const iceServers = useMemo(
     () => webRTCIceServers(config?.go2rtc?.webrtc?.ice_servers),
@@ -136,9 +147,9 @@ export function useWebRTCGloballyAvailable(): GlobalAvailability {
       JSON.stringify({
         candidates: config?.go2rtc?.webrtc?.candidates ?? [],
         iceServers,
-        testStream,
+        streams: Object.keys(config?.go2rtc?.streams ?? {}),
       }),
-    [config, iceServers, testStream],
+    [config, iceServers],
   );
 
   useEffect(() => {
@@ -153,11 +164,29 @@ export function useWebRTCGloballyAvailable(): GlobalAvailability {
   }, [probeSignature]);
 
   useEffect(() => {
-    if (!browserOk || !configured || !testStream) {
+    if (probe.state !== "fail") {
+      return;
+    }
+
+    const onVisibilityChange = () => {
+      if (document.visibilityState === "visible") {
+        setRetryToken((token) => token + 1);
+      }
+    };
+
+    document.addEventListener("visibilitychange", onVisibilityChange);
+    return () =>
+      document.removeEventListener("visibilitychange", onVisibilityChange);
+  }, [probe.state]);
+
+  useEffect(() => {
+    const streams = testStreamsKey ? testStreamsKey.split("\n") : [];
+
+    if (!browserOk || !configured || streams.length === 0) {
       return;
     }
     let cancelled = false;
-    probeWebRTCAvailability(testStream, iceServers).then((result) => {
+    probeWebRTCAvailability(streams, iceServers).then((result) => {
       if (!cancelled) {
         setProbe({
           state: result.ok ? "pass" : "fail",
@@ -168,7 +197,7 @@ export function useWebRTCGloballyAvailable(): GlobalAvailability {
     return () => {
       cancelled = true;
     };
-  }, [browserOk, configured, testStream, iceServers]);
+  }, [browserOk, configured, testStreamsKey, iceServers, retryToken]);
 
   const availability = useMemo<GlobalAvailability>(() => {
     if (!browserOk) {
@@ -196,9 +225,9 @@ export function useWebRTCGloballyAvailable(): GlobalAvailability {
     logWebRTCUnavailable(
       undefined,
       reason,
-      describeGlobalReason(reason, testStream, probe.detail),
+      describeGlobalReason(reason, probe.detail),
     );
-  }, [config, availability, testStream, probe.detail]);
+  }, [config, availability, probe.detail]);
 
   return availability;
 }
@@ -208,7 +237,8 @@ export function useWebRTCAvailableForStream(
   metadata: LiveStreamMetadata | null | undefined,
   streamName?: string,
 ): StreamAvailability {
-  const { globallyAvailable, globalReason } = useWebRTCGloballyAvailable();
+  const { globallyAvailable, globalReason } =
+    useWebRTCGloballyAvailable(streamName);
 
   const availability = useMemo(
     () =>

web/src/utils/webrtcProbe.ts

@@ -1,18 +1,28 @@
 import { baseUrl } from "@/api/baseUrl";
 
 /**
- * Performs a single real WebRTC handshake against go2rtc to verify that a
- * media connection can actually be established (validates candidates, port
- * 8555 reachability, and STUN/TURN end-to-end). Result is cached per page
- * session via a module-level promise.
+ * Performs a real WebRTC handshake against go2rtc to verify that a media
+ * connection can actually be established (validates candidates, port 8555
+ * reachability, and STUN/TURN end-to-end). A success is cached for the page
+ * session, and a failure for PROBE_FAILURE_TTL_MS and only for probes that
+ * start from the same stream.
  */
 
 export type WebRTCProbeResult = {
   ok: boolean;
   detail?: string;
+  // go2rtc could not open the stream's source, which says nothing about
+  // whether WebRTC itself can connect
+  streamError?: boolean;
 };
 
-let probePromise: Promise<WebRTCProbeResult> | null = null;
+const PROBE_FAILURE_TTL_MS = 30_000;
+
+let cachedProbe: {
+  promise: Promise<WebRTCProbeResult>;
+  firstStream: string;
+  failedAt?: number;
+} | null = null;
 
 function describeError(err: unknown): string {
   return err instanceof Error ? err.message : String(err);
@@ -118,23 +128,83 @@ function runProbe(
         pc.setRemoteDescription({ type: "answer", sdp: msg.value }).catch(
           (err) => fail(`remote answer rejected: ${describeError(err)}`),
         );
+      } else if (msg.type === "error") {
+        cleanup({ ok: false, streamError: true, detail: msg.value });
       }
     });
   });
 }
 
+async function runProbes(
+  testStreams: string[],
+  iceServers: RTCIceServer[],
+  timeoutMs: number,
+): Promise<WebRTCProbeResult> {
+  let result: WebRTCProbeResult = { ok: false, detail: "no stream to probe" };
+
+  for (const stream of testStreams) {
+    result = await runProbe(stream, iceServers, timeoutMs);
+
+    if (!result.ok) {
+      result.detail = `stream '${stream}': ${result.detail}`;
+    }
+
+    if (!result.streamError) {
+      break;
+    }
+  }
+
+  return result;
+}
+
+/** Probes the streams in order, moving on only when go2rtc refuses one. */
 export function probeWebRTCAvailability(
-  testStream: string,
+  testStreams: string[],
   iceServers: RTCIceServer[],
   timeoutMs: number = 5000,
 ): Promise<WebRTCProbeResult> {
-  if (!probePromise) {
-    probePromise = runProbe(testStream, iceServers, timeoutMs);
+  const firstStream = testStreams[0];
+
+  if (cachedProbe) {
+    const { promise, failedAt } = cachedProbe;
+
+    if (
+      cachedProbe.firstStream === firstStream &&
+      (failedAt === undefined || Date.now() - failedAt < PROBE_FAILURE_TTL_MS)
+    ) {
+      return promise;
+    }
+
+    // a pass from any stream proves the connection, but another stream's
+    // failure says nothing about this one
+    if (failedAt === undefined) {
+      return promise.then((result) =>
+        result.ok
+          ? result
+          : probeWebRTCAvailability(testStreams, iceServers, timeoutMs),
+      );
+    }
   }
-  return probePromise;
+
+  const entry: NonNullable<typeof cachedProbe> = {
+    firstStream,
+    promise: runProbes(testStreams, iceServers, timeoutMs)
+      .catch((err): WebRTCProbeResult => ({
+        ok: false,
+        detail: describeError(err),
+      }))
+      .then((result) => {
+        if (!result.ok) {
+          entry.failedAt = Date.now();
+        }
+        return result;
+      }),
+  };
+  cachedProbe = entry;
+  return entry.promise;
 }
 
 /** Clears the cached probe result (e.g. when go2rtc config changes). */
 export function resetWebRTCProbe(): void {
-  probePromise = null;
+  cachedProbe = null;
 }

web/src/views/live/LiveCameraView.tsx

@@ -27,10 +27,7 @@ import {
 } from "@/components/ui/popover";
 import { useResizeObserver } from "@/hooks/resize-observer";
 import useKeyboardListener from "@/hooks/use-keyboard-listener";
-import {
-  useWebRTCAvailableForStream,
-  useWebRTCGloballyAvailable,
-} from "@/hooks/use-webrtc-availability";
+import { useWebRTCAvailableForStream } from "@/hooks/use-webrtc-availability";
 import { CameraConfig, FrigateConfig } from "@/types/frigateConfig";
 import {
   LivePlayerError,
@@ -219,15 +216,16 @@ export default function LiveCameraView({
   );
   const isWebRTCAvailable = webRTCAvailability.available;
 
-  // Two-way talk is the sendonly backchannel: global, not per-stream.
-  const { globallyAvailable: webRTCGloballyAvailable } =
-    useWebRTCGloballyAvailable();
-
   // "checking" means the probe has not answered yet, and it re-enters that on
   // every mount, so treating it as unavailable downgrades the saved choice.
   const webRTCVerdictPending = webRTCAvailability.reason === "checking";
   const webRTCUsable = isWebRTCAvailable || webRTCVerdictPending;
 
+  // Two-way talk is a sendonly backchannel, so playback audio that WebRTC
+  // can't carry doesn't rule it out, but the stream's video must connect
+  const talkAvailable =
+    isWebRTCAvailable || webRTCAvailability.reason === "audio-codec";
+
   // Resolves the saved preference without overwriting it. Transient error
   // fallbacks layer on top in preferredLiveMode.
   const resolvedUserMode = useMemo<LivePlayerMode>(() => {
@@ -398,6 +396,14 @@ export default function LiveCameraView({
 
   const [audio, setAudio] = useSessionPersistence("liveAudio", false);
   const [mic, setMic] = useState(false);
+
+  // the mic only connects through the WebRTC player, so it can't stay on
+  // for a stream that has lost it
+  useEffect(() => {
+    if (!talkAvailable) {
+      setMic(false);
+    }
+  }, [talkAvailable]);
   const [webRTC, setWebRTC] = useState(false);
   const [pip, setPip] = useState(false);
   const [lowBandwidth, setLowBandwidth] = useState(false);
@@ -424,7 +430,7 @@ export default function LiveCameraView({
   });
 
   const preferredLiveMode = useMemo(() => {
-    if (mic && isWebRTCAvailable) {
+    if (mic && talkAvailable) {
       return "webrtc";
     }
 
@@ -456,6 +462,7 @@ export default function LiveCameraView({
     lowBandwidth,
     forceLowBandwidth,
     mic,
+    talkAvailable,
     webRTC,
     isRestreamed,
     resolvedUserMode,
@@ -489,7 +496,7 @@ export default function LiveCameraView({
         }
         break;
       case "t":
-        if (supports2WayTalk) {
+        if (supports2WayTalk && talkAvailable) {
           setMic(!mic);
           return true;
         }
@@ -737,11 +744,15 @@ export default function LiveCameraView({
                 Icon={mic ? FaMicrophone : FaMicrophoneSlash}
                 isActive={mic}
                 title={
-                  !webRTCGloballyAvailable
-                    ? t("twoWayTalk.requiresWebRTC", { ns: "views/live" })
-                    : mic
-                      ? t("twoWayTalk.disable", { ns: "views/live" })
-                      : t("twoWayTalk.enable", { ns: "views/live" })
+                  webRTCVerdictPending
+                    ? t("stream.technology.unavailable.checking", {
+                        ns: "views/live",
+                      })
+                    : !talkAvailable
+                      ? t("twoWayTalk.requiresWebRTC", { ns: "views/live" })
+                      : mic
+                        ? t("twoWayTalk.disable", { ns: "views/live" })
+                        : t("twoWayTalk.enable", { ns: "views/live" })
                 }
                 onClick={() => {
                   setMic(!mic);
@@ -749,7 +760,7 @@ export default function LiveCameraView({
                     setAudio(true);
                   }
                 }}
-                disabled={!cameraEnabled || debug || !webRTCGloballyAvailable}
+                disabled={!cameraEnabled || debug || !talkAvailable}
               />
             )}
             {supportsAudioOutput && preferredLiveMode != "jsmpeg" && (