fix clip download deadlock from unread ffmpeg stderr (#24032)

ffmpeg's stderr was piped but never read, so recording segments that generate more than 64 KB of ffmpeg warnings blocked ffmpeg mid-write, stranding the streaming thread and its anyio threadpool token for good. Enough of those and every sync endpoint stops responding until restart. The trigger is how noisy the segments are, not how long the clip is.

Send stderr to a temp file instead, and guarantee ffmpeg teardown and playlist cleanup on every exit path, including client disconnect.

Also fixes two bugs the deadlock hid: the failure branch was unreachable because returncode is None mid-loop, so the playlist file leaked and ffmpeg's logs were never reported. Playlist files now get a unique name so concurrent requests for one range cannot delete each other's input.

Extracts the terminate helper motion search already had into frigate/util/ffmpeg.py, now shared by both streaming call sites.
This commit is contained in:
Josh Hawkins
2026-08-27 20:30:35 -05:00
parent a94b532655
commit fa4002cbe2
4 changed files with 253 additions and 43 deletions
+2 -20
View File
@@ -17,6 +17,7 @@ import numpy as np
from frigate.config import CameraConfig
from frigate.ffmpeg_presets import parse_preset_hardware_acceleration_decode
from frigate.util.ffmpeg import terminate_ffmpeg_stream
from frigate.util.services import auto_detect_hwaccel
logger = logging.getLogger(__name__)
@@ -88,25 +89,6 @@ def _read_exact(stream: IO[bytes], size: int) -> bytes | None:
return bytes(buf)
def _terminate(proc: sp.Popen[bytes]) -> None:
"""Stop an ffmpeg decode process promptly."""
# Close the read end first so a blocked ffmpeg write unblocks (ffmpeg then
# sees a broken pipe), then signal it. The resulting ffmpeg write error is
# harmless and goes to the captured stderr.
if proc.stdout is not None:
try:
proc.stdout.close()
except OSError:
pass
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except sp.TimeoutExpired:
proc.kill()
proc.wait()
KEYFRAME_MAX_GAP_SECONDS = 2.0
@@ -222,7 +204,7 @@ def _run_vod_decode(
count += 1
yield frame
finally:
_terminate(proc)
terminate_ffmpeg_stream(proc)
stderr_file.close()
if count == 0 and software_retry and not should_stop():