frigate/frigate
Tom 36c61a3607 Corroborate record watchdog staleness against the cache dir before restarting
The record watchdog treats a stale maintainer heartbeat as a dead
recorder. But the heartbeat is published by the recording maintainer,
so whenever the maintainer lags (e.g. "Unable to keep up with recording
segments in cache", #9661) every camera looks stale at once and all
record processes restart together - while recording was actually
healthy. The restart churn then produces more, shorter segments,
making the maintainer fall further behind.

Before restarting on staleness, check the camera's newest cache
segment on disk: if a segment is fresher than the staleness threshold,
the recorder is demonstrably writing - log a warning, adopt the disk
mtime as the heartbeat, and skip the restart. The invalid-segment path
is untouched.

Validated on a 26-camera production deployment (0.17.1 backport of
this change): synchronized mass restarts went from 52/hour to zero,
with heartbeat-stale events still occurring ~2/hour but now correctly
identified as maintainer lag instead of recording failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 16:41:33 +01:00
..
api Fix chat tool calling and prompt breaking (#23457) 2026-06-12 07:48:43 -05:00
camera Debug replay resolution (#23287) 2026-05-22 08:39:52 -06:00
comms Restore runtime state on startup (#23326) 2026-05-27 12:03:09 -06:00
config Miscellaneous fixes (#23445) 2026-06-11 05:36:30 -06:00
data_processing Miscellaneous fixes (#23177) 2026-05-12 10:20:39 -06:00
db More mypy cleanup (#22658) 2026-03-26 12:54:12 -06:00
detectors Miscellaneous fixes (#23394) 2026-06-04 12:48:58 -06:00
embeddings Debug replay fixes (#23276) 2026-05-21 08:12:53 -06:00
events Miscellaneous fixes (#23335) 2026-05-29 06:53:17 -06:00
genai Miscellaneous fixes (#23295) 2026-05-24 06:48:52 -06:00
images Replace green screen with error message and force camera_fps to 0 (#4544) 2022-11-28 21:47:20 -06:00
jobs Refactor motion search (#23378) 2026-06-01 12:08:46 -05:00
motion Optimize more mypy classes (#22637) 2026-03-25 12:53:19 -06:00
object_detection Miscellaneous fixes (#23279) 2026-05-22 07:52:01 -06:00
output Refactor move_preview_frames function (#23264) 2026-05-20 10:52:47 -06:00
ptz Miscellaneous fixes (#23279) 2026-05-22 07:52:01 -06:00
record Upgrade ffmpeg to 8 by default (#23393) 2026-06-03 12:28:28 -05:00
review Inverse mypy and more mypy fixes (#22645) 2026-03-25 19:30:59 -05:00
service_manager Add metrics page for embeddings and face / license plate processing times (#15818) 2025-02-08 12:47:01 -06:00
stats Miscellaneous fixes (#23317) 2026-05-27 09:19:11 -06:00
test Add recording keyframe analysis to camera probe dialog (#23453) 2026-06-11 14:16:41 -06:00
track Miscellaneous fixes (#23238) 2026-05-18 22:52:40 -05:00
util Add recording keyframe analysis to camera probe dialog (#23453) 2026-06-11 14:16:41 -06:00
video Corroborate record watchdog staleness against the cache dir before restarting 2026-06-12 16:41:33 +01:00
__init__.py app container and config schema 2021-01-26 21:40:33 -06:00
__main__.py Handle SIGINT with forkserver (#18860) 2025-08-16 10:20:33 -05:00
app.py Miscellaneous fixes (#23358) 2026-05-30 21:35:03 -06:00
const.py Improve credential redaction handling (#23265) 2026-05-20 15:59:01 -06:00
debug_replay.py Miscellaneous fixes (#23317) 2026-05-27 09:19:11 -06:00
ffmpeg_presets.py Upgrade ffmpeg to 8 by default (#23393) 2026-06-03 12:28:28 -05:00
log.py Add languages (#21870) 2026-02-03 13:29:52 -06:00
models.py Improve motion review and add motion search (#22253) 2026-03-05 17:53:48 -06:00
mypy.ini More mypy cleanup (#22658) 2026-03-26 12:54:12 -06:00
plus.py fix: upload_image parses response body before checking HTTP status (#22475) 2026-03-16 17:34:30 -06:00
storage.py Improve readability by removing trailing digits caused by floating number conversion (#22934) 2026-04-20 06:35:48 -06:00
timeline.py Display more scores in Tracking Details (#22799) 2026-04-07 10:06:22 -06:00
types.py Media sync API refactor and UI (#21542) 2026-02-26 21:27:56 -07:00
watchdog.py Miscellaneous fixes (#23032) 2026-04-29 16:20:19 -06:00