Improve History's seek startup time and recordings query performance (#24011)

* serve a segment startup ladder so seeks begin playing sooner

nginx-vod was handed one 10s segment per recording file, so every playlist start had to download and decode a full segment before the first frame. Declare real keyframe data per clip and let nginx cut short leading segments from it.

- add vod_bootstrap_segment_durations 1000/2000/4000 so each playlist starts with 1s/2s/4s segments before settling at 10s
- emit real clip-relative keyFrameDurations (plus firstKeyFrameOffset when nonzero) from the recording keyframe index; rows without an index keep the whole-clip declaration, the only safe cut without keyframe knowledge
- drop the manifest's segment_duration field, which was always inert: nginx-vod parses only camelCase segmentDuration
- rebuild the player source at the seek target, quantized to a 10s grid, so the ladder applies to every seek and seek URLs stay repeatable for nginx's mapping and response caches
- route the seek model, in-range checks, and the stale-report guard through the source window rather than the chunk range
- bridge repositioning seeks (>2s from the last played timestamp) through the preview player and hold the release anchor one commit, so neither path paints a stale frame
- clear a pending loading timer before replacing it; an orphaned timer escaped onPlaying's clearTimeout and flashed loading mid-playback

* keep recordings queries on their indexes

Several recordings queries degraded into full scans or large sorts on big databases: the planner ignored index order, or the query shape gave it nothing tight to seek on. Reshape them into bounded seeks and add the composite index the per-stream lookups need.

- index recordings on (camera, stream_type, start_time DESC) and drop the (camera, stream_type) index it supersedes
- walk the recordings summary day by day with EXISTS probes and per-camera MIN/MAX seeks, skipping ahead over empty gaps instead of bucketing every row for the requested cameras
- run the summary endpoint on the event loop rather than the threadpool
- bound the unavailable-recordings query by start_time per camera and merge the results in Python
- bound the expire query's start_time so it seeks the retention window instead of scanning a camera's whole history
- enumerate deleted cameras with one index seek each rather than a camera NOT IN (...) scan
- compute bandwidth with segment_size filtered in a CASE projection; as a WHERE predicate it baited the planner into the (camera, segment_size) index plus a full sort of the camera's history
- fall back to a 1000-segment window when the recent 100 are all zero-size, so an ingest glitch doesn't report zero bandwidth
- limit the needs_refresh count instead of counting every segment
- cover sub-only and sparse calendar days, midnight-spanning day attribution, multi-camera gap merging, deleted-camera expiry, and zero-size segment runs

* fix mypy
This commit is contained in:
Josh Hawkins
2026-09-12 07:30:04 -06:00
committed by Nicolas Mowen
parent 50e7b76eb5
commit e77155a4f1
12 changed files with 654 additions and 120 deletions
+39 -1
View File
@@ -310,11 +310,18 @@ class ClipPlan:
clip_from_ms is the keyframe-snapped clipFrom, or None when the whole
file is served. skipped means the clip is omitted from the manifest.
key_frame_durations / first_key_frame_offset_ms carry clip-relative
keyframe data for nginx-vod sub-file segmentation; None means no
usable index, and the manifest declares one whole-clip segment (the
only safe cut without keyframe knowledge).
"""
clip_from_ms: int | None
duration_ms: int
skipped: bool
key_frame_durations: list[int] | None = None
first_key_frame_offset_ms: int = 0
def plan_clip(row: Any, start: float, end: float) -> ClipPlan:
@@ -378,7 +385,38 @@ def plan_clip(row: Any, start: float, end: float) -> ClipPlan:
logger.warning(f"Recording clip is missing or empty: {row.path}")
return ClipPlan(None, 0, True)
return ClipPlan(clip_from, duration, False)
return ClipPlan(
clip_from,
duration,
False,
*_clip_key_frames(row.keyframes, clip_from, duration),
)
def _clip_key_frames(
keyframes: Any, clip_from: int | None, duration: int
) -> tuple[list[int] | None, int]:
"""Clip-relative keyframe gaps and first offset for a served range.
nginx-vod validates firstKeyFrameOffset against the clip duration,
so offsets are clip-relative. A keyframe exactly on the clip end is
excluded (it would declare a zero-length tail), and fewer than two
keyframes returns (None, 0): one cut point cannot split anything.
"""
if not keyframes:
return None, 0
clip_start = clip_from if clip_from is not None else 0
relative = [
int(k) - clip_start
for k in keyframes
if clip_start <= k < clip_start + duration
]
if len(relative) < 2:
return None, 0
return [b - a for a, b in zip(relative, relative[1:])], relative[0]
def realized_timeline(