mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-09-24 18:26:51 +03:00
Improve History's seek startup time and recordings query performance (#24011)
* serve a segment startup ladder so seeks begin playing sooner nginx-vod was handed one 10s segment per recording file, so every playlist start had to download and decode a full segment before the first frame. Declare real keyframe data per clip and let nginx cut short leading segments from it. - add vod_bootstrap_segment_durations 1000/2000/4000 so each playlist starts with 1s/2s/4s segments before settling at 10s - emit real clip-relative keyFrameDurations (plus firstKeyFrameOffset when nonzero) from the recording keyframe index; rows without an index keep the whole-clip declaration, the only safe cut without keyframe knowledge - drop the manifest's segment_duration field, which was always inert: nginx-vod parses only camelCase segmentDuration - rebuild the player source at the seek target, quantized to a 10s grid, so the ladder applies to every seek and seek URLs stay repeatable for nginx's mapping and response caches - route the seek model, in-range checks, and the stale-report guard through the source window rather than the chunk range - bridge repositioning seeks (>2s from the last played timestamp) through the preview player and hold the release anchor one commit, so neither path paints a stale frame - clear a pending loading timer before replacing it; an orphaned timer escaped onPlaying's clearTimeout and flashed loading mid-playback * keep recordings queries on their indexes Several recordings queries degraded into full scans or large sorts on big databases: the planner ignored index order, or the query shape gave it nothing tight to seek on. Reshape them into bounded seeks and add the composite index the per-stream lookups need. - index recordings on (camera, stream_type, start_time DESC) and drop the (camera, stream_type) index it supersedes - walk the recordings summary day by day with EXISTS probes and per-camera MIN/MAX seeks, skipping ahead over empty gaps instead of bucketing every row for the requested cameras - run the summary endpoint on the event loop rather than the threadpool - bound the unavailable-recordings query by start_time per camera and merge the results in Python - bound the expire query's start_time so it seeks the retention window instead of scanning a camera's whole history - enumerate deleted cameras with one index seek each rather than a camera NOT IN (...) scan - compute bandwidth with segment_size filtered in a CASE projection; as a WHERE predicate it baited the planner into the (camera, segment_size) index plus a full sort of the camera's history - fall back to a 1000-segment window when the recent 100 are all zero-size, so an ingest glitch doesn't report zero bandwidth - limit the needs_refresh count instead of counting every segment - cover sub-only and sparse calendar days, midnight-spanning day attribution, multi-camera gap merging, deleted-camera expiry, and zero-size segment runs * fix mypy
This commit is contained in:
committed by
Nicolas Mowen
parent
50e7b76eb5
commit
e77155a4f1
@@ -0,0 +1,43 @@
|
||||
"""Peewee migrations -- 037_add_recordings_stream_index.py.
|
||||
|
||||
Some examples (model - class or model name)::
|
||||
|
||||
> Model = migrator.orm['model_name'] # Return model in current state by name
|
||||
|
||||
> migrator.sql(sql) # Run custom SQL
|
||||
> migrator.python(func, *args, **kwargs) # Run python code
|
||||
> migrator.create_model(Model) # Create a model (could be used as decorator)
|
||||
> migrator.remove_model(model, cascade=True) # Remove a model
|
||||
> migrator.add_fields(model, **fields) # Add fields to a model
|
||||
> migrator.change_fields(model, **fields) # Change fields
|
||||
> migrator.remove_fields(model, *field_names, cascade=True)
|
||||
> migrator.rename_field(model, old_field_name, new_field_name)
|
||||
> migrator.rename_table(model, new_table_name)
|
||||
> migrator.add_index(model, *col_names, unique=False)
|
||||
> migrator.drop_index(model, *col_names)
|
||||
> migrator.add_not_null(model, *field_names)
|
||||
> migrator.drop_not_null(model, *field_names)
|
||||
> migrator.add_default(model, field_name, default)
|
||||
|
||||
"""
|
||||
|
||||
import peewee as pw
|
||||
|
||||
SQL = pw.SQL
|
||||
|
||||
|
||||
def migrate(migrator, database, fake=False, **kwargs):
|
||||
# time-ordered per-stream lookups (bandwidth estimation, pinned-stream
|
||||
# queries) need this ordering; without it, a per-stream ORDER BY
|
||||
# start_time query on a camera with no rows for that stream walks the
|
||||
# camera's entire history via per-row table lookups
|
||||
migrator.sql(
|
||||
'CREATE INDEX IF NOT EXISTS "recordings_camera_stream_type_start_time" ON "recordings" ("camera", "stream_type", "start_time" DESC)'
|
||||
)
|
||||
# the 036 index is a strict prefix of the one above, so keeping it
|
||||
# would only add write amplification on the hottest table
|
||||
migrator.sql('DROP INDEX IF EXISTS "recordings_camera_stream_type"')
|
||||
|
||||
|
||||
def rollback(migrator, database, fake=False, **kwargs):
|
||||
pass
|
||||
Reference in New Issue
Block a user