Misc backend performance improvements (#23244)

* Add additional indicies on event and review tables.  Every events or timeline endpoint filters on event start time and camera, this should speed things up by avoiding a range scan on the table.

* Rewrite to use a CTE to leverage speedups by using sqllite internal optimization to do a single query instead of a starter query to get distinct labels and a subsequent loop of querys per distinct event labels.

Frigate is currently shipping sqlite 3.46.1, which is above the minimum version 3.25 needed for CTEs.

* Collapse a few sequential queries into a single one.

* Use peewee instead of rw sql for the CTE query.

* Slightly simplify review logic and avoid duplicating the json response for empty review IDs.

* Rerun ruff formatting.

* Remove 2x unnecessary index on reviewsegment, remove reference to prior code implementation in comment in event.py

* Editor fail, re-ruff format.

* Remove the CTE and restore the generator with sub-queries, which is more performance (thanks Nick and Blake for testing against your larger DB!)

* Update peewee index migration description

* Add on_conflict_ignore, replacing the try/catch/pass on IntegrityError

* Add a testcase for validating that on_conflict_ignore bypasses what was formerly an IntegrityError

* Change testcase to clarify that it covers the peewee behavior of on_conflict_ignore

---------

Co-authored-by: Greg <{ID}+{username}@users.noreply.github.com>
This commit is contained in:
gwmullin
2026-09-12 07:30:04 -06:00
committed by Nicolas Mowen
co-authored by Greg
parent f3a31e2fb4
commit 11b4d34f93
4 changed files with 147 additions and 46 deletions
+21
View File
@@ -0,0 +1,21 @@
"""Peewee migrations -- 036_add_perf_indexes.py.
Adds composite/single-column indexes to speed up single-camera queries
issued by the web UI.
"""
import peewee as pw
SQL = pw.SQL
def migrate(migrator, database, fake=False, **kwargs):
migrator.sql(
'CREATE INDEX IF NOT EXISTS "event_camera_start_time" '
'ON "event" ("camera", "start_time" DESC)'
)
def rollback(migrator, database, fake=False, **kwargs):
migrator.sql('DROP INDEX IF EXISTS "event_camera_start_time"')