Misc backend performance improvements (#23244)

* Add additional indicies on event and review tables.  Every events or timeline endpoint filters on event start time and camera, this should speed things up by avoiding a range scan on the table.

* Rewrite to use a CTE to leverage speedups by using sqllite internal optimization to do a single query instead of a starter query to get distinct labels and a subsequent loop of querys per distinct event labels.

Frigate is currently shipping sqlite 3.46.1, which is above the minimum version 3.25 needed for CTEs.

* Collapse a few sequential queries into a single one.

* Use peewee instead of rw sql for the CTE query.

* Slightly simplify review logic and avoid duplicating the json response for empty review IDs.

* Rerun ruff formatting.

* Remove 2x unnecessary index on reviewsegment, remove reference to prior code implementation in comment in event.py

* Editor fail, re-ruff format.

* Remove the CTE and restore the generator with sub-queries, which is more performance (thanks Nick and Blake for testing against your larger DB!)

* Update peewee index migration description

* Add on_conflict_ignore, replacing the try/catch/pass on IntegrityError

* Add a testcase for validating that on_conflict_ignore bypasses what was formerly an IntegrityError

* Change testcase to clarify that it covers the peewee behavior of on_conflict_ignore

---------

Co-authored-by: Greg <{ID}+{username}@users.noreply.github.com>
This commit is contained in:
gwmullin
2026-09-12 07:30:04 -06:00
committed by Nicolas Mowen
co-authored by Greg
parent f3a31e2fb4
commit 11b4d34f93
4 changed files with 147 additions and 46 deletions
+37
View File
@@ -497,6 +497,43 @@ class TestHttpReview(BaseTestHttp):
)
assert user_review.has_been_reviewed == True
def test_reviews_concurrent_insert_peewee_ignore(self):
"""Validates that on_conflict_ignore() silently skips a duplicate insert
Two requests can both SELECT and find no existing status, then both try
to INSERT, hitting the unique (user_id, review_segment) constraint.
on_conflict_ignore() must silently skip the duplicate instead of raising
an IntegrityError (which was previously caught with try/except).
"""
id = "123456.random"
with AuthTestClient(self.app):
super().insert_mock_review_segment(id)
# Simulate the first request having already committed its insert.
self._insert_user_review_status(id, reviewed=True)
# Simulate the second concurrent request attempting the same insert.
UserReviewStatus.insert_many(
[
{
"user_id": self.user_id,
"review_segment_id": id,
"has_been_reviewed": True,
}
]
).on_conflict_ignore().execute()
# Exactly one row should exist; no exception should have been raised.
count = (
UserReviewStatus.select()
.where(
(UserReviewStatus.user_id == self.user_id)
& (UserReviewStatus.review_segment == id)
)
.count()
)
assert count == 1
####################################################################################################################
################################### POST reviews/delete Endpoint ################################################
####################################################################################################################