mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-11-15 18:13:01 +03:00
Misc backend performance improvements (#23244)
* Add additional indicies on event and review tables. Every events or timeline endpoint filters on event start time and camera, this should speed things up by avoiding a range scan on the table.
* Rewrite to use a CTE to leverage speedups by using sqllite internal optimization to do a single query instead of a starter query to get distinct labels and a subsequent loop of querys per distinct event labels.
Frigate is currently shipping sqlite 3.46.1, which is above the minimum version 3.25 needed for CTEs.
* Collapse a few sequential queries into a single one.
* Use peewee instead of rw sql for the CTE query.
* Slightly simplify review logic and avoid duplicating the json response for empty review IDs.
* Rerun ruff formatting.
* Remove 2x unnecessary index on reviewsegment, remove reference to prior code implementation in comment in event.py
* Editor fail, re-ruff format.
* Remove the CTE and restore the generator with sub-queries, which is more performance (thanks Nick and Blake for testing against your larger DB!)
* Update peewee index migration description
* Add on_conflict_ignore, replacing the try/catch/pass on IntegrityError
* Add a testcase for validating that on_conflict_ignore bypasses what was formerly an IntegrityError
* Change testcase to clarify that it covers the peewee behavior of on_conflict_ignore
---------
Co-authored-by: Greg <{ID}+{username}@users.noreply.github.com>
This commit is contained in:
committed by
Nicolas Mowen
co-authored by
Greg
parent
f3a31e2fb4
commit
11b4d34f93
@@ -497,6 +497,43 @@ class TestHttpReview(BaseTestHttp):
|
||||
)
|
||||
assert user_review.has_been_reviewed == True
|
||||
|
||||
def test_reviews_concurrent_insert_peewee_ignore(self):
|
||||
"""Validates that on_conflict_ignore() silently skips a duplicate insert
|
||||
|
||||
Two requests can both SELECT and find no existing status, then both try
|
||||
to INSERT, hitting the unique (user_id, review_segment) constraint.
|
||||
on_conflict_ignore() must silently skip the duplicate instead of raising
|
||||
an IntegrityError (which was previously caught with try/except).
|
||||
"""
|
||||
id = "123456.random"
|
||||
with AuthTestClient(self.app):
|
||||
super().insert_mock_review_segment(id)
|
||||
|
||||
# Simulate the first request having already committed its insert.
|
||||
self._insert_user_review_status(id, reviewed=True)
|
||||
|
||||
# Simulate the second concurrent request attempting the same insert.
|
||||
UserReviewStatus.insert_many(
|
||||
[
|
||||
{
|
||||
"user_id": self.user_id,
|
||||
"review_segment_id": id,
|
||||
"has_been_reviewed": True,
|
||||
}
|
||||
]
|
||||
).on_conflict_ignore().execute()
|
||||
|
||||
# Exactly one row should exist; no exception should have been raised.
|
||||
count = (
|
||||
UserReviewStatus.select()
|
||||
.where(
|
||||
(UserReviewStatus.user_id == self.user_id)
|
||||
& (UserReviewStatus.review_segment == id)
|
||||
)
|
||||
.count()
|
||||
)
|
||||
assert count == 1
|
||||
|
||||
####################################################################################################################
|
||||
################################### POST reviews/delete Endpoint ################################################
|
||||
####################################################################################################################
|
||||
|
||||
Reference in New Issue
Block a user