Implement annotated frames for GenAI Review (#24379)
CI / AMD64 Build (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s

* Implement annotated frames mode for GenAI reviews to improve models with lacking temporal understanding

* Updates

* Improve debug sharing

* Do not number objects

* Fix assumptions

* Remove unhelpful content

* Improve object data sent as part of prompt

* Cleanup ollama dumbness

* Bind db

* Fixes

* Cleanup
This commit is contained in:
Nicolas Mowen
2026-09-17 10:28:37 -06:00
committed by GitHub
parent 10a0d5ea37
commit eccd10cd94
19 changed files with 1128 additions and 118 deletions
+13
View File
@@ -9,6 +9,7 @@ __all__ = [
"DetectionsConfig",
"AlertsConfig",
"ImageSourceEnum",
"ReviewFrameModeEnum",
"ReviewResponseStyleEnum",
]
@@ -20,6 +21,13 @@ class ImageSourceEnum(str, Enum):
recordings = "recordings"
class ReviewFrameModeEnum(str, Enum):
"""How review frames are presented to the GenAI provider."""
frames = "frames"
annotated_frames = "annotated_frames"
class ReviewResponseStyleEnum(str, Enum):
"""Writing style presets for GenAI review descriptions."""
@@ -153,6 +161,11 @@ class GenAIReviewConfig(FrigateBaseModel):
description="Preferred language to request from the GenAI provider for generated responses.",
default=None,
)
frame_mode: ReviewFrameModeEnum = Field(
default=ReviewFrameModeEnum.frames,
title="Frame mode",
description="How frames are presented to the model. 'frames' sends the prompt followed by the frames, which suits models that track a sequence well on their own. 'annotated_frames' labels each frame and interleaves notes derived from object tracking, which helps models that lose track of activity that repeats or reverses.",
)
response_style: ReviewResponseStyleEnum = Field(
default=ReviewResponseStyleEnum.default,
title="Response style",