mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-04-27 17:17:40 +03:00
Call out recognized objects more specifically
This commit is contained in:
parent
34bf1b21df
commit
ec5b2913f0
@ -46,20 +46,34 @@ class GenAIClient:
|
|||||||
debug_save: bool,
|
debug_save: bool,
|
||||||
) -> ReviewMetadata | None:
|
) -> ReviewMetadata | None:
|
||||||
"""Generate a description for the review item activity."""
|
"""Generate a description for the review item activity."""
|
||||||
|
|
||||||
|
def get_concern_prompt() -> str:
|
||||||
if concerns:
|
if concerns:
|
||||||
concern_list = "\n - ".join(concerns)
|
concern_list = "\n - ".join(concerns)
|
||||||
concern_prompt = f"""
|
return f"""
|
||||||
- `other_concerns` (list of strings): Include a list of any of the following concerns that are occurring:
|
- `other_concerns` (list of strings): Include a list of any of the following concerns that are occurring:
|
||||||
- {concern_list}
|
- {concern_list}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
else:
|
else:
|
||||||
concern_prompt = ""
|
return ""
|
||||||
|
|
||||||
|
def get_recognized_objects_prompt() -> str:
|
||||||
|
if not review_data["recognized_objects"]:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
return f"""
|
||||||
|
Recognized Objects:
|
||||||
|
- These are people, vehicles, or other entities that the system has positively identified.
|
||||||
|
- Always use these names or labels directly in the scene description instead of generic terms.
|
||||||
|
|
||||||
|
Recognized objects: {list(set(review_data["recognized_objects"]))}
|
||||||
|
"""
|
||||||
|
|
||||||
|
def get_language_prompt() -> str:
|
||||||
if preferred_language:
|
if preferred_language:
|
||||||
language_prompt = f"Provide your answer in {preferred_language}"
|
return f"Provide your answer in {preferred_language}"
|
||||||
else:
|
else:
|
||||||
language_prompt = ""
|
return ""
|
||||||
|
|
||||||
context_prompt = f"""
|
context_prompt = f"""
|
||||||
Please analyze the sequence of images ({len(thumbnails)} total) taken in chronological order from the perspective of the {review_data["camera"].replace("_", " ")} security camera.
|
Please analyze the sequence of images ({len(thumbnails)} total) taken in chronological order from the perspective of the {review_data["camera"].replace("_", " ")} security camera.
|
||||||
@ -77,11 +91,13 @@ When forming your description:
|
|||||||
- Focus on behaviors that are uncharacteristic of innocent activity: loitering without clear purpose, avoiding cameras, inspecting vehicles/doors, changing behavior when lights activate, scanning surroundings without an apparent benign reason.
|
- Focus on behaviors that are uncharacteristic of innocent activity: loitering without clear purpose, avoiding cameras, inspecting vehicles/doors, changing behavior when lights activate, scanning surroundings without an apparent benign reason.
|
||||||
- **Benign context override**: If scanning or looking around is clearly part of an innocent activity (such as playing with a dog, gardening, supervising children, or watching for a pet), do not treat it as suspicious.
|
- **Benign context override**: If scanning or looking around is clearly part of an innocent activity (such as playing with a dog, gardening, supervising children, or watching for a pet), do not treat it as suspicious.
|
||||||
|
|
||||||
|
{get_recognized_objects_prompt()}
|
||||||
|
|
||||||
Your response MUST be a flat JSON object with:
|
Your response MUST be a flat JSON object with:
|
||||||
- `scene` (string): A full description including setting, entities, actions, and any plausible supported inferences.
|
- `scene` (string): A full description including setting, entities, actions, and any plausible supported inferences.
|
||||||
- `confidence` (float): 0-1 confidence in the analysis.
|
- `confidence` (float): 0-1 confidence in the analysis.
|
||||||
- `potential_threat_level` (integer): 0, 1, or 2 as defined below.
|
- `potential_threat_level` (integer): 0, 1, or 2 as defined below.
|
||||||
{concern_prompt}
|
{get_concern_prompt()}
|
||||||
|
|
||||||
Threat-level definitions:
|
Threat-level definitions:
|
||||||
- 0 — Typical or expected activity for this location/time (includes residents, guests, or known animals engaged in normal activities, even if they glance around or scan surroundings).
|
- 0 — Typical or expected activity for this location/time (includes residents, guests, or known animals engaged in normal activities, even if they glance around or scan surroundings).
|
||||||
@ -92,12 +108,11 @@ Sequence details:
|
|||||||
- Frame 1 = earliest, Frame 10 = latest
|
- Frame 1 = earliest, Frame 10 = latest
|
||||||
- Activity occurred at {review_data["timestamp"].strftime("%I:%M %p")}
|
- Activity occurred at {review_data["timestamp"].strftime("%I:%M %p")}
|
||||||
- Detected objects: {list(set(review_data["objects"]))}
|
- Detected objects: {list(set(review_data["objects"]))}
|
||||||
- Recognized objects: {list(set(review_data["recognized_objects"])) or "None"}
|
|
||||||
- Zones involved: {review_data["zones"]}
|
- Zones involved: {review_data["zones"]}
|
||||||
|
|
||||||
**IMPORTANT:**
|
**IMPORTANT:**
|
||||||
- Values must be plain strings, floats, or integers — no nested objects, no extra commentary.
|
- Values must be plain strings, floats, or integers — no nested objects, no extra commentary.
|
||||||
{language_prompt}
|
{get_language_prompt()}
|
||||||
"""
|
"""
|
||||||
logger.debug(
|
logger.debug(
|
||||||
f"Sending {len(thumbnails)} images to create review description on {review_data['camera']}"
|
f"Sending {len(thumbnails)} images to create review description on {review_data['camera']}"
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user