From f74825e1345ba7763afd68cbd3fa779be0a72f35 Mon Sep 17 00:00:00 2001
From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
Date: Wed, 8 Apr 2026 15:24:46 -0500
Subject: [PATCH] Teach chat system prompt when to use find_similar_objects

---
 frigate/api/chat.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/frigate/api/chat.py b/frigate/api/chat.py
index 219d4d6f0..e12106eb7 100644
--- a/frigate/api/chat.py
+++ b/frigate/api/chat.py
@@ -1438,7 +1438,9 @@ Do not start your response with phrases like "I will check...", "Let me see...",
 Always present times to the user in the server's local timezone. When tool results include start_time_local and end_time_local, use those exact strings when listing or describing detection times—do not convert or invent timestamps. Do not use UTC or ISO format with Z for the user-facing answer unless the tool result only provides Unix timestamps without local time fields.
 When users ask about "today", "yesterday", "this week", etc., use the current date above as reference.
 When searching for objects or events, use ISO 8601 format for dates (e.g., {current_date_str}T00:00:00Z for the start of today).
-Always be accurate with time calculations based on the current date provided.{cameras_section}"""
+Always be accurate with time calculations based on the current date provided.
+
+When a user refers to a specific object they have seen or describe with identifying details ("that green car", "the person in the red jacket", "a package left today"), prefer the find_similar_objects tool over search_objects. Use search_objects first only to locate the anchor event, then pass its id to find_similar_objects. For generic queries like "show me all cars today", keep using search_objects.{cameras_section}"""
 
     conversation.append(
         {