Triggers tweaks (#20339)

* backend

* frontend

* use correct camera name param

* i18n

* change log message to debug level

* docs tweaks
This commit is contained in:
Josh Hawkins 2025-10-03 07:36:14 -05:00 committed by GitHub
parent 2d45ea271e
commit d818dbb6ba
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 313 additions and 190 deletions

View File

@ -912,6 +912,8 @@ cameras:
trigger_name:
# Required: Enable or disable the trigger. (default: shown below)
enabled: true
# Optional: A friendly name or descriptive text for the trigger
friendly_name: Unique name or descriptive text
# Type of trigger, either `thumbnail` for image-based matching or `description` for text-based matching. (default: none)
type: thumbnail
# Reference data for matching, either an event ID for `thumbnail` or a text string for `description`. (default: none)

View File

@ -109,11 +109,19 @@ See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_
## Triggers
Triggers utilize semantic search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes a specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
Triggers utilize Semantic Search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes a specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
:::note
Semantic Search must be enabled to use Triggers.
:::
### Configuration
Triggers are defined within the `semantic_search` configuration for each camera in your Frigate configuration file or through the UI. Each trigger consists of a `type` (either `thumbnail` or `description`), a `data` field (the reference image event ID or text), a `threshold` for similarity matching, and a list of `actions` to perform when the trigger fires.
Triggers are defined within the `semantic_search` configuration for each camera in your Frigate configuration file or through the UI. Each trigger consists of a `friendly_name`, a `type` (either `thumbnail` or `description`), a `data` field (the reference image event ID or text), a `threshold` for similarity matching, and a list of `actions` to perform when the trigger fires.
Triggers are best configured through the Frigate UI.
#### Managing Triggers in the UI
@ -122,6 +130,7 @@ Triggers are defined within the `semantic_search` configuration for each camera
3. Click **Add Trigger** to create a new trigger or use the pencil icon to edit an existing one.
4. In the **Create Trigger** dialog:
- Enter a **Name** for the trigger (e.g., "red_car_alert").
- Enter a descriptive **Friendly Name** for the trigger (e.g., "Red car on the driveway camera").
- Select the **Type** (`Thumbnail` or `Description`).
- For `Thumbnail`, select an image to trigger this action when a similar thumbnail image is detected, based on the threshold.
- For `Description`, enter text to trigger this action when a similar tracked object description is detected.
@ -149,6 +158,6 @@ When a trigger fires, the UI highlights the trigger with a blue outline for 3 se
#### Why can't I create a trigger on thumbnails for some text, like "person with a blue shirt" and have it trigger when a person with a blue shirt is detected?
TL;DR: Text-to-image triggers arent supported because CLIP can confuse similar images and give inconsistent scores, making automation unreliable.
TL;DR: Text-to-image triggers arent supported because CLIP can confuse similar images and give inconsistent scores, making automation unreliable. The same wordimage pair can give different scores and the score ranges can be too close together to set a clear cutoff.
Text-to-image triggers are not supported due to fundamental limitations of CLIP-based similarity search. While CLIP works well for exploratory, manual queries, it is unreliable for automated triggers based on a threshold. Issues include embedding drift (the same textimage pair can yield different cosine distances over time), lack of true semantic grounding (visually similar but incorrect matches), and unstable thresholding (distance distributions are dataset-dependent and often too tightly clustered to separate relevant from irrelevant results). Instead, it is recommended to set up a workflow with thumbnail triggers: first use text search to manually select 35 representative reference tracked objects, then configure thumbnail triggers based on that visual similarity. This provides robust automation without the semantic ambiguity of text to image matching.

View File

@ -138,6 +138,9 @@ class SemanticSearchConfig(FrigateBaseModel):
class TriggerConfig(FrigateBaseModel):
friendly_name: Optional[str] = Field(
None, title="Trigger friendly name used in the Frigate UI."
)
enabled: bool = Field(default=True, title="Enable this trigger")
type: TriggerType = Field(default=TriggerType.DESCRIPTION, title="Type of trigger")
data: str = Field(title="Trigger content (text phrase or image ID)")

View File

@ -159,7 +159,7 @@ class SemanticTriggerProcessor(PostProcessorApi):
# Check if similarity meets threshold
if similarity >= trigger["threshold"]:
logger.info(
logger.debug(
f"Trigger {trigger['name']} activated with similarity {similarity:.4f}"
)

View File

@ -720,6 +720,10 @@
},
"triggers": {
"documentTitle": "Triggers",
"semanticSearch": {
"title": "Semantic Search is disabled",
"desc": "Semantic Search must be enabled to use Triggers."
},
"management": {
"title": "Trigger Management",
"desc": "Manage triggers for {{camera}}. Use the thumbnail type to trigger on similar thumbnails to your selected tracked object, and the description type to trigger on similar descriptions to text you specify."
@ -774,6 +778,11 @@
"title": "Type",
"placeholder": "Select trigger type"
},
"friendly_name": {
"title": "Friendly Name",
"placeholder": "Name or describe this trigger",
"description": "An optional friendly name or descriptive text for this trigger."
},
"content": {
"title": "Content",
"imagePlaceholder": "Select an image",

View File

@ -60,6 +60,7 @@ type CreateTriggerDialogProps = {
data: string,
threshold: number,
actions: TriggerAction[],
friendly_name: string,
) => void;
onEdit: (trigger: Trigger) => void;
onCancel: () => void;
@ -102,6 +103,7 @@ export default function CreateTriggerDialog({
!existingTriggerNames.includes(value) || value === trigger?.name,
t("triggers.dialog.form.name.error.alreadyExists"),
),
friendly_name: z.string().optional(),
type: z.enum(["thumbnail", "description"]),
data: z.string().min(1, t("triggers.dialog.form.content.error.required")),
threshold: z
@ -117,6 +119,7 @@ export default function CreateTriggerDialog({
defaultValues: {
enabled: trigger?.enabled ?? true,
name: trigger?.name ?? "",
friendly_name: trigger?.friendly_name ?? "",
type: trigger?.type ?? "description",
data: trigger?.data ?? "",
threshold: trigger?.threshold ?? 0.5,
@ -135,6 +138,7 @@ export default function CreateTriggerDialog({
values.data,
values.threshold,
values.actions,
values.friendly_name ?? "",
);
}
};
@ -144,6 +148,7 @@ export default function CreateTriggerDialog({
form.reset({
enabled: true,
name: "",
friendly_name: "",
type: "description",
data: "",
threshold: 0.5,
@ -154,6 +159,7 @@ export default function CreateTriggerDialog({
{
enabled: trigger.enabled,
name: trigger.name,
friendly_name: trigger.friendly_name ?? "",
type: trigger.type,
data: trigger.data,
threshold: trigger.threshold,
@ -231,6 +237,31 @@ export default function CreateTriggerDialog({
)}
/>
<FormField
control={form.control}
name="friendly_name"
render={({ field }) => (
<FormItem>
<FormLabel>
{t("triggers.dialog.form.friendly_name.title")}
</FormLabel>
<FormControl>
<Input
placeholder={t(
"triggers.dialog.form.friendly_name.placeholder",
)}
className="h-10"
{...field}
/>
</FormControl>
<FormDescription>
{t("triggers.dialog.form.friendly_name.description")}
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="enabled"

View File

@ -237,6 +237,7 @@ export interface CameraConfig {
data: string;
threshold: number;
actions: TriggerAction[];
friendly_name: string;
};
};
};

View File

@ -8,4 +8,5 @@ export type Trigger = {
data: string;
threshold: number;
actions: TriggerAction[];
friendly_name?: string;
};

View File

@ -1,9 +1,10 @@
import { useCallback, useEffect, useMemo, useState } from "react";
import { useTranslation } from "react-i18next";
import { Trans, useTranslation } from "react-i18next";
import { Toaster, toast } from "sonner";
import useSWR from "swr";
import axios from "axios";
import { Button } from "@/components/ui/button";
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
import Heading from "@/components/ui/heading";
import { Badge } from "@/components/ui/badge";
import {
@ -12,7 +13,13 @@ import {
TooltipProvider,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { LuPlus, LuTrash, LuPencil, LuSearch } from "react-icons/lu";
import {
LuPlus,
LuTrash,
LuPencil,
LuSearch,
LuExternalLink,
} from "react-icons/lu";
import ActivityIndicator from "@/components/indicators/activity-indicator";
import CreateTriggerDialog from "@/components/overlay/CreateTriggerDialog";
import DeleteTriggerDialog from "@/components/overlay/DeleteTriggerDialog";
@ -24,6 +31,8 @@ import { formatUnixTimestampToDateTime } from "@/utils/dateUtil";
import { Link } from "react-router-dom";
import { useTriggers } from "@/api/ws";
import { useCameraFriendlyName } from "@/hooks/use-camera-friendly-name";
import { CiCircleAlert } from "react-icons/ci";
import { useDocDomain } from "@/hooks/use-doc-domain";
type ConfigSetBody = {
requires_restart: number;
@ -39,6 +48,7 @@ type ConfigSetBody = {
data: string;
threshold: number;
actions: string[];
friendly_name?: string;
}
| "";
};
@ -80,6 +90,10 @@ export default function TriggerView({
const [isLoading, setIsLoading] = useState(false);
const cameraName = useCameraFriendlyName(selectedCamera);
const isSemanticSearchEnabled = config?.semantic_search?.enabled ?? false;
const { getLocaleDocUrl } = useDocDomain();
const triggers = useMemo(() => {
if (
!config ||
@ -93,6 +107,7 @@ export default function TriggerView({
).map(([name, trigger]) => ({
enabled: trigger.enabled,
name,
friendly_name: trigger.friendly_name,
type: trigger.type,
data: trigger.data,
threshold: trigger.threshold,
@ -139,11 +154,12 @@ export default function TriggerView({
const saveToConfig = useCallback(
(trigger: Trigger, isEdit: boolean) => {
setIsLoading(true);
const { enabled, name, type, data, threshold, actions } = trigger;
const { enabled, name, type, data, threshold, actions, friendly_name } =
trigger;
const embeddingBody: TriggerEmbeddingBody = { type, data, threshold };
const embeddingUrl = isEdit
? `/trigger/embedding/${selectedCamera}/${name}`
: `/trigger/embedding?camera=${selectedCamera}&name=${name}`;
: `/trigger/embedding?camera_name=${selectedCamera}&name=${name}`;
const embeddingMethod = isEdit ? axios.put : axios.post;
embeddingMethod(embeddingUrl, embeddingBody)
@ -162,6 +178,7 @@ export default function TriggerView({
data,
threshold,
actions,
friendly_name,
},
},
},
@ -220,9 +237,21 @@ export default function TriggerView({
data: string,
threshold: number,
actions: TriggerAction[],
friendly_name: string,
) => {
setUnsavedChanges(true);
saveToConfig({ enabled, name, type, data, threshold, actions }, false);
saveToConfig(
{
enabled,
name,
type,
data,
threshold,
actions,
friendly_name,
},
false,
);
},
[saveToConfig, setUnsavedChanges],
);
@ -359,7 +388,7 @@ export default function TriggerView({
// for adding a trigger with event id via explore context menu
useSearchEffect("event_id", (eventId: string) => {
if (!config || isLoading) {
if (!config || isLoading || !isSemanticSearchEnabled) {
return false;
}
setShowCreate(true);
@ -386,6 +415,41 @@ export default function TriggerView({
<div className="flex size-full flex-col md:flex-row">
<Toaster position="top-center" closeButton={true} />
<div className="scrollbar-container order-last mb-10 mt-2 flex h-full w-full flex-col overflow-y-auto rounded-lg border-[1px] border-secondary-foreground bg-background_alt p-2 md:order-none md:mb-0 md:mr-2 md:mt-0">
{!isSemanticSearchEnabled ? (
<div className="mb-5 flex flex-row items-center justify-between gap-2">
<div className="flex flex-col items-start">
<Heading as="h3" className="my-2">
{t("triggers.management.title")}
</Heading>
<p className="mb-5 text-sm text-muted-foreground">
{t("triggers.management.desc", {
camera: cameraName,
})}
</p>
<Alert variant="destructive">
<CiCircleAlert className="size-5" />
<AlertTitle>{t("triggers.semanticSearch.title")}</AlertTitle>
<AlertDescription>
<Trans ns="views/settings">
triggers.semanticSearch.desc
</Trans>
<div className="mt-3 flex items-center">
<Link
to={getLocaleDocUrl("configuration/semantic_search")}
target="_blank"
rel="noopener noreferrer"
className="inline"
>
{t("readTheDocumentation", { ns: "common" })}{" "}
<LuExternalLink className="ml-2 inline-flex size-3" />
</Link>
</div>
</AlertDescription>
</Alert>
</div>
</div>
) : (
<>
<div className="mb-5 flex flex-row items-center justify-between gap-2">
<div className="flex flex-col items-start">
<Heading as="h3" className="my-2">
@ -443,7 +507,7 @@ export default function TriggerView({
!trigger.enabled && "opacity-60",
)}
>
{trigger.name}
{trigger.friendly_name || trigger.name}
</h3>
<div
className={cn(
@ -473,7 +537,8 @@ export default function TriggerView({
className={cn(
"text-sm",
!trigger_status?.triggers[trigger.name]
?.triggering_event_id && "pointer-events-none",
?.triggering_event_id &&
"pointer-events-none",
)}
>
<div className="flex flex-row items-center">
@ -569,6 +634,8 @@ export default function TriggerView({
</div>
</div>
</div>
</>
)}
</div>
<CreateTriggerDialog
show={showCreate}