Miscellaneous Fixes (0.17 beta) (#21474)

* disable modal on dropdown menu in explore

* add another example case for when classification overrides a sub label

* update ollama docs link

* Improve handling of automatic playback for recordings

* Improve ollama documentation

* Don't fall out when all recording segments exist

* clarify coral docs

* improve initial scroll to active item in detail stream

* i18n fixes

* remove console warning

* detail stream scrolling fixes for HA/iOS

* Improve usability of GenAI summary dialog and make clicking on the description directly open it

* Review card too

* Use empty card with dynamic text for review based on the user's config

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
This commit is contained in:
Josh Hawkins 2025-12-31 06:48:56 -06:00 committed by GitHub
parent fb9604fbcc
commit e0d6365f62
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 172 additions and 51 deletions

View File

@ -39,7 +39,7 @@ For object classification:
:::note
A tracked object can only have a single sub label. If you are using Triggers or Face Recognition and you configure an object classification model for `person` using the sub label type, your sub label may not be assigned correctly as it depends on which enrichment completes its analysis first. Consider using the `attribute` type instead.
A tracked object can only have a single sub label. If you are using Triggers or Face Recognition and you configure an object classification model for `person` using the sub label type, your sub label may not be assigned correctly as it depends on which enrichment completes its analysis first. This could also occur with `car` objects that are assigned a sub label for a delivery carrier. Consider using the `attribute` type instead.
:::

View File

@ -48,15 +48,29 @@ Using Ollama on CPU is not recommended, high inference times make using Generati
:::
[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It provides a nice API over [llama.cpp](https://github.com/ggerganov/llama.cpp). It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available.
Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests).
Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://docs.ollama.com/faq#how-does-ollama-handle-concurrent-requests).
### Model Types: Instruct vs Thinking
Most vision-language models are available as **instruct** models, which are fine-tuned to follow instructions and respond concisely to prompts. However, some models (such as certain Qwen-VL or minigpt variants) offer both **instruct** and **thinking** versions.
- **Instruct models** are always recommended for use with Frigate. These models generate direct, relevant, actionable descriptions that best fit Frigate's object and event summary use case.
- **Thinking models** are fine-tuned for more free-form, open-ended, and speculative outputs, which are typically not concise and may not provide the practical summaries Frigate expects. For this reason, Frigate does **not** recommend or support using thinking models.
Some models are labeled as **hybrid** (capable of both thinking and instruct tasks). In these cases, Frigate will always use instruct-style prompts and specifically disables thinking-mode behaviors to ensure concise, useful responses.
**Recommendation:**
Always select the `-instruct` or documented instruct/tagged variant of any model you use in your Frigate configuration. If in doubt, refer to your model providers documentation or model library for guidance on the correct model variant to use.
### Supported Models
You must use a vision capable model with Frigate. Current model variants can be found [in their model library](https://ollama.com/library). Note that Frigate will not automatically download the model you specify in your config, you must download the model to your local instance of Ollama first i.e. by running `ollama pull llava:7b` on your Ollama server/Docker container. Note that the model specified in Frigate's config must match the downloaded model tag.
You must use a vision capable model with Frigate. Current model variants can be found [in their model library](https://ollama.com/search?c=vision). Note that Frigate will not automatically download the model you specify in your config, you must download the model to your local instance of Ollama first i.e. by running `ollama pull qwen3-vl:2b-instruct` on your Ollama server/Docker container. Note that the model specified in Frigate's config must match the downloaded model tag.
:::note

View File

@ -157,7 +157,7 @@ A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite`
#### YOLOv9
[YOLOv9](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite) models that are compiled for Tensorflow Lite and properly quantized are supported, but not included by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`. Note that the model may require a custom label file (eg. [use this 17 label file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) for the model linked above.)
YOLOv9 models that are compiled for TensorFlow Lite and properly quantized are supported, but not included by default. [Download the model](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite), bind mount the file into the container, and provide the path with `model.path`. Note that the linked model requires a 17-label [labelmap file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) that includes only 17 COCO classes.
<details>
<summary>YOLOv9 Setup & Config</summary>
@ -178,7 +178,7 @@ model:
labelmap_path: /config/labels-coco17.txt
```
Note that the labelmap uses a subset of the complete COCO label set that has only 17 objects.
Note that due to hardware limitations of the Coral, the labelmap is a subset of the COCO labels and includes only 17 object classes.
</details>

View File

@ -9,7 +9,11 @@
"empty": {
"alert": "There are no alerts to review",
"detection": "There are no detections to review",
"motion": "No motion data found"
"motion": "No motion data found",
"recordingsDisabled": {
"title": "Recordings must be enabled",
"description": "Review items can only be created for a camera when recordings are enabled for that camera."
}
},
"timeline": "Timeline",
"timeline.aria": "Select timeline",

View File

@ -166,6 +166,9 @@
"tips": {
"descriptionSaved": "Successfully saved description",
"saveDescriptionFailed": "Failed to update the description: {{errorMessage}}"
},
"title": {
"label": "Title"
}
},
"itemMenu": {

View File

@ -2,15 +2,18 @@ import React from "react";
import { Button } from "../ui/button";
import Heading from "../ui/heading";
import { Link } from "react-router-dom";
import { cn } from "@/lib/utils";
type EmptyCardProps = {
className?: string;
icon: React.ReactNode;
title: string;
description: string;
description?: string;
buttonText?: string;
link?: string;
};
export function EmptyCard({
className,
icon,
title,
description,
@ -18,10 +21,12 @@ export function EmptyCard({
link,
}: EmptyCardProps) {
return (
<div className="flex flex-col items-center gap-2">
<div className={cn("flex flex-col items-center gap-2", className)}>
{icon}
<Heading as="h4">{title}</Heading>
<div className="mb-3 text-secondary-foreground">{description}</div>
{description && (
<div className="mb-3 text-secondary-foreground">{description}</div>
)}
{buttonText?.length && (
<Button size="sm" variant="select">
<Link to={link ?? "#"}>{buttonText}</Link>

View File

@ -39,6 +39,7 @@ import { Trans, useTranslation } from "react-i18next";
import { cn } from "@/lib/utils";
import { LuCircle } from "react-icons/lu";
import { MdAutoAwesome } from "react-icons/md";
import { GenAISummaryDialog } from "../overlay/chip/GenAISummaryChip";
type ReviewCardProps = {
event: ReviewSegment;
@ -219,12 +220,14 @@ export default function ReviewCard({
/>
</div>
{event.data.metadata?.title && (
<div className="flex items-center gap-1.5 rounded bg-secondary/50">
<MdAutoAwesome className="size-3 shrink-0 text-primary" />
<span className="truncate text-xs text-primary">
{event.data.metadata.title}
</span>
</div>
<GenAISummaryDialog review={event}>
<div className="flex items-center gap-1.5 rounded bg-secondary/50 hover:underline">
<MdAutoAwesome className="size-3 shrink-0 text-primary" />
<span className="truncate text-xs text-primary">
{event.data.metadata.title}
</span>
</div>
</GenAISummaryDialog>
)}
</div>
);

View File

@ -195,7 +195,7 @@ export default function SearchResultActions({
</ContextMenu>
) : (
<>
<DropdownMenu>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<BlurredIconButton aria-label={t("itemMenu.more.aria")}>
<FiMoreVertical className="size-5" />

View File

@ -6,16 +6,15 @@ import {
ThreatLevel,
THREAT_LEVEL_LABELS,
} from "@/types/review";
import { useEffect, useMemo, useState } from "react";
import React, { useEffect, useMemo, useState } from "react";
import { isDesktop } from "react-device-detect";
import { useTranslation } from "react-i18next";
import { MdAutoAwesome } from "react-icons/md";
type GenAISummaryChipProps = {
review?: ReviewSegment;
onClick: () => void;
};
export function GenAISummaryChip({ review, onClick }: GenAISummaryChipProps) {
export function GenAISummaryChip({ review }: GenAISummaryChipProps) {
const [isVisible, setIsVisible] = useState(false);
useEffect(() => {
@ -29,7 +28,6 @@ export function GenAISummaryChip({ review, onClick }: GenAISummaryChipProps) {
isVisible ? "translate-y-0 opacity-100" : "-translate-y-4 opacity-0",
isDesktop ? "bg-card" : "bg-secondary-foreground",
)}
onClick={onClick}
>
<MdAutoAwesome className="shrink-0" />
<span className="truncate">{review?.data.metadata?.title}</span>
@ -40,10 +38,12 @@ export function GenAISummaryChip({ review, onClick }: GenAISummaryChipProps) {
type GenAISummaryDialogProps = {
review?: ReviewSegment;
onOpen?: (open: boolean) => void;
children: React.ReactNode;
};
export function GenAISummaryDialog({
review,
onOpen,
children,
}: GenAISummaryDialogProps) {
const { t } = useTranslation(["views/explore"]);
@ -104,7 +104,7 @@ export function GenAISummaryDialog({
return (
<Overlay open={open} onOpenChange={setOpen}>
<Trigger asChild>
<GenAISummaryChip review={review} onClick={() => setOpen(true)} />
<div>{children}</div>
</Trigger>
<Content
className={cn(
@ -115,6 +115,10 @@ export function GenAISummaryDialog({
)}
>
{t("aiAnalysis.title")}
<div className="text-sm text-primary/40">
{t("details.title.label")}
</div>
<div className="text-sm">{aiAnalysis.title}</div>
<div className="text-sm text-primary/40">
{t("details.description.label")}
</div>

View File

@ -25,10 +25,13 @@ import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
import { Link } from "react-router-dom";
import { Switch } from "@/components/ui/switch";
import { useUserPersistence } from "@/hooks/use-user-persistence";
import { isDesktop } from "react-device-detect";
import { isDesktop, isIOS, isMobile } from "react-device-detect";
import { resolveZoneName } from "@/hooks/use-zone-friendly-name";
import { PiSlidersHorizontalBold } from "react-icons/pi";
import { MdAutoAwesome } from "react-icons/md";
import { isPWA } from "@/utils/isPWA";
import { isInIframe } from "@/utils/isIFrame";
import { GenAISummaryDialog } from "../overlay/chip/GenAISummaryChip";
type DetailStreamProps = {
reviewItems?: ReviewSegment[];
@ -100,7 +103,25 @@ export default function DetailStream({
}
}, [reviewItems, activeReviewId, effectiveTime]);
// Auto-scroll to current time
// Initial scroll to active review (runs immediately when user selects, not during playback)
useEffect(() => {
if (!scrollRef.current || !activeReviewId || userInteracting || isPlaying)
return;
const element = scrollRef.current.querySelector(
`[data-review-id="${activeReviewId}"]`,
) as HTMLElement;
if (element) {
setProgrammaticScroll();
scrollIntoView(element, {
scrollMode: "if-needed",
behavior: isMobile && isIOS && !isPWA && isInIframe ? "auto" : "smooth",
});
}
}, [activeReviewId, setProgrammaticScroll, userInteracting, isPlaying]);
// Auto-scroll to current time during playback
useEffect(() => {
if (!scrollRef.current || userInteracting || !isPlaying) return;
// Prefer the review whose range contains the effectiveTime. If none
@ -145,7 +166,8 @@ export default function DetailStream({
setProgrammaticScroll();
scrollIntoView(element, {
scrollMode: "if-needed",
behavior: "smooth",
behavior:
isMobile && isIOS && !isPWA && isInIframe ? "auto" : "smooth",
});
}
}
@ -417,7 +439,18 @@ function ReviewGroup({
{review.data.metadata.title}
</TooltipContent>
</Tooltip>
<span className="truncate">{review.data.metadata.title}</span>
<GenAISummaryDialog
review={review}
onOpen={(open) => {
if (open) {
onSeek(review.start_time, false);
}
}}
>
<span className="truncate hover:underline">
{review.data.metadata.title}
</span>
</GenAISummaryDialog>
</div>
)}
<div className="flex flex-row items-center gap-1.5">
@ -782,21 +815,27 @@ function LifecycleItem({
<div className="flex flex-col gap-1">
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("trackingDetails.lifecycleItemDesc.header.score")}
{t("trackingDetails.lifecycleItemDesc.header.score", {
ns: "views/explore",
})}
</span>
<span className="font-medium text-foreground">{score}</span>
</div>
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("trackingDetails.lifecycleItemDesc.header.ratio")}
{t("trackingDetails.lifecycleItemDesc.header.ratio", {
ns: "views/explore",
})}
</span>
<span className="font-medium text-foreground">{ratio}</span>
</div>
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("trackingDetails.lifecycleItemDesc.header.area")}{" "}
{t("trackingDetails.lifecycleItemDesc.header.area", {
ns: "views/explore",
})}{" "}
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<span className="text-muted-foreground">
@ -806,7 +845,7 @@ function LifecycleItem({
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-foreground">
{areaPx} {t("pixels", { ns: "common" })}{" "}
{areaPx} {t("information.pixels", { ns: "common" })}{" "}
<span className="text-secondary-foreground">·</span>{" "}
{areaPct}%
</span>
@ -819,7 +858,9 @@ function LifecycleItem({
attributeAreaPct !== undefined && (
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("trackingDetails.lifecycleItemDesc.header.area")}{" "}
{t("trackingDetails.lifecycleItemDesc.header.area", {
ns: "views/explore",
})}{" "}
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<span className="text-muted-foreground">
@ -828,7 +869,8 @@ function LifecycleItem({
)}
</span>
<span className="font-medium text-foreground">
{attributeAreaPx} {t("pixels", { ns: "common" })}{" "}
{attributeAreaPx}{" "}
{t("information.pixels", { ns: "common" })}{" "}
<span className="text-secondary-foreground">·</span>{" "}
{attributeAreaPct}%
</span>

View File

@ -111,7 +111,7 @@ export function MotionReviewTimeline({
const getRecordingAvailability = useCallback(
(time: number): boolean | undefined => {
if (!noRecordingRanges?.length) return undefined;
if (noRecordingRanges == undefined) return undefined;
return !noRecordingRanges.some(
(range) => time >= range.start_time && time < range.end_time,

4
web/src/types/card.ts Normal file
View File

@ -0,0 +1,4 @@
export type EmptyCardData = {
title: string;
description?: string;
};

View File

@ -79,9 +79,6 @@ i18n
parseMissingKeyHandler: (key: string) => {
const parts = key.split(".");
// eslint-disable-next-line no-console
console.warn(`Missing translation key: ${key}`);
if (parts[0] === "time" && parts[1]?.includes("formattedTimestamp")) {
// Extract the format type from the last part (12hour, 24hour)
const formatType = parts[parts.length - 1];

View File

@ -56,6 +56,8 @@ import { GiSoundWaves } from "react-icons/gi";
import useKeyboardListener from "@/hooks/use-keyboard-listener";
import { useTimelineZoom } from "@/hooks/use-timeline-zoom";
import { useTranslation } from "react-i18next";
import { EmptyCard } from "@/components/card/EmptyCard";
import { EmptyCardData } from "@/types/card";
type EventViewProps = {
reviewItems?: SegmentedReviewData;
@ -132,6 +134,24 @@ export default function EventView({
}
}, [filter, showReviewed, reviewSummary]);
const emptyCardData: EmptyCardData = useMemo(() => {
if (
!config ||
Object.values(config.cameras).find(
(cam) => cam.record.enabled_in_config,
) != undefined
) {
return {
title: t("empty." + severity.replace(/_/g, " ")),
};
}
return {
title: t("empty.recordingsDisabled.title"),
description: t("empty.recordingsDisabled.description"),
};
}, [config, severity, t]);
// review interaction
const [selectedReviews, setSelectedReviews] = useState<ReviewSegment[]>([]);
@ -412,6 +432,7 @@ export default function EventView({
timeRange={timeRange}
startTime={startTime}
loading={severity != severityToggle}
emptyCardData={emptyCardData}
markItemAsReviewed={markItemAsReviewed}
markAllItemsAsReviewed={markAllItemsAsReviewed}
onSelectReview={onSelectReview}
@ -430,6 +451,7 @@ export default function EventView({
startTime={startTime}
filter={filter}
motionOnly={motionOnly}
emptyCardData={emptyCardData}
onOpenRecording={onOpenRecording}
/>
)}
@ -455,6 +477,7 @@ type DetectionReviewProps = {
timeRange: { before: number; after: number };
startTime?: number;
loading: boolean;
emptyCardData: EmptyCardData;
markItemAsReviewed: (review: ReviewSegment) => void;
markAllItemsAsReviewed: (currentItems: ReviewSegment[]) => void;
onSelectReview: (
@ -478,6 +501,7 @@ function DetectionReview({
timeRange,
startTime,
loading,
emptyCardData,
markItemAsReviewed,
markAllItemsAsReviewed,
onSelectReview,
@ -737,10 +761,12 @@ function DetectionReview({
)}
{!loading && currentItems?.length === 0 && (
<div className="absolute left-1/2 top-1/2 flex -translate-x-1/2 -translate-y-1/2 flex-col items-center justify-center text-center">
<LuFolderCheck className="size-16" />
{t("empty." + severity.replace(/_/g, " "))}
</div>
<EmptyCard
className="y-translate-1/2 absolute left-[50%] top-[50%] -translate-x-1/2"
title={emptyCardData.title}
description={emptyCardData.description}
icon={<LuFolderCheck className="size-16" />}
/>
)}
<div
@ -875,6 +901,7 @@ type MotionReviewProps = {
startTime?: number;
filter?: ReviewFilter;
motionOnly?: boolean;
emptyCardData: EmptyCardData;
onOpenRecording: (data: RecordingStartingPoint) => void;
};
function MotionReview({
@ -885,9 +912,9 @@ function MotionReview({
startTime,
filter,
motionOnly = false,
emptyCardData,
onOpenRecording,
}: MotionReviewProps) {
const { t } = useTranslation(["views/events"]);
const segmentDuration = 30;
const { data: config } = useSWR<FrigateConfig>("config");
@ -1080,9 +1107,12 @@ function MotionReview({
if (motionData?.length === 0) {
return (
<div className="absolute left-1/2 top-1/2 flex -translate-x-1/2 -translate-y-1/2 flex-col items-center justify-center text-center">
<LuFolderX className="size-16" />
{t("empty.motion")}
<div className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2">
<EmptyCard
title={emptyCardData.title}
description={emptyCardData.description}
icon={<LuFolderX className="size-16" />}
/>
</div>
);
}

View File

@ -66,7 +66,10 @@ import {
import { CameraNameLabel } from "@/components/camera/FriendlyNameLabel";
import { useAllowedCameras } from "@/hooks/use-allowed-cameras";
import { DetailStreamProvider } from "@/context/detail-stream-context";
import { GenAISummaryDialog } from "@/components/overlay/chip/GenAISummaryChip";
import {
GenAISummaryDialog,
GenAISummaryChip,
} from "@/components/overlay/chip/GenAISummaryChip";
const DATA_REFRESH_TIME = 600000; // 10 minutes
@ -309,10 +312,18 @@ export function RecordingView({
currentTimeRange.after <= currentTime &&
currentTimeRange.before >= currentTime
) {
mainControllerRef.current?.seekToTimestamp(
currentTime,
mainControllerRef.current.isPlaying(),
);
if (mainControllerRef.current != undefined) {
let shouldPlayback = true;
if (timelineType == "detail") {
shouldPlayback = mainControllerRef.current.isPlaying();
}
mainControllerRef.current.seekToTimestamp(
currentTime,
shouldPlayback,
);
}
} else {
updateSelectedSegment(currentTime, true);
}
@ -731,7 +742,9 @@ export function RecordingView({
<GenAISummaryDialog
review={activeReviewItem}
onOpen={onAnalysisOpen}
/>
>
<GenAISummaryChip review={activeReviewItem} />
</GenAISummaryDialog>
)}
<DynamicVideoPlayer
@ -989,7 +1002,9 @@ function Timeline({
)}
>
{isMobile && timelineType == "timeline" && (
<GenAISummaryDialog review={activeReviewItem} onOpen={onAnalysisOpen} />
<GenAISummaryDialog review={activeReviewItem} onOpen={onAnalysisOpen}>
<GenAISummaryChip review={activeReviewItem} />
</GenAISummaryDialog>
)}
{timelineType != "detail" && (