From 039ab1ccd759314cc8c11e68478348bc069ec2a5 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Sat, 5 Oct 2024 14:51:05 -0500 Subject: [PATCH 001/479] add docs for yolonas plus models (#14161) * add docs for yolonas plus models * typo --- docs/docs/plus/first_model.md | 4 ++-- docs/docs/plus/improving_model.md | 31 +++++++++++++++---------------- docs/docs/plus/index.md | 25 ++++++++++++++++++++++--- 3 files changed, 39 insertions(+), 21 deletions(-) diff --git a/docs/docs/plus/first_model.md b/docs/docs/plus/first_model.md index bbaf9cacb..6978bb491 100644 --- a/docs/docs/plus/first_model.md +++ b/docs/docs/plus/first_model.md @@ -5,7 +5,7 @@ title: Requesting your first model ## Step 1: Upload and annotate your images -Before requesting your first model, you will need to upload at least 10 images to Frigate+. But for the best results, you should provide at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. +Before requesting your first model, you will need to upload and verify at least 1 image to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. It is recommended to submit **both** true positives and false positives. This will help the model differentiate between what is and isn't correct. You should aim for a target of 80% true positive submissions and 20% false positives across all of your images. If you are experiencing false positives in a specific area, submitting true positives for any object type near that area in similar lighting conditions will help teach the model what that area looks like when no objects are present. @@ -13,7 +13,7 @@ For more detailed recommendations, you can refer to the docs on [improving your ## Step 2: Submit a model request -Once you have an initial set of verified images, you can request a model on the Models page. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. +Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [this part of the documentation](./index.md#available-model-types). Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. ![Plus Models Page](/img/plus/plus-models.jpg) ## Step 3: Set your model id in the config diff --git a/docs/docs/plus/improving_model.md b/docs/docs/plus/improving_model.md index 0e97b21a5..37a765994 100644 --- a/docs/docs/plus/improving_model.md +++ b/docs/docs/plus/improving_model.md @@ -3,7 +3,7 @@ id: improving_model title: Improving your model --- -You may find that Frigate+ models result in more false positives initially, but by submitting true and false positives, the model will improve. Because a limited number of users submitted images to Frigate+ prior to this launch, you may need to submit several hundred images per camera to see good results. With all the new images now being submitted, future base models will improve as more and more users (including you) submit examples to Frigate+. Note that only verified images will be used when training your model. Submitting an image from Frigate as a true or false positive will not verify the image. You still must verify the image in Frigate+ in order for it to be used in training. +You may find that Frigate+ models result in more false positives initially, but by submitting true and false positives, the model will improve. With all the new images now being submitted by subscribers, future base models will improve as more and more examples are incorporated. Note that only images with at least one verified label will be used when training your model. Submitting an image from Frigate as a true or false positive will not verify the image. You still must verify the image in Frigate+ in order for it to be used in training. - **Submit both true positives and false positives**. This will help the model differentiate between what is and isn't correct. You should aim for a target of 80% true positive submissions and 20% false positives across all of your images. If you are experiencing false positives in a specific area, submitting true positives for any object type near that area in similar lighting conditions will help teach the model what that area looks like when no objects are present. - **Lower your thresholds a little in order to generate more false/true positives near the threshold value**. For example, if you have some false positives that are scoring at 68% and some true positives scoring at 72%, you can try lowering your threshold to 65% and submitting both true and false positives within that range. This will help the model learn and widen the gap between true and false positive scores. @@ -36,18 +36,17 @@ Misidentified objects should have a correct label added. For example, if a perso ## Shortcuts for a faster workflow -|Shortcut Key|Description| -|-----|--------| -|`?`|Show all keyboard shortcuts| -|`w`|Add box| -|`d`|Toggle difficult| -|`s`|Switch to the next label| -|`tab`|Select next largest box| -|`del`|Delete current box| -|`esc`|Deselect/Cancel| -|`← ↑ → ↓`|Move box| -|`Shift + ← ↑ → ↓`|Resize box| -|`-`|Zoom out| -|`=`|Zoom in| -|`f`|Hide/show all but current box| -|`spacebar`|Verify and save| +| Shortcut Key | Description | +| ----------------- | ----------------------------- | +| `?` | Show all keyboard shortcuts | +| `w` | Add box | +| `d` | Toggle difficult | +| `s` | Switch to the next label | +| `tab` | Select next largest box | +| `del` | Delete current box | +| `esc` | Deselect/Cancel | +| `← ↑ → ↓` | Move box | +| `Shift + ← ↑ → ↓` | Resize box | +| `scrollwheel` | Zoom in/out | +| `f` | Hide/show all but current box | +| `spacebar` | Verify and save | diff --git a/docs/docs/plus/index.md b/docs/docs/plus/index.md index 35189ed3f..b05f4f306 100644 --- a/docs/docs/plus/index.md +++ b/docs/docs/plus/index.md @@ -15,17 +15,36 @@ With a subscription, 12 model trainings per year are included. If you cancel you Information on how to integrate Frigate+ with Frigate can be found in the [integration docs](../integrations/plus.md). +## Available model types + +There are two model types offered in Frigate+: `mobiledet` and `yolonas`. Both of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types). + +Not all model types are supported by all detectors, so it's important to choose a model type to match your detector as shown in the table under [supported detector types](#supported-detector-types). + +| Model Type | Description | +| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `mobiledet` | Based on the same architecture as the default model included with Frigate. Runs on Google Coral devices and CPUs. | +| `yolonas` | A newer architecture that offers slightly higher accuracy and improved detection of small objects. Runs on Intel, NVidia GPUs, and AMD GPUs. | + ## Supported detector types +Currently, Frigate+ models support CPU (`cpu`), Google Coral (`edgetpu`), OpenVino (`openvino`), ONNX (`onnx`), and ROCm (`rocm`) detectors. + :::warning -Frigate+ models are not supported for TensorRT or OpenVino yet. +Using Frigate+ models with `onnx` and `rocm` is only available with Frigate 0.15, which is still under development. ::: -Currently, Frigate+ models only support CPU (`cpu`) and Coral (`edgetpu`) models. OpenVino is next in line to gain support. +| Hardware | Recommended Detector Type | Recommended Model Type | +| ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ---------------------- | +| [CPU](/configuration/object_detectors.md#cpu-detector-not-recommended) | `cpu` | `mobiledet` | +| [Coral (all form factors)](/configuration/object_detectors.md#edge-tpu-detector) | `edgetpu` | `mobiledet` | +| [Intel](/configuration/object_detectors.md#openvino-detector) | `openvino` | `yolonas` | +| [NVidia GPU](https://deploy-preview-13787--frigate-docs.netlify.app/configuration/object_detectors#onnx)\* | `onnx` | `yolonas` | +| [AMD ROCm GPU](https://deploy-preview-13787--frigate-docs.netlify.app/configuration/object_detectors#amdrocm-gpu-detector)\* | `rocm` | `yolonas` | -The models are created using the same MobileDet architecture as the default model. Additional architectures will be added in future releases as needed. +_\* Requires Frigate 0.15_ ## Available label types From 2a15b95f18dd7d21f3a3d2bf260be12c5f3cc6a9 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 7 Oct 2024 14:28:24 -0600 Subject: [PATCH 002/479] Docs updates (#14202) * Clarify live docs * Link out to common config examples in getting started guide * Add tip for go2rtc name configuration * direct link --- docs/docs/configuration/reference.md | 8 +++++--- docs/docs/guides/configuring_go2rtc.md | 10 +++++++++- docs/docs/guides/getting_started.md | 4 +++- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/docs/docs/configuration/reference.md b/docs/docs/configuration/reference.md index 76a9e6158..e9b1d605f 100644 --- a/docs/docs/configuration/reference.md +++ b/docs/docs/configuration/reference.md @@ -480,10 +480,12 @@ snapshots: # Uses https://github.com/AlexxIT/go2rtc (v1.9.2) go2rtc: -# Optional: jsmpeg stream configuration for WebUI +# Optional: Live stream configuration for WebUI. +# NOTE: Can be overridden at the camera level live: - # Optional: Set the name of the stream that should be used for live view - # in frigate WebUI. (default: name of camera) + # Optional: Set the name of the stream configured in go2rtc + # that should be used for live view in frigate WebUI. (default: name of camera) + # NOTE: In most cases this should be set at the camera level only. stream_name: camera_name # Optional: Set the height of the jsmpeg stream. (default: 720) # This must be less than or equal to the height of the detect stream. Lower resolutions diff --git a/docs/docs/guides/configuring_go2rtc.md b/docs/docs/guides/configuring_go2rtc.md index 8316376f2..5c85f7d11 100644 --- a/docs/docs/guides/configuring_go2rtc.md +++ b/docs/docs/guides/configuring_go2rtc.md @@ -13,7 +13,15 @@ Use of the bundled go2rtc is optional. You can still configure FFmpeg to connect # Setup a go2rtc stream -First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. For the best experience, you should set the stream name under go2rtc to match the name of your camera so that Frigate will automatically map it and be able to use better live view options for the camera. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#module-streams), not just rtsp. +First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#module-streams), not just rtsp. + +:::tip + +For the best experience, you should set the stream name under `go2rtc` to match the name of your camera so that Frigate will automatically map it and be able to use better live view options for the camera. + +See [the live view docs](../configuration/live.md#setting-stream-for-live-ui) for more information. + +::: ```yaml go2rtc: diff --git a/docs/docs/guides/getting_started.md b/docs/docs/guides/getting_started.md index e2a4420a3..79082970b 100644 --- a/docs/docs/guides/getting_started.md +++ b/docs/docs/guides/getting_started.md @@ -306,7 +306,9 @@ By default, Frigate will retain video of all events for 10 days. The full set of ### Step 7: Complete config -At this point you have a complete config with basic functionality. You can see the [full config reference](../configuration/reference.md) for a complete list of configuration options. +At this point you have a complete config with basic functionality. +- View [common configuration examples](../configuration/index.md#common-configuration-examples) for a list of common configuration examples. +- View [full config reference](../configuration/reference.md) for a complete list of configuration options. ### Follow up From d558ac83b6ffb92a780daf5bd97ac2b421e2f61e Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 8 Oct 2024 08:01:31 -0500 Subject: [PATCH 003/479] Search fixes (#14217) * Ensure semantic search is enabled before checking model download state * Only clear similarity search when removing similarity pill --- web/src/components/input/InputWithTags.tsx | 7 ++++++- web/src/pages/Explore.tsx | 11 ++++++----- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 3e40b36c8..6c06e67e7 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -397,6 +397,11 @@ export default function InputWithTags({ setIsSimilaritySearch(false); }, [setFilters, resetSuggestions, setSearch, setInputFocused]); + const handleClearSimilarity = useCallback(() => { + removeFilter("event_id", filters.event_id!); + removeFilter("search_type", "similarity"); + }, [removeFilter, filters]); + const handleInputBlur = useCallback( (e: React.FocusEvent) => { if ( @@ -638,7 +643,7 @@ export default function InputWithTags({ Similarity Search - - - )} - {state == "uploading" && } - {state == "submitted" && ( -
- - Submitted + {search.plus_id !== "not_enabled" && ( + + +
+
+ Submit To Frigate+
- )} -
- - +
+ Objects in locations you want to avoid are not false + positives. Submitting them as false positives will confuse + the model. +
+
+ +
+ {state == "reviewing" && search.end_time && ( + <> + + + + )} + {state == "uploading" && } + {state == "submitted" && ( +
+ + Submitted +
+ )} +
+ + + )} From 66d0ad58031bcd1fc8811503313038c3e4683552 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 13 Oct 2024 12:46:40 -0500 Subject: [PATCH 037/479] See a preview when using the timeline to export footage (#14321) * custom hook and generic video player component * add export preview dialog * export preview dialog when using timeline export * refactor search detail dialog to use new generic video player component * clean up --- .../components/filter/ReviewFilterGroup.tsx | 2 + web/src/components/overlay/ExportDialog.tsx | 59 ++++++++- .../overlay/MobileReviewSettingsDrawer.tsx | 13 +- .../components/overlay/SaveExportOverlay.tsx | 28 +++-- .../overlay/detail/SearchDetailDialog.tsx | 118 +++++------------- .../components/player/GenericVideoPlayer.tsx | 52 ++++++++ web/src/hooks/use-video-dimensions.ts | 45 +++++++ web/src/views/recording/RecordingView.tsx | 5 + 8 files changed, 224 insertions(+), 98 deletions(-) create mode 100644 web/src/components/player/GenericVideoPlayer.tsx create mode 100644 web/src/hooks/use-video-dimensions.ts diff --git a/web/src/components/filter/ReviewFilterGroup.tsx b/web/src/components/filter/ReviewFilterGroup.tsx index 6d3ee010a..a52755e6c 100644 --- a/web/src/components/filter/ReviewFilterGroup.tsx +++ b/web/src/components/filter/ReviewFilterGroup.tsx @@ -241,6 +241,8 @@ export default function ReviewFilterGroup({ mode="none" setMode={() => {}} setRange={() => {}} + showExportPreview={false} + setShowExportPreview={() => {}} /> )} diff --git a/web/src/components/overlay/ExportDialog.tsx b/web/src/components/overlay/ExportDialog.tsx index c9018c579..577415420 100644 --- a/web/src/components/overlay/ExportDialog.tsx +++ b/web/src/components/overlay/ExportDialog.tsx @@ -2,6 +2,7 @@ import { useCallback, useMemo, useState } from "react"; import { Dialog, DialogContent, + DialogDescription, DialogFooter, DialogHeader, DialogTitle, @@ -22,10 +23,13 @@ import { FrigateConfig } from "@/types/frigateConfig"; import { Popover, PopoverContent, PopoverTrigger } from "../ui/popover"; import { TimezoneAwareCalendar } from "./ReviewActivityCalendar"; import { SelectSeparator } from "../ui/select"; -import { isDesktop, isIOS } from "react-device-detect"; +import { isDesktop, isIOS, isMobile } from "react-device-detect"; import { Drawer, DrawerContent, DrawerTrigger } from "../ui/drawer"; import SaveExportOverlay from "./SaveExportOverlay"; import { getUTCOffset } from "@/utils/dateUtil"; +import { baseUrl } from "@/api/baseUrl"; +import { cn } from "@/lib/utils"; +import { GenericVideoPlayer } from "../player/GenericVideoPlayer"; const EXPORT_OPTIONS = [ "1", @@ -44,8 +48,10 @@ type ExportDialogProps = { currentTime: number; range?: TimeRange; mode: ExportMode; + showPreview: boolean; setRange: (range: TimeRange | undefined) => void; setMode: (mode: ExportMode) => void; + setShowPreview: (showPreview: boolean) => void; }; export default function ExportDialog({ camera, @@ -53,10 +59,13 @@ export default function ExportDialog({ currentTime, range, mode, + showPreview, setRange, setMode, + setShowPreview, }: ExportDialogProps) { const [name, setName] = useState(""); + const onStartExport = useCallback(() => { if (!range) { toast.error("No valid time range selected", { position: "top-center" }); @@ -109,9 +118,16 @@ export default function ExportDialog({ return ( <> + setShowPreview(true)} onSave={() => onStartExport()} onCancel={() => setMode("none")} /> @@ -525,3 +541,44 @@ function CustomTimeSelector({ ); } + +type ExportPreviewDialogProps = { + camera: string; + range?: TimeRange; + showPreview: boolean; + setShowPreview: (showPreview: boolean) => void; +}; + +export function ExportPreviewDialog({ + camera, + range, + showPreview, + setShowPreview, +}: ExportPreviewDialogProps) { + if (!range) { + return null; + } + + const source = `${baseUrl}vod/${camera}/start/${range.after}/end/${range.before}/index.m3u8`; + + return ( + + + + Preview Export + + Preview Export + + + + + + ); +} diff --git a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx index c9879b8cb..fe0e13c11 100644 --- a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx +++ b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx @@ -3,7 +3,7 @@ import { Drawer, DrawerContent, DrawerTrigger } from "../ui/drawer"; import { Button } from "../ui/button"; import { FaArrowDown, FaCalendarAlt, FaCog, FaFilter } from "react-icons/fa"; import { TimeRange } from "@/types/timeline"; -import { ExportContent } from "./ExportDialog"; +import { ExportContent, ExportPreviewDialog } from "./ExportDialog"; import { ExportMode } from "@/types/filter"; import ReviewActivityCalendar from "./ReviewActivityCalendar"; import { SelectSeparator } from "../ui/select"; @@ -34,12 +34,14 @@ type MobileReviewSettingsDrawerProps = { currentTime: number; range?: TimeRange; mode: ExportMode; + showExportPreview: boolean; reviewSummary?: ReviewSummary; allLabels: string[]; allZones: string[]; onUpdateFilter: (filter: ReviewFilter) => void; setRange: (range: TimeRange | undefined) => void; setMode: (mode: ExportMode) => void; + setShowExportPreview: (showPreview: boolean) => void; }; export default function MobileReviewSettingsDrawer({ features = DEFAULT_DRAWER_FEATURES, @@ -50,12 +52,14 @@ export default function MobileReviewSettingsDrawer({ currentTime, range, mode, + showExportPreview, reviewSummary, allLabels, allZones, onUpdateFilter, setRange, setMode, + setShowExportPreview, }: MobileReviewSettingsDrawerProps) { const [drawerMode, setDrawerMode] = useState("none"); @@ -282,6 +286,13 @@ export default function MobileReviewSettingsDrawer({ show={mode == "timeline"} onSave={() => onStartExport()} onCancel={() => setMode("none")} + onPreview={() => setShowExportPreview(true)} + /> + void; onSave: () => void; onCancel: () => void; }; export default function SaveExportOverlay({ className, show, + onPreview, onSave, onCancel, }: SaveExportOverlayProps) { @@ -24,6 +26,22 @@ export default function SaveExportOverlay({ "mx-auto mt-5 text-center", )} > + + - ); diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index a04d5b8c1..45c0659d8 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -6,7 +6,7 @@ import { useFormattedTimestamp } from "@/hooks/use-date-utils"; import { getIconForLabel } from "@/utils/iconUtil"; import { useApiHost } from "@/api"; import { Button } from "../../ui/button"; -import { useCallback, useEffect, useMemo, useRef, useState } from "react"; +import { useCallback, useEffect, useMemo, useState } from "react"; import axios from "axios"; import { toast } from "sonner"; import { Textarea } from "../../ui/textarea"; @@ -21,7 +21,6 @@ import { DialogTitle, } from "@/components/ui/dialog"; import { Event } from "@/types/event"; -import HlsVideoPlayer from "@/components/player/HlsVideoPlayer"; import { baseUrl } from "@/api/baseUrl"; import { cn } from "@/lib/utils"; import ActivityIndicator from "@/components/indicators/activity-indicator"; @@ -62,8 +61,7 @@ import { TransformComponent, TransformWrapper } from "react-zoom-pan-pinch"; import { Card, CardContent } from "@/components/ui/card"; import useImageLoaded from "@/hooks/use-image-loaded"; import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator"; -import { useResizeObserver } from "@/hooks/resize-observer"; -import { VideoResolutionType } from "@/types/live"; +import { GenericVideoPlayer } from "@/components/player/GenericVideoPlayer"; const SEARCH_TABS = [ "details", @@ -599,99 +597,45 @@ function ObjectSnapshotTab({ type VideoTabProps = { search: SearchResult; }; -function VideoTab({ search }: VideoTabProps) { - const [isLoading, setIsLoading] = useState(true); - const videoRef = useRef(null); - - const endTime = useMemo(() => search.end_time ?? Date.now() / 1000, [search]); +export function VideoTab({ search }: VideoTabProps) { const navigate = useNavigate(); const { data: reviewItem } = useSWR([ `review/event/${search.id}`, ]); + const endTime = useMemo(() => search.end_time ?? Date.now() / 1000, [search]); - const containerRef = useRef(null); - - const [{ width: containerWidth, height: containerHeight }] = - useResizeObserver(containerRef); - const [videoResolution, setVideoResolution] = useState({ - width: 0, - height: 0, - }); - - const videoAspectRatio = useMemo(() => { - return videoResolution.width / videoResolution.height || 16 / 9; - }, [videoResolution]); - - const containerAspectRatio = useMemo(() => { - return containerWidth / containerHeight || 16 / 9; - }, [containerWidth, containerHeight]); - - const videoDimensions = useMemo(() => { - if (!containerWidth || !containerHeight) - return { width: "100%", height: "100%" }; - - if (containerAspectRatio > videoAspectRatio) { - const height = containerHeight; - const width = height * videoAspectRatio; - return { width: `${width}px`, height: `${height}px` }; - } else { - const width = containerWidth; - const height = width / videoAspectRatio; - return { width: `${width}px`, height: `${height}px` }; - } - }, [containerWidth, containerHeight, videoAspectRatio, containerAspectRatio]); + const source = `${baseUrl}vod/${search.camera}/start/${search.start_time}/end/${endTime}/index.m3u8`; return ( -
-
- {(isLoading || !reviewItem) && ( - - )} + + {reviewItem && (
- setIsLoading(false)} - setFullResolution={setVideoResolution} - /> - {!isLoading && reviewItem && ( -
- - - { - if (reviewItem?.id) { - const params = new URLSearchParams({ - id: reviewItem.id, - }).toString(); - navigate(`/review?${params}`); - } - }} - > - - - - View in History - -
+ className={cn( + "absolute top-2 z-10 flex items-center", + isIOS ? "right-8" : "right-2", )} + > + + + { + if (reviewItem?.id) { + const params = new URLSearchParams({ + id: reviewItem.id, + }).toString(); + navigate(`/review?${params}`); + } + }} + > + + + + View in History +
-
-
+ )} + ); } diff --git a/web/src/components/player/GenericVideoPlayer.tsx b/web/src/components/player/GenericVideoPlayer.tsx new file mode 100644 index 000000000..75f56e96f --- /dev/null +++ b/web/src/components/player/GenericVideoPlayer.tsx @@ -0,0 +1,52 @@ +import React, { useState, useRef } from "react"; +import { useVideoDimensions } from "@/hooks/use-video-dimensions"; +import HlsVideoPlayer from "./HlsVideoPlayer"; +import ActivityIndicator from "../indicators/activity-indicator"; + +type GenericVideoPlayerProps = { + source: string; + onPlaying?: () => void; + children?: React.ReactNode; +}; + +export function GenericVideoPlayer({ + source, + onPlaying, + children, +}: GenericVideoPlayerProps) { + const [isLoading, setIsLoading] = useState(true); + const videoRef = useRef(null); + const containerRef = useRef(null); + const { videoDimensions, setVideoResolution } = + useVideoDimensions(containerRef); + + return ( +
+
+ {isLoading && ( + + )} +
+ { + setIsLoading(false); + onPlaying?.(); + }} + setFullResolution={setVideoResolution} + /> + {!isLoading && children} +
+
+
+ ); +} diff --git a/web/src/hooks/use-video-dimensions.ts b/web/src/hooks/use-video-dimensions.ts new file mode 100644 index 000000000..448dd5078 --- /dev/null +++ b/web/src/hooks/use-video-dimensions.ts @@ -0,0 +1,45 @@ +import { useState, useMemo } from "react"; +import { useResizeObserver } from "./resize-observer"; + +export type VideoResolutionType = { + width: number; + height: number; +}; + +export function useVideoDimensions( + containerRef: React.RefObject, +) { + const [{ width: containerWidth, height: containerHeight }] = + useResizeObserver(containerRef); + const [videoResolution, setVideoResolution] = useState({ + width: 0, + height: 0, + }); + + const videoAspectRatio = useMemo(() => { + return videoResolution.width / videoResolution.height || 16 / 9; + }, [videoResolution]); + + const containerAspectRatio = useMemo(() => { + return containerWidth / containerHeight || 16 / 9; + }, [containerWidth, containerHeight]); + + const videoDimensions = useMemo(() => { + if (!containerWidth || !containerHeight) + return { width: "100%", height: "100%" }; + if (containerAspectRatio > videoAspectRatio) { + const height = containerHeight; + const width = height * videoAspectRatio; + return { width: `${width}px`, height: `${height}px` }; + } else { + const width = containerWidth; + const height = width / videoAspectRatio; + return { width: `${width}px`, height: `${height}px` }; + } + }, [containerWidth, containerHeight, videoAspectRatio, containerAspectRatio]); + + return { + videoDimensions, + setVideoResolution, + }; +} diff --git a/web/src/views/recording/RecordingView.tsx b/web/src/views/recording/RecordingView.tsx index 0c59cef38..535c412d4 100644 --- a/web/src/views/recording/RecordingView.tsx +++ b/web/src/views/recording/RecordingView.tsx @@ -140,6 +140,7 @@ export function RecordingView({ const [exportMode, setExportMode] = useState("none"); const [exportRange, setExportRange] = useState(); + const [showExportPreview, setShowExportPreview] = useState(false); // move to next clip @@ -412,6 +413,7 @@ export function RecordingView({ latestTime={timeRange.before} mode={exportMode} range={exportRange} + showPreview={showExportPreview} setRange={(range) => { setExportRange(range); @@ -420,6 +422,7 @@ export function RecordingView({ } }} setMode={setExportMode} + setShowPreview={setShowExportPreview} /> )} {isDesktop && ( @@ -473,11 +476,13 @@ export function RecordingView({ latestTime={timeRange.before} mode={exportMode} range={exportRange} + showExportPreview={showExportPreview} allLabels={reviewFilterList.labels} allZones={reviewFilterList.zones} onUpdateFilter={updateFilter} setRange={setExportRange} setMode={setExportMode} + setShowExportPreview={setShowExportPreview} /> From 1ec459ea3a0cf123e04514e059e9a5af0c51aac3 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 13 Oct 2024 16:25:13 -0500 Subject: [PATCH 038/479] Batch embeddings fixes (#14325) * fixes * more readable loops * more robust key check and warning message * ensure we get reindex progress on mount * use correct var for length --- frigate/embeddings/embeddings.py | 40 +++++++++++++++++----------- frigate/embeddings/functions/onnx.py | 16 ++++++++--- web/src/pages/Explore.tsx | 29 +++++++++++++++----- 3 files changed, 60 insertions(+), 25 deletions(-) diff --git a/frigate/embeddings/embeddings.py b/frigate/embeddings/embeddings.py index 8d12feb32..cb0626f7b 100644 --- a/frigate/embeddings/embeddings.py +++ b/frigate/embeddings/embeddings.py @@ -145,13 +145,18 @@ class Embeddings: ] ids = list(event_thumbs.keys()) embeddings = self.vision_embedding(images) - items = [(ids[i], serialize(embeddings[i])) for i in range(len(ids))] + + items = [] + + for i in range(len(ids)): + items.append(ids[i]) + items.append(serialize(embeddings[i])) self.db.execute_sql( """ INSERT OR REPLACE INTO vec_thumbnails(id, thumbnail_embedding) VALUES {} - """.format(", ".join(["(?, ?)"] * len(items))), + """.format(", ".join(["(?, ?)"] * len(ids))), items, ) return embeddings @@ -171,13 +176,18 @@ class Embeddings: def batch_upsert_description(self, event_descriptions: dict[str, str]) -> ndarray: embeddings = self.text_embedding(list(event_descriptions.values())) ids = list(event_descriptions.keys()) - items = [(ids[i], serialize(embeddings[i])) for i in range(len(ids))] + + items = [] + + for i in range(len(ids)): + items.append(ids[i]) + items.append(serialize(embeddings[i])) self.db.execute_sql( """ INSERT OR REPLACE INTO vec_descriptions(id, description_embedding) VALUES {} - """.format(", ".join(["(?, ?)"] * len(items))), + """.format(", ".join(["(?, ?)"] * len(ids))), items, ) @@ -196,16 +206,6 @@ class Embeddings: os.remove(os.path.join(CONFIG_DIR, ".search_stats.json")) st = time.time() - totals = { - "thumbnails": 0, - "descriptions": 0, - "processed_objects": 0, - "total_objects": 0, - "time_remaining": 0, - "status": "indexing", - } - - self.requestor.send_data(UPDATE_EMBEDDINGS_REINDEX_PROGRESS, totals) # Get total count of events to process total_events = ( @@ -216,11 +216,21 @@ class Embeddings: ) .count() ) - totals["total_objects"] = total_events batch_size = 32 current_page = 1 + totals = { + "thumbnails": 0, + "descriptions": 0, + "processed_objects": total_events - 1 if total_events < batch_size else 0, + "total_objects": total_events, + "time_remaining": 0 if total_events < batch_size else -1, + "status": "indexing", + } + + self.requestor.send_data(UPDATE_EMBEDDINGS_REINDEX_PROGRESS, totals) + events = ( Event.select() .where( diff --git a/frigate/embeddings/functions/onnx.py b/frigate/embeddings/functions/onnx.py index 765a7e88c..574822d59 100644 --- a/frigate/embeddings/functions/onnx.py +++ b/frigate/embeddings/functions/onnx.py @@ -164,8 +164,15 @@ class GenericONNXEmbedding: return [] if self.model_type == "text": + max_length = max(len(self.tokenizer.encode(text)) for text in inputs) processed_inputs = [ - self.tokenizer(text, padding=True, truncation=True, return_tensors="np") + self.tokenizer( + text, + padding="max_length", + truncation=True, + max_length=max_length, + return_tensors="np", + ) for text in inputs ] else: @@ -183,8 +190,11 @@ class GenericONNXEmbedding: if key in input_names: onnx_inputs[key].append(value[0]) - for key in onnx_inputs.keys(): - onnx_inputs[key] = np.array(onnx_inputs[key]) + for key in input_names: + if onnx_inputs.get(key): + onnx_inputs[key] = np.stack(onnx_inputs[key]) + else: + logger.warning(f"Expected input '{key}' not found in onnx_inputs") embeddings = self.runner.run(onnx_inputs)[0] return [embedding for embedding in embeddings] diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 4aebaefd1..e4bb49521 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -193,6 +193,17 @@ export default function Explore() { // embeddings reindex progress + const { send: sendReindexCommand } = useWs( + "embeddings_reindex_progress", + "embeddingsReindexProgress", + ); + + useEffect(() => { + sendReindexCommand("embeddingsReindexProgress"); + // only run on mount + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + const { payload: reindexProgress } = useEmbeddingsReindexProgress(); const embeddingsReindexing = useMemo(() => { @@ -210,10 +221,10 @@ export default function Explore() { // model states - const { send: sendCommand } = useWs("model_state", "modelState"); + const { send: sendModelCommand } = useWs("model_state", "modelState"); useEffect(() => { - sendCommand("modelState"); + sendModelCommand("modelState"); // only run on mount // eslint-disable-next-line react-hooks/exhaustive-deps }, []); @@ -299,14 +310,18 @@ export default function Explore() { />
- {reindexProgress.time_remaining >= 0 && ( + {reindexProgress.time_remaining !== null && (
- Estimated time remaining: + {reindexProgress.time_remaining === -1 + ? "Starting up..." + : "Estimated time remaining:"}
- {formatSecondsToDuration( - reindexProgress.time_remaining, - ) || "Finishing shortly"} + {reindexProgress.time_remaining >= 0 && + (formatSecondsToDuration( + reindexProgress.time_remaining, + ) || + "Finishing shortly")}
)}
From 833768172d2c60d5af5316d8b3e72c06b1f4777f Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 13 Oct 2024 16:48:54 -0500 Subject: [PATCH 039/479] UI tweaks (#14326) * small tweaks for frigate+ submission and debug object list * exclude attributes from labels colormap --- frigate/detectors/detector_config.py | 12 ++++++++++-- .../components/overlay/detail/SearchDetailDialog.tsx | 4 ++-- web/src/views/settings/ObjectSettingsView.tsx | 2 +- 3 files changed, 13 insertions(+), 5 deletions(-) diff --git a/frigate/detectors/detector_config.py b/frigate/detectors/detector_config.py index bc0a0ff11..11f08a86c 100644 --- a/frigate/detectors/detector_config.py +++ b/frigate/detectors/detector_config.py @@ -157,8 +157,16 @@ class ModelConfig(BaseModel): self._model_hash = file_hash.hexdigest() def create_colormap(self, enabled_labels: set[str]) -> None: - """Get a list of colors for enabled labels.""" - colors = generate_color_palette(len(enabled_labels)) + """Get a list of colors for enabled labels that aren't attributes.""" + colors = generate_color_palette( + len( + list( + filter( + lambda label: label not in self._all_attributes, enabled_labels + ) + ) + ) + ) self._colormap = {label: color for label, color in zip(enabled_labels, colors)} diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 45c0659d8..94d28c7c8 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -534,7 +534,7 @@ function ObjectSnapshotTab({ /> )} - {search.plus_id !== "not_enabled" && ( + {search.plus_id !== "not_enabled" && search.end_time && (
@@ -553,7 +553,7 @@ function ObjectSnapshotTab({
- {state == "reviewing" && search.end_time && ( + {state == "reviewing" && ( <>
From 4ca267ea17b161fb0d127a450e6e7a567f256b0f Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 13 Oct 2024 20:36:49 -0500 Subject: [PATCH 040/479] Search UI tweaks and bugfixes (#14328) * Publish model state and embeddings reindex in dispatcher onConnect * remove unneeded from explore * add embeddings reindex progress to statusbar * don't allow right click or show similar button if semantic search is disabled * fix status bar --- frigate/comms/dispatcher.py | 5 ++ web/src/components/Statusbar.tsx | 18 +++++++ .../overlay/detail/SearchDetailDialog.tsx | 22 ++++---- web/src/pages/Explore.tsx | 53 ++++++------------- web/src/views/search/SearchView.tsx | 6 ++- 5 files changed, 56 insertions(+), 48 deletions(-) diff --git a/frigate/comms/dispatcher.py b/frigate/comms/dispatcher.py index 4a3862eaf..1f480fa9c 100644 --- a/frigate/comms/dispatcher.py +++ b/frigate/comms/dispatcher.py @@ -179,6 +179,11 @@ class Dispatcher: } self.publish("camera_activity", json.dumps(camera_status)) + self.publish("model_state", json.dumps(self.model_state.copy())) + self.publish( + "embeddings_reindex_progress", + json.dumps(self.embeddings_reindex.copy()), + ) # Dictionary mapping topic to handlers topic_handlers = { diff --git a/web/src/components/Statusbar.tsx b/web/src/components/Statusbar.tsx index 41bd9372f..1b20b26f6 100644 --- a/web/src/components/Statusbar.tsx +++ b/web/src/components/Statusbar.tsx @@ -1,3 +1,4 @@ +import { useEmbeddingsReindexProgress } from "@/api/ws"; import { StatusBarMessagesContext, StatusMessage, @@ -41,6 +42,23 @@ export default function Statusbar() { }); }, [potentialProblems, addMessage, clearMessages]); + const { payload: reindexState } = useEmbeddingsReindexProgress(); + + useEffect(() => { + if (reindexState) { + if (reindexState.status == "indexing") { + clearMessages("embeddings-reindex"); + addMessage( + "embeddings-reindex", + `Reindexing embeddings (${Math.floor((reindexState.processed_objects / reindexState.total_objects) * 100)}% complete)`, + ); + } + if (reindexState.status === "completed") { + clearMessages("embeddings-reindex"); + } + } + }, [reindexState, addMessage, clearMessages]); + return (
diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 94d28c7c8..1cee70aaa 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -396,17 +396,19 @@ function ObjectDetailsTab({ draggable={false} src={`${apiHost}api/events/${search.id}/thumbnail.jpg`} /> - + if (setSimilarity) { + setSimilarity(); + } + }} + > + Find Similar + + )}
diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index e4bb49521..03a60a8d0 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -2,7 +2,6 @@ import { useEmbeddingsReindexProgress, useEventUpdate, useModelState, - useWs, } from "@/api/ws"; import ActivityIndicator from "@/components/indicators/activity-indicator"; import AnimatedCircularProgressBar from "@/components/ui/circular-progress-bar"; @@ -193,22 +192,11 @@ export default function Explore() { // embeddings reindex progress - const { send: sendReindexCommand } = useWs( - "embeddings_reindex_progress", - "embeddingsReindexProgress", - ); - - useEffect(() => { - sendReindexCommand("embeddingsReindexProgress"); - // only run on mount - // eslint-disable-next-line react-hooks/exhaustive-deps - }, []); - - const { payload: reindexProgress } = useEmbeddingsReindexProgress(); + const { payload: reindexState } = useEmbeddingsReindexProgress(); const embeddingsReindexing = useMemo(() => { - if (reindexProgress) { - switch (reindexProgress.status) { + if (reindexState) { + switch (reindexState.status) { case "indexing": return true; case "completed": @@ -217,18 +205,10 @@ export default function Explore() { return undefined; } } - }, [reindexProgress]); + }, [reindexState]); // model states - const { send: sendModelCommand } = useWs("model_state", "modelState"); - - useEffect(() => { - sendModelCommand("modelState"); - // only run on mount - // eslint-disable-next-line react-hooks/exhaustive-deps - }, []); - const { payload: textModelState } = useModelState( "jinaai/jina-clip-v1-text_model_fp16.onnx", ); @@ -274,7 +254,8 @@ export default function Explore() { if ( config?.semantic_search.enabled && - (!textModelState || + (!reindexState || + !textModelState || !textTokenizerState || !visionModelState || !visionFeatureExtractorState) @@ -303,24 +284,22 @@ export default function Explore() {
- {reindexProgress.time_remaining !== null && ( + {reindexState.time_remaining !== null && (
- {reindexProgress.time_remaining === -1 + {reindexState.time_remaining === -1 ? "Starting up..." : "Estimated time remaining:"}
- {reindexProgress.time_remaining >= 0 && - (formatSecondsToDuration( - reindexProgress.time_remaining, - ) || + {reindexState.time_remaining >= 0 && + (formatSecondsToDuration(reindexState.time_remaining) || "Finishing shortly")}
)} @@ -328,20 +307,20 @@ export default function Explore() { Thumbnails embedded: - {reindexProgress.thumbnails} + {reindexState.thumbnails}
Descriptions embedded: - {reindexProgress.descriptions} + {reindexState.descriptions}
Tracked objects processed: - {reindexProgress.processed_objects} /{" "} - {reindexProgress.total_objects} + {reindexState.processed_objects} /{" "} + {reindexState.total_objects}
diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 4c33f7dc8..e64affa36 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -393,7 +393,11 @@ export default function SearchView({ > setSimilaritySearch(value)} + findSimilar={() => { + if (config?.semantic_search.enabled) { + setSimilaritySearch(value); + } + }} onClick={() => onSelectSearch(value, index)} /> {(searchTerm || From 9adffa1ef5b2b8e18ce791faf6c245e86c0d5785 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 13 Oct 2024 20:34:51 -0600 Subject: [PATCH 041/479] Detection adjustments (#14329) --- frigate/detectors/detector_config.py | 16 +++++--------- frigate/util/model.py | 33 +++++++++++++--------------- 2 files changed, 21 insertions(+), 28 deletions(-) diff --git a/frigate/detectors/detector_config.py b/frigate/detectors/detector_config.py index 11f08a86c..90937d8f4 100644 --- a/frigate/detectors/detector_config.py +++ b/frigate/detectors/detector_config.py @@ -158,17 +158,13 @@ class ModelConfig(BaseModel): def create_colormap(self, enabled_labels: set[str]) -> None: """Get a list of colors for enabled labels that aren't attributes.""" - colors = generate_color_palette( - len( - list( - filter( - lambda label: label not in self._all_attributes, enabled_labels - ) - ) - ) + enabled_trackable_labels = list( + filter(lambda label: label not in self._all_attributes, enabled_labels) ) - - self._colormap = {label: color for label, color in zip(enabled_labels, colors)} + colors = generate_color_palette(len(enabled_trackable_labels)) + self._colormap = { + label: color for label, color in zip(enabled_trackable_labels, colors) + } model_config = ConfigDict(extra="forbid", protected_namespaces=()) diff --git a/frigate/util/model.py b/frigate/util/model.py index 008f5169a..685cd34ec 100644 --- a/frigate/util/model.py +++ b/frigate/util/model.py @@ -25,28 +25,23 @@ def get_ort_providers( ], ) - providers = ort.get_available_providers() + providers = [] options = [] - for provider in providers: - if provider == "TensorrtExecutionProvider": - os.makedirs("/config/model_cache/tensorrt/ort/trt-engines", exist_ok=True) - - if not requires_fp16 or os.environ.get("USE_FP_16", "True") != "False": - options.append( - { - "arena_extend_strategy": "kSameAsRequested", - "trt_fp16_enable": requires_fp16, - "trt_timing_cache_enable": True, - "trt_engine_cache_enable": True, - "trt_timing_cache_path": "/config/model_cache/tensorrt/ort", - "trt_engine_cache_path": "/config/model_cache/tensorrt/ort/trt-engines", - } - ) - else: - options.append({}) + for provider in ort.get_available_providers(): + if provider == "CUDAExecutionProvider": + providers.append(provider) + options.append( + { + "arena_extend_strategy": "kSameAsRequested", + } + ) + elif provider == "TensorrtExecutionProvider": + # TensorrtExecutionProvider uses too much memory without options to control it + pass elif provider == "OpenVINOExecutionProvider": os.makedirs("/config/model_cache/openvino/ort", exist_ok=True) + providers.append(provider) options.append( { "arena_extend_strategy": "kSameAsRequested", @@ -55,12 +50,14 @@ def get_ort_providers( } ) elif provider == "CPUExecutionProvider": + providers.append(provider) options.append( { "arena_extend_strategy": "kSameAsRequested", } ) else: + providers.append(provider) options.append({}) return (providers, options) From 72aa68cedcb5138eeec0f96eecee2fce58f27365 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 14 Oct 2024 07:23:10 -0500 Subject: [PATCH 042/479] Fix genai labels (#14330) * Publish model state and embeddings reindex in dispatcher onConnect * remove unneeded from explore * add embeddings reindex progress to statusbar * don't allow right click or show similar button if semantic search is disabled * fix status bar * Convert peewee model to dict before formatting for genai description * add embeddings reindex progress to statusbar * fix status bar * Convert peewee model to dict before formatting for genai description --- frigate/genai/__init__.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/frigate/genai/__init__.py b/frigate/genai/__init__.py index dccb74c1d..e2d509383 100644 --- a/frigate/genai/__init__.py +++ b/frigate/genai/__init__.py @@ -4,6 +4,8 @@ import importlib import os from typing import Optional +from playhouse.shortcuts import model_to_dict + from frigate.config import CameraConfig, GenAIConfig, GenAIProviderEnum from frigate.models import Event @@ -36,8 +38,9 @@ class GenAIClient: ) -> Optional[str]: """Generate a description for the frame.""" prompt = camera_config.genai.object_prompts.get( - event.label, camera_config.genai.prompt - ).format(**event) + event.label, + camera_config.genai.prompt, + ).format(**model_to_dict(event)) return self._send(prompt, thumbnails) def _init_provider(self): From 0ee32cf11006e91863b917c4faabee32e901aff8 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 14 Oct 2024 09:23:08 -0500 Subject: [PATCH 043/479] Fix yaml bug and ensure embeddings progress doesn't show until all models are loaded (#14338) --- frigate/util/builtin.py | 15 +++++---------- web/src/pages/Explore.tsx | 2 +- 2 files changed, 6 insertions(+), 11 deletions(-) diff --git a/frigate/util/builtin.py b/frigate/util/builtin.py index 7c2c5790e..6dab16206 100644 --- a/frigate/util/builtin.py +++ b/frigate/util/builtin.py @@ -183,16 +183,11 @@ def update_yaml_from_url(file_path, url): update_yaml_file(file_path, key_path, new_value_list) else: value = new_value_list[0] - if "," in value: - # Skip conversion if we're a mask or zone string - update_yaml_file(file_path, key_path, value) - else: - try: - value = ast.literal_eval(value) - except (ValueError, SyntaxError): - pass - update_yaml_file(file_path, key_path, value) - + try: + # no need to convert if we have a mask/zone string + value = ast.literal_eval(value) if "," not in value else value + except (ValueError, SyntaxError): + pass update_yaml_file(file_path, key_path, value) diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 03a60a8d0..816618fe5 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -275,7 +275,7 @@ export default function Explore() {
Search Unavailable
- {embeddingsReindexing && ( + {embeddingsReindexing && allModelsLoaded && ( <>
Search can be used after tracked object embeddings have From dd7a07bd0d01a7bef5a689ccc88661a938dff3ea Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 14 Oct 2024 10:27:50 -0500 Subject: [PATCH 044/479] Add ability to rename camera groups (#14339) * Add ability to rename camera groups * clean up * ampersand consistency --- web/src/components/filter/CameraGroupSelector.tsx | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/web/src/components/filter/CameraGroupSelector.tsx b/web/src/components/filter/CameraGroupSelector.tsx index 731f440e4..d63fcd9bf 100644 --- a/web/src/components/filter/CameraGroupSelector.tsx +++ b/web/src/components/filter/CameraGroupSelector.tsx @@ -643,6 +643,11 @@ export function CameraGroupEdit({ setIsLoading(true); + let renamingQuery = ""; + if (editingGroup && editingGroup[0] !== values.name) { + renamingQuery = `camera_groups.${editingGroup[0]}&`; + } + const order = editingGroup === undefined ? currentGroups.length + 1 @@ -655,9 +660,12 @@ export function CameraGroupEdit({ .join(""); axios - .put(`config/set?${orderQuery}&${iconQuery}${cameraQueries}`, { - requires_restart: 0, - }) + .put( + `config/set?${renamingQuery}${orderQuery}&${iconQuery}${cameraQueries}`, + { + requires_restart: 0, + }, + ) .then((res) => { if (res.status === 200) { toast.success(`Camera group (${values.name}) has been saved.`, { @@ -712,7 +720,6 @@ export function CameraGroupEdit({ From 887433fc6ab8cdab17806c2d028c16bf3b05d534 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 14 Oct 2024 15:23:02 -0600 Subject: [PATCH 045/479] Streaming download (#14346) * Send downloaded mp4 as a streaming response instead of a file * Add download button to UI * Formatting * Fix CSS and text Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> * download video button component * use download button component in review detail dialog * better filename --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --- frigate/api/media.py | 132 ++++++++---------- .../components/button/DownloadVideoButton.tsx | 75 ++++++++++ .../overlay/detail/ReviewDetailDialog.tsx | 22 ++- 3 files changed, 151 insertions(+), 78 deletions(-) create mode 100644 web/src/components/button/DownloadVideoButton.tsx diff --git a/frigate/api/media.py b/frigate/api/media.py index 5915875ab..d89774a6d 100644 --- a/frigate/api/media.py +++ b/frigate/api/media.py @@ -7,6 +7,7 @@ import os import subprocess as sp import time from datetime import datetime, timedelta, timezone +from pathlib import Path as FilePath from urllib.parse import unquote import cv2 @@ -450,8 +451,27 @@ def recording_clip( camera_name: str, start_ts: float, end_ts: float, - download: bool = False, ): + def run_download(ffmpeg_cmd: list[str], file_path: str): + with sp.Popen( + ffmpeg_cmd, + stderr=sp.PIPE, + stdout=sp.PIPE, + text=False, + ) as ffmpeg: + while True: + data = ffmpeg.stdout.read(1024) + if data is not None: + yield data + else: + if ffmpeg.returncode and ffmpeg.returncode != 0: + logger.error( + f"Failed to generate clip, ffmpeg logs: {ffmpeg.stderr.read()}" + ) + else: + FilePath(file_path).unlink(missing_ok=True) + break + recordings = ( Recordings.select( Recordings.path, @@ -467,18 +487,18 @@ def recording_clip( .order_by(Recordings.start_time.asc()) ) - playlist_lines = [] - clip: Recordings - for clip in recordings: - playlist_lines.append(f"file '{clip.path}'") - # if this is the starting clip, add an inpoint - if clip.start_time < start_ts: - playlist_lines.append(f"inpoint {int(start_ts - clip.start_time)}") - # if this is the ending clip, add an outpoint - if clip.end_time > end_ts: - playlist_lines.append(f"outpoint {int(end_ts - clip.start_time)}") - - file_name = sanitize_filename(f"clip_{camera_name}_{start_ts}-{end_ts}.mp4") + file_name = sanitize_filename(f"playlist_{camera_name}_{start_ts}-{end_ts}.txt") + file_path = f"/tmp/cache/{file_name}" + with open(file_path, "w") as file: + clip: Recordings + for clip in recordings: + file.write(f"file '{clip.path}'\n") + # if this is the starting clip, add an inpoint + if clip.start_time < start_ts: + file.write(f"inpoint {int(start_ts - clip.start_time)}\n") + # if this is the ending clip, add an outpoint + if clip.end_time > end_ts: + file.write(f"outpoint {int(end_ts - clip.start_time)}\n") if len(file_name) > 1000: return JSONResponse( @@ -489,67 +509,32 @@ def recording_clip( status_code=403, ) - path = os.path.join(CLIPS_DIR, f"cache/{file_name}") - config: FrigateConfig = request.app.frigate_config - if not os.path.exists(path): - ffmpeg_cmd = [ - config.ffmpeg.ffmpeg_path, - "-hide_banner", - "-y", - "-protocol_whitelist", - "pipe,file", - "-f", - "concat", - "-safe", - "0", - "-i", - "/dev/stdin", - "-c", - "copy", - "-movflags", - "+faststart", - path, - ] - p = sp.run( - ffmpeg_cmd, - input="\n".join(playlist_lines), - encoding="ascii", - capture_output=True, - ) + ffmpeg_cmd = [ + config.ffmpeg.ffmpeg_path, + "-hide_banner", + "-y", + "-protocol_whitelist", + "pipe,file", + "-f", + "concat", + "-safe", + "0", + "-i", + file_path, + "-c", + "copy", + "-movflags", + "frag_keyframe+empty_moov", + "-f", + "mp4", + "pipe:", + ] - if p.returncode != 0: - logger.error(p.stderr) - return JSONResponse( - content={ - "success": False, - "message": "Could not create clip from recordings", - }, - status_code=500, - ) - else: - logger.debug( - f"Ignoring subsequent request for {path} as it already exists in the cache." - ) - - headers = { - "Content-Description": "File Transfer", - "Cache-Control": "no-cache", - "Content-Type": "video/mp4", - "Content-Length": str(os.path.getsize(path)), - # nginx: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers - "X-Accel-Redirect": f"/clips/cache/{file_name}", - } - - if download: - headers["Content-Disposition"] = "attachment; filename=%s" % file_name - - return FileResponse( - path, + return StreamingResponse( + run_download(ffmpeg_cmd, file_path), media_type="video/mp4", - filename=file_name, - headers=headers, ) @@ -1028,7 +1013,7 @@ def event_snapshot_clean(request: Request, event_id: str, download: bool = False @router.get("/events/{event_id}/clip.mp4") -def event_clip(request: Request, event_id: str, download: bool = False): +def event_clip(request: Request, event_id: str): try: event: Event = Event.get(Event.id == event_id) except DoesNotExist: @@ -1048,7 +1033,7 @@ def event_clip(request: Request, event_id: str, download: bool = False): end_ts = ( datetime.now().timestamp() if event.end_time is None else event.end_time ) - return recording_clip(request, event.camera, event.start_time, end_ts, download) + return recording_clip(request, event.camera, event.start_time, end_ts) headers = { "Content-Description": "File Transfer", @@ -1059,9 +1044,6 @@ def event_clip(request: Request, event_id: str, download: bool = False): "X-Accel-Redirect": f"/clips/{file_name}", } - if download: - headers["Content-Disposition"] = "attachment; filename=%s" % file_name - return FileResponse( clip_path, media_type="video/mp4", diff --git a/web/src/components/button/DownloadVideoButton.tsx b/web/src/components/button/DownloadVideoButton.tsx new file mode 100644 index 000000000..ffb50098e --- /dev/null +++ b/web/src/components/button/DownloadVideoButton.tsx @@ -0,0 +1,75 @@ +import { useState } from "react"; +import { Button } from "@/components/ui/button"; +import { toast } from "sonner"; +import ActivityIndicator from "../indicators/activity-indicator"; +import { FaDownload } from "react-icons/fa"; +import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; + +type DownloadVideoButtonProps = { + source: string; + camera: string; + startTime: number; +}; + +export function DownloadVideoButton({ + source, + camera, + startTime, +}: DownloadVideoButtonProps) { + const [isDownloading, setIsDownloading] = useState(false); + + const handleDownload = async () => { + setIsDownloading(true); + const formattedDate = formatUnixTimestampToDateTime(startTime, { + strftime_fmt: "%D-%T", + time_style: "medium", + date_style: "medium", + }); + const filename = `${camera}_${formattedDate}.mp4`; + + try { + const response = await fetch(source); + const blob = await response.blob(); + const url = window.URL.createObjectURL(blob); + const a = document.createElement("a"); + a.style.display = "none"; + a.href = url; + a.download = filename; + document.body.appendChild(a); + a.click(); + window.URL.revokeObjectURL(url); + toast.success( + "Your review item video has been downloaded successfully.", + { + position: "top-center", + }, + ); + } catch (error) { + toast.error( + "There was an error downloading the review item video. Please try again.", + { + position: "top-center", + }, + ); + } finally { + setIsDownloading(false); + } + }; + + return ( +
+ +
+ ); +} diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index ae0397470..fb3a95b57 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -38,6 +38,8 @@ import { MobilePageTitle, } from "@/components/mobile/MobilePage"; import { useOverlayState } from "@/hooks/use-overlay-state"; +import { DownloadVideoButton } from "@/components/button/DownloadVideoButton"; +import { TooltipPortal } from "@radix-ui/react-tooltip"; type ReviewDetailDialogProps = { review?: ReviewSegment; @@ -143,7 +145,7 @@ export default function ReviewDetailDialog({ Review item details
- Share this review item + + Share this review item + + + + + + + + Download +
@@ -180,7 +196,7 @@ export default function ReviewDetailDialog({
-
+
Objects
{events?.map((event) => { From 3879fde06d5f0b9402528efe141902f8d6434e3f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 14 Oct 2024 16:11:43 -0600 Subject: [PATCH 046/479] Don't allow unlimited unprocessed segments to stay in cache (#14341) * Don't allow unlimited unprocessed frames to stay in cache * Formatting --- frigate/record/maintainer.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/frigate/record/maintainer.py b/frigate/record/maintainer.py index f43d1424f..314ff3646 100644 --- a/frigate/record/maintainer.py +++ b/frigate/record/maintainer.py @@ -142,6 +142,8 @@ class RecordingMaintainer(threading.Thread): ) ) ) + + # see if the recording mover is too slow and segments need to be deleted if processed_segment_count > keep_count: logger.warning( f"Unable to keep up with recording segments in cache for {camera}. Keeping the {keep_count} most recent segments out of {processed_segment_count} and discarding the rest..." @@ -153,6 +155,21 @@ class RecordingMaintainer(threading.Thread): self.end_time_cache.pop(cache_path, None) grouped_recordings[camera] = grouped_recordings[camera][-keep_count:] + # see if detection has failed and unprocessed segments need to be deleted + unprocessed_segment_count = ( + len(grouped_recordings[camera]) - processed_segment_count + ) + if unprocessed_segment_count > keep_count: + logger.warning( + f"Too many unprocessed recording segments in cache for {camera}. This likely indicates an issue with the detect stream, keeping the {keep_count} most recent segments out of {unprocessed_segment_count} and discarding the rest..." + ) + to_remove = grouped_recordings[camera][:-keep_count] + for rec in to_remove: + cache_path = rec["cache_path"] + Path(cache_path).unlink(missing_ok=True) + self.end_time_cache.pop(cache_path, None) + grouped_recordings[camera] = grouped_recordings[camera][-keep_count:] + tasks = [] for camera, recordings in grouped_recordings.items(): # clear out all the object recording info for old frames From 0abd514064e9ad3e324cc4d9c364e8d5729cc711 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 14 Oct 2024 18:53:25 -0500 Subject: [PATCH 047/479] Use direct download link instead of blob method (#14347) --- .../components/button/DownloadVideoButton.tsx | 72 ++++++++----------- 1 file changed, 31 insertions(+), 41 deletions(-) diff --git a/web/src/components/button/DownloadVideoButton.tsx b/web/src/components/button/DownloadVideoButton.tsx index ffb50098e..8a8e541fa 100644 --- a/web/src/components/button/DownloadVideoButton.tsx +++ b/web/src/components/button/DownloadVideoButton.tsx @@ -18,57 +18,47 @@ export function DownloadVideoButton({ }: DownloadVideoButtonProps) { const [isDownloading, setIsDownloading] = useState(false); - const handleDownload = async () => { - setIsDownloading(true); - const formattedDate = formatUnixTimestampToDateTime(startTime, { - strftime_fmt: "%D-%T", - time_style: "medium", - date_style: "medium", - }); - const filename = `${camera}_${formattedDate}.mp4`; + const formattedDate = formatUnixTimestampToDateTime(startTime, { + strftime_fmt: "%D-%T", + time_style: "medium", + date_style: "medium", + }); + const filename = `${camera}_${formattedDate}.mp4`; - try { - const response = await fetch(source); - const blob = await response.blob(); - const url = window.URL.createObjectURL(blob); - const a = document.createElement("a"); - a.style.display = "none"; - a.href = url; - a.download = filename; - document.body.appendChild(a); - a.click(); - window.URL.revokeObjectURL(url); - toast.success( - "Your review item video has been downloaded successfully.", - { - position: "top-center", - }, - ); - } catch (error) { - toast.error( - "There was an error downloading the review item video. Please try again.", - { - position: "top-center", - }, - ); - } finally { - setIsDownloading(false); - } + const handleDownloadStart = () => { + setIsDownloading(true); + toast.success("Your review item video has started downloading.", { + position: "top-center", + }); + }; + + const handleDownloadEnd = () => { + setIsDownloading(false); + toast.success("Download completed successfully.", { + position: "top-center", + }); }; return (
); From 0eccb6a610232394f60044d7dbe54c0c0b7df14f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 07:17:54 -0600 Subject: [PATCH 048/479] Db fixes (#14364) * Handle case where embeddings overflow token limit * Set notification tokens * Fix sort --- frigate/api/auth.py | 1 + frigate/embeddings/embeddings.py | 12 +++++++++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/frigate/api/auth.py b/frigate/api/auth.py index 5276eb71e..8976469f5 100644 --- a/frigate/api/auth.py +++ b/frigate/api/auth.py @@ -357,6 +357,7 @@ def create_user(request: Request, body: AppPostUsersBody): { User.username: body.username, User.password_hash: password_hash, + User.notification_tokens: [], } ).execute() return JSONResponse(content={"username": body.username}) diff --git a/frigate/embeddings/embeddings.py b/frigate/embeddings/embeddings.py index cb0626f7b..e4937f955 100644 --- a/frigate/embeddings/embeddings.py +++ b/frigate/embeddings/embeddings.py @@ -6,6 +6,7 @@ import logging import os import time +import onnxruntime as ort from numpy import ndarray from PIL import Image from playhouse.shortcuts import model_to_dict @@ -174,7 +175,16 @@ class Embeddings: return embedding def batch_upsert_description(self, event_descriptions: dict[str, str]) -> ndarray: - embeddings = self.text_embedding(list(event_descriptions.values())) + descs = list(event_descriptions.values()) + + try: + embeddings = self.text_embedding(descs) + except ort.RuntimeException: + half_size = len(descs) / 2 + embeddings = [] + embeddings.extend(self.text_embedding(descs[0:half_size])) + embeddings.extend(self.text_embedding(descs[half_size:])) + ids = list(event_descriptions.keys()) items = [] From 644069fb239f2774d500b5a4e9b2477c1c8737c0 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 15 Oct 2024 08:24:47 -0500 Subject: [PATCH 049/479] Explore layout changes (#14348) * Reset selected index on new searches * Remove right click for similarity search * Fix sub label icon * add card footer * Add Frigate+ dialog * Move buttons and menu to thumbnail footer * Add similarity search * Show object score * Implement download buttons * remove confidence score * conditionally show submenu items * Implement delete * fix icon color * Add object lifecycle button * fix score * delete confirmation * small tweaks * consistent icons --------- Co-authored-by: Nicolas Mowen --- web/src/components/card/SearchThumbnail.tsx | 57 +++-- .../components/card/SearchThumbnailFooter.tsx | 198 ++++++++++++++++++ web/src/components/input/InputWithTags.tsx | 6 +- .../overlay/detail/SearchDetailDialog.tsx | 17 +- web/src/pages/Explore.tsx | 3 +- web/src/types/frigateConfig.ts | 1 + web/src/views/search/SearchView.tsx | 97 ++++----- 7 files changed, 283 insertions(+), 96 deletions(-) create mode 100644 web/src/components/card/SearchThumbnailFooter.tsx diff --git a/web/src/components/card/SearchThumbnail.tsx b/web/src/components/card/SearchThumbnail.tsx index fe174e968..4ad7d7c5c 100644 --- a/web/src/components/card/SearchThumbnail.tsx +++ b/web/src/components/card/SearchThumbnail.tsx @@ -1,50 +1,56 @@ -import { useCallback } from "react"; +import { useCallback, useMemo } from "react"; import { useApiHost } from "@/api"; import { getIconForLabel } from "@/utils/iconUtil"; -import TimeAgo from "../dynamic/TimeAgo"; import useSWR from "swr"; import { FrigateConfig } from "@/types/frigateConfig"; import { isIOS, isSafari } from "react-device-detect"; import Chip from "@/components/indicators/Chip"; -import { useFormattedTimestamp } from "@/hooks/use-date-utils"; import useImageLoaded from "@/hooks/use-image-loaded"; import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip"; import ImageLoadingIndicator from "../indicators/ImageLoadingIndicator"; -import ActivityIndicator from "../indicators/activity-indicator"; import { capitalizeFirstLetter } from "@/utils/stringUtil"; import { SearchResult } from "@/types/search"; -import useContextMenu from "@/hooks/use-contextmenu"; import { cn } from "@/lib/utils"; import { TooltipPortal } from "@radix-ui/react-tooltip"; type SearchThumbnailProps = { searchResult: SearchResult; - findSimilar: () => void; onClick: (searchResult: SearchResult) => void; }; export default function SearchThumbnail({ searchResult, - findSimilar, onClick, }: SearchThumbnailProps) { const apiHost = useApiHost(); const { data: config } = useSWR("config"); const [imgRef, imgLoaded, onImgLoad] = useImageLoaded(); - useContextMenu(imgRef, findSimilar); + // interactions const handleOnClick = useCallback(() => { onClick(searchResult); }, [searchResult, onClick]); - // date + const objectLabel = useMemo(() => { + if ( + !config || + !searchResult.sub_label || + !config.model.attributes_map[searchResult.label] + ) { + return searchResult.label; + } - const formattedDate = useFormattedTimestamp( - searchResult.start_time, - config?.ui.time_format == "24hour" ? "%b %-d, %H:%M" : "%b %-d, %I:%M %p", - config?.ui.timezone, - ); + if ( + config.model.attributes_map[searchResult.label].includes( + searchResult.sub_label, + ) + ) { + return searchResult.sub_label; + } + + return `${searchResult.label}-verified`; + }, [config, searchResult]); return (
@@ -80,17 +86,21 @@ export default function SearchThumbnail({
onClick(searchResult)} > - {getIconForLabel(searchResult.label, "size-3 text-white")} + {getIconForLabel(objectLabel, "size-3 text-white")} + {Math.floor( + searchResult.score ?? searchResult.data.top_score * 100, + )} + %
- {[...new Set([searchResult.label])] + {[objectLabel] .filter( (item) => item !== undefined && !item.includes("-verified"), ) @@ -103,18 +113,7 @@ export default function SearchThumbnail({
-
-
- {searchResult.end_time ? ( - - ) : ( -
- -
- )} - {formattedDate} -
-
+
); diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx new file mode 100644 index 000000000..3e5dbe236 --- /dev/null +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -0,0 +1,198 @@ +import { useCallback, useState } from "react"; +import TimeAgo from "../dynamic/TimeAgo"; +import useSWR from "swr"; +import { FrigateConfig } from "@/types/frigateConfig"; +import { useFormattedTimestamp } from "@/hooks/use-date-utils"; +import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip"; +import ActivityIndicator from "../indicators/activity-indicator"; +import { SearchResult } from "@/types/search"; +import { + DropdownMenu, + DropdownMenuContent, + DropdownMenuItem, + DropdownMenuLabel, + DropdownMenuSeparator, + DropdownMenuTrigger, +} from "@/components/ui/dropdown-menu"; +import { + AlertDialog, + AlertDialogAction, + AlertDialogCancel, + AlertDialogContent, + AlertDialogDescription, + AlertDialogFooter, + AlertDialogHeader, + AlertDialogTitle, +} from "../ui/alert-dialog"; +import { LuCamera, LuDownload, LuMoreVertical, LuTrash2 } from "react-icons/lu"; +import FrigatePlusIcon from "@/components/icons/FrigatePlusIcon"; +import { FrigatePlusDialog } from "../overlay/dialog/FrigatePlusDialog"; +import { Event } from "@/types/event"; +import { FaArrowsRotate } from "react-icons/fa6"; +import { baseUrl } from "@/api/baseUrl"; +import axios from "axios"; +import { toast } from "sonner"; +import { MdImageSearch } from "react-icons/md"; + +type SearchThumbnailProps = { + searchResult: SearchResult; + findSimilar: () => void; + refreshResults: () => void; + showObjectLifecycle: () => void; +}; + +export default function SearchThumbnailFooter({ + searchResult, + findSimilar, + refreshResults, + showObjectLifecycle, +}: SearchThumbnailProps) { + const { data: config } = useSWR("config"); + + // interactions + + const [showFrigatePlus, setShowFrigatePlus] = useState(false); + const [deleteDialogOpen, setDeleteDialogOpen] = useState(false); + + const handleDelete = useCallback(() => { + axios + .delete(`events/${searchResult.id}`) + .then((resp) => { + if (resp.status == 200) { + toast.success("Tracked object deleted successfully.", { + position: "top-center", + }); + refreshResults(); + } + }) + .catch(() => { + toast.error("Failed to delete tracked object.", { + position: "top-center", + }); + }); + }, [searchResult, refreshResults]); + + // date + + const formattedDate = useFormattedTimestamp( + searchResult.start_time, + config?.ui.time_format == "24hour" ? "%b %-d, %H:%M" : "%b %-d, %I:%M %p", + config?.ui.timezone, + ); + + return ( + <> + setDeleteDialogOpen(!deleteDialogOpen)} + > + + + Confirm Delete + + + Are you sure you want to delete this tracked object? + + + Cancel + + Delete + + + + + setShowFrigatePlus(false)} + onEventUploaded={() => {}} + /> + +
+ {searchResult.end_time ? ( + + ) : ( +
+ +
+ )} + {formattedDate} +
+
+ {config?.plus?.enabled && + searchResult.has_snapshot && + searchResult.end_time && ( + + + setShowFrigatePlus(true)} + /> + + Submit to Frigate+ + + )} + + {config?.semantic_search?.enabled && ( + + + + + Find similar + + )} + + + + + + + + Tracked Object Actions + + + {searchResult.has_clip && ( + + + + Download video + + + )} + {searchResult.has_snapshot && ( + + + + Download snapshot + + + )} + + + View object lifecycle + + setDeleteDialogOpen(true)}> + + Delete + + + +
+ + ); +} diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 5d0786346..9ca1e4093 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -2,7 +2,6 @@ import React, { useState, useRef, useEffect, useCallback } from "react"; import { LuX, LuFilter, - LuImage, LuChevronDown, LuChevronUp, LuTrash2, @@ -44,6 +43,7 @@ import { import { toast } from "sonner"; import useSWR from "swr"; import { FrigateConfig } from "@/types/frigateConfig"; +import { MdImageSearch } from "react-icons/md"; type InputWithTagsProps = { inputFocused: boolean; @@ -514,7 +514,7 @@ export default function InputWithTags({ onFocus={handleInputFocus} onBlur={handleInputBlur} onKeyDown={handleInputKeyDown} - className="text-md h-9 pr-24" + className="text-md h-9 pr-32" placeholder="Search..." />
@@ -549,7 +549,7 @@ export default function InputWithTags({ {isSimilaritySearch && ( - diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 1cee70aaa..23734ea90 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -69,16 +69,20 @@ const SEARCH_TABS = [ "video", "object lifecycle", ] as const; -type SearchTab = (typeof SEARCH_TABS)[number]; +export type SearchTab = (typeof SEARCH_TABS)[number]; type SearchDetailDialogProps = { search?: SearchResult; + page: SearchTab; setSearch: (search: SearchResult | undefined) => void; + setSearchPage: (page: SearchTab) => void; setSimilarity?: () => void; }; export default function SearchDetailDialog({ search, + page, setSearch, + setSearchPage, setSimilarity, }: SearchDetailDialogProps) { const { data: config } = useSWR("config", { @@ -87,8 +91,11 @@ export default function SearchDetailDialog({ // tabs - const [page, setPage] = useState("details"); - const [pageToggle, setPageToggle] = useOptimisticState(page, setPage, 100); + const [pageToggle, setPageToggle] = useOptimisticState( + page, + setSearchPage, + 100, + ); // dialog and mobile page @@ -130,9 +137,9 @@ export default function SearchDetailDialog({ } if (!searchTabs.includes(pageToggle)) { - setPage("details"); + setSearchPage("details"); } - }, [pageToggle, searchTabs]); + }, [pageToggle, searchTabs, setSearchPage]); if (!search) { return; diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 816618fe5..ffbef1060 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -384,6 +384,7 @@ export default function Explore() { searchFilter={searchFilter} searchResults={searchResults} isLoading={(isLoadingInitialData || isLoadingMore) ?? true} + hasMore={!isReachingEnd} setSearch={setSearch} setSimilaritySearch={(search) => { setSearchFilter({ @@ -395,7 +396,7 @@ export default function Explore() { setSearchFilter={setSearchFilter} onUpdateFilter={setSearchFilter} loadMore={loadMore} - hasMore={!isReachingEnd} + refresh={mutate} /> )} diff --git a/web/src/types/frigateConfig.ts b/web/src/types/frigateConfig.ts index fe889ed9d..2c54b289e 100644 --- a/web/src/types/frigateConfig.ts +++ b/web/src/types/frigateConfig.ts @@ -340,6 +340,7 @@ export interface FrigateConfig { path: string | null; width: number; colormap: { [key: string]: [number, number, number] }; + attributes_map: { [key: string]: [string] }; }; motion: Record | null; diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index e64affa36..bc4f5b54d 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -1,8 +1,9 @@ import SearchThumbnail from "@/components/card/SearchThumbnail"; import SearchFilterGroup from "@/components/filter/SearchFilterGroup"; import ActivityIndicator from "@/components/indicators/activity-indicator"; -import Chip from "@/components/indicators/Chip"; -import SearchDetailDialog from "@/components/overlay/detail/SearchDetailDialog"; +import SearchDetailDialog, { + SearchTab, +} from "@/components/overlay/detail/SearchDetailDialog"; import { Toaster } from "@/components/ui/sonner"; import { Tooltip, @@ -14,7 +15,7 @@ import { FrigateConfig } from "@/types/frigateConfig"; import { SearchFilter, SearchResult, SearchSource } from "@/types/search"; import { useCallback, useEffect, useMemo, useRef, useState } from "react"; import { isDesktop, isMobileOnly } from "react-device-detect"; -import { LuColumns, LuImage, LuSearchX, LuText } from "react-icons/lu"; +import { LuColumns, LuSearchX } from "react-icons/lu"; import useSWR from "swr"; import ExploreView from "../explore/ExploreView"; import useKeyboardListener, { @@ -25,7 +26,6 @@ import InputWithTags from "@/components/input/InputWithTags"; import { ScrollArea, ScrollBar } from "@/components/ui/scroll-area"; import { isEqual } from "lodash"; import { formatDateToLocaleString } from "@/utils/dateUtil"; -import { TooltipPortal } from "@radix-ui/react-tooltip"; import { Slider } from "@/components/ui/slider"; import { Popover, @@ -33,6 +33,7 @@ import { PopoverTrigger, } from "@/components/ui/popover"; import { usePersistence } from "@/hooks/use-persistence"; +import SearchThumbnailFooter from "@/components/card/SearchThumbnailFooter"; type SearchViewProps = { search: string; @@ -40,12 +41,13 @@ type SearchViewProps = { searchFilter?: SearchFilter; searchResults?: SearchResult[]; isLoading: boolean; + hasMore: boolean; setSearch: (search: string) => void; setSimilaritySearch: (search: SearchResult) => void; setSearchFilter: (filter: SearchFilter) => void; onUpdateFilter: (filter: SearchFilter) => void; loadMore: () => void; - hasMore: boolean; + refresh: () => void; }; export default function SearchView({ search, @@ -53,12 +55,13 @@ export default function SearchView({ searchFilter, searchResults, isLoading, + hasMore, setSearch, setSimilaritySearch, setSearchFilter, onUpdateFilter, loadMore, - hasMore, + refresh, }: SearchViewProps) { const contentRef = useRef(null); const { data: config } = useSWR("config", { @@ -76,8 +79,6 @@ export default function SearchView({ "sm:grid-cols-4": effectiveColumnCount === 4, "sm:grid-cols-5": effectiveColumnCount === 5, "sm:grid-cols-6": effectiveColumnCount === 6, - "sm:grid-cols-7": effectiveColumnCount === 7, - "sm:grid-cols-8": effectiveColumnCount >= 8, }); // suggestions values @@ -161,16 +162,25 @@ export default function SearchView({ // detail const [searchDetail, setSearchDetail] = useState(); + const [page, setPage] = useState("details"); // search interaction const [selectedIndex, setSelectedIndex] = useState(null); const itemRefs = useRef<(HTMLDivElement | null)[]>([]); - const onSelectSearch = useCallback((item: SearchResult, index: number) => { - setSearchDetail(item); - setSelectedIndex(index); - }, []); + const onSelectSearch = useCallback( + (item: SearchResult, index: number, page: SearchTab = "details") => { + setPage(page); + setSearchDetail(item); + setSelectedIndex(index); + }, + [], + ); + + useEffect(() => { + setSelectedIndex(0); + }, [searchTerm, searchFilter]); // update search detail when results change @@ -187,21 +197,6 @@ export default function SearchView({ } }, [searchResults, searchDetail]); - // confidence score - - const zScoreToConfidence = (score: number) => { - // Normalizing is not needed for similarity searches - // Sigmoid function for normalized: 1 / (1 + e^x) - // Cosine for similarity - if (searchFilter) { - const notNormalized = searchFilter?.search_type?.includes("similarity"); - - const confidence = notNormalized ? 1 - score : 1 / (1 + Math.exp(score)); - - return Math.round(confidence * 100); - } - }; - const hasExistingSearch = useMemo( () => searchResults != undefined || searchFilter != undefined, [searchResults, searchFilter], @@ -310,7 +305,9 @@ export default function SearchView({ setSimilaritySearch(searchDetail)) } @@ -388,47 +385,31 @@ export default function SearchView({ >
onSelectSearch(value, index)} + /> +
+
+
+ { if (config?.semantic_search.enabled) { setSimilaritySearch(value); } }} - onClick={() => onSelectSearch(value, index)} + refreshResults={refresh} + showObjectLifecycle={() => + onSelectSearch(value, index, "object lifecycle") + } /> - {(searchTerm || - searchFilter?.search_type?.includes("similarity")) && ( -
- - - - {value.search_source == "thumbnail" ? ( - - ) : ( - - )} - {zScoreToConfidence(value.search_distance)}% - - - - - Matched {value.search_source} at{" "} - {zScoreToConfidence(value.search_distance)}% - - - -
- )}
-
); })} @@ -467,7 +448,7 @@ export default function SearchView({ setColumnCount(value)} - max={8} + max={6} min={2} step={1} className="flex-grow" From 25043278ab9a05cf7b21a37ea6ce5c5dea5cf5d9 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 07:40:45 -0600 Subject: [PATCH 050/479] Always run embedding descs one by one (#14365) --- frigate/embeddings/embeddings.py | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/frigate/embeddings/embeddings.py b/frigate/embeddings/embeddings.py index e4937f955..f6901614f 100644 --- a/frigate/embeddings/embeddings.py +++ b/frigate/embeddings/embeddings.py @@ -175,15 +175,11 @@ class Embeddings: return embedding def batch_upsert_description(self, event_descriptions: dict[str, str]) -> ndarray: - descs = list(event_descriptions.values()) + # upsert embeddings one by one to avoid token limit + embeddings = [] - try: - embeddings = self.text_embedding(descs) - except ort.RuntimeException: - half_size = len(descs) / 2 - embeddings = [] - embeddings.extend(self.text_embedding(descs[0:half_size])) - embeddings.extend(self.text_embedding(descs[half_size:])) + for desc in event_descriptions.values(): + embeddings.append(self.text_embedding([desc])) ids = list(event_descriptions.keys()) From b75efcbca2fbacfd119cf4171e422bd16f7f7f6a Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 09:37:04 -0600 Subject: [PATCH 051/479] UI tweaks (#14369) * Adjust text size * Make cursor consistent * Fix lint --- frigate/embeddings/embeddings.py | 1 - web/src/components/card/SearchThumbnailFooter.tsx | 12 +++++++++--- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/frigate/embeddings/embeddings.py b/frigate/embeddings/embeddings.py index f6901614f..19b28c6f8 100644 --- a/frigate/embeddings/embeddings.py +++ b/frigate/embeddings/embeddings.py @@ -6,7 +6,6 @@ import logging import os import time -import onnxruntime as ort from numpy import ndarray from PIL import Image from playhouse.shortcuts import model_to_dict diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index 3e5dbe236..7947b7642 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -112,7 +112,7 @@ export default function SearchThumbnailFooter({ onEventUploaded={() => {}} /> -
+
{searchResult.end_time ? ( ) : ( @@ -182,11 +182,17 @@ export default function SearchThumbnailFooter({ )} - + View object lifecycle - setDeleteDialogOpen(true)}> + setDeleteDialogOpen(true)} + > Delete From af844ea9d5c0ec54166df98c88b763affe81dc95 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 09:39:31 -0600 Subject: [PATCH 052/479] Update coral troubleshooting docs (#14370) * Update coral docs for latest ubuntu * capitalization Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --- docs/docs/troubleshooting/edgetpu.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/docs/troubleshooting/edgetpu.md b/docs/docs/troubleshooting/edgetpu.md index 8c917fa2f..33e00f11a 100644 --- a/docs/docs/troubleshooting/edgetpu.md +++ b/docs/docs/troubleshooting/edgetpu.md @@ -49,7 +49,10 @@ The USB Coral can become stuck and need to be restarted, this can happen for a n ## PCIe Coral Not Detected -The most common reason for the PCIe coral not being detected is that the driver has not been installed. See [the coral docs](https://coral.ai/docs/m2/get-started/#2-install-the-pcie-driver-and-edge-tpu-runtime) for how to install the driver for the PCIe based coral. +The most common reason for the PCIe Coral not being detected is that the driver has not been installed. This process varies based on what OS and kernel that is being run. + +- In most cases [the Coral docs](https://coral.ai/docs/m2/get-started/#2-install-the-pcie-driver-and-edge-tpu-runtime) show how to install the driver for the PCIe based Coral. +- For Ubuntu 22.04+ https://github.com/jnicolson/gasket-builder can be used to build and install the latest version of the driver. ## Only One PCIe Coral Is Detected With Coral Dual EdgeTPU From 3f1ab668999d3c27ee77e251200c0f3da81ec22f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 18:25:59 -0600 Subject: [PATCH 053/479] Embeddings UI updates (#14378) * Handle Frigate+ submitted case * Add search settings and rename general to ui settings * Add platform aware sheet component * use two columns on mobile view * Add cameras page to more filters * clean up search settings view * Add time range to side filter * better match with ui settings * fix icon size * use two columns on mobile view * clean up search settings view * Add zones and saving logic * Add all filters to side panel * better match with ui settings * fix icon size * Fix mobile fitler page * Fix embeddings access * Cleanup * Fix scroll * fix double scrollbars and add separators on mobile too * two columns on mobile * italics for emphasis --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --- frigate/embeddings/embeddings.py | 2 +- .../components/card/SearchThumbnailFooter.tsx | 33 +- .../components/filter/CamerasFilterButton.tsx | 100 ++- .../components/filter/SearchFilterGroup.tsx | 754 +----------------- .../overlay/dialog/PlatformAwareDialog.tsx | 52 ++ .../overlay/dialog/SearchFilterDialog.tsx | 448 +++++++++++ web/src/pages/Settings.tsx | 15 +- web/src/types/frigateConfig.ts | 5 +- web/src/views/search/SearchView.tsx | 18 +- web/src/views/settings/SearchSettingsView.tsx | 288 +++++++ ...ralSettingsView.tsx => UiSettingsView.tsx} | 2 +- 11 files changed, 919 insertions(+), 798 deletions(-) create mode 100644 web/src/components/overlay/dialog/SearchFilterDialog.tsx create mode 100644 web/src/views/settings/SearchSettingsView.tsx rename web/src/views/settings/{GeneralSettingsView.tsx => UiSettingsView.tsx} (99%) diff --git a/frigate/embeddings/embeddings.py b/frigate/embeddings/embeddings.py index 19b28c6f8..9ee508823 100644 --- a/frigate/embeddings/embeddings.py +++ b/frigate/embeddings/embeddings.py @@ -178,7 +178,7 @@ class Embeddings: embeddings = [] for desc in event_descriptions.values(): - embeddings.append(self.text_embedding([desc])) + embeddings.append(self.text_embedding([desc])[0]) ids = list(event_descriptions.keys()) diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index 7947b7642..1a16b3ad0 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -10,8 +10,6 @@ import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, - DropdownMenuLabel, - DropdownMenuSeparator, DropdownMenuTrigger, } from "@/components/ui/dropdown-menu"; import { @@ -33,6 +31,7 @@ import { baseUrl } from "@/api/baseUrl"; import axios from "axios"; import { toast } from "sonner"; import { MdImageSearch } from "react-icons/md"; +import { isMobileOnly } from "react-device-detect"; type SearchThumbnailProps = { searchResult: SearchResult; @@ -109,7 +108,9 @@ export default function SearchThumbnailFooter({ showFrigatePlus ? (searchResult as unknown as Event) : undefined } onClose={() => setShowFrigatePlus(false)} - onEventUploaded={() => {}} + onEventUploaded={() => { + searchResult.plus_id = "submitted"; + }} />
@@ -122,10 +123,12 @@ export default function SearchThumbnailFooter({ )} {formattedDate}
-
- {config?.plus?.enabled && +
+ {!isMobileOnly && + config?.plus?.enabled && searchResult.has_snapshot && - searchResult.end_time && ( + searchResult.end_time && + !searchResult.plus_id && ( - - Tracked Object Actions - - {searchResult.has_clip && ( View object lifecycle + + {isMobileOnly && + config?.plus?.enabled && + searchResult.has_snapshot && + searchResult.end_time && + !searchResult.plus_id && ( + setShowFrigatePlus(true)} + > + + Submit to Frigate+ + + )} setDeleteDialogOpen(true)} diff --git a/web/src/components/filter/CamerasFilterButton.tsx b/web/src/components/filter/CamerasFilterButton.tsx index 563af8752..94f1a838e 100644 --- a/web/src/components/filter/CamerasFilterButton.tsx +++ b/web/src/components/filter/CamerasFilterButton.tsx @@ -69,6 +69,70 @@ export function CamerasFilterButton({ ); const content = ( + + ); + + if (isMobile) { + return ( + { + if (!open) { + setCurrentCameras(selectedCameras); + } + + setOpen(open); + }} + > + {trigger} + + {content} + + + ); + } + + return ( + { + if (!open) { + setCurrentCameras(selectedCameras); + } + setOpen(open); + }} + > + {trigger} + {content} + + ); +} + +type CamerasFilterContentProps = { + allCameras: string[]; + currentCameras: string[] | undefined; + groups: [string, CameraGroupConfig][]; + setCurrentCameras: (cameras: string[] | undefined) => void; + setOpen: (open: boolean) => void; + updateCameraFilter: (cameras: string[] | undefined) => void; +}; +export function CamerasFilterContent({ + allCameras, + currentCameras, + groups, + setCurrentCameras, + setOpen, + updateCameraFilter, +}: CamerasFilterContentProps) { + return ( <> {isMobile && ( <> @@ -158,40 +222,4 @@ export function CamerasFilterButton({
); - - if (isMobile) { - return ( - { - if (!open) { - setCurrentCameras(selectedCameras); - } - - setOpen(open); - }} - > - {trigger} - - {content} - - - ); - } - - return ( - { - if (!open) { - setCurrentCameras(selectedCameras); - } - setOpen(open); - }} - > - {trigger} - {content} - - ); } diff --git a/web/src/components/filter/SearchFilterGroup.tsx b/web/src/components/filter/SearchFilterGroup.tsx index 8ddb3fee6..5fe301f19 100644 --- a/web/src/components/filter/SearchFilterGroup.tsx +++ b/web/src/components/filter/SearchFilterGroup.tsx @@ -1,5 +1,4 @@ import { Button } from "../ui/button"; -import { Popover, PopoverContent, PopoverTrigger } from "../ui/popover"; import useSWR from "swr"; import { FrigateConfig } from "@/types/frigateConfig"; import { useCallback, useEffect, useMemo, useState } from "react"; @@ -10,25 +9,19 @@ import { Switch } from "../ui/switch"; import { Label } from "../ui/label"; import FilterSwitch from "./FilterSwitch"; import { FilterList } from "@/types/filter"; -import { CalendarRangeFilterButton } from "./CalendarFilterButton"; import { CamerasFilterButton } from "./CamerasFilterButton"; import { DEFAULT_SEARCH_FILTERS, SearchFilter, SearchFilters, SearchSource, - DEFAULT_TIME_RANGE_AFTER, - DEFAULT_TIME_RANGE_BEFORE, } from "@/types/search"; import { DateRange } from "react-day-picker"; import { cn } from "@/lib/utils"; -import SubFilterIcon from "../icons/SubFilterIcon"; -import { FaLocationDot } from "react-icons/fa6"; import { MdLabel } from "react-icons/md"; -import SearchSourceIcon from "../icons/SearchSourceIcon"; import PlatformAwareDialog from "../overlay/dialog/PlatformAwareDialog"; -import { FaArrowRight, FaClock } from "react-icons/fa"; -import { useFormattedHour } from "@/hooks/use-date-utils"; +import SearchFilterDialog from "../overlay/dialog/SearchFilterDialog"; +import { CalendarRangeFilterButton } from "./CalendarFilterButton"; type SearchFilterGroupProps = { className: string; @@ -79,8 +72,6 @@ export default function SearchFilterGroup({ return [...labels].sort(); }, [config, filterList, filter]); - const { data: allSubLabels } = useSWR(["sub_labels", { split_joined: 1 }]); - const allZones = useMemo(() => { if (filterList?.zones) { return filterList.zones; @@ -159,6 +150,15 @@ export default function SearchFilterGroup({ }} /> )} + {filters.includes("general") && ( + { + onUpdateFilter({ ...filter, labels: newLabels }); + }} + /> + )} {filters.includes("date") && ( )} - {filters.includes("time") && ( - - onUpdateFilter({ ...filter, time_range }) - } - /> - )} - {filters.includes("zone") && allZones.length > 0 && ( - - onUpdateFilter({ ...filter, zones: newZones }) - } - /> - )} - {filters.includes("general") && ( - { - onUpdateFilter({ ...filter, labels: newLabels }); - }} - /> - )} - {filters.includes("sub") && ( - - onUpdateFilter({ ...filter, sub_labels: newSubLabels }) - } - /> - )} - {config?.semantic_search?.enabled && - filters.includes("source") && - !filter?.search_type?.includes("similarity") && ( - - onUpdateFilter({ ...filter, search_type: newSearchSource }) - } - /> - )} +
); } @@ -397,681 +355,3 @@ export function GeneralFilterContent({ ); } - -type TimeRangeFilterButtonProps = { - config?: FrigateConfig; - timeRange?: string; - updateTimeRange: (range: string | undefined) => void; -}; -function TimeRangeFilterButton({ - config, - timeRange, - updateTimeRange, -}: TimeRangeFilterButtonProps) { - const [open, setOpen] = useState(false); - const [startOpen, setStartOpen] = useState(false); - const [endOpen, setEndOpen] = useState(false); - - const [afterHour, beforeHour] = useMemo(() => { - if (!timeRange || !timeRange.includes(",")) { - return [DEFAULT_TIME_RANGE_AFTER, DEFAULT_TIME_RANGE_BEFORE]; - } - - return timeRange.split(","); - }, [timeRange]); - - const [selectedAfterHour, setSelectedAfterHour] = useState(afterHour); - const [selectedBeforeHour, setSelectedBeforeHour] = useState(beforeHour); - - // format based on locale - - const formattedAfter = useFormattedHour(config, afterHour); - const formattedBefore = useFormattedHour(config, beforeHour); - const formattedSelectedAfter = useFormattedHour(config, selectedAfterHour); - const formattedSelectedBefore = useFormattedHour(config, selectedBeforeHour); - - useEffect(() => { - setSelectedAfterHour(afterHour); - setSelectedBeforeHour(beforeHour); - // only refresh when state changes - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [timeRange]); - - const trigger = ( - - ); - const content = ( -
-
- { - if (!open) { - setStartOpen(false); - } - }} - > - - - - - { - const clock = e.target.value; - const [hour, minute, _] = clock.split(":"); - setSelectedAfterHour(`${hour}:${minute}`); - }} - /> - - - - { - if (!open) { - setEndOpen(false); - } - }} - > - - - - - { - const clock = e.target.value; - const [hour, minute, _] = clock.split(":"); - setSelectedBeforeHour(`${hour}:${minute}`); - }} - /> - - -
- -
- - -
-
- ); - - return ( - { - setOpen(open); - }} - /> - ); -} - -type ZoneFilterButtonProps = { - allZones: string[]; - selectedZones?: string[]; - updateZoneFilter: (zones: string[] | undefined) => void; -}; -function ZoneFilterButton({ - allZones, - selectedZones, - updateZoneFilter, -}: ZoneFilterButtonProps) { - const [open, setOpen] = useState(false); - - const [currentZones, setCurrentZones] = useState( - selectedZones, - ); - - const buttonText = useMemo(() => { - if (isMobile) { - return "Zones"; - } - - if (!selectedZones || selectedZones.length == 0) { - return "All Zones"; - } - - if (selectedZones.length == 1) { - return selectedZones[0]; - } - - return `${selectedZones.length} Zones`; - }, [selectedZones]); - - // ui - - useEffect(() => { - setCurrentZones(selectedZones); - // only refresh when state changes - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [selectedZones]); - - const trigger = ( - - ); - const content = ( - setOpen(false)} - /> - ); - - return ( - { - if (!open) { - setCurrentZones(selectedZones); - } - - setOpen(open); - }} - /> - ); -} - -type ZoneFilterContentProps = { - allZones?: string[]; - selectedZones?: string[]; - currentZones?: string[]; - updateZoneFilter?: (zones: string[] | undefined) => void; - setCurrentZones?: (zones: string[] | undefined) => void; - onClose: () => void; -}; -export function ZoneFilterContent({ - allZones, - selectedZones, - currentZones, - updateZoneFilter, - setCurrentZones, - onClose, -}: ZoneFilterContentProps) { - return ( - <> -
- {allZones && setCurrentZones && ( - <> - {isDesktop && } -
- - { - if (isChecked) { - setCurrentZones(undefined); - } - }} - /> -
-
- {allZones.map((item) => ( - { - if (isChecked) { - const updatedZones = currentZones - ? [...currentZones] - : []; - - updatedZones.push(item); - setCurrentZones(updatedZones); - } else { - const updatedZones = currentZones - ? [...currentZones] - : []; - - // can not deselect the last item - if (updatedZones.length > 1) { - updatedZones.splice(updatedZones.indexOf(item), 1); - setCurrentZones(updatedZones); - } - } - }} - /> - ))} -
- - )} -
- {isDesktop && } -
- - -
- - ); -} - -type SubFilterButtonProps = { - allSubLabels: string[]; - selectedSubLabels: string[] | undefined; - updateSubLabelFilter: (labels: string[] | undefined) => void; -}; -function SubFilterButton({ - allSubLabels, - selectedSubLabels, - updateSubLabelFilter, -}: SubFilterButtonProps) { - const [open, setOpen] = useState(false); - const [currentSubLabels, setCurrentSubLabels] = useState< - string[] | undefined - >(selectedSubLabels); - - const buttonText = useMemo(() => { - if (isMobile) { - return "Sub Labels"; - } - - if (!selectedSubLabels || selectedSubLabels.length == 0) { - return "All Sub Labels"; - } - - if (selectedSubLabels.length == 1) { - return selectedSubLabels[0]; - } - - return `${selectedSubLabels.length} Sub Labels`; - }, [selectedSubLabels]); - - const trigger = ( - - ); - const content = ( - setOpen(false)} - /> - ); - - return ( - { - if (!open) { - setCurrentSubLabels(selectedSubLabels); - } - - setOpen(open); - }} - /> - ); -} - -type SubFilterContentProps = { - allSubLabels: string[]; - selectedSubLabels: string[] | undefined; - currentSubLabels: string[] | undefined; - updateSubLabelFilter: (labels: string[] | undefined) => void; - setCurrentSubLabels: (labels: string[] | undefined) => void; - onClose: () => void; -}; -export function SubFilterContent({ - allSubLabels, - selectedSubLabels, - currentSubLabels, - updateSubLabelFilter, - setCurrentSubLabels, - onClose, -}: SubFilterContentProps) { - return ( - <> -
-
- - { - if (isChecked) { - setCurrentSubLabels(undefined); - } - }} - /> -
-
- {allSubLabels.map((item) => ( - { - if (isChecked) { - const updatedLabels = currentSubLabels - ? [...currentSubLabels] - : []; - - updatedLabels.push(item); - setCurrentSubLabels(updatedLabels); - } else { - const updatedLabels = currentSubLabels - ? [...currentSubLabels] - : []; - - // can not deselect the last item - if (updatedLabels.length > 1) { - updatedLabels.splice(updatedLabels.indexOf(item), 1); - setCurrentSubLabels(updatedLabels); - } - } - }} - /> - ))} -
-
- {isDesktop && } -
- - -
- - ); -} - -type SearchTypeButtonProps = { - selectedSearchSources: SearchSource[] | undefined; - updateSearchSourceFilter: (sources: SearchSource[] | undefined) => void; -}; -function SearchTypeButton({ - selectedSearchSources, - updateSearchSourceFilter, -}: SearchTypeButtonProps) { - const [open, setOpen] = useState(false); - - const buttonText = useMemo(() => { - if (isMobile) { - return "Sources"; - } - - if ( - !selectedSearchSources || - selectedSearchSources.length == 0 || - selectedSearchSources.length == 2 - ) { - return "All Search Sources"; - } - - if (selectedSearchSources.length == 1) { - return selectedSearchSources[0]; - } - - return `${selectedSearchSources.length} Search Sources`; - }, [selectedSearchSources]); - - const trigger = ( - - ); - const content = ( - setOpen(false)} - /> - ); - - return ( - - ); -} - -type SearchTypeContentProps = { - selectedSearchSources: SearchSource[] | undefined; - updateSearchSourceFilter: (sources: SearchSource[] | undefined) => void; - onClose: () => void; -}; -export function SearchTypeContent({ - selectedSearchSources, - updateSearchSourceFilter, - onClose, -}: SearchTypeContentProps) { - const [currentSearchSources, setCurrentSearchSources] = useState< - SearchSource[] | undefined - >(selectedSearchSources); - - return ( - <> -
-
- { - const updatedSources = currentSearchSources - ? [...currentSearchSources] - : []; - - if (isChecked) { - updatedSources.push("thumbnail"); - setCurrentSearchSources(updatedSources); - } else { - if (updatedSources.length > 1) { - const index = updatedSources.indexOf("thumbnail"); - if (index !== -1) updatedSources.splice(index, 1); - setCurrentSearchSources(updatedSources); - } - } - }} - /> - { - const updatedSources = currentSearchSources - ? [...currentSearchSources] - : []; - - if (isChecked) { - updatedSources.push("description"); - setCurrentSearchSources(updatedSources); - } else { - if (updatedSources.length > 1) { - const index = updatedSources.indexOf("description"); - if (index !== -1) updatedSources.splice(index, 1); - setCurrentSearchSources(updatedSources); - } - } - }} - /> -
- {isDesktop && } -
- - -
-
- - ); -} diff --git a/web/src/components/overlay/dialog/PlatformAwareDialog.tsx b/web/src/components/overlay/dialog/PlatformAwareDialog.tsx index cd84d299c..79ee64f71 100644 --- a/web/src/components/overlay/dialog/PlatformAwareDialog.tsx +++ b/web/src/components/overlay/dialog/PlatformAwareDialog.tsx @@ -1,9 +1,16 @@ +import { + MobilePage, + MobilePageContent, + MobilePageHeader, + MobilePageTitle, +} from "@/components/mobile/MobilePage"; import { Drawer, DrawerContent, DrawerTrigger } from "@/components/ui/drawer"; import { Popover, PopoverContent, PopoverTrigger, } from "@/components/ui/popover"; +import { Sheet, SheetContent, SheetTrigger } from "@/components/ui/sheet"; import { isMobile } from "react-device-detect"; type PlatformAwareDialogProps = { @@ -42,3 +49,48 @@ export default function PlatformAwareDialog({ ); } + +type PlatformAwareSheetProps = { + trigger: JSX.Element; + content: JSX.Element; + triggerClassName?: string; + contentClassName?: string; + open: boolean; + onOpenChange: (open: boolean) => void; +}; +export function PlatformAwareSheet({ + trigger, + content, + triggerClassName = "", + contentClassName = "", + open, + onOpenChange, +}: PlatformAwareSheetProps) { + if (isMobile) { + return ( +
+
onOpenChange(true)}>{trigger}
+ + + onOpenChange(false)} + > + More Filters + +
{content}
+ + +
+ ); + } + + return ( + + + {trigger} + + {content} + + ); +} diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx new file mode 100644 index 000000000..676e86cff --- /dev/null +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -0,0 +1,448 @@ +import { FaArrowRight, FaCog } from "react-icons/fa"; + +import { useEffect, useMemo, useState } from "react"; +import { PlatformAwareSheet } from "./PlatformAwareDialog"; +import { Button } from "@/components/ui/button"; +import useSWR from "swr"; +import { + DEFAULT_TIME_RANGE_AFTER, + DEFAULT_TIME_RANGE_BEFORE, + SearchFilter, + SearchSource, +} from "@/types/search"; +import { FrigateConfig } from "@/types/frigateConfig"; +import { + Popover, + PopoverContent, + PopoverTrigger, +} from "@/components/ui/popover"; +import { isDesktop, isMobileOnly } from "react-device-detect"; +import { useFormattedHour } from "@/hooks/use-date-utils"; +import Heading from "@/components/ui/heading"; +import FilterSwitch from "@/components/filter/FilterSwitch"; +import { Switch } from "@/components/ui/switch"; +import { Label } from "@/components/ui/label"; +import { DropdownMenuSeparator } from "@/components/ui/dropdown-menu"; +import { cn } from "@/lib/utils"; + +type SearchFilterDialogProps = { + config?: FrigateConfig; + filter?: SearchFilter; + filterValues: { + cameras: string[]; + labels: string[]; + zones: string[]; + search_type: SearchSource[]; + }; + onUpdateFilter: (filter: SearchFilter) => void; +}; +export default function SearchFilterDialog({ + config, + filter, + filterValues, + onUpdateFilter, +}: SearchFilterDialogProps) { + // data + + const [currentFilter, setCurrentFilter] = useState(filter ?? {}); + const { data: allSubLabels } = useSWR(["sub_labels", { split_joined: 1 }]); + + // state + + const [open, setOpen] = useState(false); + + const trigger = ( + + ); + const content = ( +
+ + setCurrentFilter({ ...currentFilter, time_range: newRange }) + } + /> + + setCurrentFilter({ ...currentFilter, zones: newZones }) + } + /> + + setCurrentFilter({ ...currentFilter, sub_labels: newSubLabels }) + } + /> + + onUpdateFilter({ ...currentFilter, search_type: newSearchSource }) + } + /> + {isDesktop && } +
+ + +
+
+ ); + + return ( + { + if (!open) { + setCurrentFilter(filter ?? {}); + } + + setOpen(open); + }} + /> + ); +} + +type TimeRangeFilterContentProps = { + config?: FrigateConfig; + timeRange?: string; + updateTimeRange: (range: string | undefined) => void; +}; +function TimeRangeFilterContent({ + config, + timeRange, + updateTimeRange, +}: TimeRangeFilterContentProps) { + const [startOpen, setStartOpen] = useState(false); + const [endOpen, setEndOpen] = useState(false); + + const [afterHour, beforeHour] = useMemo(() => { + if (!timeRange || !timeRange.includes(",")) { + return [DEFAULT_TIME_RANGE_AFTER, DEFAULT_TIME_RANGE_BEFORE]; + } + + return timeRange.split(","); + }, [timeRange]); + + const [selectedAfterHour, setSelectedAfterHour] = useState(afterHour); + const [selectedBeforeHour, setSelectedBeforeHour] = useState(beforeHour); + + // format based on locale + + const formattedSelectedAfter = useFormattedHour(config, selectedAfterHour); + const formattedSelectedBefore = useFormattedHour(config, selectedBeforeHour); + + useEffect(() => { + setSelectedAfterHour(afterHour); + setSelectedBeforeHour(beforeHour); + // only refresh when state changes + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [timeRange]); + + useEffect(() => { + if ( + selectedAfterHour == DEFAULT_TIME_RANGE_AFTER && + selectedBeforeHour == DEFAULT_TIME_RANGE_BEFORE + ) { + updateTimeRange(undefined); + } else { + updateTimeRange(`${selectedAfterHour},${selectedBeforeHour}`); + } + // only refresh when state changes + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [selectedAfterHour, selectedBeforeHour]); + + return ( +
+ Time Range +
+ { + if (!open) { + setStartOpen(false); + } + }} + > + + + + + { + const clock = e.target.value; + const [hour, minute, _] = clock.split(":"); + setSelectedAfterHour(`${hour}:${minute}`); + }} + /> + + + + { + if (!open) { + setEndOpen(false); + } + }} + > + + + + + { + const clock = e.target.value; + const [hour, minute, _] = clock.split(":"); + setSelectedBeforeHour(`${hour}:${minute}`); + }} + /> + + +
+
+ ); +} + +type ZoneFilterContentProps = { + allZones?: string[]; + zones?: string[]; + updateZones: (zones: string[] | undefined) => void; +}; +export function ZoneFilterContent({ + allZones, + zones, + updateZones, +}: ZoneFilterContentProps) { + return ( + <> +
+ + Zones + {allZones && ( + <> +
+ + { + if (isChecked) { + updateZones(undefined); + } + }} + /> +
+
+ {allZones.map((item) => ( + { + if (isChecked) { + const updatedZones = zones ? [...zones] : []; + + updatedZones.push(item); + updateZones(updatedZones); + } else { + const updatedZones = zones ? [...zones] : []; + + // can not deselect the last item + if (updatedZones.length > 1) { + updatedZones.splice(updatedZones.indexOf(item), 1); + updateZones(updatedZones); + } + } + }} + /> + ))} +
+ + )} +
+ + ); +} + +type SubFilterContentProps = { + allSubLabels: string[]; + subLabels: string[] | undefined; + setSubLabels: (labels: string[] | undefined) => void; +}; +export function SubFilterContent({ + allSubLabels, + subLabels, + setSubLabels, +}: SubFilterContentProps) { + return ( +
+ + Sub Labels +
+ + { + if (isChecked) { + setSubLabels(undefined); + } + }} + /> +
+
+ {allSubLabels.map((item) => ( + { + if (isChecked) { + const updatedLabels = subLabels ? [...subLabels] : []; + + updatedLabels.push(item); + setSubLabels(updatedLabels); + } else { + const updatedLabels = subLabels ? [...subLabels] : []; + + // can not deselect the last item + if (updatedLabels.length > 1) { + updatedLabels.splice(updatedLabels.indexOf(item), 1); + setSubLabels(updatedLabels); + } + } + }} + /> + ))} +
+
+ ); +} + +type SearchTypeContentProps = { + searchSources: SearchSource[] | undefined; + setSearchSources: (sources: SearchSource[] | undefined) => void; +}; +export function SearchTypeContent({ + searchSources, + setSearchSources, +}: SearchTypeContentProps) { + return ( + <> +
+ + Search Sources +
+ { + const updatedSources = searchSources ? [...searchSources] : []; + + if (isChecked) { + updatedSources.push("thumbnail"); + setSearchSources(updatedSources); + } else { + if (updatedSources.length > 1) { + const index = updatedSources.indexOf("thumbnail"); + if (index !== -1) updatedSources.splice(index, 1); + setSearchSources(updatedSources); + } + } + }} + /> + { + const updatedSources = searchSources ? [...searchSources] : []; + + if (isChecked) { + updatedSources.push("description"); + setSearchSources(updatedSources); + } else { + if (updatedSources.length > 1) { + const index = updatedSources.indexOf("description"); + if (index !== -1) updatedSources.splice(index, 1); + setSearchSources(updatedSources); + } + } + }} + /> +
+
+ + ); +} diff --git a/web/src/pages/Settings.tsx b/web/src/pages/Settings.tsx index 6d147b332..4ba29dd08 100644 --- a/web/src/pages/Settings.tsx +++ b/web/src/pages/Settings.tsx @@ -29,16 +29,18 @@ import { ZoneMaskFilterButton } from "@/components/filter/ZoneMaskFilter"; import { PolygonType } from "@/types/canvas"; import { ScrollArea, ScrollBar } from "@/components/ui/scroll-area"; import scrollIntoView from "scroll-into-view-if-needed"; -import GeneralSettingsView from "@/views/settings/GeneralSettingsView"; import CameraSettingsView from "@/views/settings/CameraSettingsView"; import ObjectSettingsView from "@/views/settings/ObjectSettingsView"; import MotionTunerView from "@/views/settings/MotionTunerView"; import MasksAndZonesView from "@/views/settings/MasksAndZonesView"; import AuthenticationView from "@/views/settings/AuthenticationView"; import NotificationView from "@/views/settings/NotificationsSettingsView"; +import SearchSettingsView from "@/views/settings/SearchSettingsView"; +import UiSettingsView from "@/views/settings/UiSettingsView"; const allSettingsViews = [ - "general", + "UI settings", + "search settings", "camera settings", "masks / zones", "motion tuner", @@ -49,7 +51,7 @@ const allSettingsViews = [ type SettingsType = (typeof allSettingsViews)[number]; export default function Settings() { - const [page, setPage] = useState("general"); + const [page, setPage] = useState("UI settings"); const [pageToggle, setPageToggle] = useOptimisticState(page, setPage, 100); const tabsRef = useRef(null); @@ -140,7 +142,7 @@ export default function Settings() { {Object.values(settingsViews).map((item) => (
- {page == "general" && } + {page == "UI settings" && } + {page == "search settings" && ( + + )} {page == "debug" && ( )} diff --git a/web/src/types/frigateConfig.ts b/web/src/types/frigateConfig.ts index 2c54b289e..76d9cfa67 100644 --- a/web/src/types/frigateConfig.ts +++ b/web/src/types/frigateConfig.ts @@ -27,6 +27,8 @@ export const ATTRIBUTE_LABELS = [ "ups", ]; +export type SearchModelSize = "small" | "large"; + export interface CameraConfig { audio: { enabled: boolean; @@ -418,7 +420,8 @@ export interface FrigateConfig { semantic_search: { enabled: boolean; - model_size: string; + reindex: boolean; + model_size: SearchModelSize; }; snapshots: { diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index bc4f5b54d..9427cdcff 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -73,13 +73,17 @@ export default function SearchView({ const [columnCount, setColumnCount] = usePersistence("exploreGridColumns", 4); const effectiveColumnCount = useMemo(() => columnCount ?? 4, [columnCount]); - const gridClassName = cn("grid w-full gap-2 px-1 gap-2 lg:gap-4 md:mx-2", { - "sm:grid-cols-2": effectiveColumnCount <= 2, - "sm:grid-cols-3": effectiveColumnCount === 3, - "sm:grid-cols-4": effectiveColumnCount === 4, - "sm:grid-cols-5": effectiveColumnCount === 5, - "sm:grid-cols-6": effectiveColumnCount === 6, - }); + const gridClassName = cn( + "grid w-full gap-2 px-1 gap-2 lg:gap-4 md:mx-2", + isMobileOnly && "grid-cols-2", + { + "sm:grid-cols-2": effectiveColumnCount <= 2, + "sm:grid-cols-3": effectiveColumnCount === 3, + "sm:grid-cols-4": effectiveColumnCount === 4, + "sm:grid-cols-5": effectiveColumnCount === 5, + "sm:grid-cols-6": effectiveColumnCount === 6, + }, + ); // suggestions values diff --git a/web/src/views/settings/SearchSettingsView.tsx b/web/src/views/settings/SearchSettingsView.tsx new file mode 100644 index 000000000..a08816675 --- /dev/null +++ b/web/src/views/settings/SearchSettingsView.tsx @@ -0,0 +1,288 @@ +import Heading from "@/components/ui/heading"; +import { FrigateConfig, SearchModelSize } from "@/types/frigateConfig"; +import useSWR from "swr"; +import axios from "axios"; +import ActivityIndicator from "@/components/indicators/activity-indicator"; +import { useCallback, useContext, useEffect, useState } from "react"; +import { Label } from "@/components/ui/label"; +import { Button } from "@/components/ui/button"; +import { Switch } from "@/components/ui/switch"; +import { Toaster } from "@/components/ui/sonner"; +import { toast } from "sonner"; +import { Separator } from "@/components/ui/separator"; +import { Link } from "react-router-dom"; +import { LuExternalLink } from "react-icons/lu"; +import { StatusBarMessagesContext } from "@/context/statusbar-provider"; +import { + Select, + SelectContent, + SelectGroup, + SelectItem, + SelectTrigger, +} from "@/components/ui/select"; + +type SearchSettingsViewProps = { + setUnsavedChanges: React.Dispatch>; +}; + +type SearchSettings = { + enabled?: boolean; + reindex?: boolean; + model_size?: SearchModelSize; +}; + +export default function SearchSettingsView({ + setUnsavedChanges, +}: SearchSettingsViewProps) { + const { data: config, mutate: updateConfig } = + useSWR("config"); + const [changedValue, setChangedValue] = useState(false); + const [isLoading, setIsLoading] = useState(false); + + const { addMessage, removeMessage } = useContext(StatusBarMessagesContext)!; + + const [searchSettings, setSearchSettings] = useState({ + enabled: undefined, + reindex: undefined, + model_size: undefined, + }); + + const [origSearchSettings, setOrigSearchSettings] = useState({ + enabled: undefined, + reindex: undefined, + model_size: undefined, + }); + + useEffect(() => { + if (config) { + if (searchSettings?.enabled == undefined) { + setSearchSettings({ + enabled: config.semantic_search.enabled, + reindex: config.semantic_search.reindex, + model_size: config.semantic_search.model_size, + }); + } + + setOrigSearchSettings({ + enabled: config.semantic_search.enabled, + reindex: config.semantic_search.reindex, + model_size: config.semantic_search.model_size, + }); + } + // we know that these deps are correct + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [config]); + + const handleSearchConfigChange = (newConfig: Partial) => { + setSearchSettings((prevConfig) => ({ ...prevConfig, ...newConfig })); + setUnsavedChanges(true); + setChangedValue(true); + }; + + const saveToConfig = useCallback(async () => { + setIsLoading(true); + + axios + .put( + `config/set?semantic_search.enabled=${searchSettings.enabled}&semantic_search.reindex=${searchSettings.reindex}&semantic_search.model_size=${searchSettings.model_size}`, + ) + .then((res) => { + if (res.status === 200) { + toast.success("Search settings have been saved.", { + position: "top-center", + }); + setChangedValue(false); + updateConfig(); + } else { + toast.error(`Failed to save config changes: ${res.statusText}`, { + position: "top-center", + }); + } + }) + .catch((error) => { + toast.error( + `Failed to save config changes: ${error.response.data.message}`, + { position: "top-center" }, + ); + }) + .finally(() => { + setIsLoading(false); + }); + }, [ + updateConfig, + searchSettings.enabled, + searchSettings.reindex, + searchSettings.model_size, + ]); + + const onCancel = useCallback(() => { + setSearchSettings(origSearchSettings); + setChangedValue(false); + removeMessage("search_settings", "search_settings"); + }, [origSearchSettings, removeMessage]); + + useEffect(() => { + if (changedValue) { + addMessage( + "search_settings", + `Unsaved search settings changes`, + undefined, + "search_settings", + ); + } else { + removeMessage("search_settings", "search_settings"); + } + // we know that these deps are correct + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [changedValue]); + + useEffect(() => { + document.title = "Search Settings - Frigate"; + }, []); + + if (!config) { + return ; + } + + return ( +
+ +
+ + Search Settings + + + + Semantic Search + +
+
+

+ Semantic Search in Frigate allows you to find tracked objects + within your review items using either the image itself, a + user-defined text description, or an automatically generated one. +

+ +
+ + Read the Documentation + + +
+
+
+ +
+
+ { + handleSearchConfigChange({ enabled: isChecked }); + }} + /> +
+ +
+
+
+
+ { + handleSearchConfigChange({ reindex: isChecked }); + }} + /> +
+ +
+
+
+ Re-indexing will reprocess all thumbnails and descriptions (if + enabled) and apply the embeddings on each startup.{" "} + Don't forget to disable the option after restarting! +
+
+
+
+
Model Size
+
+

+ The size of the model used for semantic search embeddings. +

+
    +
  • + Using small employs a quantized version of the + model that uses less RAM and runs faster on CPU with a very + negligible difference in embedding quality. +
    • +
  • + Using large employs the full Jina model and will + automatically run on the GPU if applicable. +
    • +
+
+
+ +
+
+ + +
+ + +
+
+
+ ); +} diff --git a/web/src/views/settings/GeneralSettingsView.tsx b/web/src/views/settings/UiSettingsView.tsx similarity index 99% rename from web/src/views/settings/GeneralSettingsView.tsx rename to web/src/views/settings/UiSettingsView.tsx index 0cb7689f6..c212073c1 100644 --- a/web/src/views/settings/GeneralSettingsView.tsx +++ b/web/src/views/settings/UiSettingsView.tsx @@ -22,7 +22,7 @@ import { const PLAYBACK_RATE_DEFAULT = isSafari ? [0.5, 1, 2] : [0.5, 1, 2, 4, 8, 16]; const WEEK_STARTS_ON = ["Sunday", "Monday"]; -export default function GeneralSettingsView() { +export default function UiSettingsView() { const { data: config } = useSWR("config"); const clearStoredLayouts = useCallback(() => { From eda52a3b8273961db60d367c25f552522c514bd6 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 16 Oct 2024 07:15:25 -0500 Subject: [PATCH 054/479] Search and search filter UI tweaks (#14381) * fix search type switches * select/unselect style for more filters button * fix reset button * fix labels scrollbar * set min width and remove modal to allow scrolling with filters open * hover colors * better match of font size * stop sheet from displaying console errors * fix detail dialog behavior --- .../components/card/SearchThumbnailFooter.tsx | 8 +-- .../components/filter/SearchFilterGroup.tsx | 8 ++- .../overlay/detail/SearchDetailDialog.tsx | 19 ++--- .../overlay/dialog/PlatformAwareDialog.tsx | 25 ++++++- .../overlay/dialog/SearchFilterDialog.tsx | 71 +++++++++++++------ 5 files changed, 86 insertions(+), 45 deletions(-) diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index 1a16b3ad0..af7606b37 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -113,7 +113,7 @@ export default function SearchThumbnailFooter({ }} /> -
+
{searchResult.end_time ? ( ) : ( @@ -132,7 +132,7 @@ export default function SearchThumbnailFooter({ setShowFrigatePlus(true)} /> @@ -144,7 +144,7 @@ export default function SearchThumbnailFooter({ @@ -154,7 +154,7 @@ export default function SearchThumbnailFooter({ - + {searchResult.has_clip && ( diff --git a/web/src/components/filter/SearchFilterGroup.tsx b/web/src/components/filter/SearchFilterGroup.tsx index 5fe301f19..ef816bb9f 100644 --- a/web/src/components/filter/SearchFilterGroup.tsx +++ b/web/src/components/filter/SearchFilterGroup.tsx @@ -253,7 +253,11 @@ function GeneralFilterButton({ { if (!open) { @@ -284,7 +288,7 @@ export function GeneralFilterContent({ }: GeneralFilterContentProps) { return ( <> -
+
- void; + setDefaultView: (view: string) => void; +}; +export default function SearchSettings({ + className, + columns, + setColumns, + defaultView, + setDefaultView, +}: SearchSettingsProps) { + const [open, setOpen] = useState(false); + + const trigger = ( + + ); + const content = ( +
+
+
+
Default Search View
+
+ When no filters are selected, display a summary of the most recent + tracked objects per label, or display an unfiltered grid. +
+
+ +
+ +
+
+
Grid Columns
+
+ Select the number of columns in the results grid. +
+
+
+ setColumns(value)} + max={6} + min={2} + step={1} + className="flex-grow" + /> + {columns} +
+
+
+ ); + + return ( + { + setOpen(open); + }} + /> + ); +} diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index ffbef1060..770b45cb8 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -7,6 +7,7 @@ import ActivityIndicator from "@/components/indicators/activity-indicator"; import AnimatedCircularProgressBar from "@/components/ui/circular-progress-bar"; import { useApiFilterArgs } from "@/hooks/use-api-filter"; import { useTimezone } from "@/hooks/use-date-utils"; +import { usePersistence } from "@/hooks/use-persistence"; import { FrigateConfig } from "@/types/frigateConfig"; import { SearchFilter, SearchQuery, SearchResult } from "@/types/search"; import { ModelState } from "@/types/ws"; @@ -28,6 +29,18 @@ export default function Explore() { revalidateOnFocus: false, }); + // grid + + const [columnCount, setColumnCount] = usePersistence("exploreGridColumns", 4); + const gridColumns = useMemo(() => columnCount ?? 4, [columnCount]); + + // default layout + + const [defaultView, setDefaultView] = usePersistence( + "exploreDefaultView", + "summary", + ); + const timezone = useTimezone(config); const [search, setSearch] = useState(""); @@ -65,7 +78,11 @@ export default function Explore() { const searchQuery: SearchQuery = useMemo(() => { // no search parameters if (searchSearchParams && Object.keys(searchSearchParams).length === 0) { - return null; + if (defaultView == "grid") { + return ["events", {}]; + } else { + return null; + } } // parameters, but no search term and not similarity @@ -117,7 +134,7 @@ export default function Explore() { include_thumbnails: 0, }, ]; - }, [searchTerm, searchSearchParams, similaritySearch, timezone]); + }, [searchTerm, searchSearchParams, similaritySearch, timezone, defaultView]); // paging @@ -385,6 +402,8 @@ export default function Explore() { searchResults={searchResults} isLoading={(isLoadingInitialData || isLoadingMore) ?? true} hasMore={!isReachingEnd} + columns={gridColumns} + defaultView={defaultView} setSearch={setSearch} setSimilaritySearch={(search) => { setSearchFilter({ @@ -395,6 +414,8 @@ export default function Explore() { }} setSearchFilter={setSearchFilter} onUpdateFilter={setSearchFilter} + setColumns={setColumnCount} + setDefaultView={setDefaultView} loadMore={loadMore} refresh={mutate} /> diff --git a/web/src/views/explore/ExploreView.tsx b/web/src/views/explore/ExploreView.tsx index e2c8e63bc..3a0b9cc7b 100644 --- a/web/src/views/explore/ExploreView.tsx +++ b/web/src/views/explore/ExploreView.tsx @@ -1,5 +1,5 @@ import { useEffect, useMemo } from "react"; -import { isIOS, isMobileOnly, isSafari } from "react-device-detect"; +import { isDesktop, isIOS, isMobileOnly, isSafari } from "react-device-detect"; import useSWR from "swr"; import { useApiHost } from "@/api"; import { cn } from "@/lib/utils"; @@ -17,6 +17,7 @@ import useImageLoaded from "@/hooks/use-image-loaded"; import ActivityIndicator from "@/components/indicators/activity-indicator"; import { useEventUpdate } from "@/api/ws"; import { isEqual } from "lodash"; +import TimeAgo from "@/components/dynamic/TimeAgo"; type ExploreViewProps = { searchDetail: SearchResult | undefined; @@ -197,6 +198,7 @@ function ExploreThumbnailImage({ className="absolute inset-0" imgLoaded={imgLoaded} /> + + {isDesktop && ( +
+ {event.end_time ? ( + + ) : ( +
+ +
+ )} +
+ )} ); } diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 9427cdcff..b22a5248a 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -5,17 +5,12 @@ import SearchDetailDialog, { SearchTab, } from "@/components/overlay/detail/SearchDetailDialog"; import { Toaster } from "@/components/ui/sonner"; -import { - Tooltip, - TooltipContent, - TooltipTrigger, -} from "@/components/ui/tooltip"; import { cn } from "@/lib/utils"; import { FrigateConfig } from "@/types/frigateConfig"; import { SearchFilter, SearchResult, SearchSource } from "@/types/search"; import { useCallback, useEffect, useMemo, useRef, useState } from "react"; -import { isDesktop, isMobileOnly } from "react-device-detect"; -import { LuColumns, LuSearchX } from "react-icons/lu"; +import { isMobileOnly } from "react-device-detect"; +import { LuSearchX } from "react-icons/lu"; import useSWR from "swr"; import ExploreView from "../explore/ExploreView"; import useKeyboardListener, { @@ -26,14 +21,8 @@ import InputWithTags from "@/components/input/InputWithTags"; import { ScrollArea, ScrollBar } from "@/components/ui/scroll-area"; import { isEqual } from "lodash"; import { formatDateToLocaleString } from "@/utils/dateUtil"; -import { Slider } from "@/components/ui/slider"; -import { - Popover, - PopoverContent, - PopoverTrigger, -} from "@/components/ui/popover"; -import { usePersistence } from "@/hooks/use-persistence"; import SearchThumbnailFooter from "@/components/card/SearchThumbnailFooter"; +import SearchSettings from "@/components/settings/SearchSettings"; type SearchViewProps = { search: string; @@ -42,12 +31,16 @@ type SearchViewProps = { searchResults?: SearchResult[]; isLoading: boolean; hasMore: boolean; + columns: number; + defaultView?: string; setSearch: (search: string) => void; setSimilaritySearch: (search: SearchResult) => void; setSearchFilter: (filter: SearchFilter) => void; onUpdateFilter: (filter: SearchFilter) => void; loadMore: () => void; refresh: () => void; + setColumns: (columns: number) => void; + setDefaultView: (name: string) => void; }; export default function SearchView({ search, @@ -56,12 +49,16 @@ export default function SearchView({ searchResults, isLoading, hasMore, + columns, + defaultView = "summary", setSearch, setSimilaritySearch, setSearchFilter, onUpdateFilter, loadMore, refresh, + setColumns, + setDefaultView, }: SearchViewProps) { const contentRef = useRef(null); const { data: config } = useSWR("config", { @@ -70,18 +67,15 @@ export default function SearchView({ // grid - const [columnCount, setColumnCount] = usePersistence("exploreGridColumns", 4); - const effectiveColumnCount = useMemo(() => columnCount ?? 4, [columnCount]); - const gridClassName = cn( "grid w-full gap-2 px-1 gap-2 lg:gap-4 md:mx-2", isMobileOnly && "grid-cols-2", { - "sm:grid-cols-2": effectiveColumnCount <= 2, - "sm:grid-cols-3": effectiveColumnCount === 3, - "sm:grid-cols-4": effectiveColumnCount === 4, - "sm:grid-cols-5": effectiveColumnCount === 5, - "sm:grid-cols-6": effectiveColumnCount === 6, + "sm:grid-cols-2": columns <= 2, + "sm:grid-cols-3": columns === 3, + "sm:grid-cols-4": columns === 4, + "sm:grid-cols-5": columns === 5, + "sm:grid-cols-6": columns === 6, }, ); @@ -342,7 +336,7 @@ export default function SearchView({ {hasExistingSearch && ( -
+
+
@@ -425,53 +425,13 @@ export default function SearchView({
{hasMore && isLoading && }
- - {isDesktop && columnCount && ( -
- - - - -
- -
- - - Adjust Grid Columns - - -
-
- Grid Columns -
-
- setColumnCount(value)} - max={6} - min={2} - step={1} - className="flex-grow" - /> - - {effectiveColumnCount} - -
-
- - -
- )} )}
{searchFilter && Object.keys(searchFilter).length === 0 && - !searchTerm && ( + !searchTerm && + defaultView == "summary" && (
Date: Wed, 16 Oct 2024 15:22:34 -0600 Subject: [PATCH 059/479] Update logos handling (#14396) * Add attribute for logos * Clean up tracked object to pass model data * Update default attributes map --- frigate/const.py | 16 +++++++++++++++- frigate/detectors/detector_config.py | 11 +++++++++++ frigate/object_processing.py | 17 +++++++---------- 3 files changed, 33 insertions(+), 11 deletions(-) diff --git a/frigate/const.py b/frigate/const.py index ad1aacd0f..c83b10e73 100644 --- a/frigate/const.py +++ b/frigate/const.py @@ -17,7 +17,21 @@ PLUS_API_HOST = "https://api.frigate.video" DEFAULT_ATTRIBUTE_LABEL_MAP = { "person": ["amazon", "face"], - "car": ["amazon", "fedex", "license_plate", "ups"], + "car": [ + "amazon", + "an_post", + "dhl", + "dpd", + "fedex", + "gls", + "license_plate", + "nzpost", + "postnl", + "postnord", + "purolator", + "ups", + "usps", + ], } LABEL_CONSOLIDATION_MAP = { "car": 0.8, diff --git a/frigate/detectors/detector_config.py b/frigate/detectors/detector_config.py index 90937d8f4..c40ef65bf 100644 --- a/frigate/detectors/detector_config.py +++ b/frigate/detectors/detector_config.py @@ -59,6 +59,7 @@ class ModelConfig(BaseModel): _merged_labelmap: Optional[Dict[int, str]] = PrivateAttr() _colormap: Dict[int, Tuple[int, int, int]] = PrivateAttr() _all_attributes: list[str] = PrivateAttr() + _all_attribute_logos: list[str] = PrivateAttr() _model_hash: str = PrivateAttr() @property @@ -73,6 +74,10 @@ class ModelConfig(BaseModel): def all_attributes(self) -> list[str]: return self._all_attributes + @property + def all_attribute_logos(self) -> list[str]: + return self._all_attribute_logos + @property def model_hash(self) -> str: return self._model_hash @@ -93,6 +98,9 @@ class ModelConfig(BaseModel): unique_attributes.update(attributes) self._all_attributes = list(unique_attributes) + self._all_attribute_logos = list( + unique_attributes - set(["face", "license_plate"]) + ) def check_and_load_plus_model( self, plus_api: PlusApi, detector: str = None @@ -140,6 +148,9 @@ class ModelConfig(BaseModel): unique_attributes.update(attributes) self._all_attributes = list(unique_attributes) + self._all_attribute_logos = list( + unique_attributes - set(["face", "license_plate"]) + ) self._merged_labelmap = { **{int(key): val for key, val in model_info["labelMap"].items()}, diff --git a/frigate/object_processing.py b/frigate/object_processing.py index a5b06b76f..6e63562a4 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -20,6 +20,7 @@ from frigate.comms.inter_process import InterProcessRequestor from frigate.config import ( CameraConfig, FrigateConfig, + ModelConfig, MqttConfig, RecordConfig, SnapshotsConfig, @@ -109,8 +110,7 @@ def is_better_thumbnail(label, current_thumb, new_obj, frame_shape) -> bool: class TrackedObject: def __init__( self, - camera, - colormap, + model_config: ModelConfig, camera_config: CameraConfig, frame_cache, obj_data: dict[str, any], @@ -120,8 +120,8 @@ class TrackedObject: del obj_data["score_history"] self.obj_data = obj_data - self.camera = camera - self.colormap = colormap + self.colormap = model_config.colormap + self.logos = model_config.all_attribute_logos self.camera_config = camera_config self.frame_cache = frame_cache self.zone_presence: dict[str, int] = {} @@ -245,9 +245,7 @@ class TrackedObject: # populate the sub_label for object with highest scoring logo if self.obj_data["label"] in ["car", "package", "person"]: recognized_logos = { - k: self.attributes[k] - for k in ["ups", "fedex", "amazon"] - if k in self.attributes + k: self.attributes[k] for k in self.logos if k in self.attributes } if len(recognized_logos) > 0: max_logo = max(recognized_logos, key=recognized_logos.get) @@ -291,7 +289,7 @@ class TrackedObject: def to_dict(self, include_thumbnail: bool = False): event = { "id": self.obj_data["id"], - "camera": self.camera, + "camera": self.camera_config.name, "frame_time": self.obj_data["frame_time"], "snapshot": self.thumbnail_data, "label": self.obj_data["label"], @@ -707,8 +705,7 @@ class CameraState: for id in new_ids: new_obj = tracked_objects[id] = TrackedObject( - self.name, - self.config.model.colormap, + self.config.model, self.camera_config, self.frame_cache, current_detections[id], From edaccd86d61d70baa34266d47faf7c8aa0eed50e Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 16 Oct 2024 18:26:47 -0600 Subject: [PATCH 060/479] Fix build (#14398) --- docker/main/install_deps.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docker/main/install_deps.sh b/docker/main/install_deps.sh index c63c015d3..46f2a5357 100755 --- a/docker/main/install_deps.sh +++ b/docker/main/install_deps.sh @@ -76,6 +76,9 @@ if [[ "${TARGETARCH}" == "amd64" ]]; then apt-get -qq install --no-install-recommends --no-install-suggests -y \ i965-va-driver-shaders + # intel packages use zst compression so we need to update dpkg + apt-get install -y dpkg + rm -f /etc/apt/sources.list.d/debian-bookworm.list # use intel apt intel packages From 8173cd77761fb740b2f19d5918d9c5145990b821 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 17 Oct 2024 06:30:52 -0500 Subject: [PATCH 061/479] Add score filter to Explore view (#14397) * backend score filtering and sorting * score filter frontend * use input for score filtering * use correct score on search thumbnail * add popover to explain top_score * revert sublabel score calc * update filters logic * fix rounding on score * wait until default view is loaded * don't turn button to selected style for similarity searches * clarify language * fix alert dialog buttons to use correct destructive variant * use root level top_score for very old events * better arrangement of thumbnail footer items on smaller screens --- frigate/api/defs/events_query_parameters.py | 3 + frigate/api/event.py | 23 ++- web/src/components/card/ReviewCard.tsx | 11 +- web/src/components/card/SearchThumbnail.tsx | 6 +- .../components/card/SearchThumbnailFooter.tsx | 188 ++++++++++-------- .../components/filter/CameraGroupSelector.tsx | 7 +- .../components/filter/ReviewActionGroup.tsx | 7 +- .../components/input/DeleteSearchDialog.tsx | 3 +- web/src/components/input/InputWithTags.tsx | 47 ++++- .../overlay/detail/SearchDetailDialog.tsx | 27 ++- .../overlay/dialog/SearchFilterDialog.tsx | 74 ++++++- web/src/components/settings/PolygonItem.tsx | 6 +- .../components/settings/SearchSettings.tsx | 48 +++-- web/src/pages/Explore.tsx | 27 ++- web/src/types/search.ts | 5 + web/src/views/search/SearchView.tsx | 7 +- 16 files changed, 353 insertions(+), 136 deletions(-) diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/events_query_parameters.py index 02bbc31ea..c4e40bd4e 100644 --- a/frigate/api/defs/events_query_parameters.py +++ b/frigate/api/defs/events_query_parameters.py @@ -45,6 +45,9 @@ class EventsSearchQueryParams(BaseModel): before: Optional[float] = None time_range: Optional[str] = DEFAULT_TIME_RANGE timezone: Optional[str] = "utc" + min_score: Optional[float] = None + max_score: Optional[float] = None + sort: Optional[str] = None class EventsSummaryQueryParams(BaseModel): diff --git a/frigate/api/event.py b/frigate/api/event.py index c716bba13..892624e53 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -348,6 +348,7 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) search_type = params.search_type include_thumbnails = params.include_thumbnails limit = params.limit + sort = params.sort # Filters cameras = params.cameras @@ -355,6 +356,8 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) zones = params.zones after = params.after before = params.before + min_score = params.min_score + max_score = params.max_score time_range = params.time_range # for similarity search @@ -430,6 +433,14 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) if before: event_filters.append((Event.start_time < before)) + if min_score is not None and max_score is not None: + event_filters.append((Event.data["score"].between(min_score, max_score))) + else: + if min_score is not None: + event_filters.append((Event.data["score"] >= min_score)) + if max_score is not None: + event_filters.append((Event.data["score"] <= max_score)) + if time_range != DEFAULT_TIME_RANGE: tz_name = params.timezone hour_modifier, minute_modifier, _ = get_tz_modifiers(tz_name) @@ -554,11 +565,19 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) processed_events.append(processed_event) - # Sort by search distance if search_results are available, otherwise by start_time + # Sort by search distance if search_results are available, otherwise by start_time as default if search_results: processed_events.sort(key=lambda x: x.get("search_distance", float("inf"))) else: - processed_events.sort(key=lambda x: x["start_time"], reverse=True) + if sort == "score_asc": + processed_events.sort(key=lambda x: x["score"]) + elif sort == "score_desc": + processed_events.sort(key=lambda x: x["score"], reverse=True) + elif sort == "date_asc": + processed_events.sort(key=lambda x: x["start_time"]) + else: + # "date_desc" default + processed_events.sort(key=lambda x: x["start_time"], reverse=True) # Limit the number of events returned processed_events = processed_events[:limit] diff --git a/web/src/components/card/ReviewCard.tsx b/web/src/components/card/ReviewCard.tsx index a28b89783..e10e009fb 100644 --- a/web/src/components/card/ReviewCard.tsx +++ b/web/src/components/card/ReviewCard.tsx @@ -34,6 +34,7 @@ import { toast } from "sonner"; import useKeyboardListener from "@/hooks/use-keyboard-listener"; import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip"; import { capitalizeFirstLetter } from "@/utils/stringUtil"; +import { buttonVariants } from "../ui/button"; type ReviewCardProps = { event: ReviewSegment; @@ -228,7 +229,10 @@ export default function ReviewCard({ setOptionsOpen(false)}> Cancel - + Delete @@ -295,7 +299,10 @@ export default function ReviewCard({ setOptionsOpen(false)}> Cancel - + Delete diff --git a/web/src/components/card/SearchThumbnail.tsx b/web/src/components/card/SearchThumbnail.tsx index 4ad7d7c5c..72a560deb 100644 --- a/web/src/components/card/SearchThumbnail.tsx +++ b/web/src/components/card/SearchThumbnail.tsx @@ -90,8 +90,10 @@ export default function SearchThumbnail({ onClick={() => onClick(searchResult)} > {getIconForLabel(objectLabel, "size-3 text-white")} - {Math.floor( - searchResult.score ?? searchResult.data.top_score * 100, + {Math.round( + (searchResult.data.score ?? + searchResult.data.top_score ?? + searchResult.top_score) * 100, )} % diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index f66aca516..b21361b18 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -32,9 +32,12 @@ import axios from "axios"; import { toast } from "sonner"; import { MdImageSearch } from "react-icons/md"; import { isMobileOnly } from "react-device-detect"; +import { buttonVariants } from "../ui/button"; +import { cn } from "@/lib/utils"; type SearchThumbnailProps = { searchResult: SearchResult; + columns: number; findSimilar: () => void; refreshResults: () => void; showObjectLifecycle: () => void; @@ -42,6 +45,7 @@ type SearchThumbnailProps = { export default function SearchThumbnailFooter({ searchResult, + columns, findSimilar, refreshResults, showObjectLifecycle, @@ -95,7 +99,7 @@ export default function SearchThumbnailFooter({ Cancel Delete @@ -113,104 +117,112 @@ export default function SearchThumbnailFooter({ }} /> -
- {searchResult.end_time ? ( - - ) : ( -
- -
+
4 && + "items-start sm:flex-col sm:gap-2 lg:flex-row lg:items-center lg:gap-1", )} - {formattedDate} -
-
- {!isMobileOnly && - config?.plus?.enabled && - searchResult.has_snapshot && - searchResult.end_time && - !searchResult.plus_id && ( + > +
+ {searchResult.end_time ? ( + + ) : ( +
+ +
+ )} + {formattedDate} +
+
+ {!isMobileOnly && + config?.plus?.enabled && + searchResult.has_snapshot && + searchResult.end_time && + !searchResult.plus_id && ( + + + setShowFrigatePlus(true)} + /> + + Submit to Frigate+ + + )} + + {config?.semantic_search?.enabled && ( - setShowFrigatePlus(true)} + onClick={findSimilar} /> - Submit to Frigate+ + Find similar )} - {config?.semantic_search?.enabled && ( - - - - - Find similar - - )} - - - - - - - {searchResult.has_clip && ( - - - - Download video - - - )} - {searchResult.has_snapshot && ( - - - - Download snapshot - - - )} - - - View object lifecycle - - - {isMobileOnly && - config?.plus?.enabled && - searchResult.has_snapshot && - searchResult.end_time && - !searchResult.plus_id && ( - setShowFrigatePlus(true)} - > - - Submit to Frigate+ + + + + + + {searchResult.has_clip && ( + + + + Download video + )} - setDeleteDialogOpen(true)} - > - - Delete - - - + {searchResult.has_snapshot && ( + + + + Download snapshot + + + )} + + + View object lifecycle + + + {isMobileOnly && + config?.plus?.enabled && + searchResult.has_snapshot && + searchResult.end_time && + !searchResult.plus_id && ( + setShowFrigatePlus(true)} + > + + Submit to Frigate+ + + )} + setDeleteDialogOpen(true)} + > + + Delete + + + +
); diff --git a/web/src/components/filter/CameraGroupSelector.tsx b/web/src/components/filter/CameraGroupSelector.tsx index d63fcd9bf..28568514e 100644 --- a/web/src/components/filter/CameraGroupSelector.tsx +++ b/web/src/components/filter/CameraGroupSelector.tsx @@ -3,7 +3,7 @@ import { isDesktop, isMobile } from "react-device-detect"; import useSWR from "swr"; import { MdHome } from "react-icons/md"; import { usePersistedOverlayState } from "@/hooks/use-overlay-state"; -import { Button } from "../ui/button"; +import { Button, buttonVariants } from "../ui/button"; import { useCallback, useMemo, useState } from "react"; import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip"; import { LuPencil, LuPlus } from "react-icons/lu"; @@ -518,7 +518,10 @@ export function CameraGroupRow({ Cancel - + Delete diff --git a/web/src/components/filter/ReviewActionGroup.tsx b/web/src/components/filter/ReviewActionGroup.tsx index c637b1e35..f2d67efd7 100644 --- a/web/src/components/filter/ReviewActionGroup.tsx +++ b/web/src/components/filter/ReviewActionGroup.tsx @@ -1,7 +1,7 @@ import { FaCircleCheck } from "react-icons/fa6"; import { useCallback, useState } from "react"; import axios from "axios"; -import { Button } from "../ui/button"; +import { Button, buttonVariants } from "../ui/button"; import { isDesktop } from "react-device-detect"; import { FaCompactDisc } from "react-icons/fa"; import { HiTrash } from "react-icons/hi"; @@ -79,7 +79,10 @@ export default function ReviewActionGroup({ Cancel - + Delete diff --git a/web/src/components/input/DeleteSearchDialog.tsx b/web/src/components/input/DeleteSearchDialog.tsx index 0aaabdde5..735f52b26 100644 --- a/web/src/components/input/DeleteSearchDialog.tsx +++ b/web/src/components/input/DeleteSearchDialog.tsx @@ -8,6 +8,7 @@ import { AlertDialogHeader, AlertDialogTitle, } from "@/components/ui/alert-dialog"; +import { buttonVariants } from "../ui/button"; type DeleteSearchDialogProps = { isOpen: boolean; @@ -35,7 +36,7 @@ export function DeleteSearchDialog({ Cancel Delete diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 9ca1e4093..45dfd6c32 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -201,10 +201,13 @@ export default function InputWithTags({ allSuggestions[type as FilterType]?.includes(value) || type == "before" || type == "after" || - type == "time_range" + type == "time_range" || + type == "min_score" || + type == "max_score" ) { const newFilters = { ...filters }; let timestamp = 0; + let score = 0; switch (type) { case "before": @@ -244,6 +247,40 @@ export default function InputWithTags({ newFilters[type] = timestamp / 1000; } break; + case "min_score": + case "max_score": + score = parseInt(value); + if (score >= 0) { + // Check for conflicts between min_score and max_score + if ( + type === "min_score" && + filters.max_score !== undefined && + score > filters.max_score * 100 + ) { + toast.error( + "The 'min_score' must be less than or equal to the 'max_score'.", + { + position: "top-center", + }, + ); + return; + } + if ( + type === "max_score" && + filters.min_score !== undefined && + score < filters.min_score * 100 + ) { + toast.error( + "The 'max_score' must be greater than or equal to the 'min_score'.", + { + position: "top-center", + }, + ); + return; + } + newFilters[type] = score / 100; + } + break; case "time_range": newFilters[type] = value; break; @@ -302,6 +339,8 @@ export default function InputWithTags({ } - ${ config?.ui.time_format === "24hour" ? endTime : convertTo12Hour(endTime) }`; + } else if (filterType === "min_score" || filterType === "max_score") { + return Math.round(Number(filterValues) * 100).toString() + "%"; } else { return filterValues as string; } @@ -320,7 +359,11 @@ export default function InputWithTags({ isValidTimeRange( trimmedValue.replace("-", ","), config?.ui.time_format, - )) + )) || + ((filterType === "min_score" || filterType === "max_score") && + !isNaN(Number(trimmedValue)) && + Number(trimmedValue) >= 50 && + Number(trimmedValue) <= 100) ) { createFilter( filterType, diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 37813645b..fe150bd56 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -62,6 +62,12 @@ import { Card, CardContent } from "@/components/ui/card"; import useImageLoaded from "@/hooks/use-image-loaded"; import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator"; import { GenericVideoPlayer } from "@/components/player/GenericVideoPlayer"; +import { + Popover, + PopoverContent, + PopoverTrigger, +} from "@/components/ui/popover"; +import { LuInfo } from "react-icons/lu"; const SEARCH_TABS = [ "details", @@ -279,7 +285,7 @@ function ObjectDetailsTab({ return 0; } - const value = search.score ?? search.data.top_score; + const value = search.data.top_score; return Math.round(value * 100); }, [search]); @@ -369,7 +375,24 @@ function ObjectDetailsTab({
-
Score
+
+
+ Top Score + + +
+ + Info +
+ + + The top score is the highest median score for the tracked + object, so this may differ from the score shown on the + search result thumbnail. + + +
+
{score}%{subLabelScore && ` (${subLabelScore}%)`}
diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx index f1fd4c676..ad9fe1c2b 100644 --- a/web/src/components/overlay/dialog/SearchFilterDialog.tsx +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -23,6 +23,8 @@ import { Switch } from "@/components/ui/switch"; import { Label } from "@/components/ui/label"; import { DropdownMenuSeparator } from "@/components/ui/dropdown-menu"; import { cn } from "@/lib/utils"; +import { DualThumbSlider } from "@/components/ui/slider"; +import { Input } from "@/components/ui/input"; type SearchFilterDialogProps = { config?: FrigateConfig; @@ -46,6 +48,12 @@ export default function SearchFilterDialog({ const [currentFilter, setCurrentFilter] = useState(filter ?? {}); const { data: allSubLabels } = useSWR(["sub_labels", { split_joined: 1 }]); + useEffect(() => { + if (filter) { + setCurrentFilter(filter); + } + }, [filter]); + // state const [open, setOpen] = useState(false); @@ -54,9 +62,12 @@ export default function SearchFilterDialog({ () => currentFilter && (currentFilter.time_range || + (currentFilter.min_score ?? 0) > 0.5 || + (currentFilter.max_score ?? 1) < 1 || (currentFilter.zones?.length ?? 0) > 0 || (currentFilter.sub_labels?.length ?? 0) > 0 || - (currentFilter.search_type?.length ?? 2) !== 2), + (!currentFilter.search_type?.includes("similarity") && + (currentFilter.search_type?.length ?? 2) !== 2)), [currentFilter], ); @@ -97,6 +108,13 @@ export default function SearchFilterDialog({ setCurrentFilter({ ...currentFilter, sub_labels: newSubLabels }) } /> + + setCurrentFilter({ ...currentFilter, min_score: min, max_score: max }) + } + /> {config?.semantic_search?.enabled && !currentFilter?.search_type?.includes("similarity") && ( @@ -420,6 +440,58 @@ export function SubFilterContent({ ); } +type ScoreFilterContentProps = { + minScore: number | undefined; + maxScore: number | undefined; + setScoreRange: (min: number | undefined, max: number | undefined) => void; +}; +export function ScoreFilterContent({ + minScore, + maxScore, + setScoreRange, +}: ScoreFilterContentProps) { + return ( +
+ +
Score
+
+ { + const value = e.target.value; + + if (value) { + setScoreRange(parseInt(value) / 100.0, maxScore ?? 1.0); + } + }} + /> + setScoreRange(min, max)} + /> + { + const value = e.target.value; + + if (value) { + setScoreRange(minScore ?? 0.5, parseInt(value) / 100.0); + } + }} + /> +
+
+ ); +} + type SearchTypeContentProps = { searchSources: SearchSource[] | undefined; setSearchSources: (sources: SearchSource[] | undefined) => void; diff --git a/web/src/components/settings/PolygonItem.tsx b/web/src/components/settings/PolygonItem.tsx index 68aa89978..bc1db92c3 100644 --- a/web/src/components/settings/PolygonItem.tsx +++ b/web/src/components/settings/PolygonItem.tsx @@ -35,6 +35,7 @@ import { FrigateConfig } from "@/types/frigateConfig"; import { reviewQueries } from "@/utils/zoneEdutUtil"; import IconWrapper from "../ui/icon-wrapper"; import { StatusBarMessagesContext } from "@/context/statusbar-provider"; +import { buttonVariants } from "../ui/button"; type PolygonItemProps = { polygon: Polygon; @@ -257,7 +258,10 @@ export default function PolygonItem({ Cancel - + Delete diff --git a/web/src/components/settings/SearchSettings.tsx b/web/src/components/settings/SearchSettings.tsx index d1c0856ef..b3a1e89d3 100644 --- a/web/src/components/settings/SearchSettings.tsx +++ b/web/src/components/settings/SearchSettings.tsx @@ -1,6 +1,6 @@ import { Button } from "../ui/button"; import { useState } from "react"; -import { isDesktop } from "react-device-detect"; +import { isDesktop, isMobileOnly } from "react-device-detect"; import { cn } from "@/lib/utils"; import PlatformAwareDialog from "../overlay/dialog/PlatformAwareDialog"; import { FaCog } from "react-icons/fa"; @@ -40,7 +40,7 @@ export default function SearchSettings({
-
Default Search View
+
Default View
When no filters are selected, display a summary of the most recent tracked objects per label, or display an unfiltered grid. @@ -68,26 +68,32 @@ export default function SearchSettings({
- -
-
-
Grid Columns
-
- Select the number of columns in the results grid. + {!isMobileOnly && ( + <> + +
+
+
Grid Columns
+
+ Select the number of columns in the grid view. +
+
+
+ setColumns(value)} + max={6} + min={2} + step={1} + className="flex-grow" + /> + + {columns} + +
-
-
- setColumns(value)} - max={6} - min={2} - step={1} - className="flex-grow" - /> - {columns} -
-
+ + )}
); diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 770b45cb8..202b079a6 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -14,6 +14,7 @@ import { ModelState } from "@/types/ws"; import { formatSecondsToDuration } from "@/utils/dateUtil"; import SearchView from "@/views/search/SearchView"; import { useCallback, useEffect, useMemo, useState } from "react"; +import { isMobileOnly } from "react-device-detect"; import { LuCheck, LuExternalLink, LuX } from "react-icons/lu"; import { TbExclamationCircle } from "react-icons/tb"; import { Link } from "react-router-dom"; @@ -32,11 +33,16 @@ export default function Explore() { // grid const [columnCount, setColumnCount] = usePersistence("exploreGridColumns", 4); - const gridColumns = useMemo(() => columnCount ?? 4, [columnCount]); + const gridColumns = useMemo(() => { + if (isMobileOnly) { + return 2; + } + return columnCount ?? 4; + }, [columnCount]); // default layout - const [defaultView, setDefaultView] = usePersistence( + const [defaultView, setDefaultView, defaultViewLoaded] = usePersistence( "exploreDefaultView", "summary", ); @@ -103,6 +109,8 @@ export default function Explore() { after: searchSearchParams["after"], time_range: searchSearchParams["time_range"], search_type: searchSearchParams["search_type"], + min_score: searchSearchParams["min_score"], + max_score: searchSearchParams["max_score"], limit: Object.keys(searchSearchParams).length == 0 ? API_LIMIT : undefined, timezone, @@ -129,6 +137,8 @@ export default function Explore() { after: searchSearchParams["after"], time_range: searchSearchParams["time_range"], search_type: searchSearchParams["search_type"], + min_score: searchSearchParams["min_score"], + max_score: searchSearchParams["max_score"], event_id: searchSearchParams["event_id"], timezone, include_thumbnails: 0, @@ -270,12 +280,13 @@ export default function Explore() { }; if ( - config?.semantic_search.enabled && - (!reindexState || - !textModelState || - !textTokenizerState || - !visionModelState || - !visionFeatureExtractorState) + !defaultViewLoaded || + (config?.semantic_search.enabled && + (!reindexState || + !textModelState || + !textTokenizerState || + !visionModelState || + !visionFeatureExtractorState)) ) { return ( diff --git a/web/src/types/search.ts b/web/src/types/search.ts index 50521878e..d7f3fb97d 100644 --- a/web/src/types/search.ts +++ b/web/src/types/search.ts @@ -35,6 +35,7 @@ export type SearchResult = { zones: string[]; search_source: SearchSource; search_distance: number; + top_score: number; // for old events data: { top_score: number; score: number; @@ -56,6 +57,8 @@ export type SearchFilter = { zones?: string[]; before?: number; after?: number; + min_score?: number; + max_score?: number; time_range?: string; search_type?: SearchSource[]; event_id?: string; @@ -71,6 +74,8 @@ export type SearchQueryParams = { zones?: string[]; before?: string; after?: string; + min_score?: number; + max_score?: number; search_type?: string; limit?: number; in_progress?: number; diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index b22a5248a..665f7a4fd 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -144,6 +144,8 @@ export default function SearchView({ : ["12:00AM-11:59PM"], before: [formatDateToLocaleString()], after: [formatDateToLocaleString(-5)], + min_score: ["50"], + max_score: ["100"], }), [config, allLabels, allZones, allSubLabels], ); @@ -385,7 +387,7 @@ export default function SearchView({ key={value.id} ref={(item) => (itemRefs.current[index] = item)} data-start={value.start_time} - className="review-item relative rounded-lg" + className="review-item relative flex flex-col rounded-lg" >
-
+
{ if (config?.semantic_search.enabled) { setSimilaritySearch(value); From 6294ce7807ab7ea151e52a7c257a2d0621cb8c12 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 17 Oct 2024 10:21:20 -0500 Subject: [PATCH 062/479] Adjust Explore settings (#14409) * Re-add search source chip without confidence percentage * add confidence to tooltip only * move search type to settings * padding tweak * docs update * docs clarity --- docs/docs/configuration/semantic_search.md | 6 +- .../overlay/dialog/SearchFilterDialog.tsx | 74 +--------------- .../components/settings/SearchSettings.tsx | 84 +++++++++++++++++++ web/src/views/search/SearchView.tsx | 50 ++++++++++- 4 files changed, 137 insertions(+), 77 deletions(-) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index a569e8f1a..18093a479 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -53,7 +53,7 @@ semantic_search: ## Usage 1. Semantic search is used in conjunction with the other filters available on the Search page. Use a combination of traditional filtering and semantic search for the best results. -2. The comparison between text and image embedding distances generally means that results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" filter to help find what you are looking for. -3. Make your search language and tone closely match your descriptions. If you are using thumbnail search, phrase your query as an image caption. +2. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the the ones without them are more relevant. +3. Make your search language and tone closely match your descriptions. If you are using thumbnail search, **phrase your query as an image caption**. For example "red car" will not work as well as "red sedan driving down a residential street on a sunny day". 4. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well. -5. Experiment! Find a tracked object you want to test and start typing keywords to see what works for you. +5. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you. diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx index ad9fe1c2b..ed091b350 100644 --- a/web/src/components/overlay/dialog/SearchFilterDialog.tsx +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -65,9 +65,7 @@ export default function SearchFilterDialog({ (currentFilter.min_score ?? 0) > 0.5 || (currentFilter.max_score ?? 1) < 1 || (currentFilter.zones?.length ?? 0) > 0 || - (currentFilter.sub_labels?.length ?? 0) > 0 || - (!currentFilter.search_type?.includes("similarity") && - (currentFilter.search_type?.length ?? 2) !== 2)), + (currentFilter.sub_labels?.length ?? 0) > 0), [currentFilter], ); @@ -115,20 +113,6 @@ export default function SearchFilterDialog({ setCurrentFilter({ ...currentFilter, min_score: min, max_score: max }) } /> - {config?.semantic_search?.enabled && - !currentFilter?.search_type?.includes("similarity") && ( - - setCurrentFilter({ - ...currentFilter, - search_type: newSearchSource, - }) - } - /> - )} {isDesktop && }
)} + {config?.semantic_search?.enabled && ( + { + setSearchSources(sources as SearchSource[]); + onUpdateFilter({ ...filter, search_type: sources }); + }} + /> + )}
); @@ -113,3 +135,65 @@ export default function SearchSettings({ /> ); } + +type SearchTypeContentProps = { + searchSources: SearchSource[] | undefined; + setSearchSources: (sources: SearchSource[] | undefined) => void; +}; +export function SearchTypeContent({ + searchSources, + setSearchSources, +}: SearchTypeContentProps) { + return ( + <> +
+ +
+
Search Source
+
+ Choose whether to search the thumbnails or descriptions of your + tracked objects. +
+
+
+ { + const updatedSources = searchSources ? [...searchSources] : []; + + if (isChecked) { + updatedSources.push("thumbnail"); + setSearchSources(updatedSources); + } else { + if (updatedSources.length > 1) { + const index = updatedSources.indexOf("thumbnail"); + if (index !== -1) updatedSources.splice(index, 1); + setSearchSources(updatedSources); + } + } + }} + /> + { + const updatedSources = searchSources ? [...searchSources] : []; + + if (isChecked) { + updatedSources.push("description"); + setSearchSources(updatedSources); + } else { + if (updatedSources.length > 1) { + const index = updatedSources.indexOf("description"); + if (index !== -1) updatedSources.splice(index, 1); + setSearchSources(updatedSources); + } + } + }} + /> +
+
+ + ); +} diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 665f7a4fd..07842fed6 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -10,7 +10,7 @@ import { FrigateConfig } from "@/types/frigateConfig"; import { SearchFilter, SearchResult, SearchSource } from "@/types/search"; import { useCallback, useEffect, useMemo, useRef, useState } from "react"; import { isMobileOnly } from "react-device-detect"; -import { LuSearchX } from "react-icons/lu"; +import { LuImage, LuSearchX, LuText } from "react-icons/lu"; import useSWR from "swr"; import ExploreView from "../explore/ExploreView"; import useKeyboardListener, { @@ -23,6 +23,13 @@ import { isEqual } from "lodash"; import { formatDateToLocaleString } from "@/utils/dateUtil"; import SearchThumbnailFooter from "@/components/card/SearchThumbnailFooter"; import SearchSettings from "@/components/settings/SearchSettings"; +import { + Tooltip, + TooltipContent, + TooltipTrigger, +} from "@/components/ui/tooltip"; +import Chip from "@/components/indicators/Chip"; +import { TooltipPortal } from "@radix-ui/react-tooltip"; type SearchViewProps = { search: string; @@ -182,6 +189,21 @@ export default function SearchView({ setSelectedIndex(0); }, [searchTerm, searchFilter]); + // confidence score + + const zScoreToConfidence = (score: number) => { + // Normalizing is not needed for similarity searches + // Sigmoid function for normalized: 1 / (1 + e^x) + // Cosine for similarity + if (searchFilter) { + const notNormalized = searchFilter?.search_type?.includes("similarity"); + + const confidence = notNormalized ? 1 - score : 1 / (1 + Math.exp(score)); + + return Math.round(confidence * 100); + } + }; + // update search detail when results change useEffect(() => { @@ -351,6 +373,8 @@ export default function SearchView({ setColumns={setColumns} defaultView={defaultView} setDefaultView={setDefaultView} + filter={searchFilter} + onUpdateFilter={onUpdateFilter} />
@@ -398,6 +422,30 @@ export default function SearchView({ searchResult={value} onClick={() => onSelectSearch(value, index)} /> + {(searchTerm || + searchFilter?.search_type?.includes("similarity")) && ( +
+ + + + {value.search_source == "thumbnail" ? ( + + ) : ( + + )} + + + + + Matched {value.search_source} at{" "} + {zScoreToConfidence(value.search_distance)}% + + + +
+ )}
Date: Thu, 17 Oct 2024 10:02:27 -0600 Subject: [PATCH 063/479] Various fixes (#14410) * Fix access * Reorganize tracked object for imports * Separate out rockchip build * Formatting * Use original ffmpeg build * Fix build * Update default search type value --- .github/workflows/ci.yml | 22 + docker/main/install_deps.sh | 5 +- frigate/api/defs/events_query_parameters.py | 2 +- frigate/object_processing.py | 456 +------------------- frigate/ptz/autotrack.py | 26 +- frigate/test/test_obects.py | 4 +- frigate/track/object_attribute.py | 44 -- frigate/track/tracked_object.py | 447 +++++++++++++++++++ frigate/util/image.py | 66 +++ frigate/video.py | 6 +- 10 files changed, 563 insertions(+), 515 deletions(-) delete mode 100644 frigate/track/object_attribute.py create mode 100644 frigate/track/tracked_object.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bbf47a57d..3a5a67041 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -155,6 +155,28 @@ jobs: tensorrt.tags=${{ steps.setup.outputs.image-name }}-tensorrt *.cache-from=type=registry,ref=${{ steps.setup.outputs.cache-name }}-amd64 *.cache-to=type=registry,ref=${{ steps.setup.outputs.cache-name }}-amd64,mode=max + arm64_extra_builds: + runs-on: ubuntu-latest + name: ARM Extra Build + needs: + - arm64_build + steps: + - name: Check out code + uses: actions/checkout@v4 + - name: Set up QEMU and Buildx + id: setup + uses: ./.github/actions/setup + with: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Build and push Rockchip build + uses: docker/bake-action@v3 + with: + push: true + targets: rk + files: docker/rockchip/rk.hcl + set: | + rk.tags=${{ steps.setup.outputs.image-name }}-rk + *.cache-from=type=gha combined_extra_builds: runs-on: ubuntu-latest name: Combined Extra Builds diff --git a/docker/main/install_deps.sh b/docker/main/install_deps.sh index 46f2a5357..2d7662053 100755 --- a/docker/main/install_deps.sh +++ b/docker/main/install_deps.sh @@ -8,6 +8,7 @@ apt-get -qq install --no-install-recommends -y \ apt-transport-https \ gnupg \ wget \ + lbzip2 \ procps vainfo \ unzip locales tzdata libxml2 xz-utils \ python3.9 \ @@ -45,7 +46,7 @@ if [[ "${TARGETARCH}" == "amd64" ]]; then wget -qO btbn-ffmpeg.tar.xz "https://github.com/NickM-27/FFmpeg-Builds/releases/download/autobuild-2022-07-31-12-37/ffmpeg-n5.1-2-g915ef932a3-linux64-gpl-5.1.tar.xz" tar -xf btbn-ffmpeg.tar.xz -C /usr/lib/ffmpeg/5.0 --strip-components 1 rm -rf btbn-ffmpeg.tar.xz /usr/lib/ffmpeg/5.0/doc /usr/lib/ffmpeg/5.0/bin/ffplay - wget -qO btbn-ffmpeg.tar.xz "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2024-09-30-15-36/ffmpeg-n7.1-linux64-gpl-7.1.tar.xz" + wget -qO btbn-ffmpeg.tar.xz "https://github.com/NickM-27/FFmpeg-Builds/releases/download/autobuild-2024-09-19-12-51/ffmpeg-n7.0.2-18-g3e6cec1286-linux64-gpl-7.0.tar.xz" tar -xf btbn-ffmpeg.tar.xz -C /usr/lib/ffmpeg/7.0 --strip-components 1 rm -rf btbn-ffmpeg.tar.xz /usr/lib/ffmpeg/7.0/doc /usr/lib/ffmpeg/7.0/bin/ffplay fi @@ -57,7 +58,7 @@ if [[ "${TARGETARCH}" == "arm64" ]]; then wget -qO btbn-ffmpeg.tar.xz "https://github.com/NickM-27/FFmpeg-Builds/releases/download/autobuild-2022-07-31-12-37/ffmpeg-n5.1-2-g915ef932a3-linuxarm64-gpl-5.1.tar.xz" tar -xf btbn-ffmpeg.tar.xz -C /usr/lib/ffmpeg/5.0 --strip-components 1 rm -rf btbn-ffmpeg.tar.xz /usr/lib/ffmpeg/5.0/doc /usr/lib/ffmpeg/5.0/bin/ffplay - wget -qO btbn-ffmpeg.tar.xz "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2024-09-30-15-36/ffmpeg-n7.1-linuxarm64-gpl-7.1.tar.xz" + wget -qO btbn-ffmpeg.tar.xz "https://github.com/NickM-27/FFmpeg-Builds/releases/download/autobuild-2024-09-19-12-51/ffmpeg-n7.0.2-18-g3e6cec1286-linuxarm64-gpl-7.0.tar.xz" tar -xf btbn-ffmpeg.tar.xz -C /usr/lib/ffmpeg/7.0 --strip-components 1 rm -rf btbn-ffmpeg.tar.xz /usr/lib/ffmpeg/7.0/doc /usr/lib/ffmpeg/7.0/bin/ffplay fi diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/events_query_parameters.py index c4e40bd4e..f4c98809c 100644 --- a/frigate/api/defs/events_query_parameters.py +++ b/frigate/api/defs/events_query_parameters.py @@ -35,7 +35,7 @@ class EventsQueryParams(BaseModel): class EventsSearchQueryParams(BaseModel): query: Optional[str] = None event_id: Optional[str] = None - search_type: Optional[str] = "thumbnail,description" + search_type: Optional[str] = "thumbnail" include_thumbnails: Optional[int] = 1 limit: Optional[int] = 50 cameras: Optional[str] = "all" diff --git a/frigate/object_processing.py b/frigate/object_processing.py index 6e63562a4..7ba3270f1 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -1,4 +1,3 @@ -import base64 import datetime import json import logging @@ -7,7 +6,6 @@ import queue import threading from collections import Counter, defaultdict from multiprocessing.synchronize import Event as MpEvent -from statistics import median from typing import Callable import cv2 @@ -18,9 +16,7 @@ from frigate.comms.dispatcher import Dispatcher from frigate.comms.events_updater import EventEndSubscriber, EventUpdatePublisher from frigate.comms.inter_process import InterProcessRequestor from frigate.config import ( - CameraConfig, FrigateConfig, - ModelConfig, MqttConfig, RecordConfig, SnapshotsConfig, @@ -29,466 +25,18 @@ from frigate.config import ( from frigate.const import CLIPS_DIR, UPDATE_CAMERA_ACTIVITY from frigate.events.types import EventStateEnum, EventTypeEnum from frigate.ptz.autotrack import PtzAutoTrackerThread +from frigate.track.tracked_object import TrackedObject from frigate.util.image import ( SharedMemoryFrameManager, - area, - calculate_region, draw_box_with_label, draw_timestamp, + is_better_thumbnail, is_label_printable, ) logger = logging.getLogger(__name__) -def on_edge(box, frame_shape): - if ( - box[0] == 0 - or box[1] == 0 - or box[2] == frame_shape[1] - 1 - or box[3] == frame_shape[0] - 1 - ): - return True - - -def has_better_attr(current_thumb, new_obj, attr_label) -> bool: - max_new_attr = max( - [0] - + [area(a["box"]) for a in new_obj["attributes"] if a["label"] == attr_label] - ) - max_current_attr = max( - [0] - + [ - area(a["box"]) - for a in current_thumb["attributes"] - if a["label"] == attr_label - ] - ) - - # if the thumb has a higher scoring attr - return max_new_attr > max_current_attr - - -def is_better_thumbnail(label, current_thumb, new_obj, frame_shape) -> bool: - # larger is better - # cutoff images are less ideal, but they should also be smaller? - # better scores are obviously better too - - # check face on person - if label == "person": - if has_better_attr(current_thumb, new_obj, "face"): - return True - # if the current thumb has a face attr, dont update unless it gets better - if any([a["label"] == "face" for a in current_thumb["attributes"]]): - return False - - # check license_plate on car - if label == "car": - if has_better_attr(current_thumb, new_obj, "license_plate"): - return True - # if the current thumb has a license_plate attr, dont update unless it gets better - if any([a["label"] == "license_plate" for a in current_thumb["attributes"]]): - return False - - # if the new_thumb is on an edge, and the current thumb is not - if on_edge(new_obj["box"], frame_shape) and not on_edge( - current_thumb["box"], frame_shape - ): - return False - - # if the score is better by more than 5% - if new_obj["score"] > current_thumb["score"] + 0.05: - return True - - # if the area is 10% larger - if new_obj["area"] > current_thumb["area"] * 1.1: - return True - - return False - - -class TrackedObject: - def __init__( - self, - model_config: ModelConfig, - camera_config: CameraConfig, - frame_cache, - obj_data: dict[str, any], - ): - # set the score history then remove as it is not part of object state - self.score_history = obj_data["score_history"] - del obj_data["score_history"] - - self.obj_data = obj_data - self.colormap = model_config.colormap - self.logos = model_config.all_attribute_logos - self.camera_config = camera_config - self.frame_cache = frame_cache - self.zone_presence: dict[str, int] = {} - self.zone_loitering: dict[str, int] = {} - self.current_zones = [] - self.entered_zones = [] - self.attributes = defaultdict(float) - self.false_positive = True - self.has_clip = False - self.has_snapshot = False - self.top_score = self.computed_score = 0.0 - self.thumbnail_data = None - self.last_updated = 0 - self.last_published = 0 - self.frame = None - self.active = True - self.pending_loitering = False - self.previous = self.to_dict() - - def _is_false_positive(self): - # once a true positive, always a true positive - if not self.false_positive: - return False - - threshold = self.camera_config.objects.filters[self.obj_data["label"]].threshold - return self.computed_score < threshold - - def compute_score(self): - """get median of scores for object.""" - return median(self.score_history) - - def update(self, current_frame_time: float, obj_data, has_valid_frame: bool): - thumb_update = False - significant_change = False - autotracker_update = False - # if the object is not in the current frame, add a 0.0 to the score history - if obj_data["frame_time"] != current_frame_time: - self.score_history.append(0.0) - else: - self.score_history.append(obj_data["score"]) - - # only keep the last 10 scores - if len(self.score_history) > 10: - self.score_history = self.score_history[-10:] - - # calculate if this is a false positive - self.computed_score = self.compute_score() - if self.computed_score > self.top_score: - self.top_score = self.computed_score - self.false_positive = self._is_false_positive() - self.active = self.is_active() - - if not self.false_positive and has_valid_frame: - # determine if this frame is a better thumbnail - if self.thumbnail_data is None or is_better_thumbnail( - self.obj_data["label"], - self.thumbnail_data, - obj_data, - self.camera_config.frame_shape, - ): - self.thumbnail_data = { - "frame_time": current_frame_time, - "box": obj_data["box"], - "area": obj_data["area"], - "region": obj_data["region"], - "score": obj_data["score"], - "attributes": obj_data["attributes"], - } - thumb_update = True - - # check zones - current_zones = [] - bottom_center = (obj_data["centroid"][0], obj_data["box"][3]) - in_loitering_zone = False - - # check each zone - for name, zone in self.camera_config.zones.items(): - # if the zone is not for this object type, skip - if len(zone.objects) > 0 and obj_data["label"] not in zone.objects: - continue - contour = zone.contour - zone_score = self.zone_presence.get(name, 0) + 1 - # check if the object is in the zone - if cv2.pointPolygonTest(contour, bottom_center, False) >= 0: - # if the object passed the filters once, dont apply again - if name in self.current_zones or not zone_filtered(self, zone.filters): - # an object is only considered present in a zone if it has a zone inertia of 3+ - if zone_score >= zone.inertia: - # if the zone has loitering time, update loitering status - if zone.loitering_time > 0: - in_loitering_zone = True - - loitering_score = self.zone_loitering.get(name, 0) + 1 - - # loitering time is configured as seconds, convert to count of frames - if loitering_score >= ( - self.camera_config.zones[name].loitering_time - * self.camera_config.detect.fps - ): - current_zones.append(name) - - if name not in self.entered_zones: - self.entered_zones.append(name) - else: - self.zone_loitering[name] = loitering_score - else: - self.zone_presence[name] = zone_score - else: - # once an object has a zone inertia of 3+ it is not checked anymore - if 0 < zone_score < zone.inertia: - self.zone_presence[name] = zone_score - 1 - - # update loitering status - self.pending_loitering = in_loitering_zone - - # maintain attributes - for attr in obj_data["attributes"]: - if self.attributes[attr["label"]] < attr["score"]: - self.attributes[attr["label"]] = attr["score"] - - # populate the sub_label for object with highest scoring logo - if self.obj_data["label"] in ["car", "package", "person"]: - recognized_logos = { - k: self.attributes[k] for k in self.logos if k in self.attributes - } - if len(recognized_logos) > 0: - max_logo = max(recognized_logos, key=recognized_logos.get) - - # don't overwrite sub label if it is already set - if ( - self.obj_data.get("sub_label") is None - or self.obj_data["sub_label"][0] == max_logo - ): - self.obj_data["sub_label"] = (max_logo, recognized_logos[max_logo]) - - # check for significant change - if not self.false_positive: - # if the zones changed, signal an update - if set(self.current_zones) != set(current_zones): - significant_change = True - - # if the position changed, signal an update - if self.obj_data["position_changes"] != obj_data["position_changes"]: - significant_change = True - - if self.obj_data["attributes"] != obj_data["attributes"]: - significant_change = True - - # if the state changed between stationary and active - if self.previous["active"] != self.active: - significant_change = True - - # update at least once per minute - if self.obj_data["frame_time"] - self.previous["frame_time"] > 60: - significant_change = True - - # update autotrack at most 3 objects per second - if self.obj_data["frame_time"] - self.previous["frame_time"] >= (1 / 3): - autotracker_update = True - - self.obj_data.update(obj_data) - self.current_zones = current_zones - return (thumb_update, significant_change, autotracker_update) - - def to_dict(self, include_thumbnail: bool = False): - event = { - "id": self.obj_data["id"], - "camera": self.camera_config.name, - "frame_time": self.obj_data["frame_time"], - "snapshot": self.thumbnail_data, - "label": self.obj_data["label"], - "sub_label": self.obj_data.get("sub_label"), - "top_score": self.top_score, - "false_positive": self.false_positive, - "start_time": self.obj_data["start_time"], - "end_time": self.obj_data.get("end_time", None), - "score": self.obj_data["score"], - "box": self.obj_data["box"], - "area": self.obj_data["area"], - "ratio": self.obj_data["ratio"], - "region": self.obj_data["region"], - "active": self.active, - "stationary": not self.active, - "motionless_count": self.obj_data["motionless_count"], - "position_changes": self.obj_data["position_changes"], - "current_zones": self.current_zones.copy(), - "entered_zones": self.entered_zones.copy(), - "has_clip": self.has_clip, - "has_snapshot": self.has_snapshot, - "attributes": self.attributes, - "current_attributes": self.obj_data["attributes"], - "pending_loitering": self.pending_loitering, - } - - if include_thumbnail: - event["thumbnail"] = base64.b64encode(self.get_thumbnail()).decode("utf-8") - - return event - - def is_active(self): - return not self.is_stationary() - - def is_stationary(self): - return ( - self.obj_data["motionless_count"] - > self.camera_config.detect.stationary.threshold - ) - - def get_thumbnail(self): - if ( - self.thumbnail_data is None - or self.thumbnail_data["frame_time"] not in self.frame_cache - ): - ret, jpg = cv2.imencode(".jpg", np.zeros((175, 175, 3), np.uint8)) - - jpg_bytes = self.get_jpg_bytes( - timestamp=False, bounding_box=False, crop=True, height=175 - ) - - if jpg_bytes: - return jpg_bytes - else: - ret, jpg = cv2.imencode(".jpg", np.zeros((175, 175, 3), np.uint8)) - return jpg.tobytes() - - def get_clean_png(self): - if self.thumbnail_data is None: - return None - - try: - best_frame = cv2.cvtColor( - self.frame_cache[self.thumbnail_data["frame_time"]], - cv2.COLOR_YUV2BGR_I420, - ) - except KeyError: - logger.warning( - f"Unable to create clean png because frame {self.thumbnail_data['frame_time']} is not in the cache" - ) - return None - - ret, png = cv2.imencode(".png", best_frame) - if ret: - return png.tobytes() - else: - return None - - def get_jpg_bytes( - self, timestamp=False, bounding_box=False, crop=False, height=None, quality=70 - ): - if self.thumbnail_data is None: - return None - - try: - best_frame = cv2.cvtColor( - self.frame_cache[self.thumbnail_data["frame_time"]], - cv2.COLOR_YUV2BGR_I420, - ) - except KeyError: - logger.warning( - f"Unable to create jpg because frame {self.thumbnail_data['frame_time']} is not in the cache" - ) - return None - - if bounding_box: - thickness = 2 - color = self.colormap[self.obj_data["label"]] - - # draw the bounding boxes on the frame - box = self.thumbnail_data["box"] - draw_box_with_label( - best_frame, - box[0], - box[1], - box[2], - box[3], - self.obj_data["label"], - f"{int(self.thumbnail_data['score']*100)}% {int(self.thumbnail_data['area'])}", - thickness=thickness, - color=color, - ) - - # draw any attributes - for attribute in self.thumbnail_data["attributes"]: - box = attribute["box"] - draw_box_with_label( - best_frame, - box[0], - box[1], - box[2], - box[3], - attribute["label"], - f"{attribute['score']:.0%}", - thickness=thickness, - color=color, - ) - - if crop: - box = self.thumbnail_data["box"] - box_size = 300 - region = calculate_region( - best_frame.shape, - box[0], - box[1], - box[2], - box[3], - box_size, - multiplier=1.1, - ) - best_frame = best_frame[region[1] : region[3], region[0] : region[2]] - - if height: - width = int(height * best_frame.shape[1] / best_frame.shape[0]) - best_frame = cv2.resize( - best_frame, dsize=(width, height), interpolation=cv2.INTER_AREA - ) - if timestamp: - color = self.camera_config.timestamp_style.color - draw_timestamp( - best_frame, - self.thumbnail_data["frame_time"], - self.camera_config.timestamp_style.format, - font_effect=self.camera_config.timestamp_style.effect, - font_thickness=self.camera_config.timestamp_style.thickness, - font_color=(color.blue, color.green, color.red), - position=self.camera_config.timestamp_style.position, - ) - - ret, jpg = cv2.imencode( - ".jpg", best_frame, [int(cv2.IMWRITE_JPEG_QUALITY), quality] - ) - if ret: - return jpg.tobytes() - else: - return None - - -def zone_filtered(obj: TrackedObject, object_config): - object_name = obj.obj_data["label"] - - if object_name in object_config: - obj_settings = object_config[object_name] - - # if the min area is larger than the - # detected object, don't add it to detected objects - if obj_settings.min_area > obj.obj_data["area"]: - return True - - # if the detected object is larger than the - # max area, don't add it to detected objects - if obj_settings.max_area < obj.obj_data["area"]: - return True - - # if the score is lower than the threshold, skip - if obj_settings.threshold > obj.computed_score: - return True - - # if the object is not proportionally wide enough - if obj_settings.min_ratio > obj.obj_data["ratio"]: - return True - - # if the object is proportionally too wide - if obj_settings.max_ratio < obj.obj_data["ratio"]: - return True - - return False - - # Maintains the state of a camera class CameraState: def __init__( diff --git a/frigate/ptz/autotrack.py b/frigate/ptz/autotrack.py index fd9933bcb..e9226f267 100644 --- a/frigate/ptz/autotrack.py +++ b/frigate/ptz/autotrack.py @@ -32,6 +32,7 @@ from frigate.const import ( CONFIG_DIR, ) from frigate.ptz.onvif import OnvifController +from frigate.track.tracked_object import TrackedObject from frigate.util.builtin import update_yaml_file from frigate.util.image import SharedMemoryFrameManager, intersection_over_union @@ -214,7 +215,7 @@ class PtzAutoTracker: ): self._autotracker_setup(camera_config, camera) - def _autotracker_setup(self, camera_config, camera): + def _autotracker_setup(self, camera_config: CameraConfig, camera: str): logger.debug(f"{camera}: Autotracker init") self.object_types[camera] = camera_config.onvif.autotracking.track @@ -852,7 +853,7 @@ class PtzAutoTracker: logger.debug(f"{camera}: Valid velocity ") return True, velocities.flatten() - def _get_distance_threshold(self, camera, obj): + def _get_distance_threshold(self, camera: str, obj: TrackedObject): # Returns true if Euclidean distance from object to center of frame is # less than 10% of the of the larger dimension (width or height) of the frame, # multiplied by a scaling factor for object size. @@ -888,7 +889,9 @@ class PtzAutoTracker: return distance_threshold - def _should_zoom_in(self, camera, obj, box, predicted_time, debug_zooming=False): + def _should_zoom_in( + self, camera: str, obj: TrackedObject, box, predicted_time, debug_zooming=False + ): # returns True if we should zoom in, False if we should zoom out, None to do nothing camera_config = self.config.cameras[camera] camera_width = camera_config.frame_shape[1] @@ -1019,7 +1022,7 @@ class PtzAutoTracker: # Don't zoom at all return None - def _autotrack_move_ptz(self, camera, obj): + def _autotrack_move_ptz(self, camera: str, obj: TrackedObject): camera_config = self.config.cameras[camera] camera_width = camera_config.frame_shape[1] camera_height = camera_config.frame_shape[0] @@ -1090,7 +1093,12 @@ class PtzAutoTracker: self._enqueue_move(camera, obj.obj_data["frame_time"], 0, 0, zoom) def _get_zoom_amount( - self, camera, obj, predicted_box, predicted_movement_time, debug_zoom=True + self, + camera: str, + obj: TrackedObject, + predicted_box, + predicted_movement_time, + debug_zoom=True, ): camera_config = self.config.cameras[camera] @@ -1186,13 +1194,13 @@ class PtzAutoTracker: return zoom - def is_autotracking(self, camera): + def is_autotracking(self, camera: str): return self.tracked_object[camera] is not None - def autotracked_object_region(self, camera): + def autotracked_object_region(self, camera: str): return self.tracked_object[camera]["region"] - def autotrack_object(self, camera, obj): + def autotrack_object(self, camera: str, obj: TrackedObject): camera_config = self.config.cameras[camera] if camera_config.onvif.autotracking.enabled: @@ -1208,7 +1216,7 @@ class PtzAutoTracker: if ( # new object self.tracked_object[camera] is None - and obj.camera == camera + and obj.camera_config.name == camera and obj.obj_data["label"] in self.object_types[camera] and set(obj.entered_zones) & set(self.required_zones[camera]) and not obj.previous["false_positive"] diff --git a/frigate/test/test_obects.py b/frigate/test/test_obects.py index f1c039ef8..8fe831980 100644 --- a/frigate/test/test_obects.py +++ b/frigate/test/test_obects.py @@ -1,11 +1,11 @@ import unittest -from frigate.track.object_attribute import ObjectAttribute +from frigate.track.tracked_object import TrackedObjectAttribute class TestAttribute(unittest.TestCase): def test_overlapping_object_selection(self) -> None: - attribute = ObjectAttribute( + attribute = TrackedObjectAttribute( ( "amazon", 0.80078125, diff --git a/frigate/track/object_attribute.py b/frigate/track/object_attribute.py deleted file mode 100644 index 54433c5f3..000000000 --- a/frigate/track/object_attribute.py +++ /dev/null @@ -1,44 +0,0 @@ -"""Object attribute.""" - -from frigate.util.object import area, box_inside - - -class ObjectAttribute: - def __init__(self, raw_data: tuple) -> None: - self.label = raw_data[0] - self.score = raw_data[1] - self.box = raw_data[2] - self.area = raw_data[3] - self.ratio = raw_data[4] - self.region = raw_data[5] - - def get_tracking_data(self) -> dict[str, any]: - """Return data saved to the object.""" - return { - "label": self.label, - "score": self.score, - "box": self.box, - } - - def find_best_object(self, objects: list[dict[str, any]]) -> str: - """Find the best attribute for each object and return its ID.""" - best_object_area = None - best_object_id = None - - for obj in objects: - if not box_inside(obj["box"], self.box): - continue - - object_area = area(obj["box"]) - - # if multiple objects have the same attribute then they - # are overlapping, it is most likely that the smaller object - # is the one with the attribute - if best_object_area is None: - best_object_area = object_area - best_object_id = obj["id"] - elif object_area < best_object_area: - best_object_area = object_area - best_object_id = obj["id"] - - return best_object_id diff --git a/frigate/track/tracked_object.py b/frigate/track/tracked_object.py new file mode 100644 index 000000000..a4b4e8426 --- /dev/null +++ b/frigate/track/tracked_object.py @@ -0,0 +1,447 @@ +"""Object attribute.""" + +import base64 +import logging +from collections import defaultdict +from statistics import median + +import cv2 +import numpy as np + +from frigate.config import ( + CameraConfig, + ModelConfig, +) +from frigate.util.image import ( + area, + calculate_region, + draw_box_with_label, + draw_timestamp, + is_better_thumbnail, +) +from frigate.util.object import box_inside + +logger = logging.getLogger(__name__) + + +class TrackedObject: + def __init__( + self, + model_config: ModelConfig, + camera_config: CameraConfig, + frame_cache, + obj_data: dict[str, any], + ): + # set the score history then remove as it is not part of object state + self.score_history = obj_data["score_history"] + del obj_data["score_history"] + + self.obj_data = obj_data + self.colormap = model_config.colormap + self.logos = model_config.all_attribute_logos + self.camera_config = camera_config + self.frame_cache = frame_cache + self.zone_presence: dict[str, int] = {} + self.zone_loitering: dict[str, int] = {} + self.current_zones = [] + self.entered_zones = [] + self.attributes = defaultdict(float) + self.false_positive = True + self.has_clip = False + self.has_snapshot = False + self.top_score = self.computed_score = 0.0 + self.thumbnail_data = None + self.last_updated = 0 + self.last_published = 0 + self.frame = None + self.active = True + self.pending_loitering = False + self.previous = self.to_dict() + + def _is_false_positive(self): + # once a true positive, always a true positive + if not self.false_positive: + return False + + threshold = self.camera_config.objects.filters[self.obj_data["label"]].threshold + return self.computed_score < threshold + + def compute_score(self): + """get median of scores for object.""" + return median(self.score_history) + + def update(self, current_frame_time: float, obj_data, has_valid_frame: bool): + thumb_update = False + significant_change = False + autotracker_update = False + # if the object is not in the current frame, add a 0.0 to the score history + if obj_data["frame_time"] != current_frame_time: + self.score_history.append(0.0) + else: + self.score_history.append(obj_data["score"]) + + # only keep the last 10 scores + if len(self.score_history) > 10: + self.score_history = self.score_history[-10:] + + # calculate if this is a false positive + self.computed_score = self.compute_score() + if self.computed_score > self.top_score: + self.top_score = self.computed_score + self.false_positive = self._is_false_positive() + self.active = self.is_active() + + if not self.false_positive and has_valid_frame: + # determine if this frame is a better thumbnail + if self.thumbnail_data is None or is_better_thumbnail( + self.obj_data["label"], + self.thumbnail_data, + obj_data, + self.camera_config.frame_shape, + ): + self.thumbnail_data = { + "frame_time": current_frame_time, + "box": obj_data["box"], + "area": obj_data["area"], + "region": obj_data["region"], + "score": obj_data["score"], + "attributes": obj_data["attributes"], + } + thumb_update = True + + # check zones + current_zones = [] + bottom_center = (obj_data["centroid"][0], obj_data["box"][3]) + in_loitering_zone = False + + # check each zone + for name, zone in self.camera_config.zones.items(): + # if the zone is not for this object type, skip + if len(zone.objects) > 0 and obj_data["label"] not in zone.objects: + continue + contour = zone.contour + zone_score = self.zone_presence.get(name, 0) + 1 + # check if the object is in the zone + if cv2.pointPolygonTest(contour, bottom_center, False) >= 0: + # if the object passed the filters once, dont apply again + if name in self.current_zones or not zone_filtered(self, zone.filters): + # an object is only considered present in a zone if it has a zone inertia of 3+ + if zone_score >= zone.inertia: + # if the zone has loitering time, update loitering status + if zone.loitering_time > 0: + in_loitering_zone = True + + loitering_score = self.zone_loitering.get(name, 0) + 1 + + # loitering time is configured as seconds, convert to count of frames + if loitering_score >= ( + self.camera_config.zones[name].loitering_time + * self.camera_config.detect.fps + ): + current_zones.append(name) + + if name not in self.entered_zones: + self.entered_zones.append(name) + else: + self.zone_loitering[name] = loitering_score + else: + self.zone_presence[name] = zone_score + else: + # once an object has a zone inertia of 3+ it is not checked anymore + if 0 < zone_score < zone.inertia: + self.zone_presence[name] = zone_score - 1 + + # update loitering status + self.pending_loitering = in_loitering_zone + + # maintain attributes + for attr in obj_data["attributes"]: + if self.attributes[attr["label"]] < attr["score"]: + self.attributes[attr["label"]] = attr["score"] + + # populate the sub_label for object with highest scoring logo + if self.obj_data["label"] in ["car", "package", "person"]: + recognized_logos = { + k: self.attributes[k] for k in self.logos if k in self.attributes + } + if len(recognized_logos) > 0: + max_logo = max(recognized_logos, key=recognized_logos.get) + + # don't overwrite sub label if it is already set + if ( + self.obj_data.get("sub_label") is None + or self.obj_data["sub_label"][0] == max_logo + ): + self.obj_data["sub_label"] = (max_logo, recognized_logos[max_logo]) + + # check for significant change + if not self.false_positive: + # if the zones changed, signal an update + if set(self.current_zones) != set(current_zones): + significant_change = True + + # if the position changed, signal an update + if self.obj_data["position_changes"] != obj_data["position_changes"]: + significant_change = True + + if self.obj_data["attributes"] != obj_data["attributes"]: + significant_change = True + + # if the state changed between stationary and active + if self.previous["active"] != self.active: + significant_change = True + + # update at least once per minute + if self.obj_data["frame_time"] - self.previous["frame_time"] > 60: + significant_change = True + + # update autotrack at most 3 objects per second + if self.obj_data["frame_time"] - self.previous["frame_time"] >= (1 / 3): + autotracker_update = True + + self.obj_data.update(obj_data) + self.current_zones = current_zones + return (thumb_update, significant_change, autotracker_update) + + def to_dict(self, include_thumbnail: bool = False): + event = { + "id": self.obj_data["id"], + "camera": self.camera_config.name, + "frame_time": self.obj_data["frame_time"], + "snapshot": self.thumbnail_data, + "label": self.obj_data["label"], + "sub_label": self.obj_data.get("sub_label"), + "top_score": self.top_score, + "false_positive": self.false_positive, + "start_time": self.obj_data["start_time"], + "end_time": self.obj_data.get("end_time", None), + "score": self.obj_data["score"], + "box": self.obj_data["box"], + "area": self.obj_data["area"], + "ratio": self.obj_data["ratio"], + "region": self.obj_data["region"], + "active": self.active, + "stationary": not self.active, + "motionless_count": self.obj_data["motionless_count"], + "position_changes": self.obj_data["position_changes"], + "current_zones": self.current_zones.copy(), + "entered_zones": self.entered_zones.copy(), + "has_clip": self.has_clip, + "has_snapshot": self.has_snapshot, + "attributes": self.attributes, + "current_attributes": self.obj_data["attributes"], + "pending_loitering": self.pending_loitering, + } + + if include_thumbnail: + event["thumbnail"] = base64.b64encode(self.get_thumbnail()).decode("utf-8") + + return event + + def is_active(self): + return not self.is_stationary() + + def is_stationary(self): + return ( + self.obj_data["motionless_count"] + > self.camera_config.detect.stationary.threshold + ) + + def get_thumbnail(self): + if ( + self.thumbnail_data is None + or self.thumbnail_data["frame_time"] not in self.frame_cache + ): + ret, jpg = cv2.imencode(".jpg", np.zeros((175, 175, 3), np.uint8)) + + jpg_bytes = self.get_jpg_bytes( + timestamp=False, bounding_box=False, crop=True, height=175 + ) + + if jpg_bytes: + return jpg_bytes + else: + ret, jpg = cv2.imencode(".jpg", np.zeros((175, 175, 3), np.uint8)) + return jpg.tobytes() + + def get_clean_png(self): + if self.thumbnail_data is None: + return None + + try: + best_frame = cv2.cvtColor( + self.frame_cache[self.thumbnail_data["frame_time"]], + cv2.COLOR_YUV2BGR_I420, + ) + except KeyError: + logger.warning( + f"Unable to create clean png because frame {self.thumbnail_data['frame_time']} is not in the cache" + ) + return None + + ret, png = cv2.imencode(".png", best_frame) + if ret: + return png.tobytes() + else: + return None + + def get_jpg_bytes( + self, timestamp=False, bounding_box=False, crop=False, height=None, quality=70 + ): + if self.thumbnail_data is None: + return None + + try: + best_frame = cv2.cvtColor( + self.frame_cache[self.thumbnail_data["frame_time"]], + cv2.COLOR_YUV2BGR_I420, + ) + except KeyError: + logger.warning( + f"Unable to create jpg because frame {self.thumbnail_data['frame_time']} is not in the cache" + ) + return None + + if bounding_box: + thickness = 2 + color = self.colormap[self.obj_data["label"]] + + # draw the bounding boxes on the frame + box = self.thumbnail_data["box"] + draw_box_with_label( + best_frame, + box[0], + box[1], + box[2], + box[3], + self.obj_data["label"], + f"{int(self.thumbnail_data['score']*100)}% {int(self.thumbnail_data['area'])}", + thickness=thickness, + color=color, + ) + + # draw any attributes + for attribute in self.thumbnail_data["attributes"]: + box = attribute["box"] + draw_box_with_label( + best_frame, + box[0], + box[1], + box[2], + box[3], + attribute["label"], + f"{attribute['score']:.0%}", + thickness=thickness, + color=color, + ) + + if crop: + box = self.thumbnail_data["box"] + box_size = 300 + region = calculate_region( + best_frame.shape, + box[0], + box[1], + box[2], + box[3], + box_size, + multiplier=1.1, + ) + best_frame = best_frame[region[1] : region[3], region[0] : region[2]] + + if height: + width = int(height * best_frame.shape[1] / best_frame.shape[0]) + best_frame = cv2.resize( + best_frame, dsize=(width, height), interpolation=cv2.INTER_AREA + ) + if timestamp: + color = self.camera_config.timestamp_style.color + draw_timestamp( + best_frame, + self.thumbnail_data["frame_time"], + self.camera_config.timestamp_style.format, + font_effect=self.camera_config.timestamp_style.effect, + font_thickness=self.camera_config.timestamp_style.thickness, + font_color=(color.blue, color.green, color.red), + position=self.camera_config.timestamp_style.position, + ) + + ret, jpg = cv2.imencode( + ".jpg", best_frame, [int(cv2.IMWRITE_JPEG_QUALITY), quality] + ) + if ret: + return jpg.tobytes() + else: + return None + + +def zone_filtered(obj: TrackedObject, object_config): + object_name = obj.obj_data["label"] + + if object_name in object_config: + obj_settings = object_config[object_name] + + # if the min area is larger than the + # detected object, don't add it to detected objects + if obj_settings.min_area > obj.obj_data["area"]: + return True + + # if the detected object is larger than the + # max area, don't add it to detected objects + if obj_settings.max_area < obj.obj_data["area"]: + return True + + # if the score is lower than the threshold, skip + if obj_settings.threshold > obj.computed_score: + return True + + # if the object is not proportionally wide enough + if obj_settings.min_ratio > obj.obj_data["ratio"]: + return True + + # if the object is proportionally too wide + if obj_settings.max_ratio < obj.obj_data["ratio"]: + return True + + return False + + +class TrackedObjectAttribute: + def __init__(self, raw_data: tuple) -> None: + self.label = raw_data[0] + self.score = raw_data[1] + self.box = raw_data[2] + self.area = raw_data[3] + self.ratio = raw_data[4] + self.region = raw_data[5] + + def get_tracking_data(self) -> dict[str, any]: + """Return data saved to the object.""" + return { + "label": self.label, + "score": self.score, + "box": self.box, + } + + def find_best_object(self, objects: list[dict[str, any]]) -> str: + """Find the best attribute for each object and return its ID.""" + best_object_area = None + best_object_id = None + + for obj in objects: + if not box_inside(obj["box"], self.box): + continue + + object_area = area(obj["box"]) + + # if multiple objects have the same attribute then they + # are overlapping, it is most likely that the smaller object + # is the one with the attribute + if best_object_area is None: + best_object_area = object_area + best_object_id = obj["id"] + elif object_area < best_object_area: + best_object_area = object_area + best_object_id = obj["id"] + + return best_object_id diff --git a/frigate/util/image.py b/frigate/util/image.py index 41024a599..484737f71 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -36,6 +36,72 @@ def transliterate_to_latin(text: str) -> str: return unidecode(text) +def on_edge(box, frame_shape): + if ( + box[0] == 0 + or box[1] == 0 + or box[2] == frame_shape[1] - 1 + or box[3] == frame_shape[0] - 1 + ): + return True + + +def has_better_attr(current_thumb, new_obj, attr_label) -> bool: + max_new_attr = max( + [0] + + [area(a["box"]) for a in new_obj["attributes"] if a["label"] == attr_label] + ) + max_current_attr = max( + [0] + + [ + area(a["box"]) + for a in current_thumb["attributes"] + if a["label"] == attr_label + ] + ) + + # if the thumb has a higher scoring attr + return max_new_attr > max_current_attr + + +def is_better_thumbnail(label, current_thumb, new_obj, frame_shape) -> bool: + # larger is better + # cutoff images are less ideal, but they should also be smaller? + # better scores are obviously better too + + # check face on person + if label == "person": + if has_better_attr(current_thumb, new_obj, "face"): + return True + # if the current thumb has a face attr, dont update unless it gets better + if any([a["label"] == "face" for a in current_thumb["attributes"]]): + return False + + # check license_plate on car + if label == "car": + if has_better_attr(current_thumb, new_obj, "license_plate"): + return True + # if the current thumb has a license_plate attr, dont update unless it gets better + if any([a["label"] == "license_plate" for a in current_thumb["attributes"]]): + return False + + # if the new_thumb is on an edge, and the current thumb is not + if on_edge(new_obj["box"], frame_shape) and not on_edge( + current_thumb["box"], frame_shape + ): + return False + + # if the score is better by more than 5% + if new_obj["score"] > current_thumb["score"] + 0.05: + return True + + # if the area is 10% larger + if new_obj["area"] > current_thumb["area"] * 1.1: + return True + + return False + + def draw_timestamp( frame, timestamp, diff --git a/frigate/video.py b/frigate/video.py index 0f051b6b2..c0341446a 100755 --- a/frigate/video.py +++ b/frigate/video.py @@ -27,7 +27,7 @@ from frigate.object_detection import RemoteObjectDetector from frigate.ptz.autotrack import ptz_moving_at_frame_time from frigate.track import ObjectTracker from frigate.track.norfair_tracker import NorfairTracker -from frigate.track.object_attribute import ObjectAttribute +from frigate.track.tracked_object import TrackedObjectAttribute from frigate.util.builtin import EventsPerSecond, get_tomorrow_at_time from frigate.util.image import ( FrameManager, @@ -734,10 +734,10 @@ def process_frames( object_tracker.update_frame_times(frame_time) # group the attribute detections based on what label they apply to - attribute_detections: dict[str, list[ObjectAttribute]] = {} + attribute_detections: dict[str, list[TrackedObjectAttribute]] = {} for label, attribute_labels in model_config.attributes_map.items(): attribute_detections[label] = [ - ObjectAttribute(d) + TrackedObjectAttribute(d) for d in consolidated_detections if d[0] in attribute_labels ] From b299652e86869dae080725fceecf9eb34efbe254 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 17 Oct 2024 11:15:44 -0500 Subject: [PATCH 064/479] Generative AI changes (#14413) * Update default genai prompt * Update docs * improve wording * clarify wording --- docs/docs/configuration/genai.md | 22 ++++++++++++++-------- docs/docs/configuration/semantic_search.md | 11 ++++++----- frigate/config/camera/genai.py | 4 ++-- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index e2f6ac318..aace224f3 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -3,7 +3,7 @@ id: genai title: Generative AI --- -Generative AI can be used to automatically generate descriptions based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate by providing detailed text descriptions as a basis of the search query. +Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Semantic Search must be enabled to use Generative AI. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail. @@ -122,12 +122,18 @@ genai: api_key: "{FRIGATE_OPENAI_API_KEY}" ``` +## Usage and Best Practices + +Frigate's thumbnail search excels at identifying specific details about tracked objects – for example, using an "image caption" approach to find a "person wearing a yellow vest," "a white dog running across the lawn," or "a red car on a residential street." To enhance this further, Frigate’s default prompts are designed to ask your AI provider about the intent behind the object's actions, rather than just describing its appearance. + +While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate’s default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what’s happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they’re moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation’s context. + ## Custom Prompts Frigate sends multiple frames from the tracked object along with a prompt to your Generative AI provider asking it to generate a description. The default prompt is as follows: ``` -Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background. +Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next. ``` :::tip @@ -144,10 +150,10 @@ genai: provider: ollama base_url: http://localhost:11434 model: llava - prompt: "Describe the {label} in these images from the {camera} security camera." + prompt: "Analyze the {label} in these images from the {camera} security camera. Focus on the actions, behavior, and potential intent of the {label}, rather than just describing its appearance." object_prompts: - person: "Describe the main person in these images (gender, age, clothing, activity, etc). Do not include where the activity is occurring (sidewalk, concrete, driveway, etc)." - car: "Label the primary vehicle in these images with just the name of the company if it is a delivery vehicle, or the color make and model." + person: "Examine the main person in these images. What are they doing and what might their actions suggest about their intent (e.g., approaching a door, leaving an area, standing still)? Do not describe the surroundings or static details." + car: "Observe the primary vehicle in these images. Focus on its movement, direction, or purpose (e.g., parking, approaching, circling). If it's a delivery vehicle, mention the company." ``` Prompts can also be overriden at the camera level to provide a more detailed prompt to the model about your specific camera, if you desire. By default, descriptions will be generated for all tracked objects and all zones. But you can also optionally specify `objects` and `required_zones` to only generate descriptions for certain tracked objects or zones. @@ -159,10 +165,10 @@ cameras: front_door: genai: use_snapshot: True - prompt: "Describe the {label} in these images from the {camera} security camera at the front door of a house, aimed outward toward the street." + prompt: "Analyze the {label} in these images from the {camera} security camera at the front door. Focus on the actions and potential intent of the {label}." object_prompts: - person: "Describe the main person in these images (gender, age, clothing, activity, etc). Do not include where the activity is occurring (sidewalk, concrete, driveway, etc). If delivering a package, include the company the package is from." - cat: "Describe the cat in these images (color, size, tail). Indicate whether or not the cat is by the flower pots. If the cat is chasing a mouse, make up a name for the mouse." + person: "Examine the person in these images. What are they doing, and how might their actions suggest their purpose (e.g., delivering something, approaching, leaving)? If they are carrying or interacting with a package, include details about its source or destination." + cat: "Observe the cat in these images. Focus on its movement and intent (e.g., wandering, hunting, interacting with objects). If the cat is near the flower pots or engaging in any specific actions, mention it." objects: - person - cat diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 18093a479..a7b35ed77 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -50,10 +50,11 @@ semantic_search: - Configuring the `large` model employs the full Jina model and will automatically run on the GPU if applicable. - Configuring the `small` model employs a quantized version of the model that uses much less RAM and runs faster on CPU with a very negligible difference in embedding quality. -## Usage +## Usage and Best Practices 1. Semantic search is used in conjunction with the other filters available on the Search page. Use a combination of traditional filtering and semantic search for the best results. -2. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the the ones without them are more relevant. -3. Make your search language and tone closely match your descriptions. If you are using thumbnail search, **phrase your query as an image caption**. For example "red car" will not work as well as "red sedan driving down a residential street on a sunny day". -4. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well. -5. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you. +2. Use the thumbnail search type when searching for particular objects in the scene. Use the description search type when attempting to discern the intent of your object. +3. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that with multi-modal (`thumbnail` and `description`) searches, results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the the ones without them are more relevant. +4. Make your search language and tone closely match exactly what you're looking for. If you are using thumbnail search, **phrase your query as an image caption**. Searching for "red car" may not work as well as "red sedan driving down a residential street on a sunny day". +5. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well. +6. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you. diff --git a/frigate/config/camera/genai.py b/frigate/config/camera/genai.py index 21c3d4525..35c26eaf8 100644 --- a/frigate/config/camera/genai.py +++ b/frigate/config/camera/genai.py @@ -23,7 +23,7 @@ class GenAICameraConfig(BaseModel): default=False, title="Use snapshots for generating descriptions." ) prompt: str = Field( - default="Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background.", + default="Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next.", title="Default caption prompt.", ) object_prompts: dict[str, str] = Field( @@ -51,7 +51,7 @@ class GenAICameraConfig(BaseModel): class GenAIConfig(FrigateBaseModel): enabled: bool = Field(default=False, title="Enable GenAI.") prompt: str = Field( - default="Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background.", + default="Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next.", title="Default caption prompt.", ) object_prompts: dict[str, str] = Field( From 5d8bcb42c647d1d598a638edcc8a78167e8dd5d9 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 17 Oct 2024 11:21:27 -0500 Subject: [PATCH 065/479] Fix autotrack to work with new tracked object package (#14414) --- frigate/ptz/autotrack.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/ptz/autotrack.py b/frigate/ptz/autotrack.py index e9226f267..24b12087d 100644 --- a/frigate/ptz/autotrack.py +++ b/frigate/ptz/autotrack.py @@ -1275,7 +1275,7 @@ class PtzAutoTracker: # If it's within bounds, start tracking that object. # Should we check region (maybe too broad) or expand the previous object's box a bit and check that? self.tracked_object[camera] is None - and obj.camera == camera + and obj.camera_config.name == camera and obj.obj_data["label"] in self.object_types[camera] and not obj.previous["false_positive"] and not obj.false_positive From b56f4c4558e553283caf9e8360d65de5758708d3 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 18 Oct 2024 09:07:29 -0500 Subject: [PATCH 066/479] Semantic search docs update (#14438) * Add minimum requirements to semantic search docs * clarify --- docs/docs/configuration/semantic_search.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index a7b35ed77..7f84fdf95 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -9,6 +9,14 @@ Frigate has support for [Jina AI's CLIP model](https://huggingface.co/jinaai/jin Semantic Search is accessed via the _Explore_ view in the Frigate UI. +## Minimum System Requirements + +Semantic Search works by running a large AI model locally on your system. Small or underpowered systems like a Raspberry Pi will not run Semantic Search reliably or at all. + +A minimum of 8GB of RAM is required to use Semantic Search. A GPU is not strictly required but will provide a significant performance increase over CPU-only systems. + +For best performance, 16GB or more of RAM and a dedicated GPU are recommended. + ## Configuration Semantic search is disabled by default, and must be enabled in your config file before it can be used. Semantic Search is a global configuration setting. From 3c591ad8a9c503f683b5e195c607a39de66f982e Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 18 Oct 2024 16:16:43 -0500 Subject: [PATCH 067/479] Explore snapshot and clip filter (#14439) * backend * add ToggleButton component * boolean type * frontend * allow setting filter in input * better padding on dual slider * use shadcn toggle group instead of custom component --- frigate/api/defs/events_query_parameters.py | 2 + frigate/api/event.py | 8 + web/src/components/input/InputWithTags.tsx | 10 ++ .../overlay/dialog/SearchFilterDialog.tsx | 169 +++++++++++++++++- web/src/pages/Explore.tsx | 4 + web/src/types/search.ts | 2 + web/src/views/search/SearchView.tsx | 2 + 7 files changed, 194 insertions(+), 3 deletions(-) diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/events_query_parameters.py index f4c98809c..fe1d2b8b2 100644 --- a/frigate/api/defs/events_query_parameters.py +++ b/frigate/api/defs/events_query_parameters.py @@ -44,6 +44,8 @@ class EventsSearchQueryParams(BaseModel): after: Optional[float] = None before: Optional[float] = None time_range: Optional[str] = DEFAULT_TIME_RANGE + has_clip: Optional[bool] = None + has_snapshot: Optional[bool] = None timezone: Optional[str] = "utc" min_score: Optional[float] = None max_score: Optional[float] = None diff --git a/frigate/api/event.py b/frigate/api/event.py index 892624e53..d15fe326c 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -359,6 +359,8 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) min_score = params.min_score max_score = params.max_score time_range = params.time_range + has_clip = params.has_clip + has_snapshot = params.has_snapshot # for similarity search event_id = params.event_id @@ -433,6 +435,12 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) if before: event_filters.append((Event.start_time < before)) + if has_clip is not None: + event_filters.append((Event.has_clip == has_clip)) + + if has_snapshot is not None: + event_filters.append((Event.has_snapshot == has_snapshot)) + if min_score is not None and max_score is not None: event_filters.append((Event.data["score"].between(min_score, max_score))) else: diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 45dfd6c32..29a6f8a31 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -296,6 +296,14 @@ export default function InputWithTags({ ); } break; + case "has_snapshot": + if (!newFilters.has_snapshot) newFilters.has_snapshot = undefined; + newFilters.has_snapshot = value == "yes" ? 1 : 0; + break; + case "has_clip": + if (!newFilters.has_clip) newFilters.has_clip = undefined; + newFilters.has_clip = value == "yes" ? 1 : 0; + break; case "event_id": newFilters.event_id = value; break; @@ -341,6 +349,8 @@ export default function InputWithTags({ }`; } else if (filterType === "min_score" || filterType === "max_score") { return Math.round(Number(filterValues) * 100).toString() + "%"; + } else if (filterType === "has_clip" || filterType === "has_snapshot") { + return filterValues ? "Yes" : "No"; } else { return filterValues as string; } diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx index ed091b350..fdc80eefd 100644 --- a/web/src/components/overlay/dialog/SearchFilterDialog.tsx +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -25,6 +25,8 @@ import { DropdownMenuSeparator } from "@/components/ui/dropdown-menu"; import { cn } from "@/lib/utils"; import { DualThumbSlider } from "@/components/ui/slider"; import { Input } from "@/components/ui/input"; +import { Checkbox } from "@/components/ui/checkbox"; +import { ToggleGroup, ToggleGroupItem } from "@/components/ui/toggle-group"; type SearchFilterDialogProps = { config?: FrigateConfig; @@ -63,6 +65,8 @@ export default function SearchFilterDialog({ currentFilter && (currentFilter.time_range || (currentFilter.min_score ?? 0) > 0.5 || + (currentFilter.has_snapshot ?? 0) === 1 || + (currentFilter.has_clip ?? 0) === 1 || (currentFilter.max_score ?? 1) < 1 || (currentFilter.zones?.length ?? 0) > 0 || (currentFilter.sub_labels?.length ?? 0) > 0), @@ -113,6 +117,26 @@ export default function SearchFilterDialog({ setCurrentFilter({ ...currentFilter, min_score: min, max_score: max }) } /> + + setCurrentFilter({ + ...currentFilter, + has_snapshot: + snapshot !== undefined ? (snapshot ? 1 : 0) : undefined, + has_clip: clip !== undefined ? (clip ? 1 : 0) : undefined, + }) + } + /> {isDesktop && }
))} @@ -184,54 +203,78 @@ function ThumbnailRow({ type ExploreThumbnailImageProps = { event: SearchResult; setSearchDetail: (search: SearchResult | undefined) => void; + mutate: () => void; + setSimilaritySearch: (search: SearchResult) => void; + onSelectSearch: (item: SearchResult, index: number, page?: SearchTab) => void; }; function ExploreThumbnailImage({ event, setSearchDetail, + mutate, + setSimilaritySearch, + onSelectSearch, }: ExploreThumbnailImageProps) { const apiHost = useApiHost(); + const { data: config } = useSWR("config"); const [imgRef, imgLoaded, onImgLoad] = useImageLoaded(); - return ( - <> - + const handleFindSimilar = () => { + if (config?.semantic_search.enabled) { + setSimilaritySearch(event); + } + }; - setSearchDetail(event)} - onLoad={() => { - onImgLoad(); - }} - /> - {isDesktop && ( -
- {event.end_time ? ( - - ) : ( -
- -
+ const handleShowObjectLifecycle = () => { + onSelectSearch(event, 0, "object lifecycle"); + }; + + return ( + +
+ + - )} - + style={ + isIOS + ? { + WebkitUserSelect: "none", + WebkitTouchCallout: "none", + } + : undefined + } + loading={isSafari ? "eager" : "lazy"} + draggable={false} + src={`${apiHost}api/events/${event.id}/thumbnail.jpg`} + onClick={() => setSearchDetail(event)} + onLoad={onImgLoad} + alt={`${event.label} thumbnail`} + /> + {isDesktop && ( +
+ {event.end_time ? ( + + ) : ( +
+ +
+ )} +
+ )} +
+ ); } diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 4b84d0ac5..5fd6c98fa 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -489,6 +489,8 @@ export default function SearchView({
)} From c7d9f8363838093ef7a5cab4e29c49b7bf6165d1 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 22 Oct 2024 16:11:05 -0500 Subject: [PATCH 082/479] UI changes and fixes (#14516) * Add camera webui link to debug view * fix optimistic description update * simplify * clean up * params --- .../overlay/detail/SearchDetailDialog.tsx | 18 ++++++------------ web/src/views/settings/ObjectSettingsView.tsx | 18 +++++++++++++++++- 2 files changed, 23 insertions(+), 13 deletions(-) diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index bbe82a0e2..a307e6668 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -321,22 +321,16 @@ function ObjectDetailsTab({ (key.includes("events") || key.includes("events/search") || key.includes("events/explore")), - (currentData: SearchResult[][] | undefined) => { + (currentData: SearchResult[][] | SearchResult[] | undefined) => { if (!currentData) return currentData; // optimistic update - return currentData.map((page) => - page.map((event) => + return currentData + .flat() + .map((event) => event.id === search.id - ? { - ...event, - data: { - ...event.data, - description: desc, - }, - } + ? { ...event, data: { ...event.data, description: desc } } : event, - ), - ); + ); }, { optimisticData: true, diff --git a/web/src/views/settings/ObjectSettingsView.tsx b/web/src/views/settings/ObjectSettingsView.tsx index 66962cca7..7b8a08d2e 100644 --- a/web/src/views/settings/ObjectSettingsView.tsx +++ b/web/src/views/settings/ObjectSettingsView.tsx @@ -21,7 +21,8 @@ import useDeepMemo from "@/hooks/use-deep-memo"; import { Card } from "@/components/ui/card"; import { getIconForLabel } from "@/utils/iconUtil"; import { capitalizeFirstLetter } from "@/utils/stringUtil"; -import { LuInfo } from "react-icons/lu"; +import { LuExternalLink, LuInfo } from "react-icons/lu"; +import { Link } from "react-router-dom"; type ObjectSettingsViewProps = { selectedCamera?: string; @@ -187,6 +188,21 @@ export default function ObjectSettingsView({ objects.

+ {config?.cameras[cameraConfig.name]?.webui_url && ( +
+
+ + Open {capitalizeFirstLetter(cameraConfig.name)}'s Web UI + + +
+
+ )} From ad308252a12ececa58184bb192173f196cba703c Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 22 Oct 2024 17:07:42 -0500 Subject: [PATCH 083/479] Accessibility features (#14518) * Add screen reader aria labels to buttons and menu items * Fix sub_label score in search detail dialog --- web/src/components/auth/AuthForm.tsx | 1 + .../components/button/DownloadVideoButton.tsx | 1 + .../components/camera/DebugCameraImage.tsx | 7 +- web/src/components/card/AnimatedEventCard.tsx | 1 + web/src/components/card/ExportCard.tsx | 2 + web/src/components/dynamic/NewReviewData.tsx | 1 + .../filter/CalendarFilterButton.tsx | 3 + .../components/filter/CameraGroupSelector.tsx | 22 ++- .../components/filter/CamerasFilterButton.tsx | 3 + web/src/components/filter/LogLevelFilter.tsx | 6 +- .../components/filter/ReviewActionGroup.tsx | 3 + .../components/filter/ReviewFilterGroup.tsx | 5 + .../components/filter/SearchFilterGroup.tsx | 3 + web/src/components/filter/ZoneMaskFilter.tsx | 1 + web/src/components/icons/IconPicker.tsx | 5 +- web/src/components/input/SaveSearchDialog.tsx | 5 +- web/src/components/menu/AccountSettings.tsx | 1 + web/src/components/menu/GeneralSettings.tsx | 19 +- .../components/menu/SearchResultActions.tsx | 24 ++- web/src/components/mobile/MobilePage.tsx | 1 + .../components/overlay/CameraInfoDialog.tsx | 6 +- .../components/overlay/CreateUserDialog.tsx | 6 +- .../components/overlay/DeleteUserDialog.tsx | 1 + web/src/components/overlay/ExportDialog.tsx | 4 + web/src/components/overlay/GPUInfoDialog.tsx | 26 ++- .../components/overlay/MobileCameraDrawer.tsx | 6 +- .../overlay/MobileReviewSettingsDrawer.tsx | 5 + .../overlay/MobileTimelineDrawer.tsx | 6 +- .../components/overlay/SaveExportOverlay.tsx | 3 + .../components/overlay/SetPasswordDialog.tsx | 1 + .../overlay/detail/AnnotationSettingsPane.tsx | 2 + .../overlay/detail/ObjectLifecycle.tsx | 2 + .../overlay/detail/ReviewDetailDialog.tsx | 1 + .../overlay/detail/SearchDetailDialog.tsx | 19 +- .../overlay/dialog/FrigatePlusDialog.tsx | 8 +- .../overlay/dialog/SearchFilterDialog.tsx | 5 + .../settings/MotionMaskEditPane.tsx | 7 +- .../settings/ObjectMaskEditPane.tsx | 7 +- .../settings/PolygonEditControls.tsx | 2 + web/src/components/settings/PolygonItem.tsx | 7 +- .../components/settings/SearchSettings.tsx | 6 +- web/src/components/settings/ZoneEditPane.tsx | 7 +- web/src/components/ui/calendar-range.tsx | 3 + web/src/components/ui/carousel.tsx | 162 +++++++++--------- web/src/pages/ConfigEditor.tsx | 3 + web/src/pages/Exports.tsx | 1 + web/src/pages/Logs.tsx | 3 + web/src/pages/Settings.tsx | 1 + web/src/views/events/EventView.tsx | 1 + web/src/views/live/LiveBirdseyeView.tsx | 1 + web/src/views/live/LiveCameraView.tsx | 13 +- web/src/views/live/LiveDashboardView.tsx | 3 + web/src/views/recording/RecordingView.tsx | 2 + web/src/views/settings/AuthenticationView.tsx | 3 + web/src/views/settings/CameraSettingsView.tsx | 2 + web/src/views/settings/MasksAndZonesView.tsx | 3 + web/src/views/settings/MotionTunerView.tsx | 7 +- .../settings/NotificationsSettingsView.tsx | 3 + web/src/views/settings/SearchSettingsView.tsx | 3 +- web/src/views/settings/UiSettingsView.tsx | 7 +- web/src/views/system/GeneralMetrics.tsx | 1 + 61 files changed, 358 insertions(+), 115 deletions(-) diff --git a/web/src/components/auth/AuthForm.tsx b/web/src/components/auth/AuthForm.tsx index f3a435828..9daa92966 100644 --- a/web/src/components/auth/AuthForm.tsx +++ b/web/src/components/auth/AuthForm.tsx @@ -121,6 +121,7 @@ export function UserAuthForm({ className, ...props }: UserAuthFormProps) { variant="select" disabled={isLoading} className="flex flex-1" + aria-label="Login" > {isLoading && } Login diff --git a/web/src/components/button/DownloadVideoButton.tsx b/web/src/components/button/DownloadVideoButton.tsx index 8a8e541fa..5ea1f8465 100644 --- a/web/src/components/button/DownloadVideoButton.tsx +++ b/web/src/components/button/DownloadVideoButton.tsx @@ -46,6 +46,7 @@ export function DownloadVideoButton({ disabled={isDownloading} className="flex items-center gap-2" size="sm" + aria-label="Download Video" > - diff --git a/web/src/components/filter/ReviewActionGroup.tsx b/web/src/components/filter/ReviewActionGroup.tsx index f2d67efd7..833039272 100644 --- a/web/src/components/filter/ReviewActionGroup.tsx +++ b/web/src/components/filter/ReviewActionGroup.tsx @@ -104,6 +104,7 @@ export default function ReviewActionGroup({ {selectedReviews.length == 1 && ( ) : ( diff --git a/web/src/components/input/SaveSearchDialog.tsx b/web/src/components/input/SaveSearchDialog.tsx index c5bf29001..89e9217d7 100644 --- a/web/src/components/input/SaveSearchDialog.tsx +++ b/web/src/components/input/SaveSearchDialog.tsx @@ -59,11 +59,14 @@ export function SaveSearchDialog({ placeholder="Enter a name for your search" /> - + diff --git a/web/src/components/menu/AccountSettings.tsx b/web/src/components/menu/AccountSettings.tsx index 1b7470b9b..0bc968061 100644 --- a/web/src/components/menu/AccountSettings.tsx +++ b/web/src/components/menu/AccountSettings.tsx @@ -72,6 +72,7 @@ export default function AccountSettings({ className }: AccountSettingsProps) { className={ isDesktop ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Log out" > diff --git a/web/src/components/menu/GeneralSettings.tsx b/web/src/components/menu/GeneralSettings.tsx index c1c46d87b..0341f2500 100644 --- a/web/src/components/menu/GeneralSettings.tsx +++ b/web/src/components/menu/GeneralSettings.tsx @@ -176,6 +176,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Log out" > @@ -194,6 +195,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex w-full items-center p-2 text-sm" } + aria-label="System metrics" > System metrics @@ -206,6 +208,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex w-full items-center p-2 text-sm" } + aria-label="System logs" > System logs @@ -224,6 +227,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex w-full items-center p-2 text-sm" } + aria-label="Settings" > Settings @@ -236,6 +240,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex w-full items-center p-2 text-sm" } + aria-label="Configuration editor" > Configuration editor @@ -269,6 +274,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Light mode" onClick={() => setTheme("light")} > {theme === "light" ? ( @@ -286,6 +292,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Dark mode" onClick={() => setTheme("dark")} > {theme === "dark" ? ( @@ -303,6 +310,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Use the system settings for light or dark mode" onClick={() => setTheme("system")} > {theme === "system" ? ( @@ -343,6 +351,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label={`Color scheme - ${scheme}`} onClick={() => setColorScheme(scheme)} > {scheme === colorScheme ? ( @@ -370,6 +379,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { className={ isDesktop ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Frigate documentation" > Documentation @@ -383,6 +393,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { className={ isDesktop ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Frigate Github" > GitHub @@ -393,6 +404,7 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { className={ isDesktop ? "cursor-pointer" : "flex items-center p-2 text-sm" } + aria-label="Restart Frigate" onClick={() => setRestartDialogOpen(true)} > @@ -446,7 +458,12 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) {

This page will reload in {countdown} seconds.

-
diff --git a/web/src/components/menu/SearchResultActions.tsx b/web/src/components/menu/SearchResultActions.tsx index d95acc5a5..8a9373bcc 100644 --- a/web/src/components/menu/SearchResultActions.tsx +++ b/web/src/components/menu/SearchResultActions.tsx @@ -86,7 +86,7 @@ export default function SearchResultActions({ const menuItems = ( <> {searchResult.has_clip && ( - + )} {searchResult.has_snapshot && ( - + )} - + View object lifecycle {config?.semantic_search?.enabled && isContextMenu && ( - + Find similar @@ -124,12 +130,18 @@ export default function SearchResultActions({ searchResult.has_snapshot && searchResult.end_time && !searchResult.plus_id && ( - setShowFrigatePlus(true)}> + setShowFrigatePlus(true)} + > Submit to Frigate+ )} - setDeleteDialogOpen(true)}> + setDeleteDialogOpen(true)} + > Delete diff --git a/web/src/components/mobile/MobilePage.tsx b/web/src/components/mobile/MobilePage.tsx index 37e54a49c..52bc4d9fe 100644 --- a/web/src/components/mobile/MobilePage.tsx +++ b/web/src/components/mobile/MobilePage.tsx @@ -154,6 +154,7 @@ export function MobilePageHeader({ >
- diff --git a/web/src/components/overlay/CreateUserDialog.tsx b/web/src/components/overlay/CreateUserDialog.tsx index 65741b65e..7d44159dd 100644 --- a/web/src/components/overlay/CreateUserDialog.tsx +++ b/web/src/components/overlay/CreateUserDialog.tsx @@ -98,7 +98,11 @@ export default function CreateUserDialog({ )} /> - diff --git a/web/src/components/overlay/DeleteUserDialog.tsx b/web/src/components/overlay/DeleteUserDialog.tsx index a1c0b2a32..8638b9145 100644 --- a/web/src/components/overlay/DeleteUserDialog.tsx +++ b/web/src/components/overlay/DeleteUserDialog.tsx @@ -27,6 +27,7 @@ export default function DeleteUserDialog({
- + @@ -88,8 +97,17 @@ export default function GPUInfoDialog({ )} - - + diff --git a/web/src/components/overlay/MobileCameraDrawer.tsx b/web/src/components/overlay/MobileCameraDrawer.tsx index 0b450ff32..c12bc0ab2 100644 --- a/web/src/components/overlay/MobileCameraDrawer.tsx +++ b/web/src/components/overlay/MobileCameraDrawer.tsx @@ -23,7 +23,11 @@ export default function MobileCameraDrawer({ return ( - diff --git a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx index fe0e13c11..d58d485b9 100644 --- a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx +++ b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx @@ -132,6 +132,7 @@ export default function MobileReviewSettingsDrawer({ {features.includes("export") && ( diff --git a/web/src/components/overlay/SaveExportOverlay.tsx b/web/src/components/overlay/SaveExportOverlay.tsx index 1ce552a9a..6bb899ed8 100644 --- a/web/src/components/overlay/SaveExportOverlay.tsx +++ b/web/src/components/overlay/SaveExportOverlay.tsx @@ -28,6 +28,7 @@ export default function SaveExportOverlay({ > regenerateDescription("snapshot")} > Regenerate from Snapshot regenerateDescription("thumbnails")} > Regenerate from Thumbnails @@ -495,7 +502,11 @@ function ObjectDetailsTab({ )}
)} -
@@ -601,6 +612,7 @@ function ObjectSnapshotTab({ <> } + {dialog && ( + + )} diff --git a/web/src/components/settings/ZoneEditPane.tsx b/web/src/components/settings/ZoneEditPane.tsx index 949cfd1ac..54799db72 100644 --- a/web/src/components/settings/ZoneEditPane.tsx +++ b/web/src/components/settings/ZoneEditPane.tsx @@ -466,13 +466,18 @@ export default function ZoneEditPane({ )} />
- diff --git a/web/src/components/ui/carousel.tsx b/web/src/components/ui/carousel.tsx index 9c2b9bf37..7667f4e83 100644 --- a/web/src/components/ui/carousel.tsx +++ b/web/src/components/ui/carousel.tsx @@ -1,43 +1,43 @@ -import * as React from "react" +import * as React from "react"; import useEmblaCarousel, { type UseEmblaCarouselType, -} from "embla-carousel-react" -import { ArrowLeft, ArrowRight } from "lucide-react" +} from "embla-carousel-react"; +import { ArrowLeft, ArrowRight } from "lucide-react"; -import { cn } from "@/lib/utils" -import { Button } from "@/components/ui/button" +import { cn } from "@/lib/utils"; +import { Button } from "@/components/ui/button"; -type CarouselApi = UseEmblaCarouselType[1] -type UseCarouselParameters = Parameters -type CarouselOptions = UseCarouselParameters[0] -type CarouselPlugin = UseCarouselParameters[1] +type CarouselApi = UseEmblaCarouselType[1]; +type UseCarouselParameters = Parameters; +type CarouselOptions = UseCarouselParameters[0]; +type CarouselPlugin = UseCarouselParameters[1]; type CarouselProps = { - opts?: CarouselOptions - plugins?: CarouselPlugin - orientation?: "horizontal" | "vertical" - setApi?: (api: CarouselApi) => void -} + opts?: CarouselOptions; + plugins?: CarouselPlugin; + orientation?: "horizontal" | "vertical"; + setApi?: (api: CarouselApi) => void; +}; type CarouselContextProps = { - carouselRef: ReturnType[0] - api: ReturnType[1] - scrollPrev: () => void - scrollNext: () => void - canScrollPrev: boolean - canScrollNext: boolean -} & CarouselProps + carouselRef: ReturnType[0]; + api: ReturnType[1]; + scrollPrev: () => void; + scrollNext: () => void; + canScrollPrev: boolean; + canScrollNext: boolean; +} & CarouselProps; -const CarouselContext = React.createContext(null) +const CarouselContext = React.createContext(null); function useCarousel() { - const context = React.useContext(CarouselContext) + const context = React.useContext(CarouselContext); if (!context) { - throw new Error("useCarousel must be used within a ") + throw new Error("useCarousel must be used within a "); } - return context + return context; } const Carousel = React.forwardRef< @@ -54,69 +54,69 @@ const Carousel = React.forwardRef< children, ...props }, - ref + ref, ) => { const [carouselRef, api] = useEmblaCarousel( { ...opts, axis: orientation === "horizontal" ? "x" : "y", }, - plugins - ) - const [canScrollPrev, setCanScrollPrev] = React.useState(false) - const [canScrollNext, setCanScrollNext] = React.useState(false) + plugins, + ); + const [canScrollPrev, setCanScrollPrev] = React.useState(false); + const [canScrollNext, setCanScrollNext] = React.useState(false); const onSelect = React.useCallback((api: CarouselApi) => { if (!api) { - return + return; } - setCanScrollPrev(api.canScrollPrev()) - setCanScrollNext(api.canScrollNext()) - }, []) + setCanScrollPrev(api.canScrollPrev()); + setCanScrollNext(api.canScrollNext()); + }, []); const scrollPrev = React.useCallback(() => { - api?.scrollPrev() - }, [api]) + api?.scrollPrev(); + }, [api]); const scrollNext = React.useCallback(() => { - api?.scrollNext() - }, [api]) + api?.scrollNext(); + }, [api]); const handleKeyDown = React.useCallback( (event: React.KeyboardEvent) => { if (event.key === "ArrowLeft") { - event.preventDefault() - scrollPrev() + event.preventDefault(); + scrollPrev(); } else if (event.key === "ArrowRight") { - event.preventDefault() - scrollNext() + event.preventDefault(); + scrollNext(); } }, - [scrollPrev, scrollNext] - ) + [scrollPrev, scrollNext], + ); React.useEffect(() => { if (!api || !setApi) { - return + return; } - setApi(api) - }, [api, setApi]) + setApi(api); + }, [api, setApi]); React.useEffect(() => { if (!api) { - return + return; } - onSelect(api) - api.on("reInit", onSelect) - api.on("select", onSelect) + onSelect(api); + api.on("reInit", onSelect); + api.on("select", onSelect); return () => { - api?.off("select", onSelect) - } - }, [api, onSelect]) + api?.off("select", onSelect); + }; + }, [api, onSelect]); return ( - ) - } -) -Carousel.displayName = "Carousel" + ); + }, +); +Carousel.displayName = "Carousel"; const CarouselContent = React.forwardRef< HTMLDivElement, React.HTMLAttributes >(({ className, ...props }, ref) => { - const { carouselRef, orientation } = useCarousel() + const { carouselRef, orientation } = useCarousel(); return (
@@ -161,20 +161,20 @@ const CarouselContent = React.forwardRef< className={cn( "flex", orientation === "horizontal" ? "-ml-4" : "-mt-4 flex-col", - className + className, )} {...props} />
- ) -}) -CarouselContent.displayName = "CarouselContent" + ); +}); +CarouselContent.displayName = "CarouselContent"; const CarouselItem = React.forwardRef< HTMLDivElement, React.HTMLAttributes >(({ className, ...props }, ref) => { - const { orientation } = useCarousel() + const { orientation } = useCarousel(); return (
- ) -}) -CarouselItem.displayName = "CarouselItem" + ); +}); +CarouselItem.displayName = "CarouselItem"; const CarouselPrevious = React.forwardRef< HTMLButtonElement, React.ComponentProps >(({ className, variant = "outline", size = "icon", ...props }, ref) => { - const { orientation, scrollPrev, canScrollPrev } = useCarousel() + const { orientation, scrollPrev, canScrollPrev } = useCarousel(); return ( - ) -}) -CarouselPrevious.displayName = "CarouselPrevious" + ); +}); +CarouselPrevious.displayName = "CarouselPrevious"; const CarouselNext = React.forwardRef< HTMLButtonElement, React.ComponentProps >(({ className, variant = "outline", size = "icon", ...props }, ref) => { - const { orientation, scrollNext, canScrollNext } = useCarousel() + const { orientation, scrollNext, canScrollNext } = useCarousel(); return ( - ) -}) -CarouselNext.displayName = "CarouselNext" + ); +}); +CarouselNext.displayName = "CarouselNext"; export { type CarouselApi, @@ -257,4 +259,4 @@ export { CarouselItem, CarouselPrevious, CarouselNext, -} +}; diff --git a/web/src/pages/ConfigEditor.tsx b/web/src/pages/ConfigEditor.tsx index 857c94900..52cb05473 100644 --- a/web/src/pages/ConfigEditor.tsx +++ b/web/src/pages/ConfigEditor.tsx @@ -192,6 +192,7 @@ function ConfigEditor() { @@ -717,6 +727,7 @@ function PtzControlPanel({ return ( sendPtz(`preset_${preset}`)} > diff --git a/web/src/views/live/LiveDashboardView.tsx b/web/src/views/live/LiveDashboardView.tsx index cac604e26..7642d5a0d 100644 --- a/web/src/views/live/LiveDashboardView.tsx +++ b/web/src/views/live/LiveDashboardView.tsx @@ -240,6 +240,7 @@ export default function LiveDashboardView({ ? "bg-blue-900 bg-opacity-60 focus:bg-blue-900 focus:bg-opacity-60" : "bg-secondary" }`} + aria-label="Use mobile grid layout" size="xs" onClick={() => setMobileLayout("grid")} > @@ -251,6 +252,7 @@ export default function LiveDashboardView({ ? "bg-blue-900 bg-opacity-60 focus:bg-blue-900 focus:bg-opacity-60" : "bg-secondary" }`} + aria-label="Use mobile list layout" size="xs" onClick={() => setMobileLayout("list")} > @@ -267,6 +269,7 @@ export default function LiveDashboardView({ ? "bg-selected text-primary" : "bg-secondary text-secondary-foreground", )} + aria-label="Enter layout editing mode" size="xs" onClick={() => setIsEditMode((prevIsEditMode) => !prevIsEditMode) diff --git a/web/src/views/recording/RecordingView.tsx b/web/src/views/recording/RecordingView.tsx index 535c412d4..374201f7c 100644 --- a/web/src/views/recording/RecordingView.tsx +++ b/web/src/views/recording/RecordingView.tsx @@ -380,6 +380,7 @@ export function RecordingView({
-
- +
diff --git a/web/src/views/system/GeneralMetrics.tsx b/web/src/views/system/GeneralMetrics.tsx index e6b04f5b7..6e85710bc 100644 --- a/web/src/views/system/GeneralMetrics.tsx +++ b/web/src/views/system/GeneralMetrics.tsx @@ -541,6 +541,7 @@ export default function GeneralMetrics({ {canGetGpuInfo && ( diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 1bb887181..5df921d4f 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -27,6 +27,7 @@ import ActivityIndicator from "@/components/indicators/activity-indicator"; import { FaCheckCircle, FaChevronDown, + FaDownload, FaHistory, FaImage, FaRegListAlt, @@ -68,6 +69,7 @@ import { PopoverTrigger, } from "@/components/ui/popover"; import { LuInfo } from "react-icons/lu"; +import { TooltipPortal } from "@radix-ui/react-tooltip"; const SEARCH_TABS = [ "details", @@ -577,16 +579,39 @@ function ObjectSnapshotTab({ }} > {search?.id && ( - {`${search?.label}`} { - onImgLoad(); - }} - /> +
+ {`${search?.label}`} { + onImgLoad(); + }} + /> +
+ + + + + + + + + + Download + + +
+
)} {search.plus_id !== "not_enabled" && search.end_time && ( @@ -669,7 +694,7 @@ export function VideoTab({ search }: VideoTabProps) { {reviewItem && (
@@ -689,7 +714,24 @@ export function VideoTab({ search }: VideoTabProps) { - View in History + + View in History + + + + + + + + + + + + Download +
)} From 7afc1e9762d20caebf2696a39c37fcf6e062adbc Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 23 Oct 2024 07:14:50 -0500 Subject: [PATCH 087/479] Improve error message when semantic search is not enabled with genai (#14528) --- frigate/api/event.py | 2 +- web/src/components/overlay/detail/SearchDetailDialog.tsx | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/frigate/api/event.py b/frigate/api/event.py index 89b2fedef..7f4f14610 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -1015,7 +1015,7 @@ def regenerate_description( content=( { "success": False, - "message": "Semantic search and generative AI are not enabled", + "message": "Semantic Search and Generative AI must be enabled to regenerate a description", } ), status_code=400, diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 5df921d4f..f443c9d44 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -368,9 +368,9 @@ function ObjectDetailsTab({ ); } }) - .catch(() => { + .catch((error) => { toast.error( - `Failed to call ${capitalizeAll(config?.genai.provider.replaceAll("_", " ") ?? "Generative AI")} for a new description`, + `Failed to call ${capitalizeAll(config?.genai.provider.replaceAll("_", " ") ?? "Generative AI")} for a new description: ${error.response.data.message}`, { position: "top-center", }, From 8bc145472a1bc86dc96940d52917924d2f0211ab Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 23 Oct 2024 08:31:48 -0500 Subject: [PATCH 088/479] Error message and search reset for explore pane (#14534) --- web/src/pages/Explore.tsx | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index eee9fce9b..8989c7b05 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -18,6 +18,7 @@ import { isMobileOnly } from "react-device-detect"; import { LuCheck, LuExternalLink, LuX } from "react-icons/lu"; import { TbExclamationCircle } from "react-icons/tb"; import { Link } from "react-router-dom"; +import { toast } from "sonner"; import useSWR from "swr"; import useSWRInfinite from "swr/infinite"; @@ -180,6 +181,18 @@ export default function Explore() { revalidateFirstPage: true, revalidateOnFocus: true, revalidateAll: false, + onError: (error) => { + toast.error( + `Error fetching tracked objects: ${error.response.data.message}`, + { + position: "top-center", + }, + ); + if (error.response.status === 404) { + // reset all filters if 404 + setSearchFilter({}); + } + }, }); const searchResults = useMemo( From fa81d87dc0f63d36aca9f42031066271fa7fad60 Mon Sep 17 00:00:00 2001 From: Rui Alves Date: Wed, 23 Oct 2024 14:35:49 +0100 Subject: [PATCH 089/479] Updated Documentation for the Review endpoints (#14401) * Updated documentation for the review endpoint * Updated documentation for the review/summary endpoint * Updated documentation for the review/summary endpoint * Documentation for the review activity audio and motion endpoints * Added responses for more review.py endpoints * Added responses for more review.py endpoints * Fixed review.py responses and proper path parameter names * Added body model for /reviews/viewed and /reviews/delete * Updated OpenAPI specification for the review controller endpoints * Run ruff format frigate * Drop significant_motion * Updated frigate-api.yaml * Deleted total_motion * Combine 2 models into generic --- docs/static/frigate-api.yaml | 428 +++++++++++++------- frigate/api/defs/generic_response.py | 6 + frigate/api/defs/review_body.py | 6 + frigate/api/defs/review_query_parameters.py | 37 +- frigate/api/defs/review_responses.py | 43 ++ frigate/api/fastapi_app.py | 4 + frigate/api/review.py | 196 ++------- frigate/models.py | 2 +- web/src/pages/Events.tsx | 4 - web/src/types/review.ts | 2 - web/src/views/events/EventView.tsx | 2 - 11 files changed, 404 insertions(+), 326 deletions(-) create mode 100644 frigate/api/defs/generic_response.py create mode 100644 frigate/api/defs/review_body.py create mode 100644 frigate/api/defs/review_responses.py diff --git a/docs/static/frigate-api.yaml b/docs/static/frigate-api.yaml index 1833aab99..9a6364e27 100644 --- a/docs/static/frigate-api.yaml +++ b/docs/static/frigate-api.yaml @@ -172,76 +172,65 @@ paths: in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Cameras - name: labels in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Labels - name: zones in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Zones - name: reviewed in: query required: false schema: - anyOf: - - type: integer - - type: 'null' + type: integer default: 0 title: Reviewed - name: limit in: query required: false schema: - anyOf: - - type: integer - - type: 'null' + type: integer title: Limit - name: severity in: query required: false schema: - anyOf: - - type: string - - type: 'null' + allOf: + - $ref: '#/components/schemas/SeverityEnum' title: Severity - name: before in: query required: false schema: - anyOf: - - type: number - - type: 'null' + type: number title: Before - name: after in: query required: false schema: - anyOf: - - type: number - - type: 'null' + type: number title: After responses: '200': description: Successful Response content: application/json: - schema: { } + schema: + type: array + items: + $ref: '#/components/schemas/ReviewSegmentResponse' + title: Response Review Review Get '422': description: Validation Error content: @@ -259,36 +248,28 @@ paths: in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Cameras - name: labels in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Labels - name: zones in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Zones - name: timezone in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: utc title: Timezone responses: @@ -296,7 +277,8 @@ paths: description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/ReviewSummaryResponse' '422': description: Validation Error content: @@ -310,17 +292,18 @@ paths: summary: Set Multiple Reviewed operationId: set_multiple_reviewed_reviews_viewed_post requestBody: + required: true content: application/json: schema: - type: object - title: Body + $ref: '#/components/schemas/ReviewSetMultipleReviewedBody' responses: '200': description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/GenericResponse' '422': description: Validation Error content: @@ -334,17 +317,18 @@ paths: summary: Delete Reviews operationId: delete_reviews_reviews_delete_post requestBody: + required: true content: application/json: schema: - type: object - title: Body + $ref: '#/components/schemas/ReviewDeleteMultipleReviewsBody' responses: '200': description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/GenericResponse' '422': description: Validation Error content: @@ -363,96 +347,38 @@ paths: in: query required: false schema: - anyOf: - - type: string - - type: 'null' + type: string default: all title: Cameras - name: before in: query required: false schema: - anyOf: - - type: number - - type: 'null' + type: number title: Before - name: after in: query required: false schema: - anyOf: - - type: number - - type: 'null' + type: number title: After - name: scale in: query required: false schema: - anyOf: - - type: integer - - type: 'null' + type: integer default: 30 title: Scale responses: '200': description: Successful Response - content: - application/json: - schema: { } - '422': - description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /review/activity/audio: - get: - tags: - - Review - summary: Audio Activity - description: Get motion and audio activity. - operationId: audio_activity_review_activity_audio_get - parameters: - - name: cameras - in: query - required: false - schema: - anyOf: - - type: string - - type: 'null' - default: all - title: Cameras - - name: before - in: query - required: false - schema: - anyOf: - - type: number - - type: 'null' - title: Before - - name: after - in: query - required: false - schema: - anyOf: - - type: number - - type: 'null' - title: After - - name: scale - in: query - required: false - schema: - anyOf: - - type: integer - - type: 'null' - default: 30 - title: Scale - responses: - '200': - description: Successful Response - content: - application/json: - schema: { } + type: array + items: + $ref: '#/components/schemas/ReviewActivityMotionResponse' + title: Response Motion Activity Review Activity Motion Get '422': description: Validation Error content: @@ -477,57 +403,60 @@ paths: description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/ReviewSegmentResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' - /review/{event_id}: + /review/{review_id}: get: tags: - Review summary: Get Review - operationId: get_review_review__event_id__get + operationId: get_review_review__review_id__get parameters: - - name: event_id + - name: review_id in: path required: true schema: type: string - title: Event Id + title: Review Id responses: '200': description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/ReviewSegmentResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' - /review/{event_id}/viewed: + /review/{review_id}/viewed: delete: tags: - Review summary: Set Not Reviewed - operationId: set_not_reviewed_review__event_id__viewed_delete + operationId: set_not_reviewed_review__review_id__viewed_delete parameters: - - name: event_id + - name: review_id in: path required: true schema: type: string - title: Event Id + title: Review Id responses: '200': description: Successful Response content: application/json: - schema: { } + schema: + $ref: '#/components/schemas/GenericResponse' '422': description: Validation Error content: @@ -763,13 +692,25 @@ paths: content: application/json: schema: { } + /nvinfo: + get: + tags: + - App + summary: Nvinfo + operationId: nvinfo_nvinfo_get + responses: + '200': + description: Successful Response + content: + application/json: + schema: { } /logs/{service}: get: tags: - App - Logs summary: Logs - description: Get logs for the requested service (frigate/nginx/go2rtc/chroma) + description: Get logs for the requested service (frigate/nginx/go2rtc) operationId: logs_logs__service__get parameters: - name: service @@ -781,7 +722,6 @@ paths: - frigate - nginx - go2rtc - - chroma title: Service - name: download in: query @@ -1042,7 +982,8 @@ paths: - Preview summary: Preview Hour description: Get all mp4 previews relevant for time period given the timezone - operationId: preview_hour_preview__year_month___day___hour___camera_name___tz_name__get + operationId: >- + preview_hour_preview__year_month___day___hour___camera_name___tz_name__get parameters: - name: year_month in: path @@ -1092,7 +1033,8 @@ paths: - Preview summary: Get Preview Frames From Cache description: Get list of cached preview frames - operationId: get_preview_frames_from_cache_preview__camera_name__start__start_ts__end__end_ts__frames_get + operationId: >- + get_preview_frames_from_cache_preview__camera_name__start__start_ts__end__end_ts__frames_get parameters: - name: camera_name in: path @@ -1177,7 +1119,8 @@ paths: tags: - Export summary: Export Recording - operationId: export_recording_export__camera_name__start__start_time__end__end_time__post + operationId: >- + export_recording_export__camera_name__start__start_time__end__end_time__post parameters: - name: camera_name in: path @@ -1656,6 +1599,30 @@ paths: - type: 'null' default: utc title: Timezone + - name: min_score + in: query + required: false + schema: + anyOf: + - type: number + - type: 'null' + title: Min Score + - name: max_score + in: query + required: false + schema: + anyOf: + - type: number + - type: 'null' + title: Max Score + - name: sort + in: query + required: false + schema: + anyOf: + - type: string + - type: 'null' + title: Sort responses: '200': description: Successful Response @@ -1942,6 +1909,15 @@ paths: schema: type: string title: Event Id + - name: source + in: query + required: false + schema: + anyOf: + - $ref: '#/components/schemas/RegenerateDescriptionEnum' + - type: 'null' + default: thumbnails + title: Source responses: '200': description: Successful Response @@ -2029,12 +2005,12 @@ paths: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' - '{camera_name}': + /{camera_name}: get: tags: - Media summary: Mjpeg Feed - operationId: mjpeg_feed_camera_name__get + operationId: mjpeg_feed__camera_name__get parameters: - name: camera_name in: path @@ -2241,7 +2217,8 @@ paths: tags: - Media summary: Get Snapshot From Recording - operationId: get_snapshot_from_recording__camera_name__recordings__frame_time__snapshot__format__get + operationId: >- + get_snapshot_from_recording__camera_name__recordings__frame_time__snapshot__format__get parameters: - name: camera_name in: path @@ -2363,7 +2340,9 @@ paths: tags: - Media summary: Recordings - description: Return specific camera recordings between the given 'after'/'end' times. If not provided the last hour will be used + description: >- + Return specific camera recordings between the given 'after'/'end' times. + If not provided the last hour will be used operationId: recordings__camera_name__recordings_get parameters: - name: camera_name @@ -2377,14 +2356,14 @@ paths: required: false schema: type: number - default: 1727542549.303557 + default: 1729274204.653048 title: After - name: before in: query required: false schema: type: number - default: 1727546149.303926 + default: 1729277804.653095 title: Before responses: '200': @@ -2423,13 +2402,6 @@ paths: schema: type: number title: End Ts - - name: download - in: query - required: false - schema: - type: boolean - default: false - title: Download responses: '200': description: Successful Response @@ -2800,13 +2772,6 @@ paths: schema: type: string title: Event Id - - name: download - in: query - required: false - schema: - type: boolean - default: false - title: Download responses: '200': description: Successful Response @@ -3121,7 +3086,9 @@ paths: tags: - Media summary: Label Snapshot - description: Returns the snapshot image from the latest event for the given camera and label combo + description: >- + Returns the snapshot image from the latest event for the given camera + and label combo operationId: label_snapshot__camera_name___label__snapshot_jpg_get parameters: - name: camera_name @@ -3193,6 +3160,32 @@ components: required: - password title: AppPutPasswordBody + DayReview: + properties: + day: + type: string + format: date-time + title: Day + reviewed_alert: + type: integer + title: Reviewed Alert + reviewed_detection: + type: integer + title: Reviewed Detection + total_alert: + type: integer + title: Total Alert + total_detection: + type: integer + title: Total Detection + type: object + required: + - day + - reviewed_alert + - reviewed_detection + - total_alert + - total_detection + title: DayReview EventsCreateBody: properties: source_type: @@ -3237,7 +3230,6 @@ components: description: anyOf: - type: string - minLength: 1 - type: 'null' title: The description of the event type: object @@ -3278,6 +3270,19 @@ components: - jpg - jpeg title: Extension + GenericResponse: + properties: + success: + type: boolean + title: Success + message: + type: string + title: Message + type: object + required: + - success + - message + title: GenericResponse HTTPValidationError: properties: detail: @@ -3287,6 +3292,133 @@ components: title: Detail type: object title: HTTPValidationError + Last24HoursReview: + properties: + reviewed_alert: + type: integer + title: Reviewed Alert + reviewed_detection: + type: integer + title: Reviewed Detection + total_alert: + type: integer + title: Total Alert + total_detection: + type: integer + title: Total Detection + type: object + required: + - reviewed_alert + - reviewed_detection + - total_alert + - total_detection + title: Last24HoursReview + RegenerateDescriptionEnum: + type: string + enum: + - thumbnails + - snapshot + title: RegenerateDescriptionEnum + ReviewActivityMotionResponse: + properties: + start_time: + type: integer + title: Start Time + motion: + type: number + title: Motion + camera: + type: string + title: Camera + type: object + required: + - start_time + - motion + - camera + title: ReviewActivityMotionResponse + ReviewDeleteMultipleReviewsBody: + properties: + ids: + items: + type: string + minLength: 1 + type: array + minItems: 1 + title: Ids + type: object + required: + - ids + title: ReviewDeleteMultipleReviewsBody + ReviewSegmentResponse: + properties: + id: + type: string + title: Id + camera: + type: string + title: Camera + start_time: + type: string + format: date-time + title: Start Time + end_time: + type: string + format: date-time + title: End Time + has_been_reviewed: + type: boolean + title: Has Been Reviewed + severity: + $ref: '#/components/schemas/SeverityEnum' + thumb_path: + type: string + title: Thumb Path + data: + title: Data + type: object + required: + - id + - camera + - start_time + - end_time + - has_been_reviewed + - severity + - thumb_path + - data + title: ReviewSegmentResponse + ReviewSetMultipleReviewedBody: + properties: + ids: + items: + type: string + minLength: 1 + type: array + minItems: 1 + title: Ids + type: object + required: + - ids + title: ReviewSetMultipleReviewedBody + ReviewSummaryResponse: + properties: + last24Hours: + $ref: '#/components/schemas/Last24HoursReview' + root: + additionalProperties: + $ref: '#/components/schemas/DayReview' + type: object + title: Root + type: object + required: + - last24Hours + - root + title: ReviewSummaryResponse + SeverityEnum: + type: string + enum: + - alert + - detection + title: SeverityEnum SubmitPlusBody: properties: include_annotation: diff --git a/frigate/api/defs/generic_response.py b/frigate/api/defs/generic_response.py new file mode 100644 index 000000000..dbf9434f9 --- /dev/null +++ b/frigate/api/defs/generic_response.py @@ -0,0 +1,6 @@ +from pydantic import BaseModel + + +class GenericResponse(BaseModel): + success: bool + message: str diff --git a/frigate/api/defs/review_body.py b/frigate/api/defs/review_body.py new file mode 100644 index 000000000..991f190f8 --- /dev/null +++ b/frigate/api/defs/review_body.py @@ -0,0 +1,6 @@ +from pydantic import BaseModel, conlist, constr + + +class ReviewModifyMultipleBody(BaseModel): + # List of string with at least one element and each element with at least one char + ids: conlist(constr(min_length=1), min_length=1) diff --git a/frigate/api/defs/review_query_parameters.py b/frigate/api/defs/review_query_parameters.py index a3f63d292..4361d313c 100644 --- a/frigate/api/defs/review_query_parameters.py +++ b/frigate/api/defs/review_query_parameters.py @@ -1,28 +1,31 @@ -from typing import Optional +from typing import Union from pydantic import BaseModel +from pydantic.json_schema import SkipJsonSchema + +from frigate.review.maintainer import SeverityEnum class ReviewQueryParams(BaseModel): - cameras: Optional[str] = "all" - labels: Optional[str] = "all" - zones: Optional[str] = "all" - reviewed: Optional[int] = 0 - limit: Optional[int] = None - severity: Optional[str] = None - before: Optional[float] = None - after: Optional[float] = None + cameras: str = "all" + labels: str = "all" + zones: str = "all" + reviewed: int = 0 + limit: Union[int, SkipJsonSchema[None]] = None + severity: Union[SeverityEnum, SkipJsonSchema[None]] = None + before: Union[float, SkipJsonSchema[None]] = None + after: Union[float, SkipJsonSchema[None]] = None class ReviewSummaryQueryParams(BaseModel): - cameras: Optional[str] = "all" - labels: Optional[str] = "all" - zones: Optional[str] = "all" - timezone: Optional[str] = "utc" + cameras: str = "all" + labels: str = "all" + zones: str = "all" + timezone: str = "utc" class ReviewActivityMotionQueryParams(BaseModel): - cameras: Optional[str] = "all" - before: Optional[float] = None - after: Optional[float] = None - scale: Optional[int] = 30 + cameras: str = "all" + before: Union[float, SkipJsonSchema[None]] = None + after: Union[float, SkipJsonSchema[None]] = None + scale: int = 30 diff --git a/frigate/api/defs/review_responses.py b/frigate/api/defs/review_responses.py new file mode 100644 index 000000000..39e078b21 --- /dev/null +++ b/frigate/api/defs/review_responses.py @@ -0,0 +1,43 @@ +from datetime import datetime +from typing import Dict + +from pydantic import BaseModel, Json + +from frigate.review.maintainer import SeverityEnum + + +class ReviewSegmentResponse(BaseModel): + id: str + camera: str + start_time: datetime + end_time: datetime + has_been_reviewed: bool + severity: SeverityEnum + thumb_path: str + data: Json + + +class Last24HoursReview(BaseModel): + reviewed_alert: int + reviewed_detection: int + total_alert: int + total_detection: int + + +class DayReview(BaseModel): + day: datetime + reviewed_alert: int + reviewed_detection: int + total_alert: int + total_detection: int + + +class ReviewSummaryResponse(BaseModel): + last24Hours: Last24HoursReview + root: Dict[str, DayReview] + + +class ReviewActivityMotionResponse(BaseModel): + start_time: int + motion: float + camera: str diff --git a/frigate/api/fastapi_app.py b/frigate/api/fastapi_app.py index 3980e0b40..e3542458e 100644 --- a/frigate/api/fastapi_app.py +++ b/frigate/api/fastapi_app.py @@ -82,6 +82,10 @@ def create_fastapi_app( database.close() return response + @app.on_event("startup") + async def startup(): + logger.info("FastAPI started") + # Rate limiter (used for login endpoint) auth.rateLimiter.set_limit(frigate_config.auth.failed_login_rate_limit or "") app.state.limiter = limiter diff --git a/frigate/api/review.py b/frigate/api/review.py index 7c05386ef..21b468640 100644 --- a/frigate/api/review.py +++ b/frigate/api/review.py @@ -12,11 +12,18 @@ from fastapi.responses import JSONResponse from peewee import Case, DoesNotExist, fn, operator from playhouse.shortcuts import model_to_dict +from frigate.api.defs.generic_response import GenericResponse +from frigate.api.defs.review_body import ReviewModifyMultipleBody from frigate.api.defs.review_query_parameters import ( ReviewActivityMotionQueryParams, ReviewQueryParams, ReviewSummaryQueryParams, ) +from frigate.api.defs.review_responses import ( + ReviewActivityMotionResponse, + ReviewSegmentResponse, + ReviewSummaryResponse, +) from frigate.api.defs.tags import Tags from frigate.models import Recordings, ReviewSegment from frigate.util.builtin import get_tz_modifiers @@ -26,7 +33,7 @@ logger = logging.getLogger(__name__) router = APIRouter(tags=[Tags.review]) -@router.get("/review") +@router.get("/review", response_model=list[ReviewSegmentResponse]) def review(params: ReviewQueryParams = Depends()): cameras = params.cameras labels = params.labels @@ -102,7 +109,7 @@ def review(params: ReviewQueryParams = Depends()): return JSONResponse(content=[r for r in review]) -@router.get("/review/summary") +@router.get("/review/summary", response_model=ReviewSummaryResponse) def review_summary(params: ReviewSummaryQueryParams = Depends()): hour_modifier, minute_modifier, seconds_offset = get_tz_modifiers(params.timezone) day_ago = (datetime.datetime.now() - datetime.timedelta(hours=24)).timestamp() @@ -173,18 +180,6 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): 0, ) ).alias("reviewed_detection"), - fn.SUM( - Case( - None, - [ - ( - (ReviewSegment.severity == "significant_motion"), - ReviewSegment.has_been_reviewed, - ) - ], - 0, - ) - ).alias("reviewed_motion"), fn.SUM( Case( None, @@ -209,18 +204,6 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): 0, ) ).alias("total_detection"), - fn.SUM( - Case( - None, - [ - ( - (ReviewSegment.severity == "significant_motion"), - 1, - ) - ], - 0, - ) - ).alias("total_motion"), ) .where(reduce(operator.and_, clauses)) .dicts() @@ -282,18 +265,6 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): 0, ) ).alias("reviewed_detection"), - fn.SUM( - Case( - None, - [ - ( - (ReviewSegment.severity == "significant_motion"), - ReviewSegment.has_been_reviewed, - ) - ], - 0, - ) - ).alias("reviewed_motion"), fn.SUM( Case( None, @@ -318,18 +289,6 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): 0, ) ).alias("total_detection"), - fn.SUM( - Case( - None, - [ - ( - (ReviewSegment.severity == "significant_motion"), - 1, - ) - ], - 0, - ) - ).alias("total_motion"), ) .where(reduce(operator.and_, clauses)) .group_by( @@ -348,19 +307,10 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): return JSONResponse(content=data) -@router.post("/reviews/viewed") -def set_multiple_reviewed(body: dict = None): - json: dict[str, any] = body or {} - list_of_ids = json.get("ids", "") - - if not list_of_ids or len(list_of_ids) == 0: - return JSONResponse( - context=({"success": False, "message": "Not a valid list of ids"}), - status_code=404, - ) - +@router.post("/reviews/viewed", response_model=GenericResponse) +def set_multiple_reviewed(body: ReviewModifyMultipleBody): ReviewSegment.update(has_been_reviewed=True).where( - ReviewSegment.id << list_of_ids + ReviewSegment.id << body.ids ).execute() return JSONResponse( @@ -369,17 +319,9 @@ def set_multiple_reviewed(body: dict = None): ) -@router.post("/reviews/delete") -def delete_reviews(body: dict = None): - json: dict[str, any] = body or {} - list_of_ids = json.get("ids", "") - - if not list_of_ids or len(list_of_ids) == 0: - return JSONResponse( - content=({"success": False, "message": "Not a valid list of ids"}), - status_code=404, - ) - +@router.post("/reviews/delete", response_model=GenericResponse) +def delete_reviews(body: ReviewModifyMultipleBody): + list_of_ids = body.ids reviews = ( ReviewSegment.select( ReviewSegment.camera, @@ -424,7 +366,9 @@ def delete_reviews(body: dict = None): ) -@router.get("/review/activity/motion") +@router.get( + "/review/activity/motion", response_model=list[ReviewActivityMotionResponse] +) def motion_activity(params: ReviewActivityMotionQueryParams = Depends()): """Get motion and audio activity.""" cameras = params.cameras @@ -498,98 +442,44 @@ def motion_activity(params: ReviewActivityMotionQueryParams = Depends()): return JSONResponse(content=normalized) -@router.get("/review/activity/audio") -def audio_activity(params: ReviewActivityMotionQueryParams = Depends()): - """Get motion and audio activity.""" - cameras = params.cameras - before = params.before or datetime.datetime.now().timestamp() - after = ( - params.after - or (datetime.datetime.now() - datetime.timedelta(hours=1)).timestamp() - ) - # get scale in seconds - scale = params.scale - - clauses = [(Recordings.start_time > after) & (Recordings.end_time < before)] - - if cameras != "all": - camera_list = cameras.split(",") - clauses.append((Recordings.camera << camera_list)) - - all_recordings: list[Recordings] = ( - Recordings.select( - Recordings.start_time, - Recordings.duration, - Recordings.objects, - Recordings.dBFS, - ) - .where(reduce(operator.and_, clauses)) - .order_by(Recordings.start_time.asc()) - .iterator() - ) - - # format is: { timestamp: segment_start_ts, motion: [0-100], audio: [0 - -100] } - # periods where active objects / audio was detected will cause audio to be scaled down - data: list[dict[str, float]] = [] - - for rec in all_recordings: - data.append( - { - "start_time": rec.start_time, - "audio": rec.dBFS if rec.objects == 0 else 0, - } - ) - - # resample data using pandas to get activity on scaled basis - df = pd.DataFrame(data, columns=["start_time", "audio"]) - df = df.astype(dtype={"audio": "float16"}) - - # set date as datetime index - df["start_time"] = pd.to_datetime(df["start_time"], unit="s") - df.set_index(["start_time"], inplace=True) - - # normalize data - df = df.resample(f"{scale}S").mean().fillna(0.0) - df["audio"] = ( - (df["audio"] - df["audio"].max()) - / (df["audio"].min() - df["audio"].max()) - * -100 - ) - - # change types for output - df.index = df.index.astype(int) // (10**9) - normalized = df.reset_index().to_dict("records") - return JSONResponse(content=normalized) - - -@router.get("/review/event/{event_id}") +@router.get("/review/event/{event_id}", response_model=ReviewSegmentResponse) def get_review_from_event(event_id: str): try: - return model_to_dict( - ReviewSegment.get( - ReviewSegment.data["detections"].cast("text") % f'*"{event_id}"*' + return JSONResponse( + model_to_dict( + ReviewSegment.get( + ReviewSegment.data["detections"].cast("text") % f'*"{event_id}"*' + ) ) ) except DoesNotExist: - return "Review item not found", 404 + return JSONResponse( + content={"success": False, "message": "Review item not found"}, + status_code=404, + ) -@router.get("/review/{event_id}") -def get_review(event_id: str): +@router.get("/review/{review_id}", response_model=ReviewSegmentResponse) +def get_review(review_id: str): try: - return model_to_dict(ReviewSegment.get(ReviewSegment.id == event_id)) + return JSONResponse( + content=model_to_dict(ReviewSegment.get(ReviewSegment.id == review_id)) + ) except DoesNotExist: - return "Review item not found", 404 + return JSONResponse( + content={"success": False, "message": "Review item not found"}, + status_code=404, + ) -@router.delete("/review/{event_id}/viewed") -def set_not_reviewed(event_id: str): +@router.delete("/review/{review_id}/viewed", response_model=GenericResponse) +def set_not_reviewed(review_id: str): try: - review: ReviewSegment = ReviewSegment.get(ReviewSegment.id == event_id) + review: ReviewSegment = ReviewSegment.get(ReviewSegment.id == review_id) except DoesNotExist: return JSONResponse( content=( - {"success": False, "message": "Review " + event_id + " not found"} + {"success": False, "message": "Review " + review_id + " not found"} ), status_code=404, ) @@ -598,6 +488,8 @@ def set_not_reviewed(event_id: str): review.save() return JSONResponse( - content=({"success": True, "message": "Reviewed " + event_id + " not viewed"}), + content=( + {"success": True, "message": "Set Review " + review_id + " as not viewed"} + ), status_code=200, ) diff --git a/frigate/models.py b/frigate/models.py index c73033b3e..62bbf0bd3 100644 --- a/frigate/models.py +++ b/frigate/models.py @@ -93,7 +93,7 @@ class ReviewSegment(Model): # type: ignore[misc] start_time = DateTimeField() end_time = DateTimeField() has_been_reviewed = BooleanField(default=False) - severity = CharField(max_length=30) # alert, detection, significant_motion + severity = CharField(max_length=30) # alert, detection thumb_path = CharField(unique=True) data = JSONField() # additional data about detection like list of labels, zone, areas of significant motion diff --git a/web/src/pages/Events.tsx b/web/src/pages/Events.tsx index fa582ca3b..8c6f3cd38 100644 --- a/web/src/pages/Events.tsx +++ b/web/src/pages/Events.tsx @@ -407,10 +407,6 @@ export default function Events() { review.severity == "detection" ? item.reviewed_detection + 1 : item.reviewed_detection, - reviewed_motion: - review.severity == "significant_motion" - ? item.reviewed_motion + 1 - : item.reviewed_motion, }, }; }, diff --git a/web/src/types/review.ts b/web/src/types/review.ts index d1d03e637..73b8fed14 100644 --- a/web/src/types/review.ts +++ b/web/src/types/review.ts @@ -42,10 +42,8 @@ type ReviewSummaryDay = { day: string; reviewed_alert: number; reviewed_detection: number; - reviewed_motion: number; total_alert: number; total_detection: number; - total_motion: number; }; export type ReviewSummary = { diff --git a/web/src/views/events/EventView.tsx b/web/src/views/events/EventView.tsx index 615bfc535..d7997bf62 100644 --- a/web/src/views/events/EventView.tsx +++ b/web/src/views/events/EventView.tsx @@ -117,13 +117,11 @@ export default function EventView({ return { alert: summary.total_alert ?? 0, detection: summary.total_detection ?? 0, - significant_motion: summary.total_motion ?? 0, }; } else { return { alert: summary.total_alert - summary.reviewed_alert, detection: summary.total_detection - summary.reviewed_detection, - significant_motion: summary.total_motion - summary.reviewed_motion, }; } }, [filter, showReviewed, reviewSummary]); From 18824830fde88a50873cba5d1f0dfeaeba970e2b Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 23 Oct 2024 07:36:52 -0600 Subject: [PATCH 090/479] Export preview via api (#14535) * Break out recording to separate function * Implement preview exporting * Formatting --- frigate/api/export.py | 67 ++++++++++++++------ frigate/record/export.py | 128 +++++++++++++++++++++++++++++++-------- 2 files changed, 152 insertions(+), 43 deletions(-) diff --git a/frigate/api/export.py b/frigate/api/export.py index d697709c5..c3299a4b0 100644 --- a/frigate/api/export.py +++ b/frigate/api/export.py @@ -13,8 +13,12 @@ from peewee import DoesNotExist from frigate.api.defs.tags import Tags from frigate.const import EXPORT_DIR -from frigate.models import Export, Recordings -from frigate.record.export import PlaybackFactorEnum, RecordingExporter +from frigate.models import Export, Previews, Recordings +from frigate.record.export import ( + PlaybackFactorEnum, + PlaybackSourceEnum, + RecordingExporter, +) logger = logging.getLogger(__name__) @@ -45,6 +49,7 @@ def export_recording( json: dict[str, any] = body or {} playback_factor = json.get("playback", "realtime") + playback_source = json.get("source", "recordings") friendly_name: Optional[str] = json.get("name") if len(friendly_name or "") > 256: @@ -55,25 +60,48 @@ def export_recording( existing_image = json.get("image_path") - recordings_count = ( - Recordings.select() - .where( - Recordings.start_time.between(start_time, end_time) - | Recordings.end_time.between(start_time, end_time) - | ((start_time > Recordings.start_time) & (end_time < Recordings.end_time)) + if playback_source == "recordings": + recordings_count = ( + Recordings.select() + .where( + Recordings.start_time.between(start_time, end_time) + | Recordings.end_time.between(start_time, end_time) + | ( + (start_time > Recordings.start_time) + & (end_time < Recordings.end_time) + ) + ) + .where(Recordings.camera == camera_name) + .count() ) - .where(Recordings.camera == camera_name) - .count() - ) - if recordings_count <= 0: - return JSONResponse( - content=( - {"success": False, "message": "No recordings found for time range"} - ), - status_code=400, + if recordings_count <= 0: + return JSONResponse( + content=( + {"success": False, "message": "No recordings found for time range"} + ), + status_code=400, + ) + else: + previews_count = ( + Previews.select() + .where( + Previews.start_time.between(start_time, end_time) + | Previews.end_time.between(start_time, end_time) + | ((start_time > Previews.start_time) & (end_time < Previews.end_time)) + ) + .where(Previews.camera == camera_name) + .count() ) + if previews_count <= 0: + return JSONResponse( + content=( + {"success": False, "message": "No previews found for time range"} + ), + status_code=400, + ) + export_id = f"{camera_name}_{''.join(random.choices(string.ascii_lowercase + string.digits, k=6))}" exporter = RecordingExporter( request.app.frigate_config, @@ -88,6 +116,11 @@ def export_recording( if playback_factor in PlaybackFactorEnum.__members__.values() else PlaybackFactorEnum.realtime ), + ( + PlaybackSourceEnum[playback_source] + if playback_source in PlaybackSourceEnum.__members__.values() + else PlaybackSourceEnum.recordings + ), ) exporter.start() return JSONResponse( diff --git a/frigate/record/export.py b/frigate/record/export.py index 395da79ea..325f08419 100644 --- a/frigate/record/export.py +++ b/frigate/record/export.py @@ -43,6 +43,11 @@ class PlaybackFactorEnum(str, Enum): timelapse_25x = "timelapse_25x" +class PlaybackSourceEnum(str, Enum): + recordings = "recordings" + preview = "preview" + + class RecordingExporter(threading.Thread): """Exports a specific set of recordings for a camera to storage as a single file.""" @@ -56,6 +61,7 @@ class RecordingExporter(threading.Thread): start_time: int, end_time: int, playback_factor: PlaybackFactorEnum, + playback_source: PlaybackSourceEnum, ) -> None: super().__init__() self.config = config @@ -66,6 +72,7 @@ class RecordingExporter(threading.Thread): self.start_time = start_time self.end_time = end_time self.playback_factor = playback_factor + self.playback_source = playback_source # ensure export thumb dir Path(os.path.join(CLIPS_DIR, "export")).mkdir(exist_ok=True) @@ -170,30 +177,7 @@ class RecordingExporter(threading.Thread): return thumb_path - def run(self) -> None: - logger.debug( - f"Beginning export for {self.camera} from {self.start_time} to {self.end_time}" - ) - export_name = ( - self.user_provided_name - or f"{self.camera.replace('_', ' ')} {self.get_datetime_from_timestamp(self.start_time)} {self.get_datetime_from_timestamp(self.end_time)}" - ) - video_path = f"{EXPORT_DIR}/{self.export_id}.mp4" - - thumb_path = self.save_thumbnail(self.export_id) - - Export.insert( - { - Export.id: self.export_id, - Export.camera: self.camera, - Export.name: export_name, - Export.date: self.start_time, - Export.video_path: video_path, - Export.thumb_path: thumb_path, - Export.in_progress: True, - } - ).execute() - + def get_record_export_command(self, video_path: str) -> list[str]: if (self.end_time - self.start_time) <= MAX_PLAYLIST_SECONDS: playlist_lines = f"http://127.0.0.1:5000/vod/{self.camera}/start/{self.start_time}/end/{self.end_time}/index.m3u8" ffmpeg_input = ( @@ -204,7 +188,10 @@ class RecordingExporter(threading.Thread): # get full set of recordings export_recordings = ( - Recordings.select() + Recordings.select( + Recordings.start_time, + Recordings.end_time, + ) .where( Recordings.start_time.between(self.start_time, self.end_time) | Recordings.end_time.between(self.start_time, self.end_time) @@ -229,6 +216,65 @@ class RecordingExporter(threading.Thread): ffmpeg_input = "-y -protocol_whitelist pipe,file,http,tcp -f concat -safe 0 -i /dev/stdin" + if self.playback_factor == PlaybackFactorEnum.realtime: + ffmpeg_cmd = ( + f"{self.config.ffmpeg.ffmpeg_path} -hide_banner {ffmpeg_input} -c copy -movflags +faststart {video_path}" + ).split(" ") + elif self.playback_factor == PlaybackFactorEnum.timelapse_25x: + ffmpeg_cmd = ( + parse_preset_hardware_acceleration_encode( + self.config.ffmpeg.ffmpeg_path, + self.config.ffmpeg.hwaccel_args, + f"-an {ffmpeg_input}", + f"{self.config.cameras[self.camera].record.export.timelapse_args} -movflags +faststart {video_path}", + EncodeTypeEnum.timelapse, + ) + ).split(" ") + + return ffmpeg_cmd, playlist_lines + + def get_preview_export_command(self, video_path: str) -> list[str]: + playlist_lines = [] + + # get full set of previews + export_previews = ( + Previews.select( + Previews.path, + Previews.start_time, + Previews.end_time, + ) + .where( + Previews.start_time.between(self.start_time, self.end_time) + | Previews.end_time.between(self.start_time, self.end_time) + | ( + (self.start_time > Previews.start_time) + & (self.end_time < Previews.end_time) + ) + ) + .where(Previews.camera == self.camera) + .order_by(Previews.start_time.asc()) + .namedtuples() + .iterator() + ) + + preview: Previews + for preview in export_previews: + playlist_lines.append(f"file '{preview.path}'") + + if preview.start_time < self.start_time: + playlist_lines.append( + f"inpoint {int(self.start_time - preview.start_time)}" + ) + + if preview.end_time > self.end_time: + playlist_lines.append( + f"outpoint {int(preview.end_time - self.end_time)}" + ) + + ffmpeg_input = ( + "-y -protocol_whitelist pipe,file,tcp -f concat -safe 0 -i /dev/stdin" + ) + if self.playback_factor == PlaybackFactorEnum.realtime: ffmpeg_cmd = ( f"{self.config.ffmpeg.ffmpeg_path} -hide_banner {ffmpeg_input} -c copy -movflags +faststart {video_path}" @@ -244,6 +290,36 @@ class RecordingExporter(threading.Thread): ) ).split(" ") + return ffmpeg_cmd, playlist_lines + + def run(self) -> None: + logger.debug( + f"Beginning export for {self.camera} from {self.start_time} to {self.end_time}" + ) + export_name = ( + self.user_provided_name + or f"{self.camera.replace('_', ' ')} {self.get_datetime_from_timestamp(self.start_time)} {self.get_datetime_from_timestamp(self.end_time)}" + ) + video_path = f"{EXPORT_DIR}/{self.export_id}.mp4" + thumb_path = self.save_thumbnail(self.export_id) + + Export.insert( + { + Export.id: self.export_id, + Export.camera: self.camera, + Export.name: export_name, + Export.date: self.start_time, + Export.video_path: video_path, + Export.thumb_path: thumb_path, + Export.in_progress: True, + } + ).execute() + + if self.playback_source == PlaybackSourceEnum.recordings: + ffmpeg_cmd, playlist_lines = self.get_record_export_command(video_path) + else: + ffmpeg_cmd, playlist_lines = self.get_preview_export_command(video_path) + p = sp.run( ffmpeg_cmd, input="\n".join(playlist_lines), @@ -254,7 +330,7 @@ class RecordingExporter(threading.Thread): if p.returncode != 0: logger.error( - f"Failed to export recording for command {' '.join(ffmpeg_cmd)}" + f"Failed to export {self.playback_source.value} for command {' '.join(ffmpeg_cmd)}" ) logger.error(p.stderr) Path(video_path).unlink(missing_ok=True) From 8fefded8dcad357be9c3dc0c56aa92b3bc056eda Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 23 Oct 2024 10:31:20 -0500 Subject: [PATCH 091/479] Fix score in search details dialog for old events (#14541) --- web/src/components/overlay/detail/SearchDetailDialog.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index f443c9d44..78796036f 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -287,7 +287,7 @@ function ObjectDetailsTab({ return 0; } - const value = search.data.top_score; + const value = search.data.top_score ?? search.top_score ?? 0; return Math.round(value * 100); }, [search]); From f9b246dbd0e951cfac8d74bbf604a86f05b0cbed Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 24 Oct 2024 07:48:14 -0600 Subject: [PATCH 092/479] Deps updates (#14556) * Update nvidia deps * Update python deps * Update web deps --- docker/main/requirements-wheels.txt | 8 +- docker/tensorrt/requirements-amd64.txt | 4 +- web/package-lock.json | 692 ++++++++++++++++--------- web/package.json | 48 +- 4 files changed, 478 insertions(+), 274 deletions(-) diff --git a/docker/main/requirements-wheels.txt b/docker/main/requirements-wheels.txt index 11ad94f3f..795456588 100644 --- a/docker/main/requirements-wheels.txt +++ b/docker/main/requirements-wheels.txt @@ -1,9 +1,9 @@ click == 8.1.* # FastAPI starlette-context == 0.3.6 -fastapi == 0.115.0 +fastapi == 0.115.* uvicorn == 0.30.* -slowapi == 0.1.9 +slowapi == 0.1.* imutils == 0.5.* joserfc == 1.0.* pathvalidate == 3.2.* @@ -16,10 +16,10 @@ paho-mqtt == 2.1.* pandas == 2.2.* peewee == 3.17.* peewee_migrate == 1.13.* -psutil == 5.9.* +psutil == 6.1.* pydantic == 2.8.* git+https://github.com/fbcotter/py3nvml#egg=py3nvml -pytz == 2024.1 +pytz == 2024.* pyzmq == 26.2.* ruamel.yaml == 0.18.* tzlocal == 5.2 diff --git a/docker/tensorrt/requirements-amd64.txt b/docker/tensorrt/requirements-amd64.txt index b5ad4fcbd..c73797516 100644 --- a/docker/tensorrt/requirements-amd64.txt +++ b/docker/tensorrt/requirements-amd64.txt @@ -9,6 +9,6 @@ nvidia-cuda-runtime-cu11 == 11.8.*; platform_machine == 'x86_64' nvidia-cublas-cu11 == 11.11.3.6; platform_machine == 'x86_64' nvidia-cudnn-cu11 == 8.6.0.*; platform_machine == 'x86_64' nvidia-cufft-cu11==10.*; platform_machine == 'x86_64' -onnx==1.14.0; platform_machine == 'x86_64' -onnxruntime-gpu==1.17.*; platform_machine == 'x86_64' +onnx==1.16.*; platform_machine == 'x86_64' +onnxruntime-gpu==1.19.*; platform_machine == 'x86_64' protobuf==3.20.3; platform_machine == 'x86_64' diff --git a/web/package-lock.json b/web/package-lock.json index 77f0bfc0f..a0971c361 100644 --- a/web/package-lock.json +++ b/web/package-lock.json @@ -10,28 +10,28 @@ "dependencies": { "@cycjimmy/jsmpeg-player": "^6.1.1", "@hookform/resolvers": "^3.9.0", - "@radix-ui/react-alert-dialog": "^1.1.1", + "@radix-ui/react-alert-dialog": "^1.1.2", "@radix-ui/react-aspect-ratio": "^1.1.0", - "@radix-ui/react-checkbox": "^1.1.1", - "@radix-ui/react-context-menu": "^2.2.1", - "@radix-ui/react-dialog": "^1.1.1", - "@radix-ui/react-dropdown-menu": "^2.1.1", - "@radix-ui/react-hover-card": "^1.1.1", + "@radix-ui/react-checkbox": "^1.1.2", + "@radix-ui/react-context-menu": "^2.2.2", + "@radix-ui/react-dialog": "^1.1.2", + "@radix-ui/react-dropdown-menu": "^2.1.2", + "@radix-ui/react-hover-card": "^1.1.2", "@radix-ui/react-label": "^2.1.0", - "@radix-ui/react-popover": "^1.1.1", - "@radix-ui/react-radio-group": "^1.2.0", - "@radix-ui/react-scroll-area": "^1.1.0", - "@radix-ui/react-select": "^2.1.1", + "@radix-ui/react-popover": "^1.1.2", + "@radix-ui/react-radio-group": "^1.2.1", + "@radix-ui/react-scroll-area": "^1.2.0", + "@radix-ui/react-select": "^2.1.2", "@radix-ui/react-separator": "^1.1.0", - "@radix-ui/react-slider": "^1.2.0", + "@radix-ui/react-slider": "^1.2.1", "@radix-ui/react-slot": "^1.1.0", - "@radix-ui/react-switch": "^1.1.0", - "@radix-ui/react-tabs": "^1.1.0", + "@radix-ui/react-switch": "^1.1.1", + "@radix-ui/react-tabs": "^1.1.1", "@radix-ui/react-toggle": "^1.1.0", "@radix-ui/react-toggle-group": "^1.1.0", - "@radix-ui/react-tooltip": "^1.1.2", + "@radix-ui/react-tooltip": "^1.1.3", "apexcharts": "^3.52.0", - "axios": "^1.7.3", + "axios": "^1.7.7", "class-variance-authority": "^0.7.0", "clsx": "^2.1.1", "cmdk": "^1.0.0", @@ -39,10 +39,10 @@ "date-fns": "^3.6.0", "embla-carousel-react": "^8.2.0", "framer-motion": "^11.5.4", - "hls.js": "^1.5.14", + "hls.js": "^1.5.17", "idb-keyval": "^6.2.1", "immer": "^10.1.1", - "konva": "^9.3.14", + "konva": "^9.3.16", "lodash": "^4.17.21", "lucide-react": "^0.407.0", "monaco-yaml": "^5.2.2", @@ -59,7 +59,7 @@ "react-konva": "^18.2.10", "react-router-dom": "^6.26.0", "react-swipeable": "^7.0.1", - "react-tracked": "^2.0.0", + "react-tracked": "^2.0.1", "react-transition-group": "^4.4.5", "react-use-websocket": "^4.8.1", "react-zoom-pan-pinch": "3.4.4", @@ -77,9 +77,9 @@ "zod": "^3.23.8" }, "devDependencies": { - "@tailwindcss/forms": "^0.5.7", - "@testing-library/jest-dom": "^6.4.6", - "@types/lodash": "^4.17.7", + "@tailwindcss/forms": "^0.5.9", + "@testing-library/jest-dom": "^6.6.2", + "@types/lodash": "^4.17.12", "@types/node": "^20.14.10", "@types/react": "^18.3.2", "@types/react-dom": "^18.3.0", @@ -89,7 +89,7 @@ "@types/strftime": "^0.9.8", "@typescript-eslint/eslint-plugin": "^7.5.0", "@typescript-eslint/parser": "^7.5.0", - "@vitejs/plugin-react-swc": "^3.6.0", + "@vitejs/plugin-react-swc": "^3.7.1", "@vitest/coverage-v8": "^2.0.5", "autoprefixer": "^10.4.20", "eslint": "^8.57.0", @@ -103,8 +103,8 @@ "jest-websocket-mock": "^2.5.0", "jsdom": "^24.1.1", "msw": "^2.3.5", - "postcss": "^8.4.39", - "prettier": "^3.3.2", + "postcss": "^8.4.47", + "prettier": "^3.3.3", "prettier-plugin-tailwindcss": "^0.6.5", "tailwindcss": "^3.4.9", "typescript": "^5.5.4", @@ -1116,15 +1116,15 @@ "license": "MIT" }, "node_modules/@radix-ui/react-alert-dialog": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-alert-dialog/-/react-alert-dialog-1.1.1.tgz", - "integrity": "sha512-wmCoJwj7byuVuiLKqDLlX7ClSUU0vd9sdCeM+2Ls+uf13+cpSJoMgwysHq1SGVVkJj5Xn0XWi1NoRCdkMpr6Mw==", + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-alert-dialog/-/react-alert-dialog-1.1.2.tgz", + "integrity": "sha512-eGSlLzPhKO+TErxkiGcCZGuvbVMnLA1MTnyBksGOeGRGkxHiiJUujsjmNTdWTm4iHVSRaUao9/4Ur671auMghQ==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-dialog": "1.1.1", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-dialog": "1.1.2", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-slot": "1.1.0" }, @@ -1143,6 +1143,21 @@ } } }, + "node_modules/@radix-ui/react-alert-dialog/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-arrow": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-arrow/-/react-arrow-1.1.0.tgz", @@ -1190,14 +1205,15 @@ } }, "node_modules/@radix-ui/react-checkbox": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-checkbox/-/react-checkbox-1.1.1.tgz", - "integrity": "sha512-0i/EKJ222Afa1FE0C6pNJxDq1itzcl3HChE9DwskA4th4KRse8ojx8a1nVcOjwJdbpDLcz7uol77yYnQNMHdKw==", + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-checkbox/-/react-checkbox-1.1.2.tgz", + "integrity": "sha512-/i0fl686zaJbDQLNKrkCbMyDm6FQMt4jg323k7HuqitoANm9sE23Ql8yOK3Wusk34HSLKDChhMux05FnP6KUkw==", + "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-controllable-state": "1.1.0", "@radix-ui/react-use-previous": "1.1.0", @@ -1218,6 +1234,21 @@ } } }, + "node_modules/@radix-ui/react-checkbox/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-collection": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-collection/-/react-collection-1.1.0.tgz", @@ -1275,14 +1306,14 @@ } }, "node_modules/@radix-ui/react-context-menu": { - "version": "2.2.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-context-menu/-/react-context-menu-2.2.1.tgz", - "integrity": "sha512-wvMKKIeb3eOrkJ96s722vcidZ+2ZNfcYZWBPRHIB1VWrF+fiF851Io6LX0kmK5wTDQFKdulCCKJk2c3SBaQHvA==", + "version": "2.2.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context-menu/-/react-context-menu-2.2.2.tgz", + "integrity": "sha512-99EatSTpW+hRYHt7m8wdDlLtkmTovEe8Z/hnxUPV+SKuuNL5HWNhQI4QSdjZqNSgXHay2z4M3Dym73j9p2Gx5Q==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-menu": "2.1.1", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-menu": "2.1.2", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-callback-ref": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0" @@ -1302,26 +1333,41 @@ } } }, - "node_modules/@radix-ui/react-dialog": { + "node_modules/@radix-ui/react-context-menu/node_modules/@radix-ui/react-context": { "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-dialog/-/react-dialog-1.1.1.tgz", - "integrity": "sha512-zysS+iU4YP3STKNS6USvFVqI4qqx8EpiwmT5TuCApVEBca+eRCbONi4EgzfNSuVnOXvC5UPHHMjs8RXO6DH9Bg==", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-dialog": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dialog/-/react-dialog-1.1.2.tgz", + "integrity": "sha512-Yj4dZtqa2o+kG61fzB0H2qUvmwBA2oyQroGLyNtBj1beo1khoQ3q1a2AO8rrQYjd8256CO9+N8L9tvsS+bnIyA==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", - "@radix-ui/react-focus-guards": "1.1.0", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-dismissable-layer": "1.1.1", + "@radix-ui/react-focus-guards": "1.1.1", "@radix-ui/react-focus-scope": "1.1.0", "@radix-ui/react-id": "1.1.0", - "@radix-ui/react-portal": "1.1.1", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-portal": "1.1.2", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-slot": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0", "aria-hidden": "^1.1.1", - "react-remove-scroll": "2.5.7" + "react-remove-scroll": "2.6.0" }, "peerDependencies": { "@types/react": "*", @@ -1338,6 +1384,21 @@ } } }, + "node_modules/@radix-ui/react-dialog/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-direction": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-direction/-/react-direction-1.1.0.tgz", @@ -1354,9 +1415,9 @@ } }, "node_modules/@radix-ui/react-dismissable-layer": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-dismissable-layer/-/react-dismissable-layer-1.1.0.tgz", - "integrity": "sha512-/UovfmmXGptwGcBQawLzvn2jOfM0t4z3/uKffoBlj724+n3FvBbZ7M0aaBOmkp6pqFYpO4yx8tSVJjx3Fl2jig==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dismissable-layer/-/react-dismissable-layer-1.1.1.tgz", + "integrity": "sha512-QSxg29lfr/xcev6kSz7MAlmDnzbP1eI/Dwn3Tp1ip0KT5CUELsxkekFEMVBEoykI3oV39hKT4TKZzBNMbcTZYQ==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", @@ -1381,16 +1442,16 @@ } }, "node_modules/@radix-ui/react-dropdown-menu": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-dropdown-menu/-/react-dropdown-menu-2.1.1.tgz", - "integrity": "sha512-y8E+x9fBq9qvteD2Zwa4397pUVhYsh9iq44b5RD5qu1GMJWBCBuVg1hMyItbc6+zH00TxGRqd9Iot4wzf3OoBQ==", + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dropdown-menu/-/react-dropdown-menu-2.1.2.tgz", + "integrity": "sha512-GVZMR+eqK8/Kes0a36Qrv+i20bAPXSn8rCBTHx30w+3ECnR5o3xixAlqcVaYvLeyKUsm0aqyhWfmUcqufM8nYA==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-id": "1.1.0", - "@radix-ui/react-menu": "2.1.1", + "@radix-ui/react-menu": "2.1.2", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-controllable-state": "1.1.0" }, @@ -1409,10 +1470,25 @@ } } }, + "node_modules/@radix-ui/react-dropdown-menu/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-focus-guards": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-guards/-/react-focus-guards-1.1.0.tgz", - "integrity": "sha512-w6XZNUPVv6xCpZUqb/yN9DL6auvpGX3C/ee6Hdi16v2UUy25HV2Q5bcflsiDyT/g5RwbPQ/GIT1vLkeRb+ITBw==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-guards/-/react-focus-guards-1.1.1.tgz", + "integrity": "sha512-pSIwfrT1a6sIoDASCSpFwOasEwKTZWDw/iBdtnqKO7v6FeOzYJ7U53cPzYFVR3geGGXgVHaH+CdngrrAzqUGxg==", "license": "MIT", "peerDependencies": { "@types/react": "*", @@ -1450,18 +1526,18 @@ } }, "node_modules/@radix-ui/react-hover-card": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-hover-card/-/react-hover-card-1.1.1.tgz", - "integrity": "sha512-IwzAOP97hQpDADYVKrEEHUH/b2LA+9MgB0LgdmnbFO2u/3M5hmEofjjr2M6CyzUblaAqJdFm6B7oFtU72DPXrA==", + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-hover-card/-/react-hover-card-1.1.2.tgz", + "integrity": "sha512-Y5w0qGhysvmqsIy6nQxaPa6mXNKznfoGjOfBgzOjocLxr2XlSjqBMYQQL+FfyogsMuX+m8cZyQGYhJxvxUzO4w==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-dismissable-layer": "1.1.1", "@radix-ui/react-popper": "1.2.0", - "@radix-ui/react-portal": "1.1.1", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-portal": "1.1.2", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-controllable-state": "1.1.0" }, @@ -1480,6 +1556,21 @@ } } }, + "node_modules/@radix-ui/react-hover-card/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-id": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-id/-/react-id-1.1.0.tgz", @@ -1522,29 +1613,29 @@ } }, "node_modules/@radix-ui/react-menu": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-menu/-/react-menu-2.1.1.tgz", - "integrity": "sha512-oa3mXRRVjHi6DZu/ghuzdylyjaMXLymx83irM7hTxutQbD+7IhPKdMdRHD26Rm+kHRrWcrUkkRPv5pd47a2xFQ==", + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-menu/-/react-menu-2.1.2.tgz", + "integrity": "sha512-lZ0R4qR2Al6fZ4yCCZzu/ReTFrylHFxIqy7OezIpWF4bL0o9biKo0pFIvkaew3TyZ9Fy5gYVrR5zCGZBVbO1zg==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-collection": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", - "@radix-ui/react-focus-guards": "1.1.0", + "@radix-ui/react-dismissable-layer": "1.1.1", + "@radix-ui/react-focus-guards": "1.1.1", "@radix-ui/react-focus-scope": "1.1.0", "@radix-ui/react-id": "1.1.0", "@radix-ui/react-popper": "1.2.0", - "@radix-ui/react-portal": "1.1.1", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-portal": "1.1.2", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-roving-focus": "1.1.0", "@radix-ui/react-slot": "1.1.0", "@radix-ui/react-use-callback-ref": "1.1.0", "aria-hidden": "^1.1.1", - "react-remove-scroll": "2.5.7" + "react-remove-scroll": "2.6.0" }, "peerDependencies": { "@types/react": "*", @@ -1561,27 +1652,42 @@ } } }, - "node_modules/@radix-ui/react-popover": { + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-context": { "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-popover/-/react-popover-1.1.1.tgz", - "integrity": "sha512-3y1A3isulwnWhvTTwmIreiB8CF4L+qRjZnK1wYLO7pplddzXKby/GnZ2M7OZY3qgnl6p9AodUIHRYGXNah8Y7g==", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-popover": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-popover/-/react-popover-1.1.2.tgz", + "integrity": "sha512-u2HRUyWW+lOiA2g0Le0tMmT55FGOEWHwPFt1EPfbLly7uXQExFo5duNKqG2DzmFXIdqOeNd+TpE8baHWJCyP9w==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", - "@radix-ui/react-focus-guards": "1.1.0", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-dismissable-layer": "1.1.1", + "@radix-ui/react-focus-guards": "1.1.1", "@radix-ui/react-focus-scope": "1.1.0", "@radix-ui/react-id": "1.1.0", "@radix-ui/react-popper": "1.2.0", - "@radix-ui/react-portal": "1.1.1", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-portal": "1.1.2", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-slot": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0", "aria-hidden": "^1.1.1", - "react-remove-scroll": "2.5.7" + "react-remove-scroll": "2.6.0" }, "peerDependencies": { "@types/react": "*", @@ -1598,6 +1704,21 @@ } } }, + "node_modules/@radix-ui/react-popover/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-popper": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-popper/-/react-popper-1.2.0.tgz", @@ -1631,9 +1752,9 @@ } }, "node_modules/@radix-ui/react-portal": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-portal/-/react-portal-1.1.1.tgz", - "integrity": "sha512-A3UtLk85UtqhzFqtoC8Q0KvR2GbXF3mtPgACSazajqq6A41mEQgo53iPzY4i6BwDxlIFqWIhiQ2G729n+2aw/g==", + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-portal/-/react-portal-1.1.2.tgz", + "integrity": "sha512-WeDYLGPxJb/5EGBoedyJbT0MpoULmwnIPMJMSldkuiMsBAv7N1cRdsTWZWht9vpPOiN3qyiGAtbK2is47/uMFg==", "license": "MIT", "dependencies": { "@radix-ui/react-primitive": "2.0.0", @@ -1655,9 +1776,9 @@ } }, "node_modules/@radix-ui/react-presence": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-presence/-/react-presence-1.1.0.tgz", - "integrity": "sha512-Gq6wuRN/asf9H/E/VzdKoUtT8GC9PQc9z40/vEr0VCJ4u5XvvhWIrSsCB6vD2/cH7ugTdSfYq9fLJCcM00acrQ==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-presence/-/react-presence-1.1.1.tgz", + "integrity": "sha512-IeFXVi4YS1K0wVZzXNrbaaUvIJ3qdY+/Ih4eHFhWA9SwGR9UDX7Ck8abvL57C4cv3wwMvUE0OG69Qc3NCcTe/A==", "license": "MIT", "dependencies": { "@radix-ui/react-compose-refs": "1.1.0", @@ -1702,16 +1823,16 @@ } }, "node_modules/@radix-ui/react-radio-group": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-radio-group/-/react-radio-group-1.2.0.tgz", - "integrity": "sha512-yv+oiLaicYMBpqgfpSPw6q+RyXlLdIpQWDHZbUKURxe+nEh53hFXPPlfhfQQtYkS5MMK/5IWIa76SksleQZSzw==", + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-radio-group/-/react-radio-group-1.2.1.tgz", + "integrity": "sha512-kdbv54g4vfRjja9DNWPMxKvXblzqbpEC8kspEkZ6dVP7kQksGCn+iZHkcCz2nb00+lPdRvxrqy4WrvvV1cNqrQ==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-roving-focus": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0", @@ -1733,6 +1854,21 @@ } } }, + "node_modules/@radix-ui/react-radio-group/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-roving-focus": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-roving-focus/-/react-roving-focus-1.1.0.tgz", @@ -1765,17 +1901,17 @@ } }, "node_modules/@radix-ui/react-scroll-area": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-scroll-area/-/react-scroll-area-1.1.0.tgz", - "integrity": "sha512-9ArIZ9HWhsrfqS765h+GZuLoxaRHD/j0ZWOWilsCvYTpYJp8XwCqNG7Dt9Nu/TItKOdgLGkOPCodQvDc+UMwYg==", + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/@radix-ui/react-scroll-area/-/react-scroll-area-1.2.0.tgz", + "integrity": "sha512-q2jMBdsJ9zB7QG6ngQNzNwlvxLQqONyL58QbEGwuyRZZb/ARQwk3uQVbCF7GvQVOtV6EU/pDxAw3zRzJZI3rpQ==", "license": "MIT", "dependencies": { "@radix-ui/number": "1.1.0", "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-callback-ref": "1.1.0", "@radix-ui/react-use-layout-effect": "1.1.0" @@ -1795,24 +1931,39 @@ } } }, + "node_modules/@radix-ui/react-scroll-area/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-select": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/@radix-ui/react-select/-/react-select-2.1.1.tgz", - "integrity": "sha512-8iRDfyLtzxlprOo9IicnzvpsO1wNCkuwzzCM+Z5Rb5tNOpCdMvcc2AkzX0Fz+Tz9v6NJ5B/7EEgyZveo4FBRfQ==", + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-select/-/react-select-2.1.2.tgz", + "integrity": "sha512-rZJtWmorC7dFRi0owDmoijm6nSJH1tVw64QGiNIZ9PNLyBDtG+iAq+XGsya052At4BfarzY/Dhv9wrrUr6IMZA==", "license": "MIT", "dependencies": { "@radix-ui/number": "1.1.0", "@radix-ui/primitive": "1.1.0", "@radix-ui/react-collection": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", - "@radix-ui/react-focus-guards": "1.1.0", + "@radix-ui/react-dismissable-layer": "1.1.1", + "@radix-ui/react-focus-guards": "1.1.1", "@radix-ui/react-focus-scope": "1.1.0", "@radix-ui/react-id": "1.1.0", "@radix-ui/react-popper": "1.2.0", - "@radix-ui/react-portal": "1.1.1", + "@radix-ui/react-portal": "1.1.2", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-slot": "1.1.0", "@radix-ui/react-use-callback-ref": "1.1.0", @@ -1821,7 +1972,7 @@ "@radix-ui/react-use-previous": "1.1.0", "@radix-ui/react-visually-hidden": "1.1.0", "aria-hidden": "^1.1.1", - "react-remove-scroll": "2.5.7" + "react-remove-scroll": "2.6.0" }, "peerDependencies": { "@types/react": "*", @@ -1838,6 +1989,21 @@ } } }, + "node_modules/@radix-ui/react-select/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-separator": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-separator/-/react-separator-1.1.0.tgz", @@ -1862,16 +2028,16 @@ } }, "node_modules/@radix-ui/react-slider": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-slider/-/react-slider-1.2.0.tgz", - "integrity": "sha512-dAHCDA4/ySXROEPaRtaMV5WHL8+JB/DbtyTbJjYkY0RXmKMO2Ln8DFZhywG5/mVQ4WqHDBc8smc14yPXPqZHYA==", + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-slider/-/react-slider-1.2.1.tgz", + "integrity": "sha512-bEzQoDW0XP+h/oGbutF5VMWJPAl/UU8IJjr7h02SOHDIIIxq+cep8nItVNoBV+OMmahCdqdF38FTpmXoqQUGvw==", "license": "MIT", "dependencies": { "@radix-ui/number": "1.1.0", "@radix-ui/primitive": "1.1.0", "@radix-ui/react-collection": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-controllable-state": "1.1.0", @@ -1894,6 +2060,21 @@ } } }, + "node_modules/@radix-ui/react-slider/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-slot": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.1.0.tgz", @@ -1913,14 +2094,14 @@ } }, "node_modules/@radix-ui/react-switch": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-switch/-/react-switch-1.1.0.tgz", - "integrity": "sha512-OBzy5WAj641k0AOSpKQtreDMe+isX0MQJ1IVyF03ucdF3DunOnROVrjWs8zsXUxC3zfZ6JL9HFVCUlMghz9dJw==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-switch/-/react-switch-1.1.1.tgz", + "integrity": "sha512-diPqDDoBcZPSicYoMWdWx+bCPuTRH4QSp9J+65IvtdS0Kuzt67bI6n32vCj8q6NZmYW/ah+2orOtMwcX5eQwIg==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-use-controllable-state": "1.1.0", "@radix-ui/react-use-previous": "1.1.0", @@ -1941,17 +2122,32 @@ } } }, + "node_modules/@radix-ui/react-switch/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-tabs": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@radix-ui/react-tabs/-/react-tabs-1.1.0.tgz", - "integrity": "sha512-bZgOKB/LtZIij75FSuPzyEti/XBhJH52ExgtdVqjCIh+Nx/FW+LhnbXtbCzIi34ccyMsyOja8T0thCzoHFXNKA==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-tabs/-/react-tabs-1.1.1.tgz", + "integrity": "sha512-3GBUDmP2DvzmtYLMsHmpA1GtR46ZDZ+OreXM/N+kkQJOPIgytFWWTfDQmBQKBvaFS0Vno0FktdbVzN28KGrMdw==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", - "@radix-ui/react-context": "1.1.0", + "@radix-ui/react-context": "1.1.1", "@radix-ui/react-direction": "1.1.0", "@radix-ui/react-id": "1.1.0", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-roving-focus": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0" @@ -1971,6 +2167,21 @@ } } }, + "node_modules/@radix-ui/react-tabs/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-toggle": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-toggle/-/react-toggle-1.1.0.tgz", @@ -2026,19 +2237,19 @@ } }, "node_modules/@radix-ui/react-tooltip": { - "version": "1.1.2", - "resolved": "https://registry.npmjs.org/@radix-ui/react-tooltip/-/react-tooltip-1.1.2.tgz", - "integrity": "sha512-9XRsLwe6Yb9B/tlnYCPVUd/TFS4J7HuOZW345DCeC6vKIxQGMZdx21RK4VoZauPD5frgkXTYVS5y90L+3YBn4w==", + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-tooltip/-/react-tooltip-1.1.3.tgz", + "integrity": "sha512-Z4w1FIS0BqVFI2c1jZvb/uDVJijJjJ2ZMuPV81oVgTZ7g3BZxobplnMVvXtFWgtozdvYJ+MFWtwkM5S2HnAong==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.0", "@radix-ui/react-compose-refs": "1.1.0", - "@radix-ui/react-context": "1.1.0", - "@radix-ui/react-dismissable-layer": "1.1.0", + "@radix-ui/react-context": "1.1.1", + "@radix-ui/react-dismissable-layer": "1.1.1", "@radix-ui/react-id": "1.1.0", "@radix-ui/react-popper": "1.2.0", - "@radix-ui/react-portal": "1.1.1", - "@radix-ui/react-presence": "1.1.0", + "@radix-ui/react-portal": "1.1.2", + "@radix-ui/react-presence": "1.1.1", "@radix-ui/react-primitive": "2.0.0", "@radix-ui/react-slot": "1.1.0", "@radix-ui/react-use-controllable-state": "1.1.0", @@ -2059,6 +2270,21 @@ } } }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-context": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@radix-ui/react-context/-/react-context-1.1.1.tgz", + "integrity": "sha512-UASk9zi+crv9WteK/NU4PLvOoL3OuE6BWVKNF6hPRBtYBDXQ2u5iu3O59zUlJiTVvkyuycnqrztsHVJwcK9K+Q==", + "license": "MIT", + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-use-callback-ref": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@radix-ui/react-use-callback-ref/-/react-use-callback-ref-1.1.0.tgz", @@ -2390,15 +2616,15 @@ "dev": true }, "node_modules/@swc/core": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core/-/core-1.5.24.tgz", - "integrity": "sha512-Eph9zvO4xvqWZGVzTdtdEJ0Vqf0VIML/o/e4Qd2RLOqtfgnlRi7avmMu5C0oqciJ0tk+hqdUKVUZ4JPoPaiGvQ==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core/-/core-1.7.39.tgz", + "integrity": "sha512-jns6VFeOT49uoTKLWIEfiQqJAlyqldNAt80kAr8f7a5YjX0zgnG3RBiLMpksx4Ka4SlK4O6TJ/lumIM3Trp82g==", "dev": true, "hasInstallScript": true, "license": "Apache-2.0", "dependencies": { "@swc/counter": "^0.1.3", - "@swc/types": "^0.1.7" + "@swc/types": "^0.1.13" }, "engines": { "node": ">=10" @@ -2408,16 +2634,16 @@ "url": "https://opencollective.com/swc" }, "optionalDependencies": { - "@swc/core-darwin-arm64": "1.5.24", - "@swc/core-darwin-x64": "1.5.24", - "@swc/core-linux-arm-gnueabihf": "1.5.24", - "@swc/core-linux-arm64-gnu": "1.5.24", - "@swc/core-linux-arm64-musl": "1.5.24", - "@swc/core-linux-x64-gnu": "1.5.24", - "@swc/core-linux-x64-musl": "1.5.24", - "@swc/core-win32-arm64-msvc": "1.5.24", - "@swc/core-win32-ia32-msvc": "1.5.24", - "@swc/core-win32-x64-msvc": "1.5.24" + "@swc/core-darwin-arm64": "1.7.39", + "@swc/core-darwin-x64": "1.7.39", + "@swc/core-linux-arm-gnueabihf": "1.7.39", + "@swc/core-linux-arm64-gnu": "1.7.39", + "@swc/core-linux-arm64-musl": "1.7.39", + "@swc/core-linux-x64-gnu": "1.7.39", + "@swc/core-linux-x64-musl": "1.7.39", + "@swc/core-win32-arm64-msvc": "1.7.39", + "@swc/core-win32-ia32-msvc": "1.7.39", + "@swc/core-win32-x64-msvc": "1.7.39" }, "peerDependencies": { "@swc/helpers": "*" @@ -2429,9 +2655,9 @@ } }, "node_modules/@swc/core-darwin-arm64": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.5.24.tgz", - "integrity": "sha512-M7oLOcC0sw+UTyAuL/9uyB9GeO4ZpaBbH76JSH6g1m0/yg7LYJZGRmplhDmwVSDAR5Fq4Sjoi1CksmmGkgihGA==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.7.39.tgz", + "integrity": "sha512-o2nbEL6scMBMCTvY9OnbyVXtepLuNbdblV9oNJEFia5v5eGj9WMrnRQiylH3Wp/G2NYkW7V1/ZVW+kfvIeYe9A==", "cpu": [ "arm64" ], @@ -2446,9 +2672,9 @@ } }, "node_modules/@swc/core-darwin-x64": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-darwin-x64/-/core-darwin-x64-1.5.24.tgz", - "integrity": "sha512-MfcFjGGYognpSBSos2pYUNYJSmqEhuw5ceGr6qAdME7ddbjGXliza4W6FggsM+JnWwpqa31+e7/R+GetW4WkaQ==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-darwin-x64/-/core-darwin-x64-1.7.39.tgz", + "integrity": "sha512-qMlv3XPgtPi/Fe11VhiPDHSLiYYk2dFYl747oGsHZPq+6tIdDQjIhijXPcsUHIXYDyG7lNpODPL8cP/X1sc9MA==", "cpu": [ "x64" ], @@ -2463,9 +2689,9 @@ } }, "node_modules/@swc/core-linux-arm-gnueabihf": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.5.24.tgz", - "integrity": "sha512-amI2pwtcWV3E/m/nf+AQtn1LWDzKLZyjCmWd3ms7QjEueWYrY8cU1Y4Wp7wNNsxIoPOi8zek1Uj2wwFD/pttNQ==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.7.39.tgz", + "integrity": "sha512-NP+JIkBs1ZKnpa3Lk2W1kBJMwHfNOxCUJXuTa2ckjFsuZ8OUu2gwdeLFkTHbR43dxGwH5UzSmuGocXeMowra/Q==", "cpu": [ "arm" ], @@ -2480,9 +2706,9 @@ } }, "node_modules/@swc/core-linux-arm64-gnu": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.5.24.tgz", - "integrity": "sha512-sTSvmqMmgT1ynH/nP75Pc51s+iT4crZagHBiDOf5cq+kudUYjda9lWMs7xkXB/TUKFHPCRK0HGunl8bkwiIbuw==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.7.39.tgz", + "integrity": "sha512-cPc+/HehyHyHcvAsk3ML/9wYcpWVIWax3YBaA+ScecJpSE04l/oBHPfdqKUPslqZ+Gcw0OWnIBGJT/fBZW2ayw==", "cpu": [ "arm64" ], @@ -2497,9 +2723,9 @@ } }, "node_modules/@swc/core-linux-arm64-musl": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.5.24.tgz", - "integrity": "sha512-vd2/hfOBGbrX21FxsFdXCUaffjkHvlZkeE2UMRajdXifwv79jqOHIJg3jXG1F3ZrhCghCzirFts4tAZgcG8XWg==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.7.39.tgz", + "integrity": "sha512-8RxgBC6ubFem66bk9XJ0vclu3exJ6eD7x7CwDhp5AD/tulZslTYXM7oNPjEtje3xxabXuj/bEUMNvHZhQRFdqA==", "cpu": [ "arm64" ], @@ -2514,9 +2740,9 @@ } }, "node_modules/@swc/core-linux-x64-gnu": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.5.24.tgz", - "integrity": "sha512-Zrdzi7NqzQxm2BvAG5KyOSBEggQ7ayrxh599AqqevJmsUXJ8o2nMiWQOBvgCGp7ye+Biz3pvZn1EnRzAp+TpUg==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.7.39.tgz", + "integrity": "sha512-3gtCPEJuXLQEolo9xsXtuPDocmXQx12vewEyFFSMSjOfakuPOBmOQMa0sVL8Wwius8C1eZVeD1fgk0omMqeC+Q==", "cpu": [ "x64" ], @@ -2531,9 +2757,9 @@ } }, "node_modules/@swc/core-linux-x64-musl": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.5.24.tgz", - "integrity": "sha512-1F8z9NRi52jdZQCGc5sflwYSctL6omxiVmIFVp8TC9nngjQKc00TtX/JC2Eo2HwvgupkFVl5YQJidAck9YtmJw==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.7.39.tgz", + "integrity": "sha512-mg39pW5x/eqqpZDdtjZJxrUvQNSvJF4O8wCl37fbuFUqOtXs4TxsjZ0aolt876HXxxhsQl7rS+N4KioEMSgTZw==", "cpu": [ "x64" ], @@ -2548,9 +2774,9 @@ } }, "node_modules/@swc/core-win32-arm64-msvc": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.5.24.tgz", - "integrity": "sha512-cKpP7KvS6Xr0jFSTBXY53HZX/YfomK5EMQYpCVDOvfsZeYHN20sQSKXfpVLvA/q2igVt1zzy1XJcOhpJcgiKLg==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.7.39.tgz", + "integrity": "sha512-NZwuS0mNJowH3e9bMttr7B1fB8bW5svW/yyySigv9qmV5VcQRNz1kMlCvrCLYRsa93JnARuiaBI6FazSeG8mpA==", "cpu": [ "arm64" ], @@ -2565,9 +2791,9 @@ } }, "node_modules/@swc/core-win32-ia32-msvc": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.5.24.tgz", - "integrity": "sha512-IoPWfi0iwqjZuf7gE223+B97/ZwkKbu7qL5KzGP7g3hJrGSKAvv7eC5Y9r2iKKtLKyv5R/T6Ho0kFR/usi7rHw==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.7.39.tgz", + "integrity": "sha512-qFmvv5UExbJPXhhvCVDBnjK5Duqxr048dlVB6ZCgGzbRxuarOlawCzzLK4N172230pzlAWGLgn9CWl3+N6zfHA==", "cpu": [ "ia32" ], @@ -2582,9 +2808,9 @@ } }, "node_modules/@swc/core-win32-x64-msvc": { - "version": "1.5.24", - "resolved": "https://registry.npmjs.org/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.5.24.tgz", - "integrity": "sha512-zHgF2k1uVJL8KIW+PnVz1To4a3Cz9THbh2z2lbehaF/gKHugH4c3djBozU4das1v35KOqf5jWIEviBLql2wDLQ==", + "version": "1.7.39", + "resolved": "https://registry.npmjs.org/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.7.39.tgz", + "integrity": "sha512-o+5IMqgOtj9+BEOp16atTfBgCogVak9svhBpwsbcJQp67bQbxGYhAPPDW/hZ2rpSSF7UdzbY9wudoX9G4trcuQ==", "cpu": [ "x64" ], @@ -2606,9 +2832,9 @@ "license": "Apache-2.0" }, "node_modules/@swc/types": { - "version": "0.1.7", - "resolved": "https://registry.npmjs.org/@swc/types/-/types-0.1.7.tgz", - "integrity": "sha512-scHWahbHF0eyj3JsxG9CFJgFdFNaVQCNAimBlT6PzS3n/HptxqREjsm4OH6AN3lYcffZYSPxXW8ua2BEHp0lJQ==", + "version": "0.1.13", + "resolved": "https://registry.npmjs.org/@swc/types/-/types-0.1.13.tgz", + "integrity": "sha512-JL7eeCk6zWCbiYQg2xQSdLXQJl8Qoc9rXmG2cEKvHe3CKwMHwHGpfOb8frzNLmbycOo6I51qxnLnn9ESf4I20Q==", "dev": true, "license": "Apache-2.0", "dependencies": { @@ -2616,26 +2842,26 @@ } }, "node_modules/@tailwindcss/forms": { - "version": "0.5.7", - "resolved": "https://registry.npmjs.org/@tailwindcss/forms/-/forms-0.5.7.tgz", - "integrity": "sha512-QE7X69iQI+ZXwldE+rzasvbJiyV/ju1FGHH0Qn2W3FKbuYtqp8LKcy6iSw79fVUT5/Vvf+0XgLCeYVG+UV6hOw==", + "version": "0.5.9", + "resolved": "https://registry.npmjs.org/@tailwindcss/forms/-/forms-0.5.9.tgz", + "integrity": "sha512-tM4XVr2+UVTxXJzey9Twx48c1gcxFStqn1pQz0tRsX8o3DvxhN5oY5pvyAbUx7VTaZxpej4Zzvc6h+1RJBzpIg==", "dev": true, + "license": "MIT", "dependencies": { "mini-svg-data-uri": "^1.2.3" }, "peerDependencies": { - "tailwindcss": ">=3.0.0 || >= 3.0.0-alpha.1" + "tailwindcss": ">=3.0.0 || >= 3.0.0-alpha.1 || >= 4.0.0-alpha.20" } }, "node_modules/@testing-library/jest-dom": { - "version": "6.4.6", - "resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-6.4.6.tgz", - "integrity": "sha512-8qpnGVincVDLEcQXWaHOf6zmlbwTKc6Us6PPu4CRnPXCzo2OGBS5cwgMMOWdxDpEz1mkbvXHpEy99M5Yvt682w==", + "version": "6.6.2", + "resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-6.6.2.tgz", + "integrity": "sha512-P6GJD4yqc9jZLbe98j/EkyQDTPgqftohZF5FBkHY5BUERZmcf4HeO2k0XaefEg329ux2p21i1A1DmyQ1kKw2Jw==", "dev": true, "license": "MIT", "dependencies": { "@adobe/css-tools": "^4.4.0", - "@babel/runtime": "^7.9.2", "aria-query": "^5.0.0", "chalk": "^3.0.0", "css.escape": "^1.5.1", @@ -2647,30 +2873,6 @@ "node": ">=14", "npm": ">=6", "yarn": ">=1" - }, - "peerDependencies": { - "@jest/globals": ">= 28", - "@types/bun": "latest", - "@types/jest": ">= 28", - "jest": ">= 28", - "vitest": ">= 0.32" - }, - "peerDependenciesMeta": { - "@jest/globals": { - "optional": true - }, - "@types/bun": { - "optional": true - }, - "@types/jest": { - "optional": true - }, - "jest": { - "optional": true - }, - "vitest": { - "optional": true - } } }, "node_modules/@testing-library/jest-dom/node_modules/chalk": { @@ -2699,9 +2901,9 @@ "dev": true }, "node_modules/@types/lodash": { - "version": "4.17.7", - "resolved": "https://registry.npmjs.org/@types/lodash/-/lodash-4.17.7.tgz", - "integrity": "sha512-8wTvZawATi/lsmNu10/j2hk1KEP0IvjubqPE3cu1Xz7xfXXt5oCq3SNUz4fMIP4XGF9Ky+Ue2tBA3hcS7LSBlA==", + "version": "4.17.12", + "resolved": "https://registry.npmjs.org/@types/lodash/-/lodash-4.17.12.tgz", + "integrity": "sha512-sviUmCE8AYdaF/KIHLDJBQgeYzPBI0vf/17NaYehBJfYD1j6/L95Slh07NlyK2iNyBNaEkb3En2jRt+a8y3xZQ==", "dev": true, "license": "MIT" }, @@ -3036,13 +3238,13 @@ "license": "ISC" }, "node_modules/@vitejs/plugin-react-swc": { - "version": "3.7.0", - "resolved": "https://registry.npmjs.org/@vitejs/plugin-react-swc/-/plugin-react-swc-3.7.0.tgz", - "integrity": "sha512-yrknSb3Dci6svCd/qhHqhFPDSw0QtjumcqdKMoNNzmOl5lMXTTiqzjWtG4Qask2HdvvzaNgSunbQGet8/GrKdA==", + "version": "3.7.1", + "resolved": "https://registry.npmjs.org/@vitejs/plugin-react-swc/-/plugin-react-swc-3.7.1.tgz", + "integrity": "sha512-vgWOY0i1EROUK0Ctg1hwhtC3SdcDjZcdit4Ups4aPkDcB1jYhmo+RMYWY87cmXMhvtD5uf8lV89j2w16vkdSVg==", "dev": true, "license": "MIT", "dependencies": { - "@swc/core": "^1.5.7" + "@swc/core": "^1.7.26" }, "peerDependencies": { "vite": "^4 || ^5" @@ -3398,9 +3600,9 @@ } }, "node_modules/axios": { - "version": "1.7.3", - "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.3.tgz", - "integrity": "sha512-Ar7ND9pU99eJ9GpoGQKhKf58GpUOgnzuaB7ueNQ5BMi0p+LZ5oaEnfF999fAArcTIBwXTCHAmGcHOZJaWPq9Nw==", + "version": "1.7.7", + "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.7.tgz", + "integrity": "sha512-S4kL7XrjgBmvdGut0sN3yJxqYzrDOnivkBiN0OFs6hLiUam3UPvswUo0kqGyhqUZGEOytHyumEdXsAkgCOUf3Q==", "license": "MIT", "dependencies": { "follow-redirects": "^1.15.6", @@ -5303,9 +5505,9 @@ "dev": true }, "node_modules/hls.js": { - "version": "1.5.14", - "resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.5.14.tgz", - "integrity": "sha512-5wLiQ2kWJMui6oUslaq8PnPOv1vjuee5gTxjJD0DSsccY12OXtDT0h137UuqjczNeHzeEYR0ROZQibKNMr7Mzg==", + "version": "1.5.17", + "resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.5.17.tgz", + "integrity": "sha512-wA66nnYFvQa1o4DO/BFgLNRKnBTVXpNeldGRBJ2Y0SvFtdwvFKCbqa9zhHoZLoxHhZ+jYsj3aIBkWQQCPNOhMw==", "license": "Apache-2.0" }, "node_modules/html-encoding-sniffer": { @@ -5828,9 +6030,9 @@ } }, "node_modules/konva": { - "version": "9.3.14", - "resolved": "https://registry.npmjs.org/konva/-/konva-9.3.14.tgz", - "integrity": "sha512-Gmm5lyikGYJyogKQA7Fy6dKkfNh350V6DwfZkid0RVrGYP2cfCsxuMxgF5etKeCv7NjXYpJxKqi1dYkIkX/dcA==", + "version": "9.3.16", + "resolved": "https://registry.npmjs.org/konva/-/konva-9.3.16.tgz", + "integrity": "sha512-qa47cefGDDHzkToGRGDsy24f/Njrz7EHP56jQ8mlDcjAPO7vkfTDeoBDIfmF7PZtpfzDdooafQmEUJMDU2F7FQ==", "funding": [ { "type": "patreon", @@ -6582,9 +6784,9 @@ } }, "node_modules/picocolors": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.0.1.tgz", - "integrity": "sha512-anP1Z8qwhkbmu7MFP5iTt+wQKXgwzf7zTyGlcdzabySa9vd0Xt392U0rVmz9poOaBj0uHJKyyo9/upk0HrEQew==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", "license": "ISC" }, "node_modules/picomatch": { @@ -6615,9 +6817,9 @@ } }, "node_modules/postcss": { - "version": "8.4.41", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.41.tgz", - "integrity": "sha512-TesUflQ0WKZqAvg52PWL6kHgLKP6xB6heTOdoYM0Wt2UHyxNa4K25EZZMgKns3BH1RLVbZCREPpLY0rhnNoHVQ==", + "version": "8.4.47", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.47.tgz", + "integrity": "sha512-56rxCq7G/XfB4EkXq9Egn5GCqugWvDFjafDOThIdMBsI15iqPqR5r15TfSr1YPYeEI19YeaXMCbY6u88Y76GLQ==", "funding": [ { "type": "opencollective", @@ -6635,8 +6837,8 @@ "license": "MIT", "dependencies": { "nanoid": "^3.3.7", - "picocolors": "^1.0.1", - "source-map-js": "^1.2.0" + "picocolors": "^1.1.0", + "source-map-js": "^1.2.1" }, "engines": { "node": "^10 || ^12 || >=14" @@ -6763,9 +6965,9 @@ } }, "node_modules/prettier": { - "version": "3.3.2", - "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.3.2.tgz", - "integrity": "sha512-rAVeHYMcv8ATV5d508CFdn+8/pHPpXeIid1DdrPwXnaAdH7cqjVbpJaT5eq4yRAFU/lsbwYwSF/n5iNrdJHPQA==", + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.3.3.tgz", + "integrity": "sha512-i2tDNA0O5IrMO757lfrdQZCc2jPNDVntV0m/+4whiDfWaTKfMNgR7Qz0NAeGz/nRqF4m5/6CLzbP4/liHt12Ew==", "dev": true, "license": "MIT", "bin": { @@ -7132,12 +7334,12 @@ } }, "node_modules/react-remove-scroll": { - "version": "2.5.7", - "resolved": "https://registry.npmjs.org/react-remove-scroll/-/react-remove-scroll-2.5.7.tgz", - "integrity": "sha512-FnrTWO4L7/Bhhf3CYBNArEG/yROV0tKmTv7/3h9QCFvH6sndeFf1wPqOcbFVu5VAulS5dV1wGT3GZZ/1GawqiA==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/react-remove-scroll/-/react-remove-scroll-2.6.0.tgz", + "integrity": "sha512-I2U4JVEsQenxDAKaVa3VZ/JeJZe0/2DxPWL8Tj8yLKctQJQiZM52pn/GWFpSp8dftjM3pSAHVJZscAnC/y+ySQ==", "license": "MIT", "dependencies": { - "react-remove-scroll-bar": "^2.3.4", + "react-remove-scroll-bar": "^2.3.6", "react-style-singleton": "^2.2.1", "tslib": "^2.1.0", "use-callback-ref": "^1.3.0", @@ -7254,9 +7456,10 @@ } }, "node_modules/react-tracked": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/react-tracked/-/react-tracked-2.0.0.tgz", - "integrity": "sha512-Px8Ms9zhQKzAj3gnwQm6L+sJwzB0uPa8/BgHKOhB8bIuQEgB2iJfryM7GVja9oviiGAa7vtgEBtM+poT1E7V2w==", + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/react-tracked/-/react-tracked-2.0.1.tgz", + "integrity": "sha512-qjbmtkO2IcW+rB2cFskRWDTjKs/w9poxvNnduacjQA04LWxOoLy9J8WfIEq1ahifQ/tVJQECrQPBm+UEzKRDtg==", + "license": "MIT", "dependencies": { "proxy-compare": "^3.0.0", "use-context-selector": "^2.0.0" @@ -7740,9 +7943,10 @@ } }, "node_modules/source-map-js": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.0.tgz", - "integrity": "sha512-itJW8lvSA0TXEphiRoawsCksnlf8SyvmFzIhltqAHluXd88pkCd+cXJVHTDwdCr0IzwptSm035IHQktUu1QUMg==", + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "license": "BSD-3-Clause", "engines": { "node": ">=0.10.0" } diff --git a/web/package.json b/web/package.json index f8f4bc306..73b2ed309 100644 --- a/web/package.json +++ b/web/package.json @@ -16,28 +16,28 @@ "dependencies": { "@cycjimmy/jsmpeg-player": "^6.1.1", "@hookform/resolvers": "^3.9.0", - "@radix-ui/react-alert-dialog": "^1.1.1", + "@radix-ui/react-alert-dialog": "^1.1.2", "@radix-ui/react-aspect-ratio": "^1.1.0", - "@radix-ui/react-checkbox": "^1.1.1", - "@radix-ui/react-context-menu": "^2.2.1", - "@radix-ui/react-dialog": "^1.1.1", - "@radix-ui/react-dropdown-menu": "^2.1.1", - "@radix-ui/react-hover-card": "^1.1.1", + "@radix-ui/react-checkbox": "^1.1.2", + "@radix-ui/react-context-menu": "^2.2.2", + "@radix-ui/react-dialog": "^1.1.2", + "@radix-ui/react-dropdown-menu": "^2.1.2", + "@radix-ui/react-hover-card": "^1.1.2", "@radix-ui/react-label": "^2.1.0", - "@radix-ui/react-popover": "^1.1.1", - "@radix-ui/react-radio-group": "^1.2.0", - "@radix-ui/react-scroll-area": "^1.1.0", - "@radix-ui/react-select": "^2.1.1", + "@radix-ui/react-popover": "^1.1.2", + "@radix-ui/react-radio-group": "^1.2.1", + "@radix-ui/react-scroll-area": "^1.2.0", + "@radix-ui/react-select": "^2.1.2", "@radix-ui/react-separator": "^1.1.0", - "@radix-ui/react-slider": "^1.2.0", + "@radix-ui/react-slider": "^1.2.1", "@radix-ui/react-slot": "^1.1.0", - "@radix-ui/react-switch": "^1.1.0", - "@radix-ui/react-tabs": "^1.1.0", + "@radix-ui/react-switch": "^1.1.1", + "@radix-ui/react-tabs": "^1.1.1", "@radix-ui/react-toggle": "^1.1.0", "@radix-ui/react-toggle-group": "^1.1.0", - "@radix-ui/react-tooltip": "^1.1.2", + "@radix-ui/react-tooltip": "^1.1.3", "apexcharts": "^3.52.0", - "axios": "^1.7.3", + "axios": "^1.7.7", "class-variance-authority": "^0.7.0", "clsx": "^2.1.1", "cmdk": "^1.0.0", @@ -45,10 +45,10 @@ "date-fns": "^3.6.0", "embla-carousel-react": "^8.2.0", "framer-motion": "^11.5.4", - "hls.js": "^1.5.14", + "hls.js": "^1.5.17", "idb-keyval": "^6.2.1", "immer": "^10.1.1", - "konva": "^9.3.14", + "konva": "^9.3.16", "lodash": "^4.17.21", "lucide-react": "^0.407.0", "monaco-yaml": "^5.2.2", @@ -65,7 +65,7 @@ "react-konva": "^18.2.10", "react-router-dom": "^6.26.0", "react-swipeable": "^7.0.1", - "react-tracked": "^2.0.0", + "react-tracked": "^2.0.1", "react-transition-group": "^4.4.5", "react-use-websocket": "^4.8.1", "react-zoom-pan-pinch": "3.4.4", @@ -83,9 +83,9 @@ "zod": "^3.23.8" }, "devDependencies": { - "@tailwindcss/forms": "^0.5.7", - "@testing-library/jest-dom": "^6.4.6", - "@types/lodash": "^4.17.7", + "@tailwindcss/forms": "^0.5.9", + "@testing-library/jest-dom": "^6.6.2", + "@types/lodash": "^4.17.12", "@types/node": "^20.14.10", "@types/react": "^18.3.2", "@types/react-dom": "^18.3.0", @@ -95,7 +95,7 @@ "@types/strftime": "^0.9.8", "@typescript-eslint/eslint-plugin": "^7.5.0", "@typescript-eslint/parser": "^7.5.0", - "@vitejs/plugin-react-swc": "^3.6.0", + "@vitejs/plugin-react-swc": "^3.7.1", "@vitest/coverage-v8": "^2.0.5", "autoprefixer": "^10.4.20", "eslint": "^8.57.0", @@ -109,8 +109,8 @@ "jest-websocket-mock": "^2.5.0", "jsdom": "^24.1.1", "msw": "^2.3.5", - "postcss": "^8.4.39", - "prettier": "^3.3.2", + "postcss": "^8.4.47", + "prettier": "^3.3.3", "prettier-plugin-tailwindcss": "^0.6.5", "tailwindcss": "^3.4.9", "typescript": "^5.5.4", From f9fba948631eea04da2064e7963b08d06d6b5237 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 24 Oct 2024 12:17:11 -0600 Subject: [PATCH 093/479] Slightly downgrade onnxruntime-gpu (#14558) --- docker/tensorrt/requirements-amd64.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/tensorrt/requirements-amd64.txt b/docker/tensorrt/requirements-amd64.txt index c73797516..df276a613 100644 --- a/docker/tensorrt/requirements-amd64.txt +++ b/docker/tensorrt/requirements-amd64.txt @@ -10,5 +10,5 @@ nvidia-cublas-cu11 == 11.11.3.6; platform_machine == 'x86_64' nvidia-cudnn-cu11 == 8.6.0.*; platform_machine == 'x86_64' nvidia-cufft-cu11==10.*; platform_machine == 'x86_64' onnx==1.16.*; platform_machine == 'x86_64' -onnxruntime-gpu==1.19.*; platform_machine == 'x86_64' +onnxruntime-gpu==1.18.*; platform_machine == 'x86_64' protobuf==3.20.3; platform_machine == 'x86_64' From 4ff0c8a8d1e63ef2860523ee7407b1e17826ba20 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 24 Oct 2024 16:00:39 -0600 Subject: [PATCH 094/479] Better review sub-labels (#14563) * Better review sub-labels * Handle init --- frigate/review/maintainer.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/frigate/review/maintainer.py b/frigate/review/maintainer.py index bdb749dc8..d87e1d33c 100644 --- a/frigate/review/maintainer.py +++ b/frigate/review/maintainer.py @@ -51,7 +51,7 @@ class PendingReviewSegment: frame_time: float, severity: SeverityEnum, detections: dict[str, str], - sub_labels: set[str], + sub_labels: dict[str, str], zones: list[str], audio: set[str], ): @@ -135,7 +135,7 @@ class PendingReviewSegment: ReviewSegment.data.name: { "detections": list(set(self.detections.keys())), "objects": list(set(self.detections.values())), - "sub_labels": list(self.sub_labels), + "sub_labels": list(self.sub_labels.values()), "zones": self.zones, "audio": list(self.audio), }, @@ -261,7 +261,7 @@ class ReviewSegmentMaintainer(threading.Thread): segment.detections[object["id"]] = object["sub_label"][0] else: segment.detections[object["id"]] = f'{object["label"]}-verified' - segment.sub_labels.add(object["sub_label"][0]) + segment.sub_labels[object["id"]] = object["sub_label"][0] # if object is alert label # and has entered required zones or required zones is not set @@ -347,7 +347,7 @@ class ReviewSegmentMaintainer(threading.Thread): if len(active_objects) > 0: detections: dict[str, str] = {} - sub_labels = set() + sub_labels = dict[str, str] = {} zones: list[str] = [] severity = None @@ -358,7 +358,7 @@ class ReviewSegmentMaintainer(threading.Thread): detections[object["id"]] = object["sub_label"][0] else: detections[object["id"]] = f'{object["label"]}-verified' - sub_labels.add(object["sub_label"][0]) + sub_labels[object["id"]] = object["sub_label"][0] # if object is alert label # and has entered required zones or required zones is not set @@ -566,7 +566,7 @@ class ReviewSegmentMaintainer(threading.Thread): frame_time, severity, {}, - set(), + {}, [], detections, ) @@ -576,7 +576,7 @@ class ReviewSegmentMaintainer(threading.Thread): frame_time, SeverityEnum.alert, {manual_info["event_id"]: manual_info["label"]}, - set(), + {}, [], set(), ) From 2d27e72ed9907ccac352557d93c46e591909eeac Mon Sep 17 00:00:00 2001 From: Corwin Date: Fri, 25 Oct 2024 14:07:01 +0200 Subject: [PATCH 095/479] fix: hailo driver wrong version name (#14575) --- docker/hailo8l/user_installation.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/hailo8l/user_installation.sh b/docker/hailo8l/user_installation.sh index 734c640f9..853652ffa 100644 --- a/docker/hailo8l/user_installation.sh +++ b/docker/hailo8l/user_installation.sh @@ -38,7 +38,7 @@ cd ../../ if [ ! -d /lib/firmware/hailo ]; then sudo mkdir /lib/firmware/hailo fi -sudo mv hailo8_fw.4.17.0.bin /lib/firmware/hailo/hailo8_fw.bin +sudo mv hailo8_fw.4.18.0.bin /lib/firmware/hailo/hailo8_fw.bin # Install udev rules sudo cp ./linux/pcie/51-hailo-udev.rules /etc/udev/rules.d/ From 4dadf6d35337f2b402784abd99de6085ba198f45 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 25 Oct 2024 07:24:04 -0500 Subject: [PATCH 096/479] Bugfixes (#14587) * Ensure review and search item mobile pages reopen correctly * disable pan/pinch/zoom when native browser video controls are displayed * report 0 for storage usage when api returns null --- .../components/graph/CombinedStorageGraph.tsx | 2 +- web/src/components/mobile/MobilePage.tsx | 8 ++++- .../overlay/detail/ReviewDetailDialog.tsx | 31 ++++++++++++------- .../overlay/detail/SearchDetailDialog.tsx | 25 +++++++++------ web/src/components/player/HlsVideoPlayer.tsx | 1 + 5 files changed, 44 insertions(+), 23 deletions(-) diff --git a/web/src/components/graph/CombinedStorageGraph.tsx b/web/src/components/graph/CombinedStorageGraph.tsx index ff605ac02..2a52d82b6 100644 --- a/web/src/components/graph/CombinedStorageGraph.tsx +++ b/web/src/components/graph/CombinedStorageGraph.tsx @@ -216,7 +216,7 @@ export function CombinedStorageGraph({ )} - {getUnitSize(item.usage)} + {getUnitSize(item.usage ?? 0)} {item.data[0].toFixed(2)}% {item.name === "Unused" diff --git a/web/src/components/mobile/MobilePage.tsx b/web/src/components/mobile/MobilePage.tsx index 52bc4d9fe..169b5e524 100644 --- a/web/src/components/mobile/MobilePage.tsx +++ b/web/src/components/mobile/MobilePage.tsx @@ -25,7 +25,13 @@ export function MobilePage({ const [uncontrolledOpen, setUncontrolledOpen] = useState(false); const open = controlledOpen ?? uncontrolledOpen; - const setOpen = onOpenChange ?? setUncontrolledOpen; + const setOpen = (value: boolean) => { + if (onOpenChange) { + onOpenChange(value); + } else { + setUncontrolledOpen(value); + } + }; return ( diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index 0b20ff9bc..2230046f3 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -13,7 +13,7 @@ import { getIconForLabel } from "@/utils/iconUtil"; import { useApiHost } from "@/api"; import { ReviewDetailPaneType, ReviewSegment } from "@/types/review"; import { Event } from "@/types/event"; -import { useEffect, useMemo, useRef, useState } from "react"; +import { useCallback, useEffect, useMemo, useRef, useState } from "react"; import { cn } from "@/lib/utils"; import { FrigatePlusDialog } from "../dialog/FrigatePlusDialog"; import ObjectLifecycle from "./ObjectLifecycle"; @@ -91,6 +91,22 @@ export default function ReviewDetailDialog({ review != undefined, ); + const handleOpenChange = useCallback( + (open: boolean) => { + setIsOpen(open); + if (!open) { + // short timeout to allow the mobile page animation + // to complete before updating the state + setTimeout(() => { + setReview(undefined); + setSelectedEvent(undefined); + setPane("overview"); + }, 300); + } + }, + [setReview, setIsOpen], + ); + useEffect(() => { setIsOpen(review != undefined); // we know that these deps are correct @@ -109,16 +125,7 @@ export default function ReviewDetailDialog({ return ( <> - { - if (!open) { - setReview(undefined); - setSelectedEvent(undefined); - setPane("overview"); - } - }} - > + setUpload(undefined)} @@ -140,7 +147,7 @@ export default function ReviewDetailDialog({ > {pane == "overview" && ( -
setIsOpen(false)}> +
Review Item Details Review item details
{ + setIsOpen(open); + if (!open) { + // short timeout to allow the mobile page animation + // to complete before updating the state + setTimeout(() => { + setSearch(undefined); + }, 300); + } + }, + [setSearch], + ); + useEffect(() => { if (search) { setIsOpen(search != undefined); @@ -158,14 +172,7 @@ export default function SearchDetailDialog({ const Description = isDesktop ? DialogDescription : MobilePageDescription; return ( - { - if (search) { - setSearch(undefined); - } - }} - > + -
setIsOpen(false)}> +
Tracked Object Details Tracked object details
diff --git a/web/src/components/player/HlsVideoPlayer.tsx b/web/src/components/player/HlsVideoPlayer.tsx index bb0c89802..0661fb0c9 100644 --- a/web/src/components/player/HlsVideoPlayer.tsx +++ b/web/src/components/player/HlsVideoPlayer.tsx @@ -190,6 +190,7 @@ export default function HlsVideoPlayer({ minScale={1.0} wheel={{ smoothStep: 0.005 }} onZoom={(zoom) => setZoomScale(zoom.state.scale)} + disabled={!frigateControls} > {frigateControls && ( Date: Fri, 25 Oct 2024 06:47:56 -0600 Subject: [PATCH 097/479] Bug fixes (#14588) * Get intel stats manually if parsing fails * Fix assignment * Clean up mqtt * Formatting * Fix logic --- frigate/comms/mqtt.py | 15 ++++------- frigate/review/maintainer.py | 2 +- frigate/util/services.py | 51 +++++++++++++++++++++++++----------- 3 files changed, 41 insertions(+), 27 deletions(-) diff --git a/frigate/comms/mqtt.py b/frigate/comms/mqtt.py index eaaadfe9f..ca9d03da7 100644 --- a/frigate/comms/mqtt.py +++ b/frigate/comms/mqtt.py @@ -17,7 +17,8 @@ class MqttClient(Communicator): # type: ignore[misc] def __init__(self, config: FrigateConfig) -> None: self.config = config self.mqtt_config = config.mqtt - self.connected: bool = False + self.connected = False + self.started = False def subscribe(self, receiver: Callable) -> None: """Wrapper for allowing dispatcher to subscribe.""" @@ -27,7 +28,8 @@ class MqttClient(Communicator): # type: ignore[misc] def publish(self, topic: str, payload: Any, retain: bool = False) -> None: """Wrapper for publishing when client is in valid state.""" if not self.connected: - logger.error(f"Unable to publish to {topic}: client is not connected") + if self.started: + logger.error(f"Unable to publish to {topic}: client is not connected") return self.client.publish( @@ -197,14 +199,6 @@ class MqttClient(Communicator): # type: ignore[misc] for name in self.config.cameras.keys(): for callback in callback_types: - # We need to pre-clear existing set topics because in previous - # versions the webUI retained on the /set topic but this is - # no longer the case. - self.client.publish( - f"{self.mqtt_config.topic_prefix}/{name}/{callback}/set", - None, - retain=True, - ) self.client.message_callback_add( f"{self.mqtt_config.topic_prefix}/{name}/{callback}/set", self.on_mqtt_command, @@ -253,6 +247,7 @@ class MqttClient(Communicator): # type: ignore[misc] # with connect_async, retries are handled automatically self.client.connect_async(self.mqtt_config.host, self.mqtt_config.port, 60) self.client.loop_start() + self.started = True except Exception as e: logger.error(f"Unable to connect to MQTT server: {e}") return diff --git a/frigate/review/maintainer.py b/frigate/review/maintainer.py index d87e1d33c..38ed59294 100644 --- a/frigate/review/maintainer.py +++ b/frigate/review/maintainer.py @@ -347,7 +347,7 @@ class ReviewSegmentMaintainer(threading.Thread): if len(active_objects) > 0: detections: dict[str, str] = {} - sub_labels = dict[str, str] = {} + sub_labels: dict[str, str] = {} zones: list[str] = [] severity = None diff --git a/frigate/util/services.py b/frigate/util/services.py index 7ff46f039..a71729263 100644 --- a/frigate/util/services.py +++ b/frigate/util/services.py @@ -257,6 +257,40 @@ def get_amd_gpu_stats() -> dict[str, str]: def get_intel_gpu_stats() -> dict[str, str]: """Get stats using intel_gpu_top.""" + + def get_stats_manually(output: str) -> dict[str, str]: + """Find global stats via regex when json fails to parse.""" + reading = "".join(output) + results: dict[str, str] = {} + + # render is used for qsv + render = [] + for result in re.findall(r'"Render/3D/0":{[a-z":\d.,%]+}', reading): + packet = json.loads(result[14:]) + single = packet.get("busy", 0.0) + render.append(float(single)) + + if render: + render_avg = sum(render) / len(render) + else: + render_avg = 1 + + # video is used for vaapi + video = [] + for result in re.findall(r'"Video/\d":{[a-z":\d.,%]+}', reading): + packet = json.loads(result[10:]) + single = packet.get("busy", 0.0) + video.append(float(single)) + + if video: + video_avg = sum(video) / len(video) + else: + video_avg = 1 + + results["gpu"] = f"{round((video_avg + render_avg) / 2, 2)}%" + results["mem"] = "-%" + return results + intel_gpu_top_command = [ "timeout", "0.5s", @@ -284,22 +318,7 @@ def get_intel_gpu_stats() -> dict[str, str]: try: data = json.loads(f"[{output}]") except json.JSONDecodeError: - data = None - - # json is incomplete, remove characters until we get to valid json - while True: - while output and output[-1] != "}": - output = output[:-1] - - if not output: - return {"gpu": "", "mem": ""} - - try: - data = json.loads(f"[{output}]") - break - except json.JSONDecodeError: - output = output[:-1] - continue + return get_stats_manually(output) results: dict[str, str] = {} render = {"global": []} From 4c75440af4cebf44c2e3975d02083b73aa0cf91e Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 25 Oct 2024 07:09:25 -0600 Subject: [PATCH 098/479] Docs updates (#14590) * Update motion docs to make note of recordings * Make note of genai on CPU --- docs/docs/configuration/genai.md | 8 +++++++- docs/docs/configuration/motion_detection.md | 8 +++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index cdaf0adbe..8e2e4fbc2 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -29,7 +29,13 @@ cameras: ## Ollama -[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It provides a nice API over [llama.cpp](https://github.com/ggerganov/llama.cpp). It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance. CPU inference is not recommended. +:::warning + +Using Ollama on CPU is not recommended, high inference times make using generative AI impractical. + +::: + +[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It provides a nice API over [llama.cpp](https://github.com/ggerganov/llama.cpp). It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance. Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [docker container](https://hub.docker.com/r/ollama/ollama) available. diff --git a/docs/docs/configuration/motion_detection.md b/docs/docs/configuration/motion_detection.md index 0844c04a8..7621489ff 100644 --- a/docs/docs/configuration/motion_detection.md +++ b/docs/docs/configuration/motion_detection.md @@ -92,10 +92,16 @@ motion: lightning_threshold: 0.8 ``` -:::tip +:::warning Some cameras like doorbell cameras may have missed detections when someone walks directly in front of the camera and the lightning_threshold causes motion detection to be re-calibrated. In this case, it may be desirable to increase the `lightning_threshold` to ensure these objects are not missed. ::: +:::note + +Lightning threshold does not stop motion based recordings from being saved. + +::: + Large changes in motion like PTZ moves and camera switches between Color and IR mode should result in no motion detection. This is done via the `lightning_threshold` configuration. It is defined as the percentage of the image used to detect lightning or other substantial changes where motion detection needs to recalibrate. Increasing this value will make motion detection more likely to consider lightning or IR mode changes as valid motion. Decreasing this value will make motion detection more likely to ignore large amounts of motion such as a person approaching a doorbell camera. From eca504cb07a939213cc3b1945b217861f189b88d Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 25 Oct 2024 08:45:11 -0600 Subject: [PATCH 099/479] More bug fixes (#14593) * Adjust mqtt logging behavior * Set disconnect * Only consider intel gpu stats error if None is returned --- frigate/comms/mqtt.py | 6 ++---- frigate/stats/util.py | 8 ++++---- 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/frigate/comms/mqtt.py b/frigate/comms/mqtt.py index ca9d03da7..198ec2176 100644 --- a/frigate/comms/mqtt.py +++ b/frigate/comms/mqtt.py @@ -18,7 +18,6 @@ class MqttClient(Communicator): # type: ignore[misc] self.config = config self.mqtt_config = config.mqtt self.connected = False - self.started = False def subscribe(self, receiver: Callable) -> None: """Wrapper for allowing dispatcher to subscribe.""" @@ -28,8 +27,7 @@ class MqttClient(Communicator): # type: ignore[misc] def publish(self, topic: str, payload: Any, retain: bool = False) -> None: """Wrapper for publishing when client is in valid state.""" if not self.connected: - if self.started: - logger.error(f"Unable to publish to {topic}: client is not connected") + logger.debug(f"Unable to publish to {topic}: client is not connected") return self.client.publish( @@ -175,6 +173,7 @@ class MqttClient(Communicator): # type: ignore[misc] client_id=self.mqtt_config.client_id, ) self.client.on_connect = self._on_connect + self.client.on_disconnect = self._on_disconnect self.client.will_set( self.mqtt_config.topic_prefix + "/available", payload="offline", @@ -247,7 +246,6 @@ class MqttClient(Communicator): # type: ignore[misc] # with connect_async, retries are handled automatically self.client.connect_async(self.mqtt_config.host, self.mqtt_config.port, 60) self.client.loop_start() - self.started = True except Exception as e: logger.error(f"Unable to connect to MQTT server: {e}") return diff --git a/frigate/stats/util.py b/frigate/stats/util.py index 2a0f251fc..c2d6586ad 100644 --- a/frigate/stats/util.py +++ b/frigate/stats/util.py @@ -197,8 +197,8 @@ async def set_gpu_stats( # intel QSV GPU intel_usage = get_intel_gpu_stats() - if intel_usage: - stats["intel-qsv"] = intel_usage + if intel_usage is not None: + stats["intel-qsv"] = intel_usage or {"gpu": "", "mem": ""} else: stats["intel-qsv"] = {"gpu": "", "mem": ""} hwaccel_errors.append(args) @@ -222,8 +222,8 @@ async def set_gpu_stats( # intel VAAPI GPU intel_usage = get_intel_gpu_stats() - if intel_usage: - stats["intel-vaapi"] = intel_usage + if intel_usage is not None: + stats["intel-vaapi"] = intel_usage or {"gpu": "", "mem": ""} else: stats["intel-vaapi"] = {"gpu": "", "mem": ""} hwaccel_errors.append(args) From 33825f6d96911656b10ac6bae03c857997968460 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 28 Oct 2024 20:00:14 -0500 Subject: [PATCH 100/479] Add h8l and rocm to release workflow (#14648) * Add h8l to release workflow * Add rocm to release workflow * Variants --- .github/workflows/release.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 36ff3326c..6e90b9c78 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -34,14 +34,14 @@ jobs: STABLE_TAG=${BASE}:stable PULL_TAG=${BASE}:${BUILD_TAG} docker run --rm -v $HOME/.docker/config.json:/config.json quay.io/skopeo/stable:latest copy --authfile /config.json --multi-arch all docker://${PULL_TAG} docker://${VERSION_TAG} - for variant in standard-arm64 tensorrt tensorrt-jp4 tensorrt-jp5 rk; do + for variant in standard-arm64 tensorrt tensorrt-jp4 tensorrt-jp5 rk h8l rocm; do docker run --rm -v $HOME/.docker/config.json:/config.json quay.io/skopeo/stable:latest copy --authfile /config.json --multi-arch all docker://${PULL_TAG}-${variant} docker://${VERSION_TAG}-${variant} done # stable tag if [[ "${BUILD_TYPE}" == "stable" ]]; then docker run --rm -v $HOME/.docker/config.json:/config.json quay.io/skopeo/stable:latest copy --authfile /config.json --multi-arch all docker://${PULL_TAG} docker://${STABLE_TAG} - for variant in standard-arm64 tensorrt tensorrt-jp4 tensorrt-jp5 rk; do + for variant in standard-arm64 tensorrt tensorrt-jp4 tensorrt-jp5 rk h8l rocm; do docker run --rm -v $HOME/.docker/config.json:/config.json quay.io/skopeo/stable:latest copy --authfile /config.json --multi-arch all docker://${PULL_TAG}-${variant} docker://${STABLE_TAG}-${variant} done fi From 8aeb5977808f93321aebe93b3a2083f60d0fa3b6 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 29 Oct 2024 08:06:19 -0500 Subject: [PATCH 101/479] Fix sublabel and icon spacing (#14651) --- web/src/components/card/SearchThumbnail.tsx | 10 +--------- web/src/components/card/SearchThumbnailFooter.tsx | 7 +++---- 2 files changed, 4 insertions(+), 13 deletions(-) diff --git a/web/src/components/card/SearchThumbnail.tsx b/web/src/components/card/SearchThumbnail.tsx index 72a560deb..e96632400 100644 --- a/web/src/components/card/SearchThumbnail.tsx +++ b/web/src/components/card/SearchThumbnail.tsx @@ -41,14 +41,6 @@ export default function SearchThumbnail({ return searchResult.label; } - if ( - config.model.attributes_map[searchResult.label].includes( - searchResult.sub_label, - ) - ) { - return searchResult.sub_label; - } - return `${searchResult.label}-verified`; }, [config, searchResult]); @@ -102,7 +94,7 @@ export default function SearchThumbnail({
- {[objectLabel] + {[searchResult.sub_label ?? objectLabel] .filter( (item) => item !== undefined && !item.includes("-verified"), ) diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index 8c4fb82b5..b959a82c5 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -34,9 +34,8 @@ export default function SearchThumbnailFooter({ return (
4 && - "items-start sm:flex-col sm:gap-2 lg:flex-row lg:items-center lg:gap-1", + "flex w-full flex-row items-center justify-between gap-2", + columns > 4 && "items-start sm:flex-col lg:flex-row lg:items-center", )} >
@@ -49,7 +48,7 @@ export default function SearchThumbnailFooter({ )} {formattedDate}
-
+
Date: Tue, 29 Oct 2024 14:31:03 +0000 Subject: [PATCH 102/479] Update create_config.py (#14658) --- docker/main/rootfs/usr/local/go2rtc/create_config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/main/rootfs/usr/local/go2rtc/create_config.py b/docker/main/rootfs/usr/local/go2rtc/create_config.py index ae2c0128e..1038af64c 100644 --- a/docker/main/rootfs/usr/local/go2rtc/create_config.py +++ b/docker/main/rootfs/usr/local/go2rtc/create_config.py @@ -165,7 +165,7 @@ if config.get("birdseye", {}).get("restream", False): birdseye: dict[str, any] = config.get("birdseye") input = f"-f rawvideo -pix_fmt yuv420p -video_size {birdseye.get('width', 1280)}x{birdseye.get('height', 720)} -r 10 -i {BIRDSEYE_PIPE}" - ffmpeg_cmd = f"exec:{parse_preset_hardware_acceleration_encode(ffmpeg_path, config.get('ffmpeg', {}).get('hwaccel_args'), input, '-rtsp_transport tcp -f rtsp {output}')}" + ffmpeg_cmd = f"exec:{parse_preset_hardware_acceleration_encode(ffmpeg_path, config.get('ffmpeg', {}).get('hwaccel_args', ''), input, '-rtsp_transport tcp -f rtsp {output}')}" if go2rtc_config.get("streams"): go2rtc_config["streams"]["birdseye"] = ffmpeg_cmd From 4e25bebdd0fa5ab44ee0ec3628d65af985e4446a Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 29 Oct 2024 09:28:05 -0600 Subject: [PATCH 103/479] Add ability to configure model input dtype (#14659) * Add input type for dtype * Add ability to manually enable TRT execution provider * Formatting --- frigate/detectors/detector_config.py | 8 ++++++++ frigate/detectors/plugins/onnx.py | 2 +- frigate/object_detection.py | 17 ++++++++++++++--- frigate/util/model.py | 24 +++++++++++++++++++++--- 4 files changed, 44 insertions(+), 7 deletions(-) diff --git a/frigate/detectors/detector_config.py b/frigate/detectors/detector_config.py index c40ef65bf..45875e2e6 100644 --- a/frigate/detectors/detector_config.py +++ b/frigate/detectors/detector_config.py @@ -27,6 +27,11 @@ class InputTensorEnum(str, Enum): nhwc = "nhwc" +class InputDTypeEnum(str, Enum): + float = "float" + int = "int" + + class ModelTypeEnum(str, Enum): ssd = "ssd" yolox = "yolox" @@ -53,6 +58,9 @@ class ModelConfig(BaseModel): input_pixel_format: PixelFormatEnum = Field( default=PixelFormatEnum.rgb, title="Model Input Pixel Color Format" ) + input_dtype: InputDTypeEnum = Field( + default=InputDTypeEnum.int, title="Model Input D Type" + ) model_type: ModelTypeEnum = Field( default=ModelTypeEnum.ssd, title="Object Detection Model Type" ) diff --git a/frigate/detectors/plugins/onnx.py b/frigate/detectors/plugins/onnx.py index 3e58df72a..7004f28fa 100644 --- a/frigate/detectors/plugins/onnx.py +++ b/frigate/detectors/plugins/onnx.py @@ -54,7 +54,7 @@ class ONNXDetector(DetectionApi): logger.info(f"ONNX: {path} loaded") - def detect_raw(self, tensor_input): + def detect_raw(self, tensor_input: np.ndarray): model_input_name = self.model.get_inputs()[0].name tensor_output = self.model.run(None, {model_input_name: tensor_input}) diff --git a/frigate/object_detection.py b/frigate/object_detection.py index eaa3b4e04..0af32034e 100644 --- a/frigate/object_detection.py +++ b/frigate/object_detection.py @@ -12,7 +12,11 @@ from setproctitle import setproctitle import frigate.util as util from frigate.detectors import create_detector -from frigate.detectors.detector_config import BaseDetectorConfig, InputTensorEnum +from frigate.detectors.detector_config import ( + BaseDetectorConfig, + InputDTypeEnum, + InputTensorEnum, +) from frigate.detectors.plugins.rocm import DETECTOR_KEY as ROCM_DETECTOR_KEY from frigate.util.builtin import EventsPerSecond, load_labels from frigate.util.image import SharedMemoryFrameManager @@ -55,12 +59,15 @@ class LocalObjectDetector(ObjectDetector): self.input_transform = tensor_transform( detector_config.model.input_tensor ) + + self.dtype = detector_config.model.input_dtype else: self.input_transform = None + self.dtype = InputDTypeEnum.int self.detect_api = create_detector(detector_config) - def detect(self, tensor_input, threshold=0.4): + def detect(self, tensor_input: np.ndarray, threshold=0.4): detections = [] raw_detections = self.detect_raw(tensor_input) @@ -77,9 +84,13 @@ class LocalObjectDetector(ObjectDetector): self.fps.update() return detections - def detect_raw(self, tensor_input): + def detect_raw(self, tensor_input: np.ndarray): if self.input_transform: tensor_input = np.transpose(tensor_input, self.input_transform) + + if self.dtype == InputDTypeEnum.float: + tensor_input = tensor_input.astype(np.float32) + return self.detect_api.detect_raw(tensor_input=tensor_input) diff --git a/frigate/util/model.py b/frigate/util/model.py index 7aefe8b42..2aa06d0b2 100644 --- a/frigate/util/model.py +++ b/frigate/util/model.py @@ -13,7 +13,7 @@ except ImportError: def get_ort_providers( - force_cpu: bool = False, openvino_device: str = "AUTO", requires_fp16: bool = False + force_cpu: bool = False, device: str = "AUTO", requires_fp16: bool = False ) -> tuple[list[str], list[dict[str, any]]]: if force_cpu: return ( @@ -38,7 +38,25 @@ def get_ort_providers( ) elif provider == "TensorrtExecutionProvider": # TensorrtExecutionProvider uses too much memory without options to control it - pass + # so it is not enabled by default + if device == "Tensorrt": + os.makedirs( + "/config/model_cache/tensorrt/ort/trt-engines", exist_ok=True + ) + providers.append(provider) + options.append( + { + "arena_extend_strategy": "kSameAsRequested", + "trt_fp16_enable": requires_fp16 + and os.environ.get("USE_FP_16", "True") != "False", + "trt_timing_cache_enable": True, + "trt_engine_cache_enable": True, + "trt_timing_cache_path": "/config/model_cache/tensorrt/ort", + "trt_engine_cache_path": "/config/model_cache/tensorrt/ort/trt-engines", + } + ) + else: + continue elif provider == "OpenVINOExecutionProvider": os.makedirs("/config/model_cache/openvino/ort", exist_ok=True) providers.append(provider) @@ -46,7 +64,7 @@ def get_ort_providers( { "arena_extend_strategy": "kSameAsRequested", "cache_dir": "/config/model_cache/openvino/ort", - "device_type": openvino_device, + "device_type": device, } ) elif provider == "CPUExecutionProvider": From e67b7a6d5e6a23f5e722a3e817c6e650acb08143 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 29 Oct 2024 10:28:17 -0500 Subject: [PATCH 104/479] Add ability to use carousel buttons to scroll through object lifecycle elements (#14662) --- .../overlay/detail/ObjectLifecycle.tsx | 24 +++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/web/src/components/overlay/detail/ObjectLifecycle.tsx b/web/src/components/overlay/detail/ObjectLifecycle.tsx index bb80b5c7f..fee15af4a 100644 --- a/web/src/components/overlay/detail/ObjectLifecycle.tsx +++ b/web/src/components/overlay/detail/ObjectLifecycle.tsx @@ -203,6 +203,20 @@ export default function ObjectLifecycle({ setCurrent(index); }; + const handleThumbnailNavigation = useCallback( + (direction: "next" | "previous") => { + if (!mainApi || !thumbnailApi || !eventSequence) return; + const newIndex = + direction === "next" + ? Math.min(current + 1, eventSequence.length - 1) + : Math.max(current - 1, 0); + mainApi.scrollTo(newIndex); + thumbnailApi.scrollTo(newIndex); + setCurrent(newIndex); + }, + [mainApi, thumbnailApi, current, eventSequence], + ); + useEffect(() => { if (eventSequence && eventSequence.length > 0) { setTimeIndex(eventSequence?.[current].timestamp); @@ -545,8 +559,14 @@ export default function ObjectLifecycle({ ))} - - + handleThumbnailNavigation("previous")} + /> + handleThumbnailNavigation("next")} + />
From 73da3d9b203a0af9986cecae0f0168fd28220a5f Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:33:09 -0500 Subject: [PATCH 105/479] Use strict equality check for annotation offset in object lifecycle settings (#14667) --- .../components/overlay/detail/AnnotationSettingsPane.tsx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/web/src/components/overlay/detail/AnnotationSettingsPane.tsx b/web/src/components/overlay/detail/AnnotationSettingsPane.tsx index dd5898c34..79d078c1f 100644 --- a/web/src/components/overlay/detail/AnnotationSettingsPane.tsx +++ b/web/src/components/overlay/detail/AnnotationSettingsPane.tsx @@ -111,13 +111,13 @@ export function AnnotationSettingsPane({ function onApply(values: z.infer) { if ( !values || - values.annotationOffset == null || - values.annotationOffset == "" || + values.annotationOffset === null || + values.annotationOffset === "" || !config ) { return; } - setAnnotationOffset(values.annotationOffset); + setAnnotationOffset(values.annotationOffset ?? 0); } return ( From 357ce0382ecebba1437cdb381ac77766e2763035 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 29 Oct 2024 14:34:07 -0600 Subject: [PATCH 106/479] Fixes (#14668) * Fix environment vars reading * fix yaml returning none * Assume rocm model is onnx despite file extension --- frigate/config/env.py | 2 +- frigate/detectors/plugins/rocm.py | 6 ++---- frigate/util/config.py | 4 ++++ 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/frigate/config/env.py b/frigate/config/env.py index e4e6842cb..0a9b92e8f 100644 --- a/frigate/config/env.py +++ b/frigate/config/env.py @@ -23,7 +23,7 @@ EnvString = Annotated[str, AfterValidator(validate_env_string)] def validate_env_vars(v: dict[str, str], info: ValidationInfo) -> dict[str, str]: if isinstance(info.context, dict) and info.context.get("install", False): - for k, v in v: + for k, v in v.items(): os.environ[k] = v return v diff --git a/frigate/detectors/plugins/rocm.py b/frigate/detectors/plugins/rocm.py index 655973dfd..60118d129 100644 --- a/frigate/detectors/plugins/rocm.py +++ b/frigate/detectors/plugins/rocm.py @@ -98,9 +98,7 @@ class ROCmDetector(DetectionApi): else: logger.info(f"AMD/ROCm: loading model from {path}") - if path.endswith(".onnx"): - self.model = migraphx.parse_onnx(path) - elif ( + if ( path.endswith(".tf") or path.endswith(".tf2") or path.endswith(".tflite") @@ -108,7 +106,7 @@ class ROCmDetector(DetectionApi): # untested self.model = migraphx.parse_tf(path) else: - raise Exception(f"AMD/ROCm: unknown model format {path}") + self.model = migraphx.parse_onnx(path) logger.info("AMD/ROCm: compiling the model") diff --git a/frigate/util/config.py b/frigate/util/config.py index 729215e9e..a9a9666d9 100644 --- a/frigate/util/config.py +++ b/frigate/util/config.py @@ -29,6 +29,10 @@ def migrate_frigate_config(config_file: str): with open(config_file, "r") as f: config: dict[str, dict[str, any]] = yaml.load(f) + if config is None: + logger.error(f"Failed to load config at {config_file}") + return + previous_version = str(config.get("version", "0.13")) if previous_version == CURRENT_CONFIG_VERSION: From d12c7809ddcf1282adc68fc891b7f61f15f11ca8 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Tue, 29 Oct 2024 18:40:24 -0500 Subject: [PATCH 107/479] Update Hailo Driver to 4.19 (#14674) * update to hailo driver 4.19 * update builds for 4.19 --- docker/hailo8l/Dockerfile | 44 +++---- .../generate_wheel_conf.py | 67 ----------- .../hailo8l/pyhailort_build_scripts/setup.py | 111 ------------------ docker/hailo8l/user_installation.sh | 4 +- 4 files changed, 16 insertions(+), 210 deletions(-) delete mode 100644 docker/hailo8l/pyhailort_build_scripts/generate_wheel_conf.py delete mode 100644 docker/hailo8l/pyhailort_build_scripts/setup.py diff --git a/docker/hailo8l/Dockerfile b/docker/hailo8l/Dockerfile index 479ef9b27..701c81d0f 100644 --- a/docker/hailo8l/Dockerfile +++ b/docker/hailo8l/Dockerfile @@ -2,6 +2,9 @@ ARG DEBIAN_FRONTEND=noninteractive +# NOTE: also update user_installation.sh +ARG HAILO_VERSION=4.19.0 + # Build Python wheels FROM wheels AS h8l-wheels @@ -19,6 +22,7 @@ RUN pip3 wheel --wheel-dir=/h8l-wheels -c /requirements-wheels.txt -r /requireme # Build HailoRT and create wheel FROM wheels AS build-hailort ARG TARGETARCH +ARG HAILO_VERSION SHELL ["/bin/bash", "-c"] @@ -50,55 +54,35 @@ RUN PYTHON_VERSION=$(python3 --version 2>&1 | awk '{print $2}' | cut -d. -f1,2) RUN . /etc/environment && \ git clone https://github.com/hailo-ai/hailort.git /opt/hailort && \ cd /opt/hailort && \ - git checkout v4.18.0 && \ - cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Release -DHAILO_BUILD_PYBIND=1 -DPYBIND11_PYTHON_VERSION=${PYTHON_VERSION} && \ + git checkout v${HAILO_VERSION} && \ + cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Release && \ cmake --build build --config release --target libhailort && \ - cmake --build build --config release --target _pyhailort && \ - cp build/hailort/libhailort/bindings/python/src/_pyhailort.cpython-${PYTHON_VERSION_NO_DOT}-$(if [ $TARGETARCH == "amd64" ]; then echo 'x86_64'; else echo 'aarch64'; fi )-linux-gnu.so hailort/libhailort/bindings/python/platform/hailo_platform/pyhailort/ && \ - cp build/hailort/libhailort/src/libhailort.so hailort/libhailort/bindings/python/platform/hailo_platform/pyhailort/ - -RUN ls -ahl /opt/hailort/build/hailort/libhailort/src/ -RUN ls -ahl /opt/hailort/hailort/libhailort/bindings/python/platform/hailo_platform/pyhailort/ - -# Remove the existing setup.py if it exists in the target directory -RUN rm -f /opt/hailort/hailort/libhailort/bindings/python/platform/setup.py - -# Copy generate_wheel_conf.py and setup.py -COPY docker/hailo8l/pyhailort_build_scripts/generate_wheel_conf.py /opt/hailort/hailort/libhailort/bindings/python/platform/generate_wheel_conf.py -COPY docker/hailo8l/pyhailort_build_scripts/setup.py /opt/hailort/hailort/libhailort/bindings/python/platform/setup.py - -# Run the generate_wheel_conf.py script -RUN python3 /opt/hailort/hailort/libhailort/bindings/python/platform/generate_wheel_conf.py + cmake --build build --config release --target hailortcli && \ + cmake --build build --config release --target install # Create a wheel file using pip3 wheel RUN cd /opt/hailort/hailort/libhailort/bindings/python/platform && \ python3 setup.py bdist_wheel --dist-dir /hailo-wheels +RUN mkdir -p /rootfs/usr/local/lib /rootfs/usr/local/bin && \ + cp /usr/local/lib/libhailort.so* /rootfs/usr/local/lib && \ + cp /usr/local/bin/hailortcli /rootfs/usr/local/bin + # Use deps as the base image FROM deps AS h8l-frigate +ARG HAILO_VERSION # Copy the wheels from the wheels stage COPY --from=h8l-wheels /h8l-wheels /deps/h8l-wheels COPY --from=build-hailort /hailo-wheels /deps/hailo-wheels -COPY --from=build-hailort /etc/environment /etc/environment -RUN CC=$(python3 -c "import sysconfig; import shlex; cc = sysconfig.get_config_var('CC'); cc_cmd = shlex.split(cc)[0]; print(cc_cmd[:-4] if cc_cmd.endswith('-gcc') else cc_cmd)") && \ - echo "CC=$CC" >> /etc/environment +COPY --from=build-hailort /rootfs/ / # Install the wheels RUN pip3 install -U /deps/h8l-wheels/*.whl RUN pip3 install -U /deps/hailo-wheels/*.whl -RUN . /etc/environment && \ - mv /usr/local/lib/python${PYTHON_VERSION}/dist-packages/hailo_platform/pyhailort/libhailort.so /usr/lib/${CC} && \ - cd /usr/lib/${CC}/ && \ - ln -s libhailort.so libhailort.so.4.18.0 - # Copy base files from the rootfs stage COPY --from=rootfs / / -# Set environment variables for Hailo SDK -ENV PATH="/opt/hailort/bin:${PATH}" -ENV LD_LIBRARY_PATH="/usr/lib/$(if [ $TARGETARCH == "amd64" ]; then echo 'x86_64'; else echo 'aarch64'; fi )-linux-gnu:${LD_LIBRARY_PATH}" - # Set workdir WORKDIR /opt/frigate/ diff --git a/docker/hailo8l/pyhailort_build_scripts/generate_wheel_conf.py b/docker/hailo8l/pyhailort_build_scripts/generate_wheel_conf.py deleted file mode 100644 index a0e4987f1..000000000 --- a/docker/hailo8l/pyhailort_build_scripts/generate_wheel_conf.py +++ /dev/null @@ -1,67 +0,0 @@ -import json -import os -import platform -import sys -import sysconfig - - -def extract_toolchain_info(compiler): - # Remove the "-gcc" or "-g++" suffix if present - if compiler.endswith("-gcc") or compiler.endswith("-g++"): - compiler = compiler.rsplit("-", 1)[0] - - # Extract the toolchain and ABI part (e.g., "gnu") - toolchain_parts = compiler.split("-") - abi_conventions = next( - (part for part in toolchain_parts if part in ["gnu", "musl", "eabi", "uclibc"]), - "", - ) - - return abi_conventions - - -def generate_wheel_conf(): - conf_file_path = os.path.join( - os.path.abspath(os.path.dirname(__file__)), "wheel_conf.json" - ) - - # Extract current system and Python version information - py_version = f"cp{sys.version_info.major}{sys.version_info.minor}" - arch = platform.machine() - system = platform.system().lower() - libc_version = platform.libc_ver()[1] - - # Get the compiler information - compiler = sysconfig.get_config_var("CC") - abi_conventions = extract_toolchain_info(compiler) - - # Create the new configuration data - new_conf_data = { - "py_version": py_version, - "arch": arch, - "system": system, - "libc_version": libc_version, - "abi": abi_conventions, - "extension": { - "posix": "so", - "nt": "pyd", # Windows - }[os.name], - } - - # If the file exists, load the existing data - if os.path.isfile(conf_file_path): - with open(conf_file_path, "r") as conf_file: - conf_data = json.load(conf_file) - # Update the existing data with the new data - conf_data.update(new_conf_data) - else: - # If the file does not exist, use the new data - conf_data = new_conf_data - - # Write the updated data to the file - with open(conf_file_path, "w") as conf_file: - json.dump(conf_data, conf_file, indent=4) - - -if __name__ == "__main__": - generate_wheel_conf() diff --git a/docker/hailo8l/pyhailort_build_scripts/setup.py b/docker/hailo8l/pyhailort_build_scripts/setup.py deleted file mode 100644 index 2abe07ee5..000000000 --- a/docker/hailo8l/pyhailort_build_scripts/setup.py +++ /dev/null @@ -1,111 +0,0 @@ -import json -import os - -from setuptools import find_packages, setup -from wheel.bdist_wheel import bdist_wheel as orig_bdist_wheel - - -class NonPurePythonBDistWheel(orig_bdist_wheel): - """Makes the wheel platform-dependent so it can be based on the _pyhailort architecture""" - - def finalize_options(self): - orig_bdist_wheel.finalize_options(self) - self.root_is_pure = False - - -def _get_hailort_lib_path(): - lib_filename = "libhailort.so" - lib_path = os.path.join( - os.path.abspath(os.path.dirname(__file__)), - f"hailo_platform/pyhailort/{lib_filename}", - ) - if os.path.exists(lib_path): - print(f"Found libhailort shared library at: {lib_path}") - else: - print(f"Error: libhailort shared library not found at: {lib_path}") - raise FileNotFoundError(f"libhailort shared library not found at: {lib_path}") - return lib_path - - -def _get_pyhailort_lib_path(): - conf_file_path = os.path.join( - os.path.abspath(os.path.dirname(__file__)), "wheel_conf.json" - ) - if not os.path.isfile(conf_file_path): - raise FileNotFoundError(f"Configuration file not found: {conf_file_path}") - - with open(conf_file_path, "r") as conf_file: - content = json.load(conf_file) - py_version = content["py_version"] - arch = content["arch"] - system = content["system"] - extension = content["extension"] - abi = content["abi"] - - # Construct the filename directly - lib_filename = f"_pyhailort.cpython-{py_version.split('cp')[1]}-{arch}-{system}-{abi}.{extension}" - lib_path = os.path.join( - os.path.abspath(os.path.dirname(__file__)), - f"hailo_platform/pyhailort/{lib_filename}", - ) - - if os.path.exists(lib_path): - print(f"Found _pyhailort shared library at: {lib_path}") - else: - print(f"Error: _pyhailort shared library not found at: {lib_path}") - raise FileNotFoundError( - f"_pyhailort shared library not found at: {lib_path}" - ) - - return lib_path - - -def _get_package_paths(): - packages = [] - pyhailort_lib = _get_pyhailort_lib_path() - hailort_lib = _get_hailort_lib_path() - if pyhailort_lib: - packages.append(pyhailort_lib) - if hailort_lib: - packages.append(hailort_lib) - packages.append(os.path.abspath("hailo_tutorials/notebooks/*")) - packages.append(os.path.abspath("hailo_tutorials/hefs/*")) - return packages - - -if __name__ == "__main__": - setup( - author="Hailo team", - author_email="contact@hailo.ai", - cmdclass={ - "bdist_wheel": NonPurePythonBDistWheel, - }, - description="HailoRT", - entry_points={ - "console_scripts": [ - "hailo=hailo_platform.tools.hailocli.main:main", - ] - }, - install_requires=[ - "argcomplete", - "contextlib2", - "future", - "netaddr", - "netifaces", - "verboselogs", - "numpy==1.23.3", - ], - name="hailort", - package_data={ - "hailo_platform": _get_package_paths(), - }, - packages=find_packages(), - platforms=[ - "linux_x86_64", - "linux_aarch64", - "win_amd64", - ], - url="https://hailo.ai/", - version="4.17.0", - zip_safe=False, - ) diff --git a/docker/hailo8l/user_installation.sh b/docker/hailo8l/user_installation.sh index 853652ffa..2cf44126f 100644 --- a/docker/hailo8l/user_installation.sh +++ b/docker/hailo8l/user_installation.sh @@ -13,7 +13,7 @@ else fi # Clone the HailoRT driver repository -git clone --depth 1 --branch v4.18.0 https://github.com/hailo-ai/hailort-drivers.git +git clone --depth 1 --branch v4.19.0 https://github.com/hailo-ai/hailort-drivers.git # Build and install the HailoRT driver cd hailort-drivers/linux/pcie @@ -38,7 +38,7 @@ cd ../../ if [ ! -d /lib/firmware/hailo ]; then sudo mkdir /lib/firmware/hailo fi -sudo mv hailo8_fw.4.18.0.bin /lib/firmware/hailo/hailo8_fw.bin +sudo mv hailo8_fw.*.bin /lib/firmware/hailo/hailo8_fw.bin # Install udev rules sudo cp ./linux/pcie/51-hailo-udev.rules /etc/udev/rules.d/ From e4a6b292799569b8633da44c1469b513a311c83a Mon Sep 17 00:00:00 2001 From: Evan Jarrett Date: Wed, 30 Oct 2024 04:05:58 -0700 Subject: [PATCH 108/479] fix string comparison on mqtt error message for Server unavailable (#14675) --- frigate/comms/mqtt.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/comms/mqtt.py b/frigate/comms/mqtt.py index 198ec2176..5a85a710b 100644 --- a/frigate/comms/mqtt.py +++ b/frigate/comms/mqtt.py @@ -133,7 +133,7 @@ class MqttClient(Communicator): # type: ignore[misc] """Mqtt connection callback.""" threading.current_thread().name = "mqtt" if reason_code != 0: - if reason_code == "Server Unavailable": + if reason_code == "Server unavailable": logger.error( "Unable to connect to MQTT server: MQTT Server unavailable" ) From bb80a7b2ee683165e1ca45da494cfd79ce8fff8d Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 30 Oct 2024 06:54:06 -0500 Subject: [PATCH 109/479] UI changes and bugfixes (#14669) * Home/End buttons for search input and max 8 search columns * Fix lifecycle label * remove video tab if tracked object has no clip * hide object lifecycle if there is no clip * add test for filter value to ensure only fully numeric values are set as numbers --- web/src/components/input/InputWithTags.tsx | 16 ++++++++++++++-- .../overlay/detail/ObjectLifecycle.tsx | 2 +- .../overlay/detail/SearchDetailDialog.tsx | 7 ++++++- web/src/components/settings/SearchSettings.tsx | 2 +- web/src/hooks/use-api-filter.ts | 6 +++++- web/src/views/search/SearchView.tsx | 2 ++ 6 files changed, 29 insertions(+), 6 deletions(-) diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 29a6f8a31..becc0f4e1 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -523,17 +523,29 @@ export default function InputWithTags({ const handleInputKeyDown = useCallback( (e: React.KeyboardEvent) => { + const event = e.target as HTMLInputElement; + + if (!currentFilterType && (e.key === "Home" || e.key === "End")) { + const position = e.key === "Home" ? 0 : event.value.length; + event.setSelectionRange(position, position); + } + if ( e.key === "Enter" && inputValue.trim() !== "" && filterSuggestions(suggestions).length == 0 ) { e.preventDefault(); - handleSearch(inputValue); } }, - [inputValue, handleSearch, filterSuggestions, suggestions], + [ + inputValue, + handleSearch, + filterSuggestions, + suggestions, + currentFilterType, + ], ); // effects diff --git a/web/src/components/overlay/detail/ObjectLifecycle.tsx b/web/src/components/overlay/detail/ObjectLifecycle.tsx index fee15af4a..d770f7432 100644 --- a/web/src/components/overlay/detail/ObjectLifecycle.tsx +++ b/web/src/components/overlay/detail/ObjectLifecycle.tsx @@ -641,7 +641,7 @@ function getLifecycleItemDescription(lifecycleItem: ObjectLifecycleSequence) { )} detected for ${label}`; } else { title = `${ - lifecycleItem.data.sub_label + lifecycleItem.data.label } recognized as ${lifecycleItem.data.attribute.replaceAll("_", " ")}`; } return title; diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 9159813f4..1ff784e6d 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -141,7 +141,12 @@ export default function SearchDetailDialog({ views.splice(index, 1); } - if (search.data.type != "object") { + if (!search.has_clip) { + const index = views.indexOf("video"); + views.splice(index, 1); + } + + if (search.data.type != "object" || !search.has_clip) { const index = views.indexOf("object lifecycle"); views.splice(index, 1); } diff --git a/web/src/components/settings/SearchSettings.tsx b/web/src/components/settings/SearchSettings.tsx index d10189ede..17d319ec5 100644 --- a/web/src/components/settings/SearchSettings.tsx +++ b/web/src/components/settings/SearchSettings.tsx @@ -99,7 +99,7 @@ export default function SearchSettings({ setColumns(value)} - max={6} + max={8} min={2} step={1} className="flex-grow" diff --git a/web/src/hooks/use-api-filter.ts b/web/src/hooks/use-api-filter.ts index 1048c87e7..5c707a315 100644 --- a/web/src/hooks/use-api-filter.ts +++ b/web/src/hooks/use-api-filter.ts @@ -65,7 +65,11 @@ export function useApiFilterArgs< const filter: { [key: string]: unknown } = {}; rawParams.forEach((value, key) => { - if (value != "true" && value != "false" && isNaN(parseFloat(value))) { + if ( + value != "true" && + value != "false" && + (/[^0-9,]/.test(value) || isNaN(parseFloat(value))) + ) { filter[key] = value.includes(",") ? value.split(",") : [value]; } else { if (value != undefined) { diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 5fd6c98fa..4d81a40f7 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -83,6 +83,8 @@ export default function SearchView({ "sm:grid-cols-4": columns === 4, "sm:grid-cols-5": columns === 5, "sm:grid-cols-6": columns === 6, + "sm:grid-cols-7": columns === 7, + "sm:grid-cols-8": columns === 8, }, ); From ab26aee8b23a311344af8c294587f248dbe2d1a4 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 06:16:56 -0600 Subject: [PATCH 110/479] Fix config loading (#14684) --- frigate/config/config.py | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/frigate/config/config.py b/frigate/config/config.py index b2373fdcc..db5e06fce 100644 --- a/frigate/config/config.py +++ b/frigate/config/config.py @@ -67,7 +67,7 @@ logger = logging.getLogger(__name__) yaml = YAML() -DEFAULT_CONFIG_FILES = ["/config/config.yaml", "/config/config.yml"] +DEFAULT_CONFIG_FILE = "/config/config.yml" DEFAULT_CONFIG = """ mqtt: enabled: False @@ -634,20 +634,16 @@ class FrigateConfig(FrigateBaseModel): @classmethod def load(cls, **kwargs): - config_path = os.environ.get("CONFIG_FILE") + config_path = os.environ.get("CONFIG_FILE", DEFAULT_CONFIG_FILE) - # No explicit configuration file, try to find one in the default paths. - if config_path is None: - for path in DEFAULT_CONFIG_FILES: - if os.path.isfile(path): - config_path = path - break + if not os.path.isfile(config_path): + config_path = config_path.replace("yml", "yaml") # No configuration file found, create one. new_config = False - if config_path is None: + if not os.path.isfile(config_path): logger.info("No config file found, saving default config") - config_path = DEFAULT_CONFIG_FILES[-1] + config_path = DEFAULT_CONFIG_FILE new_config = True else: # Check if the config file needs to be migrated. From d10fea601262394f36d0c1ad98f7787623ced20c Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 06:23:10 -0600 Subject: [PATCH 111/479] Add specific section about GPU in semantic search (#14685) --- docs/docs/configuration/semantic_search.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 7f84fdf95..22c3ddfe9 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -43,12 +43,6 @@ The text model is used to embed tracked object descriptions and perform searches Differently weighted CLIP models are available and can be selected by setting the `model_size` config option: -:::tip - -The CLIP models are downloaded in ONNX format, which means they will be accelerated using GPU hardware when available. This depends on the Docker build that is used. See [the object detector docs](../configuration/object_detectors.md) for more information. - -::: - ```yaml semantic_search: enabled: True @@ -58,6 +52,18 @@ semantic_search: - Configuring the `large` model employs the full Jina model and will automatically run on the GPU if applicable. - Configuring the `small` model employs a quantized version of the model that uses much less RAM and runs faster on CPU with a very negligible difference in embedding quality. +### GPU Acceleration + +The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used, see [the object detector docs](../configuration/object_detectors.md) for more information. + +If the correct build is used for your GPU and the `large` model is configured, then the GPU will be automatically detected and used automatically. + +```yaml +semantic_search: + enabled: True + model_size: small +``` + ## Usage and Best Practices 1. Semantic search is used in conjunction with the other filters available on the Search page. Use a combination of traditional filtering and semantic search for the best results. From fffd9defeab9c381134ef7505d03811c90f23ead Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 06:30:00 -0600 Subject: [PATCH 112/479] Add docs update to type of change (#14686) --- .github/pull_request_template.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index db3e5541e..30d4dab7b 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -13,6 +13,7 @@ - [ ] New feature - [ ] Breaking change (fix/feature causing existing functionality to break) - [ ] Code quality improvements to existing code +- [ ] Documentation Update ## Additional information From 89ca085b94afcaa60abe092a2dd50272475d74a0 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 06:41:58 -0600 Subject: [PATCH 113/479] Add info about GPUs that are supported for semantic search (#14687) * Add specific information about GPUs that are supported for semantic search * clarity --- docs/docs/configuration/semantic_search.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 22c3ddfe9..2819f2a4c 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -54,9 +54,22 @@ semantic_search: ### GPU Acceleration -The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used, see [the object detector docs](../configuration/object_detectors.md) for more information. +The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used. -If the correct build is used for your GPU and the `large` model is configured, then the GPU will be automatically detected and used automatically. +:::info + +If the correct build is used for your GPU and the `large` model is configured, then the GPU will be detected and used automatically. + +**AMD** +- ROCm will automatically be detected and used for semantic search in the `-rocm` Frigate image. + +**Intel** +- OpenVINO will automatically be detected and used as a detector in the default Frigate image. + +**Nvidia** +- Nvidia GPUs will automatically be detected and used as a detector in the `-tensorrt` Frigate image. + +::: ```yaml semantic_search: From 03dd9b2d42e2d39a456d84b3745b6ed082bc4ca1 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 08:22:20 -0600 Subject: [PATCH 114/479] Don't open file with read permissions if there is no need to write to it (#14689) --- frigate/config/config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/config/config.py b/frigate/config/config.py index db5e06fce..8fbb9ec6c 100644 --- a/frigate/config/config.py +++ b/frigate/config/config.py @@ -650,7 +650,7 @@ class FrigateConfig(FrigateBaseModel): migrate_frigate_config(config_path) # Finally, load the resulting configuration file. - with open(config_path, "a+") as f: + with open(config_path, "a+" if new_config else "r") as f: # Only write the default config if the opened file is non-empty. This can happen as # a race condition. It's extremely unlikely, but eh. Might as well check it. if new_config and f.tell() == 0: From c7a4220d656db9dcd8e1f5393d5892a1c0bcee41 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 08:22:28 -0600 Subject: [PATCH 115/479] Jetson onnxruntime (#14688) * Add support for using onnx runtime with jetson * Update docs * Clarify --- docker/tensorrt/Dockerfile.arm64 | 7 ++++--- docker/tensorrt/requirements-arm64.txt | 1 + docs/docs/configuration/object_detectors.md | 6 +++--- docs/docs/configuration/semantic_search.md | 1 + 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docker/tensorrt/Dockerfile.arm64 b/docker/tensorrt/Dockerfile.arm64 index 70184bf9b..286a0af55 100644 --- a/docker/tensorrt/Dockerfile.arm64 +++ b/docker/tensorrt/Dockerfile.arm64 @@ -10,8 +10,8 @@ ARG DEBIAN_FRONTEND # Use a separate container to build wheels to prevent build dependencies in final image RUN apt-get -qq update \ && apt-get -qq install -y --no-install-recommends \ - python3.9 python3.9-dev \ - wget build-essential cmake git \ + python3.9 python3.9-dev \ + wget build-essential cmake git \ && rm -rf /var/lib/apt/lists/* # Ensure python3 defaults to python3.9 @@ -41,7 +41,8 @@ RUN --mount=type=bind,source=docker/tensorrt/detector/build_python_tensorrt.sh,t && TENSORRT_VER=$(cat /etc/TENSORRT_VER) /deps/build_python_tensorrt.sh COPY docker/tensorrt/requirements-arm64.txt /requirements-tensorrt.txt -RUN pip3 wheel --wheel-dir=/trt-wheels -r /requirements-tensorrt.txt +RUN pip3 uninstall -y onnxruntime \ + && pip3 wheel --wheel-dir=/trt-wheels -r /requirements-tensorrt.txt FROM build-wheels AS trt-model-wheels ARG DEBIAN_FRONTEND diff --git a/docker/tensorrt/requirements-arm64.txt b/docker/tensorrt/requirements-arm64.txt index 9b12dac33..93f100dd1 100644 --- a/docker/tensorrt/requirements-arm64.txt +++ b/docker/tensorrt/requirements-arm64.txt @@ -1 +1,2 @@ cuda-python == 11.7; platform_machine == 'aarch64' +onnxruntime @ https://nvidia.box.com/shared/static/9aemm4grzbbkfaesg5l7fplgjtmswhj8.whl; platform_machine == 'aarch64' diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index d4cee196d..5896260f4 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -22,14 +22,14 @@ Frigate supports multiple different detectors that work on different types of ha - [ONNX](#onnx): OpenVINO will automatically be detected and used as a detector in the default Frigate image when a supported ONNX model is configured. **Nvidia** -- [TensortRT](#nvidia-tensorrt-detector): TensorRT can run on Nvidia GPUs, using one of many default models. -- [ONNX](#onnx): TensorRT will automatically be detected and used as a detector in the `-tensorrt` Frigate image when a supported ONNX model is configured. +- [TensortRT](#nvidia-tensorrt-detector): TensorRT can run on Nvidia GPUs and Jetson devices, using one of many default models. +- [ONNX](#onnx): TensorRT will automatically be detected and used as a detector in the `-tensorrt` or `-tensorrt-jp(4/5)` Frigate images when a supported ONNX model is configured. **Rockchip** - [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs. **For Testing** -- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results. +- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results. ::: diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 2819f2a4c..8abd761a8 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -68,6 +68,7 @@ If the correct build is used for your GPU and the `large` model is configured, t **Nvidia** - Nvidia GPUs will automatically be detected and used as a detector in the `-tensorrt` Frigate image. +- Jetson devices will automatically be detected and used as a detector in the `-tensorrt-jp(4/5)` Frigate image. ::: From bb4e863e875574626ef6e1bdbb9afcf488f63de4 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 30 Oct 2024 18:16:28 -0600 Subject: [PATCH 116/479] Fix jetson onnxruntime (#14698) * Fix jetson onnxruntime * Remove comment --- docker/tensorrt/Dockerfile.arm64 | 7 +++++-- docker/tensorrt/requirements-arm64.txt | 3 +-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/docker/tensorrt/Dockerfile.arm64 b/docker/tensorrt/Dockerfile.arm64 index 286a0af55..23a2459ac 100644 --- a/docker/tensorrt/Dockerfile.arm64 +++ b/docker/tensorrt/Dockerfile.arm64 @@ -41,8 +41,11 @@ RUN --mount=type=bind,source=docker/tensorrt/detector/build_python_tensorrt.sh,t && TENSORRT_VER=$(cat /etc/TENSORRT_VER) /deps/build_python_tensorrt.sh COPY docker/tensorrt/requirements-arm64.txt /requirements-tensorrt.txt -RUN pip3 uninstall -y onnxruntime \ - && pip3 wheel --wheel-dir=/trt-wheels -r /requirements-tensorrt.txt +ADD https://nvidia.box.com/shared/static/9aemm4grzbbkfaesg5l7fplgjtmswhj8.whl /tmp/onnxruntime_gpu-1.15.1-cp39-cp39-linux_aarch64.whl + +RUN pip3 uninstall -y onnxruntime-openvino \ + && pip3 wheel --wheel-dir=/trt-wheels -r /requirements-tensorrt.txt \ + && pip3 install --no-deps /tmp/onnxruntime_gpu-1.15.1-cp39-cp39-linux_aarch64.whl FROM build-wheels AS trt-model-wheels ARG DEBIAN_FRONTEND diff --git a/docker/tensorrt/requirements-arm64.txt b/docker/tensorrt/requirements-arm64.txt index 93f100dd1..67489f80b 100644 --- a/docker/tensorrt/requirements-arm64.txt +++ b/docker/tensorrt/requirements-arm64.txt @@ -1,2 +1 @@ -cuda-python == 11.7; platform_machine == 'aarch64' -onnxruntime @ https://nvidia.box.com/shared/static/9aemm4grzbbkfaesg5l7fplgjtmswhj8.whl; platform_machine == 'aarch64' +cuda-python == 11.7; platform_machine == 'aarch64' \ No newline at end of file From 885485da70f8a2d4b67b32a02e9d950476374283 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 31 Oct 2024 06:58:33 -0500 Subject: [PATCH 117/479] Small tweaks (#14700) * Remove extra spacing for next/prev carousel buttons * Clarify ollama genai docs --- docs/docs/configuration/genai.md | 6 +++--- web/src/components/overlay/detail/ObjectLifecycle.tsx | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index 8e2e4fbc2..2ee27f724 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -31,15 +31,15 @@ cameras: :::warning -Using Ollama on CPU is not recommended, high inference times make using generative AI impractical. +Using Ollama on CPU is not recommended, high inference times and a lack of support for multi-modal parallel requests will make using Generative AI impractical. ::: [Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It provides a nice API over [llama.cpp](https://github.com/ggerganov/llama.cpp). It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance. -Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [docker container](https://hub.docker.com/r/ollama/ollama) available. +Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available. -Parallel requests also come with some caveats. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests). +Parallel requests also come with some caveats, and multi-modal parallel requests are currently not supported by Ollama. Depending on your hardware and the number of requests made to the Ollama API, these limitations may prevent Ollama from being an optimal solution for many users. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests). ### Supported Models diff --git a/web/src/components/overlay/detail/ObjectLifecycle.tsx b/web/src/components/overlay/detail/ObjectLifecycle.tsx index d770f7432..bebf5e2f5 100644 --- a/web/src/components/overlay/detail/ObjectLifecycle.tsx +++ b/web/src/components/overlay/detail/ObjectLifecycle.tsx @@ -509,7 +509,7 @@ export default function ObjectLifecycle({ containScroll: "keepSnaps", dragFree: true, }} - className="w-full max-w-[72%] md:max-w-[85%]" + className="max-w-[72%] md:max-w-[85%]" setApi={setThumbnailApi} > Date: Thu, 31 Oct 2024 06:31:01 -0600 Subject: [PATCH 118/479] Various fixes (#14703) * Fix not retaining custom events * Fix media apis --- frigate/api/media.py | 7 +++---- frigate/config/camera/record.py | 7 +++++++ frigate/events/external.py | 2 +- frigate/record/maintainer.py | 6 +----- 4 files changed, 12 insertions(+), 10 deletions(-) diff --git a/frigate/api/media.py b/frigate/api/media.py index 9dc3411d9..dcfc44f89 100644 --- a/frigate/api/media.py +++ b/frigate/api/media.py @@ -917,7 +917,7 @@ def grid_snapshot( ret, jpg = cv2.imencode(".jpg", frame, [int(cv2.IMWRITE_JPEG_QUALITY), 70]) return Response( - jpg.tobytes, + jpg.tobytes(), media_type="image/jpeg", headers={"Cache-Control": "no-store"}, ) @@ -1453,7 +1453,6 @@ def preview_thumbnail(file_name: str): return Response( jpg_bytes, - # FIXME: Shouldn't it be either jpg or webp depending on the endpoint? media_type="image/webp", headers={ "Content-Type": "image/webp", @@ -1482,7 +1481,7 @@ def label_thumbnail(request: Request, camera_name: str, label: str): ret, jpg = cv2.imencode(".jpg", frame, [int(cv2.IMWRITE_JPEG_QUALITY), 70]) return Response( - jpg.tobytes, + jpg.tobytes(), media_type="image/jpeg", headers={"Cache-Control": "no-store"}, ) @@ -1535,6 +1534,6 @@ def label_snapshot(request: Request, camera_name: str, label: str): _, jpg = cv2.imencode(".jpg", frame, [int(cv2.IMWRITE_JPEG_QUALITY), 70]) return Response( - jpg.tobytes, + jpg.tobytes(), media_type="image/jpeg", ) diff --git a/frigate/config/camera/record.py b/frigate/config/camera/record.py index 3db61c569..dec629b6b 100644 --- a/frigate/config/camera/record.py +++ b/frigate/config/camera/record.py @@ -94,3 +94,10 @@ class RecordConfig(FrigateBaseModel): enabled_in_config: Optional[bool] = Field( default=None, title="Keep track of original state of recording." ) + + @property + def event_pre_capture(self) -> int: + return max( + self.alerts.pre_capture, + self.detections.pre_capture, + ) diff --git a/frigate/events/external.py b/frigate/events/external.py index edfb757a0..76b9e3208 100644 --- a/frigate/events/external.py +++ b/frigate/events/external.py @@ -70,7 +70,7 @@ class ExternalEventProcessor: "sub_label": sub_label, "score": score, "camera": camera, - "start_time": now, + "start_time": now - camera_config.record.event_pre_capture, "end_time": end, "thumbnail": thumbnail, "has_clip": camera_config.record.enabled and include_recording, diff --git a/frigate/record/maintainer.py b/frigate/record/maintainer.py index 314ff3646..e97fb0a44 100644 --- a/frigate/record/maintainer.py +++ b/frigate/record/maintainer.py @@ -299,16 +299,12 @@ class RecordingMaintainer(threading.Thread): # if it doesn't overlap with an event, go ahead and drop the segment # if it ends more than the configured pre_capture for the camera else: - pre_capture = max( - record_config.alerts.pre_capture, - record_config.detections.pre_capture, - ) camera_info = self.object_recordings_info[camera] most_recently_processed_frame_time = ( camera_info[-1][0] if len(camera_info) > 0 else 0 ) retain_cutoff = datetime.datetime.fromtimestamp( - most_recently_processed_frame_time - pre_capture + most_recently_processed_frame_time - record_config.event_pre_capture ).astimezone(datetime.timezone.utc) if end_time < retain_cutoff: Path(cache_path).unlink(missing_ok=True) From 9e1a50c3beadfdfed40838869ccb9fdf723c8d65 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 31 Oct 2024 14:48:26 -0500 Subject: [PATCH 119/479] Clean up copy output (#14705) * Remove extra spacing for next/prev carousel buttons * Clarify ollama genai docs * Clean up copied gpu info output * Clean up copied gpu info output * Better display when manually copying/pasting log data --- web/src/components/overlay/GPUInfoDialog.tsx | 10 +-- web/src/pages/Logs.tsx | 86 +++++++++++++++++++- web/src/pages/System.tsx | 2 + 3 files changed, 89 insertions(+), 9 deletions(-) diff --git a/web/src/components/overlay/GPUInfoDialog.tsx b/web/src/components/overlay/GPUInfoDialog.tsx index 957d1e681..3821579e2 100644 --- a/web/src/components/overlay/GPUInfoDialog.tsx +++ b/web/src/components/overlay/GPUInfoDialog.tsx @@ -10,6 +10,7 @@ import ActivityIndicator from "../indicators/activity-indicator"; import { GpuInfo, Nvinfo, Vainfo } from "@/types/stats"; import { Button } from "../ui/button"; import copy from "copy-to-clipboard"; +import { toast } from "sonner"; type GPUInfoDialogProps = { showGpuInfo: boolean; @@ -30,12 +31,11 @@ export default function GPUInfoDialog({ const onCopyInfo = async () => { copy( - JSON.stringify(gpuType == "vainfo" ? vainfo : nvinfo).replace( - /[\\\s]+/gi, - "", - ), + JSON.stringify(gpuType == "vainfo" ? vainfo : nvinfo) + .replace(/\\t/g, "\t") + .replace(/\\n/g, "\n"), ); - setShowGpuInfo(false); + toast.success("Copied GPU info to clipboard."); }; if (gpuType == "vainfo") { diff --git a/web/src/pages/Logs.tsx b/web/src/pages/Logs.tsx index 8178e9e90..949fffb8a 100644 --- a/web/src/pages/Logs.tsx +++ b/web/src/pages/Logs.tsx @@ -300,6 +300,84 @@ function Logs() { }, ); + useEffect(() => { + const handleCopy = (e: ClipboardEvent) => { + e.preventDefault(); + if (!contentRef.current) return; + + const selection = window.getSelection(); + if (!selection) return; + + const range = selection.getRangeAt(0); + const fragment = range.cloneContents(); + + const extractLogData = (element: Element) => { + const severity = + element.querySelector(".log-severity")?.textContent?.trim() || ""; + const dateStamp = + element.querySelector(".log-timestamp")?.textContent?.trim() || ""; + const section = + element.querySelector(".log-section")?.textContent?.trim() || ""; + const content = + element.querySelector(".log-content")?.textContent?.trim() || ""; + + return { severity, dateStamp, section, content }; + }; + + let copyData: { + severity: string; + dateStamp: string; + section: string; + content: string; + }[] = []; + + if (fragment.querySelectorAll(".grid").length > 0) { + // Multiple grid elements + copyData = Array.from(fragment.querySelectorAll(".grid")).map( + extractLogData, + ); + } else { + // Try to find the closest grid element or use the first child element + const gridElement = + fragment.querySelector(".grid") || (fragment.firstChild as Element); + + if (gridElement) { + const data = extractLogData(gridElement); + if (data.severity || data.dateStamp || data.section || data.content) { + copyData.push(data); + } + } + } + + if (copyData.length === 0) return; // No valid data to copy + + // Calculate maximum widths for each column + const maxWidths = { + severity: Math.max(...copyData.map((d) => d.severity.length)), + dateStamp: Math.max(...copyData.map((d) => d.dateStamp.length)), + section: Math.max(...copyData.map((d) => d.section.length)), + }; + + const pad = (str: string, length: number) => str.padEnd(length, " "); + + // Create the formatted copy text + const copyText = copyData + .map( + (d) => + `${pad(d.severity, maxWidths.severity)} | ${pad(d.dateStamp, maxWidths.dateStamp)} | ${pad(d.section, maxWidths.section)} | ${d.content}`, + ) + .join("\n"); + + e.clipboardData?.setData("text/plain", copyText); + }; + + const content = contentRef.current; + content?.addEventListener("copy", handleCopy); + return () => { + content?.removeEventListener("copy", handleCopy); + }; + }, []); + return (
@@ -467,18 +545,18 @@ function LogLineData({ )} onClick={onSelect} > -
+
-
+
{line.dateStamp}
-
+
{line.section}
-
+
{line.content}
diff --git a/web/src/pages/System.tsx b/web/src/pages/System.tsx index ab5c86f6a..23d1b7e6a 100644 --- a/web/src/pages/System.tsx +++ b/web/src/pages/System.tsx @@ -13,6 +13,7 @@ import useOptimisticState from "@/hooks/use-optimistic-state"; import CameraMetrics from "@/views/system/CameraMetrics"; import { useHashState } from "@/hooks/use-overlay-state"; import { capitalizeFirstLetter } from "@/utils/stringUtil"; +import { Toaster } from "@/components/ui/sonner"; const metrics = ["general", "storage", "cameras"] as const; type SystemMetric = (typeof metrics)[number]; @@ -42,6 +43,7 @@ function System() { return (
+
{isMobile && ( From 8c2c07fd18ea68d6da1b72ebd1307a556767c2ff Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 1 Nov 2024 07:37:52 -0500 Subject: [PATCH 120/479] UI tweaks (#14719) * Show activity indicator when search grid is revalidating * improve frigate+ button title grammar --- web/src/components/overlay/detail/SearchDetailDialog.tsx | 8 ++++++-- web/src/components/overlay/dialog/FrigatePlusDialog.tsx | 6 ++++-- web/src/pages/Explore.tsx | 4 +++- web/src/views/search/SearchView.tsx | 5 ++--- 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 1ff784e6d..f158df329 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -655,7 +655,9 @@ function ObjectSnapshotTab({ onSubmitToPlus(false); }} > - This is a {search?.label} + This is{" "} + {/^[aeiou]/i.test(search?.label || "") ? "an" : "a"}{" "} + {search?.label} )} diff --git a/web/src/components/overlay/dialog/FrigatePlusDialog.tsx b/web/src/components/overlay/dialog/FrigatePlusDialog.tsx index 18ada20fc..e98a4164a 100644 --- a/web/src/components/overlay/dialog/FrigatePlusDialog.tsx +++ b/web/src/components/overlay/dialog/FrigatePlusDialog.tsx @@ -144,7 +144,8 @@ export function FrigatePlusDialog({ onSubmitToPlus(false); }} > - This is a {upload?.label} + This is {/^[aeiou]/i.test(upload?.label || "") ? "an" : "a"}{" "} + {upload?.label} )} diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 8989c7b05..9f80241c1 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -428,7 +428,9 @@ export default function Explore() { searchTerm={searchTerm} searchFilter={searchFilter} searchResults={searchResults} - isLoading={(isLoadingInitialData || isLoadingMore) ?? true} + isLoading={ + (isLoadingInitialData || isLoadingMore || isValidating) ?? true + } hasMore={!isReachingEnd} columns={gridColumns} defaultView={defaultView} diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 4d81a40f7..cf6640f18 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -397,11 +397,10 @@ export default function SearchView({
)} - {uniqueResults?.length == 0 && - isLoading && + {isLoading && (searchTerm || (searchFilter && Object.keys(searchFilter).length !== 0)) && ( - + )} {uniqueResults && ( From e5ebf938f6c45c607a9393935901ea7680e7d7e7 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 1 Nov 2024 06:55:55 -0600 Subject: [PATCH 121/479] Fix float input (#14720) --- frigate/object_detection.py | 1 + 1 file changed, 1 insertion(+) diff --git a/frigate/object_detection.py b/frigate/object_detection.py index 0af32034e..cc4641696 100644 --- a/frigate/object_detection.py +++ b/frigate/object_detection.py @@ -90,6 +90,7 @@ class LocalObjectDetector(ObjectDetector): if self.dtype == InputDTypeEnum.float: tensor_input = tensor_input.astype(np.float32) + tensor_input /= 255 return self.detect_api.detect_raw(tensor_input=tensor_input) From 1234003527b0a65a3e0c5eebf3ef975dc3af16eb Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 1 Nov 2024 19:30:40 -0500 Subject: [PATCH 122/479] Fix width of object lifecycle buttons (#14729) --- web/src/components/overlay/detail/ObjectLifecycle.tsx | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/web/src/components/overlay/detail/ObjectLifecycle.tsx b/web/src/components/overlay/detail/ObjectLifecycle.tsx index bebf5e2f5..d687915ca 100644 --- a/web/src/components/overlay/detail/ObjectLifecycle.tsx +++ b/web/src/components/overlay/detail/ObjectLifecycle.tsx @@ -521,10 +521,7 @@ export default function ObjectLifecycle({ {eventSequence.map((item, index) => ( handleThumbnailClick(index)} >
From 11068aa9d035c3833b104954819686cc91f1c153 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 1 Nov 2024 20:52:00 -0500 Subject: [PATCH 123/479] Fix validation activity indicator (#14730) * Don't show two spinners when loading/revalidating search results * clarify --- web/src/pages/Explore.tsx | 5 ++--- web/src/views/search/SearchView.tsx | 7 +++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 9f80241c1..4efcb81b8 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -428,9 +428,8 @@ export default function Explore() { searchTerm={searchTerm} searchFilter={searchFilter} searchResults={searchResults} - isLoading={ - (isLoadingInitialData || isLoadingMore || isValidating) ?? true - } + isLoading={(isLoadingInitialData || isLoadingMore) ?? true} + isValidating={isValidating} hasMore={!isReachingEnd} columns={gridColumns} defaultView={defaultView} diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index cf6640f18..836ae8f4f 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -37,6 +37,7 @@ type SearchViewProps = { searchFilter?: SearchFilter; searchResults?: SearchResult[]; isLoading: boolean; + isValidating: boolean; hasMore: boolean; columns: number; defaultView?: string; @@ -55,6 +56,7 @@ export default function SearchView({ searchFilter, searchResults, isLoading, + isValidating, hasMore, columns, defaultView = "summary", @@ -397,8 +399,9 @@ export default function SearchView({
)} - {isLoading && - (searchTerm || + {((isLoading && uniqueResults?.length == 0) || // show on initial load + (isValidating && !isLoading)) && // or revalidation + (searchTerm || // or change of filter/search term (searchFilter && Object.keys(searchFilter).length !== 0)) && ( )} From d7935abc14245eef33592dae1b4585cf9142596c Mon Sep 17 00:00:00 2001 From: joshjryan Date: Fri, 1 Nov 2024 20:01:38 -0600 Subject: [PATCH 124/479] Set the loglevel for OpenCV ffmpeg messages to fatal (#14728) * Set the loglevel for OpenCV ffmpeg messages to fatal * Set OPENCV_FFMPEG_LOGLEVEL in Dockerfile --- docker/main/Dockerfile | 3 +++ frigate/util/services.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/docker/main/Dockerfile b/docker/main/Dockerfile index ac4d277bd..9d2f442f0 100644 --- a/docker/main/Dockerfile +++ b/docker/main/Dockerfile @@ -211,6 +211,9 @@ ENV TOKENIZERS_PARALLELISM=true # https://github.com/huggingface/transformers/issues/27214 ENV TRANSFORMERS_NO_ADVISORY_WARNINGS=1 +# Set OpenCV ffmpeg loglevel to fatal: https://ffmpeg.org/doxygen/trunk/log_8h.html +ENV OPENCV_FFMPEG_LOGLEVEL=8 + ENV PATH="/usr/local/go2rtc/bin:/usr/local/tempio/bin:/usr/local/nginx/sbin:${PATH}" ENV LIBAVFORMAT_VERSION_MAJOR=60 diff --git a/frigate/util/services.py b/frigate/util/services.py index a71729263..9ee6da999 100644 --- a/frigate/util/services.py +++ b/frigate/util/services.py @@ -584,7 +584,7 @@ async def get_video_properties( width = height = 0 try: - # Open the video stream + # Open the video stream using OpenCV video = cv2.VideoCapture(url) # Check if the video stream was opened successfully From 27ef661fec25ff6d42fcafba11f238eaedc8a8c6 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Sat, 2 Nov 2024 07:13:28 -0500 Subject: [PATCH 125/479] simplify hailort (#14734) --- docker/hailo8l/Dockerfile | 58 +++---------------------------- docker/hailo8l/h8l.hcl | 7 ++++ docker/hailo8l/install_hailort.sh | 21 +++++++++++ 3 files changed, 33 insertions(+), 53 deletions(-) create mode 100755 docker/hailo8l/install_hailort.sh diff --git a/docker/hailo8l/Dockerfile b/docker/hailo8l/Dockerfile index 701c81d0f..959e7692e 100644 --- a/docker/hailo8l/Dockerfile +++ b/docker/hailo8l/Dockerfile @@ -2,9 +2,6 @@ ARG DEBIAN_FRONTEND=noninteractive -# NOTE: also update user_installation.sh -ARG HAILO_VERSION=4.19.0 - # Build Python wheels FROM wheels AS h8l-wheels @@ -19,63 +16,18 @@ RUN mkdir /h8l-wheels # Build the wheels RUN pip3 wheel --wheel-dir=/h8l-wheels -c /requirements-wheels.txt -r /requirements-wheels-h8l.txt -# Build HailoRT and create wheel -FROM wheels AS build-hailort +FROM wget AS hailort ARG TARGETARCH -ARG HAILO_VERSION - -SHELL ["/bin/bash", "-c"] - -# Install necessary APT packages -RUN apt-get -qq update \ - && apt-get -qq install -y \ - apt-transport-https \ - gnupg \ - wget \ - # the key fingerprint can be obtained from https://ftp-master.debian.org/keys.html - && wget -qO- "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xA4285295FC7B1A81600062A9605C66F00D6C9793" | \ - gpg --dearmor > /usr/share/keyrings/debian-archive-bullseye-stable.gpg \ - && echo "deb [signed-by=/usr/share/keyrings/debian-archive-bullseye-stable.gpg] http://deb.debian.org/debian bullseye main contrib non-free" | \ - tee /etc/apt/sources.list.d/debian-bullseye-nonfree.list \ - && apt-get -qq update \ - && apt-get -qq install -y \ - python3.9 \ - python3.9-dev \ - build-essential cmake git \ - && rm -rf /var/lib/apt/lists/* - -# Extract Python version and set environment variables -RUN PYTHON_VERSION=$(python3 --version 2>&1 | awk '{print $2}' | cut -d. -f1,2) && \ - PYTHON_VERSION_NO_DOT=$(echo $PYTHON_VERSION | sed 's/\.//') && \ - echo "PYTHON_VERSION=$PYTHON_VERSION" > /etc/environment && \ - echo "PYTHON_VERSION_NO_DOT=$PYTHON_VERSION_NO_DOT" >> /etc/environment - -# Clone and build HailoRT -RUN . /etc/environment && \ - git clone https://github.com/hailo-ai/hailort.git /opt/hailort && \ - cd /opt/hailort && \ - git checkout v${HAILO_VERSION} && \ - cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Release && \ - cmake --build build --config release --target libhailort && \ - cmake --build build --config release --target hailortcli && \ - cmake --build build --config release --target install - -# Create a wheel file using pip3 wheel -RUN cd /opt/hailort/hailort/libhailort/bindings/python/platform && \ - python3 setup.py bdist_wheel --dist-dir /hailo-wheels - -RUN mkdir -p /rootfs/usr/local/lib /rootfs/usr/local/bin && \ - cp /usr/local/lib/libhailort.so* /rootfs/usr/local/lib && \ - cp /usr/local/bin/hailortcli /rootfs/usr/local/bin +RUN --mount=type=bind,source=docker/hailo8l/install_hailort.sh,target=/deps/install_hailort.sh \ + /deps/install_hailort.sh # Use deps as the base image FROM deps AS h8l-frigate -ARG HAILO_VERSION # Copy the wheels from the wheels stage COPY --from=h8l-wheels /h8l-wheels /deps/h8l-wheels -COPY --from=build-hailort /hailo-wheels /deps/hailo-wheels -COPY --from=build-hailort /rootfs/ / +COPY --from=hailort /hailo-wheels /deps/hailo-wheels +COPY --from=hailort /rootfs/ / # Install the wheels RUN pip3 install -U /deps/h8l-wheels/*.whl diff --git a/docker/hailo8l/h8l.hcl b/docker/hailo8l/h8l.hcl index a1eb82fb5..91f6d13c6 100644 --- a/docker/hailo8l/h8l.hcl +++ b/docker/hailo8l/h8l.hcl @@ -1,3 +1,9 @@ +target wget { + dockerfile = "docker/main/Dockerfile" + platforms = ["linux/arm64","linux/amd64"] + target = "wget" +} + target wheels { dockerfile = "docker/main/Dockerfile" platforms = ["linux/arm64","linux/amd64"] @@ -19,6 +25,7 @@ target rootfs { target h8l { dockerfile = "docker/hailo8l/Dockerfile" contexts = { + wget = "target:wget" wheels = "target:wheels" deps = "target:deps" rootfs = "target:rootfs" diff --git a/docker/hailo8l/install_hailort.sh b/docker/hailo8l/install_hailort.sh new file mode 100755 index 000000000..004db86c9 --- /dev/null +++ b/docker/hailo8l/install_hailort.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +set -euxo pipefail + +hailo_version="4.19.0" + +if [[ "${TARGETARCH}" == "amd64" ]]; then + arch="x86_64" +elif [[ "${TARGETARCH}" == "arm64" ]]; then + arch="aarch64" +fi + +mkdir -p /rootfs + +wget -qO- "https://github.com/frigate-nvr/hailort/releases/download/v${hailo_version}/hailort-${TARGETARCH}.tar.gz" | + tar -C /rootfs/ -xzf - + +mkdir -p /hailo-wheels + +wget -P /hailo-wheels/ "https://github.com/frigate-nvr/hailort/releases/download/v${hailo_version}/hailort-${hailo_version}-cp39-cp39-linux_${arch}.whl" + From 7d3313e7322f99beb308c07b1a06a23c1e99e28c Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sat, 2 Nov 2024 18:16:07 -0500 Subject: [PATCH 126/479] Add ability to view tracked objects in Explore from review item details pane (#14744) --- frigate/api/defs/events_query_parameters.py | 1 + frigate/api/event.py | 4 ++++ .../overlay/detail/ReviewDetailDialog.tsx | 18 ++++++++++++++++++ web/src/pages/Explore.tsx | 1 + 4 files changed, 24 insertions(+) diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/events_query_parameters.py index fe1d2b8b2..4639b7f59 100644 --- a/frigate/api/defs/events_query_parameters.py +++ b/frigate/api/defs/events_query_parameters.py @@ -28,6 +28,7 @@ class EventsQueryParams(BaseModel): is_submitted: Optional[int] = None min_length: Optional[float] = None max_length: Optional[float] = None + event_id: Optional[str] = None sort: Optional[str] = None timezone: Optional[str] = "utc" diff --git a/frigate/api/event.py b/frigate/api/event.py index 7f4f14610..869b61aaf 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -88,6 +88,7 @@ def events(params: EventsQueryParams = Depends()): is_submitted = params.is_submitted min_length = params.min_length max_length = params.max_length + event_id = params.event_id sort = params.sort @@ -230,6 +231,9 @@ def events(params: EventsQueryParams = Depends()): elif is_submitted > 0: clauses.append((Event.plus_id != "")) + if event_id is not None: + clauses.append((Event.id == event_id)) + if len(clauses) == 0: clauses.append((True)) diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index 2230046f3..d40f68a1e 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -40,6 +40,7 @@ import { import { useOverlayState } from "@/hooks/use-overlay-state"; import { DownloadVideoButton } from "@/components/button/DownloadVideoButton"; import { TooltipPortal } from "@radix-ui/react-tooltip"; +import { LuSearch } from "react-icons/lu"; type ReviewDetailDialogProps = { review?: ReviewSegment; @@ -53,6 +54,8 @@ export default function ReviewDetailDialog({ revalidateOnFocus: false, }); + const navigate = useNavigate(); + // upload const [upload, setUpload] = useState(); @@ -219,6 +222,21 @@ export default function ReviewDetailDialog({ )} {event.sub_label ?? event.label} ( {Math.round(event.data.top_score * 100)}%) + + +
{ + navigate(`/explore?event_id=${event.id}`); + }} + > + +
+ + + View in Explore + +
); })} diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 4efcb81b8..e8889e3ee 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -114,6 +114,7 @@ export default function Explore() { max_score: searchSearchParams["max_score"], has_snapshot: searchSearchParams["has_snapshot"], has_clip: searchSearchParams["has_clip"], + event_id: searchSearchParams["event_id"], limit: Object.keys(searchSearchParams).length == 0 ? API_LIMIT : undefined, timezone, From 3a8c290f91403f022900c60b00bd2d4bc1e80d29 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Sun, 3 Nov 2024 06:10:38 -0600 Subject: [PATCH 127/479] update docs for new labels (#14739) --- docs/docs/plus/first_model.md | 2 +- docs/docs/plus/improving_model.md | 4 ++-- docs/docs/plus/index.md | 16 ++++++++++++---- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/docs/docs/plus/first_model.md b/docs/docs/plus/first_model.md index 6978bb491..e68fd388d 100644 --- a/docs/docs/plus/first_model.md +++ b/docs/docs/plus/first_model.md @@ -5,7 +5,7 @@ title: Requesting your first model ## Step 1: Upload and annotate your images -Before requesting your first model, you will need to upload and verify at least 1 image to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. +Before requesting your first model, you will need to upload and verify at least 10 images to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. It is recommended to submit **both** true positives and false positives. This will help the model differentiate between what is and isn't correct. You should aim for a target of 80% true positive submissions and 20% false positives across all of your images. If you are experiencing false positives in a specific area, submitting true positives for any object type near that area in similar lighting conditions will help teach the model what that area looks like when no objects are present. diff --git a/docs/docs/plus/improving_model.md b/docs/docs/plus/improving_model.md index 37a765994..578f4512c 100644 --- a/docs/docs/plus/improving_model.md +++ b/docs/docs/plus/improving_model.md @@ -13,7 +13,7 @@ You may find that Frigate+ models result in more false positives initially, but For the best results, follow the following guidelines. -**Label every object in the image**: It is important that you label all objects in each image before verifying. If you don't label a car for example, the model will be taught that part of the image is _not_ a car and it will start to get confused. +**Label every object in the image**: It is important that you label all objects in each image before verifying. If you don't label a car for example, the model will be taught that part of the image is _not_ a car and it will start to get confused. You can exclude labels that you don't want detected on any of your cameras. **Make tight bounding boxes**: Tighter bounding boxes improve the recognition and ensure that accurate bounding boxes are predicted at runtime. @@ -21,7 +21,7 @@ For the best results, follow the following guidelines. **Label objects hard to identify as difficult**: When objects are truly difficult to make out, such as a car barely visible through a bush, or a dog that is hard to distinguish from the background at night, flag it as 'difficult'. This is not used in the model training as of now, but will in the future. -**`amazon`, `ups`, and `fedex` should label the logo**: For a Fedex truck, label the truck as a `car` and make a different bounding box just for the Fedex logo. If there are multiple logos, label each of them. +**Delivery logos such as `amazon`, `ups`, and `fedex` should label the logo**: For a Fedex truck, label the truck as a `car` and make a different bounding box just for the Fedex logo. If there are multiple logos, label each of them. ![Fedex Logo](/img/plus/fedex-logo.jpg) diff --git a/docs/docs/plus/index.md b/docs/docs/plus/index.md index b05f4f306..78305544e 100644 --- a/docs/docs/plus/index.md +++ b/docs/docs/plus/index.md @@ -17,7 +17,7 @@ Information on how to integrate Frigate+ with Frigate can be found in the [integ ## Available model types -There are two model types offered in Frigate+: `mobiledet` and `yolonas`. Both of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types). +There are two model types offered in Frigate+, `mobiledet` and `yolonas`. Both of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types). Not all model types are supported by all detectors, so it's important to choose a model type to match your detector as shown in the table under [supported detector types](#supported-detector-types). @@ -48,11 +48,19 @@ _\* Requires Frigate 0.15_ ## Available label types -Frigate+ models support a more relevant set of objects for security cameras. Currently, only the following objects are supported: `person`, `face`, `car`, `license_plate`, `amazon`, `ups`, `fedex`, `package`, `dog`, `cat`, `deer`. Other object types available in the default Frigate model are not available. Additional object types will be added in future releases. +Frigate+ models support a more relevant set of objects for security cameras. Currently, the following objects are supported: + +- **People**: `person`, `face` +- **Vehicles**: `car`, `motorcycle`, `bicycle`, `boat`, `license_plate` +- **Delivery Logos**: `amazon`, `usps`, `ups`, `fedex`, `dhl`, `an_post`, `purolator`, `postnl`, `nzpost`, `postnord`, `gls`, `dpd` +- **Animals**: `dog`, `cat`, `deer`, `horse`, `bird`, `raccoon`, `fox`, `bear`, `cow`, `squirrel`, `goat`, `rabbit` +- **Other**: `package`, `waste_bin`, `bbq_grill`, `robot_lawnmower`, `umbrella` + +Other object types available in the default Frigate model are not available. Additional object types will be added in future releases. ### Label attributes -Frigate has special handling for some labels when using Frigate+ models. `face`, `license_plate`, `amazon`, `ups`, and `fedex` are considered attribute labels which are not tracked like regular objects and do not generate events. In addition, the `threshold` filter will have no effect on these labels. You should adjust the `min_score` and other filter values as needed. +Frigate has special handling for some labels when using Frigate+ models. `face`, `license_plate`, and delivery logos such as `amazon`, `ups`, and `fedex` are considered attribute labels which are not tracked like regular objects and do not generate events. In addition, the `threshold` filter will have no effect on these labels. You should adjust the `min_score` and other filter values as needed. In order to have Frigate start using these attribute labels, you will need to add them to the list of objects to track: @@ -75,6 +83,6 @@ When using Frigate+ models, Frigate will choose the snapshot of a person object ![Face Attribute](/img/plus/attribute-example-face.jpg) -`amazon`, `ups`, and `fedex` labels are used to automatically assign a sub label to car objects. +Delivery logos such as `amazon`, `ups`, and `fedex` labels are used to automatically assign a sub label to car objects. ![Fedex Attribute](/img/plus/attribute-example-fedex.jpg) From 44f40966e7b8273eb86fa3ad4389c3ffb8e7dc0e Mon Sep 17 00:00:00 2001 From: leccelecce <24962424+leccelecce@users.noreply.github.com> Date: Sun, 3 Nov 2024 12:16:59 +0000 Subject: [PATCH 128/479] Docs: correct go2rtc version used (#14753) --- docs/docs/configuration/camera_specific.md | 2 +- docs/docs/configuration/restream.md | 4 ++-- docs/docs/guides/configuring_go2rtc.md | 6 +++--- docs/sidebars.ts | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/docs/configuration/camera_specific.md b/docs/docs/configuration/camera_specific.md index 70638b69e..072c20bb5 100644 --- a/docs/docs/configuration/camera_specific.md +++ b/docs/docs/configuration/camera_specific.md @@ -181,7 +181,7 @@ go2rtc: - rtspx://192.168.1.1:7441/abcdefghijk ``` -[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#source-rtsp) +[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#source-rtsp) In the Unifi 2.0 update Unifi Protect Cameras had a change in audio sample rate which causes issues for ffmpeg. The input rate needs to be set for record if used directly with unifi protect. diff --git a/docs/docs/configuration/restream.md b/docs/docs/configuration/restream.md index 211050972..1ad09cc8d 100644 --- a/docs/docs/configuration/restream.md +++ b/docs/docs/configuration/restream.md @@ -7,7 +7,7 @@ title: Restream Frigate can restream your video feed as an RTSP feed for other applications such as Home Assistant to utilize it at `rtsp://:8554/`. Port 8554 must be open. [This allows you to use a video feed for detection in Frigate and Home Assistant live view at the same time without having to make two separate connections to the camera](#reduce-connections-to-camera). The video feed is copied from the original video feed directly to avoid re-encoding. This feed does not include any annotation by Frigate. -Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.4) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#configuration) for more advanced configurations and features. +Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.2) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#configuration) for more advanced configurations and features. :::note @@ -134,7 +134,7 @@ cameras: ## Advanced Restream Configurations -The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below: +The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below: NOTE: The output will need to be passed with two curly braces `{{output}}` diff --git a/docs/docs/guides/configuring_go2rtc.md b/docs/docs/guides/configuring_go2rtc.md index 5c85f7d11..8667ead77 100644 --- a/docs/docs/guides/configuring_go2rtc.md +++ b/docs/docs/guides/configuring_go2rtc.md @@ -13,7 +13,7 @@ Use of the bundled go2rtc is optional. You can still configure FFmpeg to connect # Setup a go2rtc stream -First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#module-streams), not just rtsp. +First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#module-streams), not just rtsp. :::tip @@ -47,8 +47,8 @@ After adding this to the config, restart Frigate and try to watch the live strea - Check Video Codec: - If the camera stream works in go2rtc but not in your browser, the video codec might be unsupported. - - If using H265, switch to H264. Refer to [video codec compatibility](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#codecs-madness) in go2rtc documentation. - - If unable to switch from H265 to H264, or if the stream format is different (e.g., MJPEG), re-encode the video using [FFmpeg parameters](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#source-ffmpeg). It supports rotating and resizing video feeds and hardware acceleration. Keep in mind that transcoding video from one format to another is a resource intensive task and you may be better off using the built-in jsmpeg view. + - If using H265, switch to H264. Refer to [video codec compatibility](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#codecs-madness) in go2rtc documentation. + - If unable to switch from H265 to H264, or if the stream format is different (e.g., MJPEG), re-encode the video using [FFmpeg parameters](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#source-ffmpeg). It supports rotating and resizing video feeds and hardware acceleration. Keep in mind that transcoding video from one format to another is a resource intensive task and you may be better off using the built-in jsmpeg view. ```yaml go2rtc: streams: diff --git a/docs/sidebars.ts b/docs/sidebars.ts index f8e8780b6..4ed41d2ad 100644 --- a/docs/sidebars.ts +++ b/docs/sidebars.ts @@ -26,7 +26,7 @@ const sidebars: SidebarsConfig = { { type: 'link', label: 'Go2RTC Configuration Reference', - href: 'https://github.com/AlexxIT/go2rtc/tree/v1.9.4#configuration', + href: 'https://github.com/AlexxIT/go2rtc/tree/v1.9.2#configuration', } as PropSidebarItemLink, ], Detectors: [ From 189d4b459fa83f93b782e4a4f00f039ba76d3371 Mon Sep 17 00:00:00 2001 From: leccelecce <24962424+leccelecce@users.noreply.github.com> Date: Sun, 3 Nov 2024 15:28:19 +0000 Subject: [PATCH 129/479] Avoid divide by zero in shm_frame_count (#14750) --- frigate/app.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/frigate/app.py b/frigate/app.py index bc4f626e0..2f771ec4d 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -512,6 +512,9 @@ class FrigateApp: 1, ) + if cam_total_frame_size == 0.0: + return 0 + shm_frame_count = min(50, int(available_shm / (cam_total_frame_size))) logger.debug( From 77ec86d31afb48770b49945d43ca972facc7a075 Mon Sep 17 00:00:00 2001 From: Felipe Santos Date: Sun, 3 Nov 2024 12:52:27 -0300 Subject: [PATCH 130/479] Fix devcontainer when there is no ~/.ssh/know_hosts file (#14758) --- .devcontainer/post_create.sh | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/.devcontainer/post_create.sh b/.devcontainer/post_create.sh index ee0888016..ec33ffb86 100755 --- a/.devcontainer/post_create.sh +++ b/.devcontainer/post_create.sh @@ -3,10 +3,12 @@ set -euxo pipefail # Cleanup the old github host key -sed -i -e '/AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31\/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi\/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==/d' ~/.ssh/known_hosts -# Add new github host key -curl -L https://api.github.com/meta | jq -r '.ssh_keys | .[]' | \ - sed -e 's/^/github.com /' >> ~/.ssh/known_hosts +if [[ -f ~/.ssh/known_hosts ]]; then + # Add new github host key + sed -i -e '/AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31\/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi\/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==/d' ~/.ssh/known_hosts + curl -L https://api.github.com/meta | jq -r '.ssh_keys | .[]' | \ + sed -e 's/^/github.com /' >> ~/.ssh/known_hosts +fi # Frigate normal container runs as root, so it have permission to create # the folders. But the devcontainer runs as the host user, so we need to From 9755fa05376c7f183bcad60563d3b77dd0a2d57a Mon Sep 17 00:00:00 2001 From: Felipe Santos Date: Sun, 3 Nov 2024 14:00:12 -0300 Subject: [PATCH 131/479] Fix exports migration when there is none (#14761) --- frigate/util/config.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/frigate/util/config.py b/frigate/util/config.py index a9a9666d9..3f3c45aa6 100644 --- a/frigate/util/config.py +++ b/frigate/util/config.py @@ -50,14 +50,15 @@ def migrate_frigate_config(config_file: str): previous_version = "0.14" logger.info("Migrating export file names...") - for file in os.listdir(EXPORT_DIR): - if "@" not in file: - continue + if os.path.isdir(EXPORT_DIR): + for file in os.listdir(EXPORT_DIR): + if "@" not in file: + continue - new_name = file.replace("@", "_") - os.rename( - os.path.join(EXPORT_DIR, file), os.path.join(EXPORT_DIR, new_name) - ) + new_name = file.replace("@", "_") + os.rename( + os.path.join(EXPORT_DIR, file), os.path.join(EXPORT_DIR, new_name) + ) if previous_version < "0.15-0": logger.info(f"Migrating frigate config from {previous_version} to 0.15-0...") From 959ca0f412f3fd4b117b32c07c2141e8634c6a91 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 3 Nov 2024 18:41:31 -0600 Subject: [PATCH 132/479] Fix object processing logic for detections (#14766) --- frigate/object_processing.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/frigate/object_processing.py b/frigate/object_processing.py index fab7bbc6c..ca7ea47b8 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -714,7 +714,8 @@ class TrackedObjectProcessor(threading.Thread): ) and ( not review_config.detections.required_zones - or set(obj.entered_zones) & set(review_config.alerts.required_zones) + or set(obj.entered_zones) + & set(review_config.detections.required_zones) ) ) ): From 156e7cc628a1b60ce299e063652cf02ef294d95e Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 3 Nov 2024 18:49:13 -0700 Subject: [PATCH 133/479] Clarify semantic search GPU (#14767) * Clarify semantic search GPU * clarity Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> * fix wording --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --- docs/docs/configuration/semantic_search.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 8abd761a8..61873478d 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -60,15 +60,17 @@ The CLIP models are downloaded in ONNX format, and the `large` model can be acce If the correct build is used for your GPU and the `large` model is configured, then the GPU will be detected and used automatically. -**AMD** -- ROCm will automatically be detected and used for semantic search in the `-rocm` Frigate image. +**NOTE:** Object detection and Semantic Search are independent features. If you want to use your GPU with Semantic Search, you must choose the appropriate Frigate Docker image for your GPU. -**Intel** -- OpenVINO will automatically be detected and used as a detector in the default Frigate image. +- **AMD** + - ROCm will automatically be detected and used for semantic search in the `-rocm` Frigate image. -**Nvidia** -- Nvidia GPUs will automatically be detected and used as a detector in the `-tensorrt` Frigate image. -- Jetson devices will automatically be detected and used as a detector in the `-tensorrt-jp(4/5)` Frigate image. +- **Intel** + - OpenVINO will automatically be detected and used for semantic search in the default Frigate image. + +- **Nvidia** + - Nvidia GPUs will automatically be detected and used for semantic search in the `-tensorrt` Frigate image. + - Jetson devices will automatically be detected and used for semantic search in the `-tensorrt-jp(4/5)` Frigate image. ::: From a13b9815f61089e13d678599e1cae398a18437d3 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 4 Nov 2024 07:07:57 -0700 Subject: [PATCH 134/479] Various fixes (#14786) * Catch openvino error * Remove clip deletion * Update deletion text * Fix timeline not respecting timezone config * Tweaks * More timezone fixes * Fix * More timezone fixes * Fix shm docs --- docs/docs/frigate/installation.md | 14 +- frigate/api/event.py | 3 - frigate/app.py | 4 +- frigate/util/model.py | 33 ++- web/src/components/bar/TimelineBar.tsx | 191 ------------------ web/src/components/graph/CameraGraph.tsx | 15 +- web/src/components/graph/SystemGraph.tsx | 17 +- .../components/menu/SearchResultActions.tsx | 8 +- .../overlay/detail/ObjectLifecycle.tsx | 1 + .../components/timeline/segment-metadata.tsx | 46 +++-- web/src/hooks/use-draggable-element.ts | 28 ++- 11 files changed, 103 insertions(+), 257 deletions(-) delete mode 100644 web/src/components/bar/TimelineBar.tsx diff --git a/docs/docs/frigate/installation.md b/docs/docs/frigate/installation.md index 10c83b013..e3599e628 100644 --- a/docs/docs/frigate/installation.md +++ b/docs/docs/frigate/installation.md @@ -81,15 +81,15 @@ You can calculate the **minimum** shm size for each camera with the following fo ```console # Replace and -$ python -c 'print("{:.2f}MB".format(( * * 1.5 * 10 + 270480) / 1048576))' +$ python -c 'print("{:.2f}MB".format(( * * 1.5 * 20 + 270480) / 1048576))' -# Example for 1280x720 -$ python -c 'print("{:.2f}MB".format((1280 * 720 * 1.5 * 10 + 270480) / 1048576))' -13.44MB +# Example for 1280x720, including logs +$ python -c 'print("{:.2f}MB".format((1280 * 720 * 1.5 * 20 + 270480) / 1048576)) + 40' +46.63MB # Example for eight cameras detecting at 1280x720, including logs -$ python -c 'print("{:.2f}MB".format(((1280 * 720 * 1.5 * 10 + 270480) / 1048576) * 8 + 40))' -136.99MB +$ python -c 'print("{:.2f}MB".format(((1280 * 720 * 1.5 * 20 + 270480) / 1048576) * 8 + 40))' +253MB ``` The shm size cannot be set per container for Home Assistant add-ons. However, this is probably not required since by default Home Assistant Supervisor allocates `/dev/shm` with half the size of your total memory. If your machine has 8GB of memory, chances are that Frigate will have access to up to 4GB without any additional configuration. @@ -194,7 +194,7 @@ services: privileged: true # this may not be necessary for all setups restart: unless-stopped image: ghcr.io/blakeblackshear/frigate:stable - shm_size: "64mb" # update for your cameras based on calculation above + shm_size: "512mb" # update for your cameras based on calculation above devices: - /dev/bus/usb:/dev/bus/usb # Passes the USB Coral, needs to be modified for other versions - /dev/apex_0:/dev/apex_0 # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux diff --git a/frigate/api/event.py b/frigate/api/event.py index 869b61aaf..ac414cdde 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -1042,9 +1042,6 @@ def delete_event(request: Request, event_id: str): media.unlink(missing_ok=True) media = Path(f"{os.path.join(CLIPS_DIR, media_name)}-clean.png") media.unlink(missing_ok=True) - if event.has_clip: - media = Path(f"{os.path.join(CLIPS_DIR, media_name)}.mp4") - media.unlink(missing_ok=True) event.delete_instance() Timeline.delete().where(Timeline.source_id == event_id).execute() diff --git a/frigate/app.py b/frigate/app.py index 2f771ec4d..d0d3ab8a1 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -521,9 +521,9 @@ class FrigateApp: f"Calculated total camera size {available_shm} / {cam_total_frame_size} :: {shm_frame_count} frames for each camera in SHM" ) - if shm_frame_count < 10: + if shm_frame_count < 20: logger.warning( - f"The current SHM size of {total_shm}MB is too small, recommend increasing it to at least {round(min_req_shm + cam_total_frame_size * 10)}MB." + f"The current SHM size of {total_shm}MB is too small, recommend increasing it to at least {round(min_req_shm + cam_total_frame_size * 20)}MB." ) return shm_frame_count diff --git a/frigate/util/model.py b/frigate/util/model.py index 2aa06d0b2..091bb0833 100644 --- a/frigate/util/model.py +++ b/frigate/util/model.py @@ -1,5 +1,6 @@ """Model Utils""" +import logging import os from typing import Any @@ -11,6 +12,8 @@ except ImportError: # openvino is not included pass +logger = logging.getLogger(__name__) + def get_ort_providers( force_cpu: bool = False, device: str = "AUTO", requires_fp16: bool = False @@ -89,19 +92,27 @@ class ONNXModelRunner: self.ort: ort.InferenceSession = None self.ov: ov.Core = None providers, options = get_ort_providers(device == "CPU", device, requires_fp16) + self.interpreter = None if "OpenVINOExecutionProvider" in providers: - # use OpenVINO directly - self.type = "ov" - self.ov = ov.Core() - self.ov.set_property( - {ov.properties.cache_dir: "/config/model_cache/openvino"} - ) - self.interpreter = self.ov.compile_model( - model=model_path, device_name=device - ) - else: - # Use ONNXRuntime + try: + # use OpenVINO directly + self.type = "ov" + self.ov = ov.Core() + self.ov.set_property( + {ov.properties.cache_dir: "/config/model_cache/openvino"} + ) + self.interpreter = self.ov.compile_model( + model=model_path, device_name=device + ) + except Exception as e: + logger.warning( + f"OpenVINO failed to build model, using CPU instead: {e}" + ) + self.interpreter = None + + # Use ONNXRuntime + if self.interpreter is None: self.type = "ort" self.ort = ort.InferenceSession( model_path, diff --git a/web/src/components/bar/TimelineBar.tsx b/web/src/components/bar/TimelineBar.tsx deleted file mode 100644 index fe05b876f..000000000 --- a/web/src/components/bar/TimelineBar.tsx +++ /dev/null @@ -1,191 +0,0 @@ -import { FrigateConfig } from "@/types/frigateConfig"; -import { GraphDataPoint } from "@/types/graph"; -import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; -import useSWR from "swr"; -import ActivityIndicator from "../indicators/activity-indicator"; - -type TimelineBarProps = { - startTime: number; - graphData: - | { - objects: number[]; - motion: GraphDataPoint[]; - } - | undefined; - onClick?: () => void; -}; -export default function TimelineBar({ - startTime, - graphData, - onClick, -}: TimelineBarProps) { - const { data: config } = useSWR("config"); - - if (!config) { - return ; - } - - return ( -
- {graphData != undefined && ( -
- {getHourBlocks().map((idx) => { - return ( -
- ); - })} -
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:00" : "%I:00%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:05" : "%I:05%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:10" : "%I:10%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:15" : "%I:15%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:20" : "%I:20%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:25" : "%I:25%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:30" : "%I:30%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:35" : "%I:35%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:40" : "%I:40%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:45" : "%I:45%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:50" : "%I:50%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
-
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config?.ui.time_format == "24hour" ? "%H:55" : "%I:55%P", - time_style: "medium", - date_style: "medium", - })} -
-
-
- )} -
- {formatUnixTimestampToDateTime(startTime, { - strftime_fmt: - config.ui.time_format == "24hour" ? "%m/%d %H:%M" : "%m/%d %I:%M%P", - time_style: "medium", - date_style: "medium", - })} -
-
- ); -} - -function getHourBlocks() { - const arr = []; - - for (let x = 0; x <= 59; x++) { - arr.push(x); - } - - return arr; -} diff --git a/web/src/components/graph/CameraGraph.tsx b/web/src/components/graph/CameraGraph.tsx index 3289887c5..ab5d6e03f 100644 --- a/web/src/components/graph/CameraGraph.tsx +++ b/web/src/components/graph/CameraGraph.tsx @@ -1,5 +1,6 @@ import { useTheme } from "@/context/theme-provider"; import { FrigateConfig } from "@/types/frigateConfig"; +import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; import { useCallback, useEffect, useMemo } from "react"; import Chart from "react-apexcharts"; import { isMobileOnly } from "react-device-detect"; @@ -42,12 +43,14 @@ export function CameraLineGraph({ const formatTime = useCallback( (val: unknown) => { - const date = new Date(updateTimes[Math.round(val as number)] * 1000); - return date.toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - }); + return formatUnixTimestampToDateTime( + updateTimes[Math.round(val as number)], + { + timezone: config?.ui.timezone, + strftime_fmt: + config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p", + }, + ); }, [config, updateTimes], ); diff --git a/web/src/components/graph/SystemGraph.tsx b/web/src/components/graph/SystemGraph.tsx index 572eae5cd..aaf838763 100644 --- a/web/src/components/graph/SystemGraph.tsx +++ b/web/src/components/graph/SystemGraph.tsx @@ -1,6 +1,7 @@ import { useTheme } from "@/context/theme-provider"; import { FrigateConfig } from "@/types/frigateConfig"; import { Threshold } from "@/types/graph"; +import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; import { useCallback, useEffect, useMemo } from "react"; import Chart from "react-apexcharts"; import { isMobileOnly } from "react-device-detect"; @@ -50,17 +51,17 @@ export function ThresholdBarGraph({ let timeOffset = 0; if (dateIndex < 0) { - timeOffset = 5000 * Math.abs(dateIndex); + timeOffset = 5 * Math.abs(dateIndex); } - const date = new Date( - updateTimes[Math.max(1, dateIndex) - 1] * 1000 - timeOffset, + return formatUnixTimestampToDateTime( + updateTimes[Math.max(1, dateIndex) - 1] - timeOffset, + { + timezone: config?.ui.timezone, + strftime_fmt: + config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p", + }, ); - return date.toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - }); }, [config, updateTimes], ); diff --git a/web/src/components/menu/SearchResultActions.tsx b/web/src/components/menu/SearchResultActions.tsx index 8a9373bcc..a07d27240 100644 --- a/web/src/components/menu/SearchResultActions.tsx +++ b/web/src/components/menu/SearchResultActions.tsx @@ -159,7 +159,13 @@ export default function SearchResultActions({ Confirm Delete - Are you sure you want to delete this tracked object? + Deleting this tracked object removes the snapshot, any saved + embeddings, and any associated object lifecycle entries. Recorded + footage of this tracked object in History view will NOT be + deleted. +
+
+ Are you sure you want to proceed? Cancel diff --git a/web/src/components/overlay/detail/ObjectLifecycle.tsx b/web/src/components/overlay/detail/ObjectLifecycle.tsx index d687915ca..7a667529e 100644 --- a/web/src/components/overlay/detail/ObjectLifecycle.tsx +++ b/web/src/components/overlay/detail/ObjectLifecycle.tsx @@ -427,6 +427,7 @@ export default function ObjectLifecycle({
{formatUnixTimestampToDateTime(item.timestamp, { + timezone: config.ui.timezone, strftime_fmt: config.ui.time_format == "24hour" ? "%d %b %H:%M:%S" diff --git a/web/src/components/timeline/segment-metadata.tsx b/web/src/components/timeline/segment-metadata.tsx index 6ca71af24..3e3c99393 100644 --- a/web/src/components/timeline/segment-metadata.tsx +++ b/web/src/components/timeline/segment-metadata.tsx @@ -1,4 +1,6 @@ import { FrigateConfig } from "@/types/frigateConfig"; +import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; +import { useMemo } from "react"; import useSWR from "swr"; type MinimapSegmentProps = { @@ -40,22 +42,22 @@ export function MinimapBounds({ className="pointer-events-none absolute inset-0 -bottom-7 z-20 flex w-full select-none scroll-mt-8 items-center justify-center text-center text-[10px] font-medium text-primary" ref={firstMinimapSegmentRef} > - {new Date(alignedMinimapStartTime * 1000).toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - ...(!dense && { month: "short", day: "2-digit" }), + {formatUnixTimestampToDateTime(alignedMinimapStartTime, { + timezone: config?.ui.timezone, + strftime_fmt: !dense + ? `%b %d, ${config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p"}` + : `${config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p"}`, })}
)} {isLastSegmentInMinimap && (
- {new Date(alignedMinimapEndTime * 1000).toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - ...(!dense && { month: "short", day: "2-digit" }), + {formatUnixTimestampToDateTime(alignedMinimapEndTime, { + timezone: config?.ui.timezone, + strftime_fmt: !dense + ? `%b %d, ${config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p"}` + : `${config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p"}`, })}
)} @@ -92,6 +94,22 @@ export function Timestamp({ }: TimestampSegmentProps) { const { data: config } = useSWR("config"); + const formattedTimestamp = useMemo(() => { + if ( + !( + timestamp.getMinutes() % timestampSpread === 0 && + timestamp.getSeconds() === 0 + ) + ) { + return undefined; + } + + return formatUnixTimestampToDateTime(timestamp.getTime() / 1000, { + timezone: config?.ui.timezone, + strftime_fmt: config?.ui.time_format == "24hour" ? "%H:%M" : "%I:%M %p", + }); + }, [config, timestamp, timestampSpread]); + return (
{!isFirstSegmentInMinimap && !isLastSegmentInMinimap && ( @@ -99,13 +117,7 @@ export function Timestamp({ key={`${segmentKey}_timestamp`} className="pointer-events-none select-none text-[8px] text-neutral_variant dark:text-neutral" > - {timestamp.getMinutes() % timestampSpread === 0 && - timestamp.getSeconds() === 0 && - timestamp.toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - })} + {formattedTimestamp}
)}
diff --git a/web/src/hooks/use-draggable-element.ts b/web/src/hooks/use-draggable-element.ts index 0168cd5a2..73013de58 100644 --- a/web/src/hooks/use-draggable-element.ts +++ b/web/src/hooks/use-draggable-element.ts @@ -10,6 +10,7 @@ import scrollIntoView from "scroll-into-view-if-needed"; import { useTimelineUtils } from "./use-timeline-utils"; import { FrigateConfig } from "@/types/frigateConfig"; import useSWR from "swr"; +import { formatUnixTimestampToDateTime } from "@/utils/dateUtil"; type DraggableElementProps = { contentRef: React.RefObject; @@ -168,6 +169,19 @@ function useDraggableElement({ [segmentDuration, timelineStartAligned, segmentHeight], ); + const getFormattedTimestamp = useCallback( + (segmentStartTime: number) => { + return formatUnixTimestampToDateTime(segmentStartTime, { + timezone: config?.ui.timezone, + strftime_fmt: + config?.ui.time_format == "24hour" + ? `%H:%M${segmentDuration < 60 && !dense ? ":%S" : ""}` + : `%I:%M${segmentDuration < 60 && !dense ? ":%S" : ""} %p`, + }); + }, + [config, dense, segmentDuration], + ); + const updateDraggableElementPosition = useCallback( ( newElementPosition: number, @@ -184,14 +198,8 @@ function useDraggableElement({ } if (draggableElementTimeRef.current) { - draggableElementTimeRef.current.textContent = new Date( - segmentStartTime * 1000, - ).toLocaleTimeString([], { - hour12: config?.ui.time_format != "24hour", - hour: "2-digit", - minute: "2-digit", - ...(segmentDuration < 60 && !dense && { second: "2-digit" }), - }); + draggableElementTimeRef.current.textContent = + getFormattedTimestamp(segmentStartTime); if (scrollTimeline && !userInteracting) { scrollIntoView(thumb, { block: "center", @@ -208,13 +216,11 @@ function useDraggableElement({ } }, [ - segmentDuration, draggableElementTimeRef, draggableElementRef, setDraggableElementTime, setDraggableElementPosition, - dense, - config, + getFormattedTimestamp, userInteracting, ], ); From 553676aade8f66b3f8a1a05cd822c8bdbdae53cd Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 4 Nov 2024 12:04:33 -0700 Subject: [PATCH 135/479] Fix missing tensor_input (#14790) --- docs/docs/configuration/object_detectors.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index 5896260f4..7fefa74a4 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -457,6 +457,7 @@ model: width: 320 # <--- should match whatever was set in notebook height: 320 # <--- should match whatever was set in notebook input_pixel_format: bgr + input_tensor: nchw path: /config/yolo_nas_s.onnx labelmap_path: /labelmap/coco-80.txt ``` From ac762762c39c1885397ee14431d178877cf1b62e Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 4 Nov 2024 13:50:05 -0600 Subject: [PATCH 136/479] Overwrite existing saved search (#14792) * Overwrite existing saved search * simplify --- web/src/components/input/InputWithTags.tsx | 22 +++++++++++++------ web/src/components/input/SaveSearchDialog.tsx | 15 ++++++++++++- 2 files changed, 29 insertions(+), 8 deletions(-) diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index becc0f4e1..ff46375fd 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -1,4 +1,10 @@ -import React, { useState, useRef, useEffect, useCallback } from "react"; +import React, { + useState, + useRef, + useEffect, + useCallback, + useMemo, +} from "react"; import { LuX, LuFilter, @@ -88,6 +94,11 @@ export default function InputWithTags({ const [isDeleteDialogOpen, setIsDeleteDialogOpen] = useState(false); const [searchToDelete, setSearchToDelete] = useState(null); + const searchHistoryNames = useMemo( + () => searchHistory?.map((item) => item.name) ?? [], + [searchHistory], + ); + const handleSetSearchHistory = useCallback(() => { setIsSaveDialogOpen(true); }, []); @@ -96,12 +107,8 @@ export default function InputWithTags({ (name: string) => { if (searchHistoryLoaded) { setSearchHistory([ - ...(searchHistory ?? []), - { - name: name, - search: search, - filter: filters, - }, + ...(searchHistory ?? []).filter((item) => item.name !== name), + { name, search, filter: filters }, ]); } }, @@ -835,6 +842,7 @@ export default function InputWithTags({ setIsSaveDialogOpen(false)} onSave={handleSaveSearch} diff --git a/web/src/components/input/SaveSearchDialog.tsx b/web/src/components/input/SaveSearchDialog.tsx index 89e9217d7..322a76421 100644 --- a/web/src/components/input/SaveSearchDialog.tsx +++ b/web/src/components/input/SaveSearchDialog.tsx @@ -9,17 +9,19 @@ import { import { Button } from "@/components/ui/button"; import { Input } from "@/components/ui/input"; -import { useState } from "react"; +import { useMemo, useState } from "react"; import { isMobile } from "react-device-detect"; import { toast } from "sonner"; type SaveSearchDialogProps = { + existingNames: string[]; isOpen: boolean; onClose: () => void; onSave: (name: string) => void; }; export function SaveSearchDialog({ + existingNames, isOpen, onClose, onSave, @@ -37,6 +39,11 @@ export function SaveSearchDialog({ } }; + const overwrite = useMemo( + () => existingNames.includes(searchName), + [existingNames, searchName], + ); + return ( setSearchName(e.target.value)} placeholder="Enter a name for your search" /> + {overwrite && ( +
+ {searchName} already exists. Saving will overwrite the existing + value. +
+ )} )) - : filterType !== "event_id" && ( + : !(filterType == "event_id" && isSimilaritySearch) && ( - {filterType.replaceAll("_", " ")}:{" "} - {formatFilterValues(filterType, filterValues)} + {filterType === "event_id" + ? "Tracked Object ID" + : filterType.replaceAll("_", " ")} + : {formatFilterValues(filterType, filterValues)}
- {restartDialogOpen && ( - setRestartDialogOpen(false)} - > - - - - Are you sure you want to restart Frigate? - - - - Cancel - { - setRestartingSheetOpen(true); - sendRestart("restart"); - }} - > - Restart - - - - - )} - {restartingSheetOpen && ( - <> - setRestartingSheetOpen(false)} - > - e.preventDefault()} - > -
- - - - Frigate is Restarting - - -

This page will reload in {countdown} seconds.

- - - -
- - - - )} + setRestartDialogOpen(false)} + onRestart={() => sendRestart("restart")} + /> ); } diff --git a/web/src/components/overlay/dialog/RestartDialog.tsx b/web/src/components/overlay/dialog/RestartDialog.tsx new file mode 100644 index 000000000..8e1a5c129 --- /dev/null +++ b/web/src/components/overlay/dialog/RestartDialog.tsx @@ -0,0 +1,122 @@ +import { useState, useEffect } from "react"; +import { + AlertDialog, + AlertDialogAction, + AlertDialogCancel, + AlertDialogContent, + AlertDialogFooter, + AlertDialogHeader, + AlertDialogTitle, +} from "@/components/ui/alert-dialog"; +import { + Sheet, + SheetContent, + SheetHeader, + SheetTitle, + SheetDescription, +} from "@/components/ui/sheet"; +import { Button } from "@/components/ui/button"; +import ActivityIndicator from "@/components/indicators/activity-indicator"; +import { baseUrl } from "@/api/baseUrl"; + +type RestartDialogProps = { + isOpen: boolean; + onClose: () => void; + onRestart: () => void; +}; + +export default function RestartDialog({ + isOpen, + onClose, + onRestart, +}: RestartDialogProps) { + const [restartDialogOpen, setRestartDialogOpen] = useState(isOpen); + const [restartingSheetOpen, setRestartingSheetOpen] = useState(false); + const [countdown, setCountdown] = useState(60); + + useEffect(() => { + setRestartDialogOpen(isOpen); + }, [isOpen]); + + useEffect(() => { + let countdownInterval: NodeJS.Timeout; + + if (restartingSheetOpen) { + countdownInterval = setInterval(() => { + setCountdown((prevCountdown) => prevCountdown - 1); + }, 1000); + } + + return () => { + clearInterval(countdownInterval); + }; + }, [restartingSheetOpen]); + + useEffect(() => { + if (countdown === 0) { + window.location.href = baseUrl; + } + }, [countdown]); + + const handleRestart = () => { + setRestartingSheetOpen(true); + onRestart(); + }; + + const handleForceReload = () => { + window.location.href = baseUrl; + }; + + return ( + <> + { + setRestartDialogOpen(false); + onClose(); + }} + > + + + + Are you sure you want to restart Frigate? + + + + Cancel + + Restart + + + + + + setRestartingSheetOpen(false)} + > + e.preventDefault()}> +
+ + + + Frigate is Restarting + + +
This page will reload in {countdown} seconds.
+ + + +
+ + + + ); +} diff --git a/web/src/pages/ConfigEditor.tsx b/web/src/pages/ConfigEditor.tsx index 52cb05473..bcb0c4c65 100644 --- a/web/src/pages/ConfigEditor.tsx +++ b/web/src/pages/ConfigEditor.tsx @@ -13,6 +13,7 @@ import { Toaster } from "@/components/ui/sonner"; import { toast } from "sonner"; import { LuCopy, LuSave } from "react-icons/lu"; import { MdOutlineRestartAlt } from "react-icons/md"; +import RestartDialog from "@/components/overlay/dialog/RestartDialog"; type SaveOptions = "saveonly" | "restart"; @@ -33,6 +34,8 @@ function ConfigEditor() { const configRef = useRef(null); const schemaConfiguredRef = useRef(false); + const [restartDialogOpen, setRestartDialogOpen] = useState(false); + const onHandleSaveConfig = useCallback( async (save_option: SaveOptions) => { if (!editorRef.current) { @@ -202,7 +205,7 @@ function ConfigEditor() { size="sm" className="flex items-center gap-2" aria-label="Save and restart" - onClick={() => onHandleSaveConfig("restart")} + onClick={() => setRestartDialogOpen(true)} >
@@ -231,6 +234,11 @@ function ConfigEditor() {
+ setRestartDialogOpen(false)} + onRestart={() => onHandleSaveConfig("restart")} + />
); } From ffd05f90f3f16d2dcf7de983b126e973e56a6c3e Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Wed, 6 Nov 2024 06:02:42 -0600 Subject: [PATCH 141/479] update hardware recommendations (#14830) --- docs/docs/frigate/hardware.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index cc9515f67..e19ec5d0d 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -13,20 +13,19 @@ Many users have reported various issues with Reolink cameras, so I do not recomm Here are some of the camera's I recommend: -- Loryta(Dahua) T5442TM-AS-LED (affiliate link) -- Loryta(Dahua) IPC-T5442TM-AS (affiliate link) -- Amcrest IP5M-T1179EW-28MM (affiliate link) +- Loryta(Dahua) IPC-T549M-ALED-S3 (affiliate link) +- Loryta(Dahua) IPC-T54IR-AS (affiliate link) +- Amcrest IP5M-T1179EW-AI-V3 (affiliate link) I may earn a small commission for my endorsement, recommendation, testimonial, or link to any products or services from this website. ## Server -My current favorite is the Beelink EQ12 because of the efficient N100 CPU and dual NICs that allow you to setup a dedicated private network for your cameras where they can be blocked from accessing the internet. There are many used workstation options on eBay that work very well. Anything with an Intel CPU and capable of running Debian should work fine. As a bonus, you may want to look for devices with a M.2 or PCIe express slot that is compatible with the Google Coral. I may earn a small commission for my endorsement, recommendation, testimonial, or link to any products or services from this website. +My current favorite is the Beelink EQ13 because of the efficient N100 CPU and dual NICs that allow you to setup a dedicated private network for your cameras where they can be blocked from accessing the internet. There are many used workstation options on eBay that work very well. Anything with an Intel CPU and capable of running Debian should work fine. As a bonus, you may want to look for devices with a M.2 or PCIe express slot that is compatible with the Google Coral. I may earn a small commission for my endorsement, recommendation, testimonial, or link to any products or services from this website. -| Name | Coral Inference Speed | Coral Compatibility | Notes | -| ------------------------------------------------------------------------------------------------------------- | --------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -| Beelink EQ12 (Amazon) | 5-10ms | USB | Dual gigabit NICs for easy isolated camera network. Easily handles several 1080p cameras. | -| Intel NUC (Amazon) | 5-10ms | USB | Overkill for most, but great performance. Can handle many cameras at 5fps depending on typical amounts of motion. Requires extra parts. | +| Name | Coral Inference Speed | Coral Compatibility | Notes | +| ------------------------------------------------------------------------------------------------------------- | --------------------- | ------------------- | ----------------------------------------------------------------------------------------- | +| Beelink EQ13 (Amazon) | 5-10ms | USB/M.2(A+E) | Dual gigabit NICs for easy isolated camera network. Easily handles several 1080p cameras. | ## Detectors From 2eb5fbf1126f5f92b121dcd89241ad798b7c3c9b Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 6 Nov 2024 06:59:33 -0700 Subject: [PATCH 142/479] Add more debug logs for preview and output (#14833) --- frigate/output/output.py | 10 ++++++++++ frigate/output/preview.py | 9 +++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/frigate/output/output.py b/frigate/output/output.py index 7d5b6d39a..1859ebd69 100644 --- a/frigate/output/output.py +++ b/frigate/output/output.py @@ -63,6 +63,7 @@ def output_frames( birdseye: Optional[Birdseye] = None preview_recorders: dict[str, PreviewRecorder] = {} preview_write_times: dict[str, float] = {} + failed_frame_requests: dict[str, int] = {} move_preview_frames("cache") @@ -99,7 +100,16 @@ def output_frames( if frame is None: logger.debug(f"Failed to get frame {frame_id} from SHM") + failed_frame_requests[camera] = failed_frame_requests.get(camera, 0) + 1 + + if failed_frame_requests[camera] > config.cameras[camera].detect.fps: + logger.warning( + f"Failed to retrieve many frames for {camera} from SHM, consider increasing SHM size if this continues." + ) + continue + else: + failed_frame_requests[camera] = 0 # send camera frame to ffmpeg process if websockets are connected if any( diff --git a/frigate/output/preview.py b/frigate/output/preview.py index a8915f688..9eae6b7de 100644 --- a/frigate/output/preview.py +++ b/frigate/output/preview.py @@ -154,6 +154,7 @@ class PreviewRecorder: self.start_time = 0 self.last_output_time = 0 self.output_frames = [] + if config.detect.width > config.detect.height: self.out_height = PREVIEW_HEIGHT self.out_width = ( @@ -274,7 +275,7 @@ class PreviewRecorder: return False - def write_frame_to_cache(self, frame_time: float, frame) -> None: + def write_frame_to_cache(self, frame_time: float, frame: np.ndarray) -> None: # resize yuv frame small_frame = np.zeros((self.out_height * 3 // 2, self.out_width), np.uint8) copy_yuv_to_position( @@ -303,7 +304,7 @@ class PreviewRecorder: current_tracked_objects: list[dict[str, any]], motion_boxes: list[list[int]], frame_time: float, - frame, + frame: np.ndarray, ) -> bool: # check for updated record config _, updated_record_config = self.config_subscriber.check_for_update() @@ -332,6 +333,10 @@ class PreviewRecorder: self.output_frames, self.requestor, ).start() + else: + logger.debug( + f"Not saving preview for {self.config.name} because there are no saved frames." + ) # reset frame cache self.segment_end = ( From bc371acb3effe1c51b938161e3430c1ca36e476a Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 6 Nov 2024 11:05:44 -0600 Subject: [PATCH 143/479] Cleanup batching (#14836) * Implement batching for event cleanup * remove import * add debug logging --- frigate/events/cleanup.py | 42 +++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/frigate/events/cleanup.py b/frigate/events/cleanup.py index 7d3e7c456..8ae38b534 100644 --- a/frigate/events/cleanup.py +++ b/frigate/events/cleanup.py @@ -21,6 +21,9 @@ class EventCleanupType(str, Enum): snapshots = "snapshots" +CHUNK_SIZE = 50 + + class EventCleanup(threading.Thread): def __init__( self, config: FrigateConfig, stop_event: MpEvent, db: SqliteVecQueueDatabase @@ -107,6 +110,7 @@ class EventCleanup(threading.Thread): .namedtuples() .iterator() ) + logger.debug(f"{len(expired_events)} events can be expired") # delete the media from disk for expired in expired_events: media_name = f"{expired.camera}-{expired.id}" @@ -125,13 +129,34 @@ class EventCleanup(threading.Thread): logger.warning(f"Unable to delete event images: {e}") # update the clips attribute for the db entry - update_query = Event.update(update_params).where( + query = Event.select(Event.id).where( Event.camera.not_in(self.camera_keys), Event.start_time < expire_after, Event.label == event.label, Event.retain_indefinitely == False, ) - update_query.execute() + + events_to_update = [] + + for batch in query.iterator(): + events_to_update.extend([event.id for event in batch]) + if len(events_to_update) >= CHUNK_SIZE: + logger.debug( + f"Updating {update_params} for {len(events_to_update)} events" + ) + Event.update(update_params).where( + Event.id << events_to_update + ).execute() + events_to_update = [] + + # Update any remaining events + if events_to_update: + logger.debug( + f"Updating clips/snapshots attribute for {len(events_to_update)} events" + ) + Event.update(update_params).where( + Event.id << events_to_update + ).execute() events_to_update = [] @@ -196,7 +221,11 @@ class EventCleanup(threading.Thread): logger.warning(f"Unable to delete event images: {e}") # update the clips attribute for the db entry - Event.update(update_params).where(Event.id << events_to_update).execute() + for i in range(0, len(events_to_update), CHUNK_SIZE): + batch = events_to_update[i : i + CHUNK_SIZE] + logger.debug(f"Updating {update_params} for {len(batch)} events") + Event.update(update_params).where(Event.id << batch).execute() + return events_to_update def run(self) -> None: @@ -222,10 +251,11 @@ class EventCleanup(threading.Thread): .iterator() ) events_to_delete = [e.id for e in events] + logger.debug(f"Found {len(events_to_delete)} events that can be expired") if len(events_to_delete) > 0: - chunk_size = 50 - for i in range(0, len(events_to_delete), chunk_size): - chunk = events_to_delete[i : i + chunk_size] + for i in range(0, len(events_to_delete), CHUNK_SIZE): + chunk = events_to_delete[i : i + CHUNK_SIZE] + logger.debug(f"Deleting {len(chunk)} events from the database") Event.delete().where(Event.id << chunk).execute() if self.config.semantic_search.enabled: From 15bd26c9b19008eaa7b6202c793cd1f2f7229720 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 7 Nov 2024 08:25:13 -0600 Subject: [PATCH 144/479] Re-send camera states after websocket disconnects and reconnects (#14847) --- web/src/api/ws.tsx | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/web/src/api/ws.tsx b/web/src/api/ws.tsx index b0c89d5dd..c7bb74095 100644 --- a/web/src/api/ws.tsx +++ b/web/src/api/ws.tsx @@ -69,7 +69,10 @@ function useValue(): useValueReturn { ...prevState, ...cameraStates, })); - setHasCameraState(true); + + if (Object.keys(cameraStates).length > 0) { + setHasCameraState(true); + } // we only want this to run initially when the config is loaded // eslint-disable-next-line react-hooks/exhaustive-deps }, [wsState]); @@ -93,6 +96,9 @@ function useValue(): useValueReturn { retain: false, }); }, + onClose: () => { + setHasCameraState(false); + }, shouldReconnect: () => true, retryOnError: true, }); From 0d59754be29df9bb60cf589e08a4b6385ce517e0 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Thu, 7 Nov 2024 14:27:55 -0600 Subject: [PATCH 145/479] Small genai fix (#14850) * Ensure the regenerate button shows when genai is only enabled at the camera level * update docs --- docs/docs/configuration/genai.md | 8 ++++++-- frigate/api/event.py | 4 +++- frigate/genai/__init__.py | 9 ++++----- web/src/components/overlay/detail/SearchDetailDialog.tsx | 2 +- web/src/types/frigateConfig.ts | 7 +++++++ 5 files changed, 21 insertions(+), 9 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index 2ee27f724..2ec9a6276 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -3,9 +3,13 @@ id: genai title: Generative AI --- -Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. +Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail. -Semantic Search must be enabled to use Generative AI. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail. +:::info + +Semantic Search must be enabled to use Generative AI. + +::: ## Configuration diff --git a/frigate/api/event.py b/frigate/api/event.py index ac414cdde..cf0ac26cc 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -996,9 +996,11 @@ def regenerate_description( status_code=404, ) + camera_config = request.app.frigate_config.cameras[event.camera] + if ( request.app.frigate_config.semantic_search.enabled - and request.app.frigate_config.genai.enabled + and camera_config.genai.enabled ): request.app.event_metadata_updater.publish((event.id, params.source)) diff --git a/frigate/genai/__init__.py b/frigate/genai/__init__.py index e2d509383..74fae9fea 100644 --- a/frigate/genai/__init__.py +++ b/frigate/genai/__init__.py @@ -54,11 +54,10 @@ class GenAIClient: def get_genai_client(genai_config: GenAIConfig) -> Optional[GenAIClient]: """Get the GenAI client.""" - if genai_config.enabled: - load_providers() - provider = PROVIDERS.get(genai_config.provider) - if provider: - return provider(genai_config) + load_providers() + provider = PROVIDERS.get(genai_config.provider) + if provider: + return provider(genai_config) return None diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index f158df329..f56074a52 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -477,7 +477,7 @@ function ObjectDetailsTab({ onChange={(e) => setDesc(e.target.value)} />
- {config?.genai.enabled && ( + {config?.cameras[search.camera].genai.enabled && (
-
diff --git a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx index d58d485b9..3eeb639cd 100644 --- a/web/src/components/overlay/MobileReviewSettingsDrawer.tsx +++ b/web/src/components/overlay/MobileReviewSettingsDrawer.tsx @@ -4,7 +4,7 @@ import { Button } from "../ui/button"; import { FaArrowDown, FaCalendarAlt, FaCog, FaFilter } from "react-icons/fa"; import { TimeRange } from "@/types/timeline"; import { ExportContent, ExportPreviewDialog } from "./ExportDialog"; -import { ExportMode } from "@/types/filter"; +import { ExportMode, GeneralFilter } from "@/types/filter"; import ReviewActivityCalendar from "./ReviewActivityCalendar"; import { SelectSeparator } from "../ui/select"; import { ReviewFilter, ReviewSeverity, ReviewSummary } from "@/types/review"; @@ -114,12 +114,12 @@ export default function MobileReviewSettingsDrawer({ // filters - const [currentLabels, setCurrentLabels] = useState( - filter?.labels, - ); - const [currentZones, setCurrentZones] = useState( - filter?.zones, - ); + const [currentFilter, setCurrentFilter] = useState({ + labels: filter?.labels, + zones: filter?.zones, + showAll: filter?.showAll, + ...filter, + }); if (!isMobile) { return; @@ -260,23 +260,21 @@ export default function MobileReviewSettingsDrawer({ - onUpdateFilter({ ...filter, zones: newZones }) - } - setShowAll={(showAll) => { - onUpdateFilter({ ...filter, showAll }); + onUpdateFilter={setCurrentFilter} + onApply={() => { + if (currentFilter !== filter) { + onUpdateFilter(currentFilter); + } + }} + onReset={() => { + const resetFilter: GeneralFilter = {}; + setCurrentFilter(resetFilter); + onUpdateFilter(resetFilter); }} - setCurrentLabels={setCurrentLabels} - updateLabelFilter={(newLabels) => - onUpdateFilter({ ...filter, labels: newLabels }) - } onClose={() => setDrawerMode("select")} />
diff --git a/web/src/types/filter.ts b/web/src/types/filter.ts index 09ff5b99a..b7d2223c0 100644 --- a/web/src/types/filter.ts +++ b/web/src/types/filter.ts @@ -10,3 +10,9 @@ export type FilterList = { }; export const LAST_24_HOURS_KEY = "last24Hours"; + +export type GeneralFilter = { + showAll?: boolean; + labels?: string[]; + zones?: string[]; +}; From 7bae9463b25201954f1e57afc43161b51bab1b73 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 8 Nov 2024 08:49:05 -0600 Subject: [PATCH 149/479] Small general filter bugfix (#14870) --- web/src/components/filter/ReviewFilterGroup.tsx | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/web/src/components/filter/ReviewFilterGroup.tsx b/web/src/components/filter/ReviewFilterGroup.tsx index bea816203..d31596561 100644 --- a/web/src/components/filter/ReviewFilterGroup.tsx +++ b/web/src/components/filter/ReviewFilterGroup.tsx @@ -214,7 +214,9 @@ export default function ReviewFilterGroup({ showAll={filter?.showAll == true} allZones={filterValues.zones} selectedZones={filter?.zones} - onUpdateFilter={onUpdateFilter} + onUpdateFilter={(general) => { + onUpdateFilter({ ...filter, ...general }); + }} /> )} {isMobile && mobileSettingsFeatures.length > 0 && ( @@ -293,7 +295,7 @@ type GeneralFilterButtonProps = { allZones: string[]; selectedZones?: string[]; filter?: GeneralFilter; - onUpdateFilter: (filter: ReviewFilter) => void; + onUpdateFilter: (filter: GeneralFilter) => void; }; function GeneralFilterButton({ @@ -370,7 +372,11 @@ function GeneralFilterButton({ setOpen(false); }} onReset={() => { - const resetFilter: GeneralFilter = {}; + const resetFilter: GeneralFilter = { + labels: undefined, + zones: undefined, + showAll: false, + }; setCurrentFilter(resetFilter); onUpdateFilter(resetFilter); }} From 3249ffb273cd17e1cd50247c90762a4c63150b4a Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 8 Nov 2024 09:19:49 -0600 Subject: [PATCH 150/479] Auto-unmute inbound audio when enabling two way audio (#14871) * Automatically enable audio when initiating two way talk with mic * remove check --- web/src/views/live/LiveCameraView.tsx | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/web/src/views/live/LiveCameraView.tsx b/web/src/views/live/LiveCameraView.tsx index 13ed85c4c..a3bbeea06 100644 --- a/web/src/views/live/LiveCameraView.tsx +++ b/web/src/views/live/LiveCameraView.tsx @@ -434,7 +434,13 @@ export default function LiveCameraView({ Icon={mic ? FaMicrophone : FaMicrophoneSlash} isActive={mic} title={`${mic ? "Disable" : "Enable"} Two Way Talk`} - onClick={() => setMic(!mic)} + onClick={() => { + setMic(!mic); + // Turn on audio when enabling the mic if audio is currently off + if (!mic && !audio) { + setAudio(true); + } + }} /> )} {supportsAudioOutput && preferredLiveMode != "jsmpeg" && ( From 580f35112e85249caded199c724353a417eba743 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 8 Nov 2024 12:47:46 -0600 Subject: [PATCH 151/479] revert changes to audio process to prevent shutdown hang (#14872) --- frigate/app.py | 14 +++++++++----- frigate/events/audio.py | 6 +++--- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index d0d3ab8a1..7543aef30 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -63,7 +63,6 @@ from frigate.record.cleanup import RecordingCleanup from frigate.record.export import migrate_exports from frigate.record.record import manage_recordings from frigate.review.review import manage_review_segments -from frigate.service_manager import ServiceManager from frigate.stats.emitter import StatsEmitter from frigate.stats.util import stats_init from frigate.storage import StorageMaintainer @@ -79,6 +78,7 @@ logger = logging.getLogger(__name__) class FrigateApp: def __init__(self, config: FrigateConfig) -> None: + self.audio_process: Optional[mp.Process] = None self.stop_event: MpEvent = mp.Event() self.detection_queue: Queue = mp.Queue() self.detectors: dict[str, ObjectDetectProcess] = {} @@ -449,8 +449,9 @@ class FrigateApp: ] if audio_cameras: - proc = AudioProcessor(audio_cameras, self.camera_metrics).start(wait=True) - self.processes["audio_detector"] = proc.pid or 0 + self.audio_process = AudioProcessor(audio_cameras, self.camera_metrics) + self.audio_process.start() + self.processes["audio_detector"] = self.audio_process.pid or 0 def start_timeline_processor(self) -> None: self.timeline_processor = TimelineProcessor( @@ -641,6 +642,11 @@ class FrigateApp: ReviewSegment.end_time == None ).execute() + # stop the audio process + if self.audio_process: + self.audio_process.terminate() + self.audio_process.join() + # ensure the capture processes are done for camera, metrics in self.camera_metrics.items(): capture_process = metrics.capture_process @@ -709,6 +715,4 @@ class FrigateApp: shm.close() shm.unlink() - ServiceManager.current().shutdown(wait=True) - os._exit(os.EX_OK) diff --git a/frigate/events/audio.py b/frigate/events/audio.py index 45706dcc8..80d035894 100644 --- a/frigate/events/audio.py +++ b/frigate/events/audio.py @@ -9,6 +9,7 @@ from typing import Tuple import numpy as np import requests +import frigate.util as util from frigate.camera import CameraMetrics from frigate.comms.config_updater import ConfigSubscriber from frigate.comms.detections_updater import DetectionPublisher, DetectionTypeEnum @@ -25,7 +26,6 @@ from frigate.const import ( from frigate.ffmpeg_presets import parse_preset_input from frigate.log import LogPipe from frigate.object_detection import load_labels -from frigate.service_manager import ServiceProcess from frigate.util.builtin import get_ffmpeg_arg_list from frigate.video import start_or_restart_ffmpeg, stop_ffmpeg @@ -63,7 +63,7 @@ def get_ffmpeg_command(ffmpeg: FfmpegConfig) -> list[str]: ) -class AudioProcessor(ServiceProcess): +class AudioProcessor(util.Process): name = "frigate.audio_manager" def __init__( @@ -71,7 +71,7 @@ class AudioProcessor(ServiceProcess): cameras: list[CameraConfig], camera_metrics: dict[str, CameraMetrics], ): - super().__init__() + super().__init__(name="frigate.audio_manager", daemon=True) self.camera_metrics = camera_metrics self.cameras = cameras From 143bab87f15dd162d5526ea006d0522de986e488 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sat, 9 Nov 2024 07:48:53 -0600 Subject: [PATCH 152/479] Genai bugfix (#14880) * Fix genai init when disabled at global level * use genai config for class init --- frigate/embeddings/maintainer.py | 2 +- frigate/genai/__init__.py | 19 +++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index 1578a0fe3..12c8bac72 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -62,7 +62,7 @@ class EmbeddingMaintainer(threading.Thread): self.requestor = InterProcessRequestor() self.stop_event = stop_event self.tracked_events = {} - self.genai_client = get_genai_client(config.genai) + self.genai_client = get_genai_client(config) def run(self) -> None: """Maintain a SQLite-vec database for semantic search.""" diff --git a/frigate/genai/__init__.py b/frigate/genai/__init__.py index 74fae9fea..ebbea7476 100644 --- a/frigate/genai/__init__.py +++ b/frigate/genai/__init__.py @@ -6,7 +6,7 @@ from typing import Optional from playhouse.shortcuts import model_to_dict -from frigate.config import CameraConfig, GenAIConfig, GenAIProviderEnum +from frigate.config import CameraConfig, FrigateConfig, GenAIConfig, GenAIProviderEnum from frigate.models import Event PROVIDERS = {} @@ -52,12 +52,19 @@ class GenAIClient: return None -def get_genai_client(genai_config: GenAIConfig) -> Optional[GenAIClient]: +def get_genai_client(config: FrigateConfig) -> Optional[GenAIClient]: """Get the GenAI client.""" - load_providers() - provider = PROVIDERS.get(genai_config.provider) - if provider: - return provider(genai_config) + genai_config = config.genai + genai_cameras = [ + c for c in config.cameras.values() if c.enabled and c.genai.enabled + ] + + if genai_cameras: + load_providers() + provider = PROVIDERS.get(genai_config.provider) + if provider: + return provider(genai_config) + return None From 7c474e6827b4257b82d254d636d8a76c0a55e97f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 9 Nov 2024 08:09:36 -0700 Subject: [PATCH 153/479] Pin intel driver (#14884) * Pin intel driver * Use slightly older version --- docker/main/install_deps.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/main/install_deps.sh b/docker/main/install_deps.sh index 2d7662053..247d383ac 100755 --- a/docker/main/install_deps.sh +++ b/docker/main/install_deps.sh @@ -87,7 +87,7 @@ if [[ "${TARGETARCH}" == "amd64" ]]; then echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | tee /etc/apt/sources.list.d/intel-gpu-jammy.list apt-get -qq update apt-get -qq install --no-install-recommends --no-install-suggests -y \ - intel-opencl-icd intel-level-zero-gpu intel-media-va-driver-non-free \ + intel-opencl-icd intel-level-zero-gpu intel-media-va-driver-non-free=24.2.4-914~22.04 \ libmfx1 libmfxgen1 libvpl2 rm -f /usr/share/keyrings/intel-graphics.gpg From a68c7f4ef859cfb15cb2275865dd46500e17e148 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 9 Nov 2024 10:08:25 -0700 Subject: [PATCH 154/479] Pin all intel packages (#14887) --- docker/main/install_deps.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/main/install_deps.sh b/docker/main/install_deps.sh index 247d383ac..6c32ae168 100755 --- a/docker/main/install_deps.sh +++ b/docker/main/install_deps.sh @@ -87,8 +87,8 @@ if [[ "${TARGETARCH}" == "amd64" ]]; then echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | tee /etc/apt/sources.list.d/intel-gpu-jammy.list apt-get -qq update apt-get -qq install --no-install-recommends --no-install-suggests -y \ - intel-opencl-icd intel-level-zero-gpu intel-media-va-driver-non-free=24.2.4-914~22.04 \ - libmfx1 libmfxgen1 libvpl2 + intel-opencl-icd=24.35.30872.31-996~22.04 intel-level-zero-gpu=1.3.29735.27-914~22.04 intel-media-va-driver-non-free=24.3.3-996~22.04 \ + libmfx1=23.2.2-880~22.04 libmfxgen1=24.2.4-914~22.04 libvpl2=1:2.13.0.0-996~22.04 rm -f /usr/share/keyrings/intel-graphics.gpg rm -f /etc/apt/sources.list.d/intel-gpu-jammy.list From 96c0c43dc8d61bafd1be258c9cbc4b1df9204411 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 10 Nov 2024 07:43:24 -0700 Subject: [PATCH 155/479] Add support for specifying tensorrt device (#14898) --- frigate/util/model.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/frigate/util/model.py b/frigate/util/model.py index 091bb0833..ce2c9538c 100644 --- a/frigate/util/model.py +++ b/frigate/util/model.py @@ -33,10 +33,12 @@ def get_ort_providers( for provider in ort.get_available_providers(): if provider == "CUDAExecutionProvider": + device_id = 0 if not device.isdigit() else int(device) providers.append(provider) options.append( { "arena_extend_strategy": "kSameAsRequested", + "device_id": device_id, } ) elif provider == "TensorrtExecutionProvider": @@ -46,10 +48,11 @@ def get_ort_providers( os.makedirs( "/config/model_cache/tensorrt/ort/trt-engines", exist_ok=True ) + device_id = 0 if not device.isdigit() else int(device) providers.append(provider) options.append( { - "arena_extend_strategy": "kSameAsRequested", + "device_id": device_id, "trt_fp16_enable": requires_fp16 and os.environ.get("USE_FP_16", "True") != "False", "trt_timing_cache_enable": True, From c1bfc1df678a72804876a889461afce4d4ac4d27 Mon Sep 17 00:00:00 2001 From: Austin Kirsch <37373320+kirsch33@users.noreply.github.com> Date: Sun, 10 Nov 2024 17:23:32 -0500 Subject: [PATCH 156/479] fix tensorrt model generation variable (#14902) --- .../detector/rootfs/etc/s6-overlay/s6-rc.d/trt-model-prepare/run | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/tensorrt/detector/rootfs/etc/s6-overlay/s6-rc.d/trt-model-prepare/run b/docker/tensorrt/detector/rootfs/etc/s6-overlay/s6-rc.d/trt-model-prepare/run index 4d734e05a..a88da89d6 100755 --- a/docker/tensorrt/detector/rootfs/etc/s6-overlay/s6-rc.d/trt-model-prepare/run +++ b/docker/tensorrt/detector/rootfs/etc/s6-overlay/s6-rc.d/trt-model-prepare/run @@ -11,6 +11,7 @@ set -o errexit -o nounset -o pipefail MODEL_CACHE_DIR=${MODEL_CACHE_DIR:-"/config/model_cache/tensorrt"} TRT_VER=${TRT_VER:-$(cat /etc/TENSORRT_VER)} OUTPUT_FOLDER="${MODEL_CACHE_DIR}/${TRT_VER}" +YOLO_MODELS=${YOLO_MODELS:-""} # Create output folder mkdir -p ${OUTPUT_FOLDER} From 0829517b72c117583c92917c580fd93daf57841e Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 10 Nov 2024 16:57:11 -0600 Subject: [PATCH 157/479] Add ability to filter Explore by Frigate+ submission status (#14909) * backend * add is_submitted to query params * add submitted filter to dialog * allow is_submitted filter selection with input --- frigate/api/defs/events_query_parameters.py | 1 + frigate/api/event.py | 7 ++ web/src/components/input/InputWithTags.tsx | 19 ++- .../overlay/dialog/SearchFilterDialog.tsx | 114 ++++++++++++++++-- web/src/pages/Explore.tsx | 2 + web/src/types/search.ts | 1 + web/src/views/search/SearchView.tsx | 4 +- 7 files changed, 135 insertions(+), 13 deletions(-) diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/events_query_parameters.py index 4639b7f59..5a2b61d43 100644 --- a/frigate/api/defs/events_query_parameters.py +++ b/frigate/api/defs/events_query_parameters.py @@ -47,6 +47,7 @@ class EventsSearchQueryParams(BaseModel): time_range: Optional[str] = DEFAULT_TIME_RANGE has_clip: Optional[bool] = None has_snapshot: Optional[bool] = None + is_submitted: Optional[bool] = None timezone: Optional[str] = "utc" min_score: Optional[float] = None max_score: Optional[float] = None diff --git a/frigate/api/event.py b/frigate/api/event.py index cf0ac26cc..bff1edc1a 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -360,6 +360,7 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) time_range = params.time_range has_clip = params.has_clip has_snapshot = params.has_snapshot + is_submitted = params.is_submitted # for similarity search event_id = params.event_id @@ -441,6 +442,12 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) if has_snapshot is not None: event_filters.append((Event.has_snapshot == has_snapshot)) + if is_submitted is not None: + if is_submitted == 0: + event_filters.append((Event.plus_id.is_null())) + elif is_submitted > 0: + event_filters.append((Event.plus_id != "")) + if min_score is not None and max_score is not None: event_filters.append((Event.data["score"].between(min_score, max_score))) else: diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index e5b492bcc..8f60bb73e 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -194,6 +194,11 @@ export default function InputWithTags({ if (newFilters[filterType] === filterValue) { delete newFilters[filterType]; } + } else if (filterType === "has_snapshot") { + if (newFilters[filterType] === filterValue) { + delete newFilters[filterType]; + delete newFilters["is_submitted"]; + } } else { delete newFilters[filterType]; } @@ -307,6 +312,10 @@ export default function InputWithTags({ if (!newFilters.has_snapshot) newFilters.has_snapshot = undefined; newFilters.has_snapshot = value == "yes" ? 1 : 0; break; + case "is_submitted": + if (!newFilters.is_submitted) newFilters.is_submitted = undefined; + newFilters.is_submitted = value == "yes" ? 1 : 0; + break; case "has_clip": if (!newFilters.has_clip) newFilters.has_clip = undefined; newFilters.has_clip = value == "yes" ? 1 : 0; @@ -356,7 +365,11 @@ export default function InputWithTags({ }`; } else if (filterType === "min_score" || filterType === "max_score") { return Math.round(Number(filterValues) * 100).toString() + "%"; - } else if (filterType === "has_clip" || filterType === "has_snapshot") { + } else if ( + filterType === "has_clip" || + filterType === "has_snapshot" || + filterType === "is_submitted" + ) { return filterValues ? "Yes" : "No"; } else { return filterValues as string; @@ -774,7 +787,9 @@ export default function InputWithTags({ > {filterType === "event_id" ? "Tracked Object ID" - : filterType.replaceAll("_", " ")} + : filterType === "is_submitted" + ? "Submitted to Frigate+" + : filterType.replaceAll("_", " ")} : {formatFilterValues(filterType, filterValues)}
diff --git a/web/src/components/card/SearchThumbnailFooter.tsx b/web/src/components/card/SearchThumbnailFooter.tsx index b959a82c5..33db0c598 100644 --- a/web/src/components/card/SearchThumbnailFooter.tsx +++ b/web/src/components/card/SearchThumbnailFooter.tsx @@ -13,6 +13,7 @@ type SearchThumbnailProps = { findSimilar: () => void; refreshResults: () => void; showObjectLifecycle: () => void; + showSnapshot: () => void; }; export default function SearchThumbnailFooter({ @@ -21,6 +22,7 @@ export default function SearchThumbnailFooter({ findSimilar, refreshResults, showObjectLifecycle, + showSnapshot, }: SearchThumbnailProps) { const { data: config } = useSWR("config"); @@ -54,6 +56,7 @@ export default function SearchThumbnailFooter({ findSimilar={findSimilar} refreshResults={refreshResults} showObjectLifecycle={showObjectLifecycle} + showSnapshot={showSnapshot} />
diff --git a/web/src/components/menu/SearchResultActions.tsx b/web/src/components/menu/SearchResultActions.tsx index a07d27240..10f0ed623 100644 --- a/web/src/components/menu/SearchResultActions.tsx +++ b/web/src/components/menu/SearchResultActions.tsx @@ -37,15 +37,14 @@ import { TooltipContent, TooltipTrigger, } from "@/components/ui/tooltip"; -import { FrigatePlusDialog } from "@/components/overlay/dialog/FrigatePlusDialog"; import useSWR from "swr"; -import { Event } from "@/types/event"; type SearchResultActionsProps = { searchResult: SearchResult; findSimilar: () => void; refreshResults: () => void; showObjectLifecycle: () => void; + showSnapshot: () => void; isContextMenu?: boolean; children?: ReactNode; }; @@ -55,12 +54,12 @@ export default function SearchResultActions({ findSimilar, refreshResults, showObjectLifecycle, + showSnapshot, isContextMenu = false, children, }: SearchResultActionsProps) { const { data: config } = useSWR("config"); - const [showFrigatePlus, setShowFrigatePlus] = useState(false); const [deleteDialogOpen, setDeleteDialogOpen] = useState(false); const handleDelete = () => { @@ -130,10 +129,7 @@ export default function SearchResultActions({ searchResult.has_snapshot && searchResult.end_time && !searchResult.plus_id && ( - setShowFrigatePlus(true)} - > + Submit to Frigate+ @@ -178,16 +174,6 @@ export default function SearchResultActions({ - setShowFrigatePlus(false)} - onEventUploaded={() => { - searchResult.plus_id = "submitted"; - }} - /> - {isContextMenu ? ( {children} @@ -216,7 +202,7 @@ export default function SearchResultActions({ setShowFrigatePlus(true)} + onClick={showSnapshot} /> Submit to Frigate+ diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index d40f68a1e..74a9950b9 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -161,7 +161,7 @@ export default function ReviewDetailDialog({ )} > - + - )} - - - - )} - {state == "uploading" && } - -
- - ); - + if (!upload) { + return; + } if (dialog) { return ( (!open ? onClose() : null)} > - - {content} + + + Submit to Frigate+ + + Submit this snapshot to Frigate+ + + + ); } - - return content; } diff --git a/web/src/views/explore/ExploreView.tsx b/web/src/views/explore/ExploreView.tsx index 00ed19ab2..f37c37453 100644 --- a/web/src/views/explore/ExploreView.tsx +++ b/web/src/views/explore/ExploreView.tsx @@ -228,12 +228,17 @@ function ExploreThumbnailImage({ onSelectSearch(event, 0, "object lifecycle"); }; + const handleShowSnapshot = () => { + onSelectSearch(event, 0, "snapshot"); + }; + return (
diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 10ec2e02e..378b313e0 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -471,6 +471,9 @@ export default function SearchView({ showObjectLifecycle={() => onSelectSearch(value, index, "object lifecycle") } + showSnapshot={() => + onSelectSearch(value, index, "snapshot") + } />
From 99506845f70b6cf5fba14fa9073bfc2ae5254d72 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 12 Nov 2024 14:48:57 -0700 Subject: [PATCH 163/479] Update edge tpu docs for RPi 5 kernel (#14946) --- docs/docs/troubleshooting/edgetpu.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/docs/troubleshooting/edgetpu.md b/docs/docs/troubleshooting/edgetpu.md index 33e00f11a..8f3cb0db7 100644 --- a/docs/docs/troubleshooting/edgetpu.md +++ b/docs/docs/troubleshooting/edgetpu.md @@ -54,6 +54,17 @@ The most common reason for the PCIe Coral not being detected is that the driver - In most cases [the Coral docs](https://coral.ai/docs/m2/get-started/#2-install-the-pcie-driver-and-edge-tpu-runtime) show how to install the driver for the PCIe based Coral. - For Ubuntu 22.04+ https://github.com/jnicolson/gasket-builder can be used to build and install the latest version of the driver. +### Not detected on Raspberry Pi5 + +A kernel update to the RPi5 means an upate to config.txt is required, see [the raspberry pi forum for more info](https://forums.raspberrypi.com/viewtopic.php?t=363682&sid=cb59b026a412f0dc041595951273a9ca&start=25) + +Specifically, add the following to config.txt + +``` +dtoverlay=pciex1-compat-pi5,no-mip +dtoverlay=pcie-32bit-dma-pi5 +``` + ## Only One PCIe Coral Is Detected With Coral Dual EdgeTPU Coral Dual EdgeTPU is one card with two identical TPU cores. Each core has it's own PCIe interface and motherboard needs to have two PCIe busses on the m.2 slot to make them both work. From 1ffdd3201339813c735c96d1f0be8bda0e0438e0 Mon Sep 17 00:00:00 2001 From: Charles Crossan Date: Thu, 14 Nov 2024 10:13:37 -0500 Subject: [PATCH 164/479] Update authentication.md (#14980) add detail to reset_admin_password setting --- docs/docs/configuration/authentication.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/docs/configuration/authentication.md b/docs/docs/configuration/authentication.md index 47d7e85a3..91b93cc58 100644 --- a/docs/docs/configuration/authentication.md +++ b/docs/docs/configuration/authentication.md @@ -24,6 +24,11 @@ On startup, an admin user and password are generated and printed in the logs. It In the event that you are locked out of your instance, you can tell Frigate to reset the admin password and print it in the logs on next startup using the `reset_admin_password` setting in your config file. +```yaml +auth: + reset_admin_password: true +``` + ## Login failure rate limiting In order to limit the risk of brute force attacks, rate limiting is available for login failures. This is implemented with Flask-Limiter, and the string notation for valid values is available in [the documentation](https://flask-limiter.readthedocs.io/en/stable/configuration.html#rate-limit-string-notation). From 4eea541352a3c561e7dd8d86f58cfc05c1349b26 Mon Sep 17 00:00:00 2001 From: Levi Tomes <4184677+ltomes@users.noreply.github.com> Date: Fri, 15 Nov 2024 06:35:43 -0600 Subject: [PATCH 165/479] Updated Documentation: Autotracking add support details for Sunba 405-D20X 4K camera. (#14352) * Add support details for Sunba 405-D20X 4K camera. * Update cameras.md Updated changes to meet documentation goals of upstream project. --- docs/docs/configuration/cameras.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration/cameras.md b/docs/docs/configuration/cameras.md index b7c2798e1..50a8c6f93 100644 --- a/docs/docs/configuration/cameras.md +++ b/docs/docs/configuration/cameras.md @@ -109,7 +109,7 @@ This list of working and non-working PTZ cameras is based on user feedback. | Reolink E1 Zoom | ✅ | ❌ | | | Reolink RLC-823A 16x | ✅ | ❌ | | | Speco O8P32X | ✅ | ❌ | | -| Sunba 405-D20X | ✅ | ❌ | | +| Sunba 405-D20X | ✅ | ❌ | Incomplete ONVIF support reported on original, and 4k models. All models are suspected incompatable. | | Tapo | ✅ | ❌ | Many models supported, ONVIF Service Port: 2020 | | Uniview IPC672LR-AX4DUPK | ✅ | ❌ | Firmware says FOV relative movement is supported, but camera doesn't actually move when sending ONVIF commands | | Uniview IPC6612SR-X33-VG | ✅ | ✅ | Leave `calibrate_on_startup` as `False`. A user has reported that zooming with `absolute` is working. | From 7fdf42a56f452285ac296dcbaf4301c8ac154dbf Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 15 Nov 2024 09:54:59 -0700 Subject: [PATCH 166/479] Various Fixes (#15004) * Don't track shared memory in frame tracker * Don't track any instance * Don't assign sub label to objects when multiple cars are overlapping * Formatting * Fix assignment --- frigate/track/tracked_object.py | 17 ++++++--- frigate/util/image.py | 64 +++++++++++++++++++++++---------- 2 files changed, 58 insertions(+), 23 deletions(-) diff --git a/frigate/track/tracked_object.py b/frigate/track/tracked_object.py index a4b4e8426..65e7a2ed5 100644 --- a/frigate/track/tracked_object.py +++ b/frigate/track/tracked_object.py @@ -4,6 +4,7 @@ import base64 import logging from collections import defaultdict from statistics import median +from typing import Optional import cv2 import numpy as np @@ -423,10 +424,11 @@ class TrackedObjectAttribute: "box": self.box, } - def find_best_object(self, objects: list[dict[str, any]]) -> str: + def find_best_object(self, objects: list[dict[str, any]]) -> Optional[str]: """Find the best attribute for each object and return its ID.""" best_object_area = None best_object_id = None + best_object_label = None for obj in objects: if not box_inside(obj["box"], self.box): @@ -440,8 +442,15 @@ class TrackedObjectAttribute: if best_object_area is None: best_object_area = object_area best_object_id = obj["id"] - elif object_area < best_object_area: - best_object_area = object_area - best_object_id = obj["id"] + best_object_label = obj["label"] + else: + if best_object_label == "car" and obj["label"] == "car": + # if multiple cars are overlapping with the same label then the label will not be assigned + return None + elif object_area < best_object_area: + # if a car and person are overlapping then assign the label to the smaller object (which should be the person) + best_object_area = object_area + best_object_id = obj["id"] + best_object_label = obj["label"] return best_object_id diff --git a/frigate/util/image.py b/frigate/util/image.py index 484737f71..763f0dfab 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -3,8 +3,10 @@ import datetime import logging import subprocess as sp +import threading from abc import ABC, abstractmethod -from multiprocessing import shared_memory +from multiprocessing import resource_tracker as _mprt +from multiprocessing import shared_memory as _mpshm from string import printable from typing import AnyStr, Optional @@ -731,32 +733,56 @@ class FrameManager(ABC): pass -class DictFrameManager(FrameManager): - def __init__(self): - self.frames = {} +class SharedMemory(_mpshm.SharedMemory): + # https://github.com/python/cpython/issues/82300#issuecomment-2169035092 - def create(self, name, size) -> AnyStr: - mem = bytearray(size) - self.frames[name] = mem - return mem + __lock = threading.Lock() - def get(self, name, shape): - mem = self.frames[name] - return np.ndarray(shape, dtype=np.uint8, buffer=mem) + def __init__( + self, + name: Optional[str] = None, + create: bool = False, + size: int = 0, + *, + track: bool = True, + ) -> None: + self._track = track - def close(self, name): - pass + # if tracking, normal init will suffice + if track: + return super().__init__(name=name, create=create, size=size) - def delete(self, name): - del self.frames[name] + # lock so that other threads don't attempt to use the + # register function during this time + with self.__lock: + # temporarily disable registration during initialization + orig_register = _mprt.register + _mprt.register = self.__tmp_register + + # initialize; ensure original register function is + # re-instated + try: + super().__init__(name=name, create=create, size=size) + finally: + _mprt.register = orig_register + + @staticmethod + def __tmp_register(*args, **kwargs) -> None: + return + + def unlink(self) -> None: + if _mpshm._USE_POSIX and self._name: + _mpshm._posixshmem.shm_unlink(self._name) + if self._track: + _mprt.unregister(self._name, "shared_memory") class SharedMemoryFrameManager(FrameManager): def __init__(self): - self.shm_store: dict[str, shared_memory.SharedMemory] = {} + self.shm_store: dict[str, SharedMemory] = {} def create(self, name: str, size) -> AnyStr: - shm = shared_memory.SharedMemory(name=name, create=True, size=size) + shm = SharedMemory(name=name, create=True, size=size, track=False) self.shm_store[name] = shm return shm.buf @@ -765,7 +791,7 @@ class SharedMemoryFrameManager(FrameManager): if name in self.shm_store: shm = self.shm_store[name] else: - shm = shared_memory.SharedMemory(name=name) + shm = SharedMemory(name=name, track=False) self.shm_store[name] = shm return np.ndarray(shape, dtype=np.uint8, buffer=shm.buf) except FileNotFoundError: @@ -788,7 +814,7 @@ class SharedMemoryFrameManager(FrameManager): del self.shm_store[name] else: try: - shm = shared_memory.SharedMemory(name=name) + shm = SharedMemory(name=name, track=False) shm.close() shm.unlink() except FileNotFoundError: From e407ba47c200424a6983a4db521cdf504f1cfd78 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 15 Nov 2024 13:25:57 -0700 Subject: [PATCH 167/479] Increase max shm frames (#15009) --- frigate/app.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/app.py b/frigate/app.py index 7543aef30..3477a7e84 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -516,7 +516,7 @@ class FrigateApp: if cam_total_frame_size == 0.0: return 0 - shm_frame_count = min(50, int(available_shm / (cam_total_frame_size))) + shm_frame_count = min(200, int(available_shm / (cam_total_frame_size))) logger.debug( f"Calculated total camera size {available_shm} / {cam_total_frame_size} :: {shm_frame_count} frames for each camera in SHM" From 206ed0690519c4f24207b0889d6bb7431f8f04a7 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 15 Nov 2024 14:14:37 -0700 Subject: [PATCH 168/479] Make all SHM management untracked (#15011) --- frigate/app.py | 9 +++++---- frigate/object_detection.py | 10 ++++------ frigate/util/image.py | 16 ++++++++++------ 3 files changed, 19 insertions(+), 16 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index 3477a7e84..96edfbd15 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -68,6 +68,7 @@ from frigate.stats.util import stats_init from frigate.storage import StorageMaintainer from frigate.timeline import TimelineProcessor from frigate.util.builtin import empty_and_close_queue +from frigate.util.image import UntrackedSharedMemory from frigate.util.object import get_camera_regions_grid from frigate.version import VERSION from frigate.video import capture_camera, track_camera @@ -325,20 +326,20 @@ class FrigateApp: for det in self.config.detectors.values() ] ) - shm_in = mp.shared_memory.SharedMemory( + shm_in = UntrackedSharedMemory( name=name, create=True, size=largest_frame, ) except FileExistsError: - shm_in = mp.shared_memory.SharedMemory(name=name) + shm_in = UntrackedSharedMemory(name=name) try: - shm_out = mp.shared_memory.SharedMemory( + shm_out = UntrackedSharedMemory( name=f"out-{name}", create=True, size=20 * 6 * 4 ) except FileExistsError: - shm_out = mp.shared_memory.SharedMemory(name=f"out-{name}") + shm_out = UntrackedSharedMemory(name=f"out-{name}") self.detection_shms.append(shm_in) self.detection_shms.append(shm_out) diff --git a/frigate/object_detection.py b/frigate/object_detection.py index cc4641696..022e565f0 100644 --- a/frigate/object_detection.py +++ b/frigate/object_detection.py @@ -19,7 +19,7 @@ from frigate.detectors.detector_config import ( ) from frigate.detectors.plugins.rocm import DETECTOR_KEY as ROCM_DETECTOR_KEY from frigate.util.builtin import EventsPerSecond, load_labels -from frigate.util.image import SharedMemoryFrameManager +from frigate.util.image import SharedMemoryFrameManager, UntrackedSharedMemory from frigate.util.services import listen logger = logging.getLogger(__name__) @@ -122,7 +122,7 @@ def run_detector( outputs = {} for name in out_events.keys(): - out_shm = mp.shared_memory.SharedMemory(name=f"out-{name}", create=False) + out_shm = UntrackedSharedMemory(name=f"out-{name}", create=False) out_np = np.ndarray((20, 6), dtype=np.float32, buffer=out_shm.buf) outputs[name] = {"shm": out_shm, "np": out_np} @@ -212,15 +212,13 @@ class RemoteObjectDetector: self.detection_queue = detection_queue self.event = event self.stop_event = stop_event - self.shm = mp.shared_memory.SharedMemory(name=self.name, create=False) + self.shm = UntrackedSharedMemory(name=self.name, create=False) self.np_shm = np.ndarray( (1, model_config.height, model_config.width, 3), dtype=np.uint8, buffer=self.shm.buf, ) - self.out_shm = mp.shared_memory.SharedMemory( - name=f"out-{self.name}", create=False - ) + self.out_shm = UntrackedSharedMemory(name=f"out-{self.name}", create=False) self.out_np_shm = np.ndarray((20, 6), dtype=np.float32, buffer=self.out_shm.buf) def detect(self, tensor_input, threshold=0.4): diff --git a/frigate/util/image.py b/frigate/util/image.py index 763f0dfab..cf1332752 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -733,7 +733,7 @@ class FrameManager(ABC): pass -class SharedMemory(_mpshm.SharedMemory): +class UntrackedSharedMemory(_mpshm.SharedMemory): # https://github.com/python/cpython/issues/82300#issuecomment-2169035092 __lock = threading.Lock() @@ -744,7 +744,7 @@ class SharedMemory(_mpshm.SharedMemory): create: bool = False, size: int = 0, *, - track: bool = True, + track: bool = False, ) -> None: self._track = track @@ -779,10 +779,14 @@ class SharedMemory(_mpshm.SharedMemory): class SharedMemoryFrameManager(FrameManager): def __init__(self): - self.shm_store: dict[str, SharedMemory] = {} + self.shm_store: dict[str, UntrackedSharedMemory] = {} def create(self, name: str, size) -> AnyStr: - shm = SharedMemory(name=name, create=True, size=size, track=False) + shm = UntrackedSharedMemory( + name=name, + create=True, + size=size, + ) self.shm_store[name] = shm return shm.buf @@ -791,7 +795,7 @@ class SharedMemoryFrameManager(FrameManager): if name in self.shm_store: shm = self.shm_store[name] else: - shm = SharedMemory(name=name, track=False) + shm = UntrackedSharedMemory(name=name) self.shm_store[name] = shm return np.ndarray(shape, dtype=np.uint8, buffer=shm.buf) except FileNotFoundError: @@ -814,7 +818,7 @@ class SharedMemoryFrameManager(FrameManager): del self.shm_store[name] else: try: - shm = SharedMemory(name=name, track=False) + shm = UntrackedSharedMemory(name=name) shm.close() shm.unlink() except FileNotFoundError: From ad85f8882b7f0cc01ead65444fc4c5d178336cb1 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 15 Nov 2024 15:24:17 -0600 Subject: [PATCH 169/479] Update ollama docs and add genai debug logging (#15012) --- docs/docs/configuration/genai.md | 4 ++-- frigate/genai/__init__.py | 4 ++++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index 2ec9a6276..da4b8afd8 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -35,7 +35,7 @@ cameras: :::warning -Using Ollama on CPU is not recommended, high inference times and a lack of support for multi-modal parallel requests will make using Generative AI impractical. +Using Ollama on CPU is not recommended, high inference times make using Generative AI impractical. ::: @@ -43,7 +43,7 @@ Using Ollama on CPU is not recommended, high inference times and a lack of suppo Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available. -Parallel requests also come with some caveats, and multi-modal parallel requests are currently not supported by Ollama. Depending on your hardware and the number of requests made to the Ollama API, these limitations may prevent Ollama from being an optimal solution for many users. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests). +Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests). ### Supported Models diff --git a/frigate/genai/__init__.py b/frigate/genai/__init__.py index ebbea7476..2c0aadbd9 100644 --- a/frigate/genai/__init__.py +++ b/frigate/genai/__init__.py @@ -1,6 +1,7 @@ """Generative AI module for Frigate.""" import importlib +import logging import os from typing import Optional @@ -9,6 +10,8 @@ from playhouse.shortcuts import model_to_dict from frigate.config import CameraConfig, FrigateConfig, GenAIConfig, GenAIProviderEnum from frigate.models import Event +logger = logging.getLogger(__name__) + PROVIDERS = {} @@ -41,6 +44,7 @@ class GenAIClient: event.label, camera_config.genai.prompt, ).format(**model_to_dict(event)) + logger.debug(f"Sending images to genai provider with prompt: {prompt}") return self._send(prompt, thumbnails) def _init_provider(self): From f9c1600f0dd86eb532620daffaf8e98924de3a3e Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 16 Nov 2024 12:24:42 -0700 Subject: [PATCH 170/479] Duplicate onnx build info (#15020) --- docs/docs/configuration/object_detectors.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index 24888ae42..5982da2ec 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -415,6 +415,24 @@ Note that the labelmap uses a subset of the complete COCO label set that has onl ONNX is an open format for building machine learning models, Frigate supports running ONNX models on CPU, OpenVINO, and TensorRT. On startup Frigate will automatically try to use a GPU if one is available. +:::info + +If the correct build is used for your GPU then the GPU will be detected and used automatically. + +- **AMD** + + - ROCm will automatically be detected and used with the ONNX detector in the `-rocm` Frigate image. + +- **Intel** + + - OpenVINO will automatically be detected and used with the ONNX detector in the default Frigate image. + +- **Nvidia** + - Nvidia GPUs will automatically be detected and used with the ONNX detector in the `-tensorrt` Frigate image. + - Jetson devices will automatically be detected and used with the ONNX detector in the `-tensorrt-jp(4/5)` Frigate image. + +::: + :::tip When using many cameras one detector may not be enough to keep up. Multiple detectors can be defined assuming GPU resources are available. An example configuration would be: From 45e9030358acddc84911a49d07866215bd0e0ce4 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 16 Nov 2024 16:00:19 -0700 Subject: [PATCH 171/479] Round robin SHM management (#15027) * Output frame name to frames processor * Finish implementing round robin * Formatting --- frigate/object_processing.py | 23 +++++++------ frigate/output/birdseye.py | 35 +++++++++---------- frigate/output/output.py | 17 +++++---- frigate/ptz/autotrack.py | 13 ++++--- frigate/record/maintainer.py | 1 + frigate/review/maintainer.py | 32 +++++++++-------- frigate/track/__init__.py | 4 ++- frigate/track/centroid_tracker.py | 2 +- frigate/track/norfair_tracker.py | 10 +++--- frigate/util/image.py | 37 +++++++++++++++++--- frigate/video.py | 57 ++++++++++++++----------------- 11 files changed, 134 insertions(+), 97 deletions(-) diff --git a/frigate/object_processing.py b/frigate/object_processing.py index ca7ea47b8..23c84eedd 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -233,17 +233,18 @@ class CameraState: def on(self, event_type: str, callback: Callable[[dict], None]): self.callbacks[event_type].append(callback) - def update(self, frame_time, current_detections, motion_boxes, regions): - # get the new frame - frame_id = f"{self.name}{frame_time}" - + def update( + self, + frame_name: str, + frame_time: float, + current_detections: dict[str, dict[str, any]], + motion_boxes: list[tuple[int, int, int, int]], + regions: list[tuple[int, int, int, int]], + ): current_frame = self.frame_manager.get( - frame_id, self.camera_config.frame_shape_yuv + frame_name, self.camera_config.frame_shape_yuv ) - if current_frame is None: - logger.debug(f"Failed to get frame {frame_id} from SHM") - tracked_objects = self.tracked_objects.copy() current_ids = set(current_detections.keys()) previous_ids = set(tracked_objects.keys()) @@ -477,7 +478,7 @@ class CameraState: if self.previous_frame_id is not None: self.frame_manager.close(self.previous_frame_id) - self.previous_frame_id = frame_id + self.previous_frame_id = frame_name class TrackedObjectProcessor(threading.Thread): @@ -798,6 +799,7 @@ class TrackedObjectProcessor(threading.Thread): try: ( camera, + frame_name, frame_time, current_tracked_objects, motion_boxes, @@ -809,7 +811,7 @@ class TrackedObjectProcessor(threading.Thread): camera_state = self.camera_states[camera] camera_state.update( - frame_time, current_tracked_objects, motion_boxes, regions + frame_name, frame_time, current_tracked_objects, motion_boxes, regions ) self.update_mqtt_motion(camera, frame_time, motion_boxes) @@ -822,6 +824,7 @@ class TrackedObjectProcessor(threading.Thread): self.detection_publisher.publish( ( camera, + frame_name, frame_time, tracked_objects, motion_boxes, diff --git a/frigate/output/birdseye.py b/frigate/output/birdseye.py index c187c77ea..cab155b9b 100644 --- a/frigate/output/birdseye.py +++ b/frigate/output/birdseye.py @@ -268,12 +268,10 @@ class BirdsEyeFrameManager: def __init__( self, config: FrigateConfig, - frame_manager: SharedMemoryFrameManager, stop_event: mp.Event, ): self.config = config self.mode = config.birdseye.mode - self.frame_manager = frame_manager width, height = get_canvas_shape(config.birdseye.width, config.birdseye.height) self.frame_shape = (height, width) self.yuv_shape = (height * 3 // 2, width) @@ -351,18 +349,13 @@ class BirdsEyeFrameManager: logger.debug("Clearing the birdseye frame") self.frame[:] = self.blank_frame - def copy_to_position(self, position, camera=None, frame_time=None): + def copy_to_position(self, position, camera=None, frame: np.ndarray = None): if camera is None: frame = None channel_dims = None else: - frame_id = f"{camera}{frame_time}" - frame = self.frame_manager.get( - frame_id, self.config.cameras[camera].frame_shape_yuv - ) - if frame is None: - logger.debug(f"Unable to copy frame {camera}{frame_time} to birdseye.") + logger.debug(f"Unable to copy frame {camera} to birdseye.") return channel_dims = self.cameras[camera]["channel_dims"] @@ -375,8 +368,6 @@ class BirdsEyeFrameManager: channel_dims, ) - self.frame_manager.close(frame_id) - def camera_active(self, mode, object_box_count, motion_box_count): if mode == BirdseyeModeEnum.continuous: return True @@ -387,7 +378,7 @@ class BirdsEyeFrameManager: if mode == BirdseyeModeEnum.objects and object_box_count > 0: return True - def update_frame(self): + def update_frame(self, frame: np.ndarray): """Update to a new frame for birdseye.""" # determine how many cameras are tracking objects within the last inactivity_threshold seconds @@ -524,7 +515,9 @@ class BirdsEyeFrameManager: for row in self.camera_layout: for position in row: self.copy_to_position( - position[1], position[0], self.cameras[position[0]]["current_frame"] + position[1], + position[0], + frame, ) return True @@ -672,7 +665,14 @@ class BirdsEyeFrameManager: else: return standard_candidate_layout - def update(self, camera, object_count, motion_count, frame_time, frame) -> bool: + def update( + self, + camera: str, + object_count: int, + motion_count: int, + frame_time: float, + frame: np.ndarray, + ) -> bool: # don't process if birdseye is disabled for this camera camera_config = self.config.cameras[camera].birdseye @@ -700,7 +700,7 @@ class BirdsEyeFrameManager: return False try: - updated_frame = self.update_frame() + updated_frame = self.update_frame(frame) except Exception: updated_frame = False self.active_cameras = [] @@ -737,12 +737,11 @@ class Birdseye: self.broadcaster = BroadcastThread( "birdseye", self.converter, websocket_server, stop_event ) - frame_manager = SharedMemoryFrameManager() - self.birdseye_manager = BirdsEyeFrameManager(config, frame_manager, stop_event) + self.birdseye_manager = BirdsEyeFrameManager(config, stop_event) self.config_subscriber = ConfigSubscriber("config/birdseye/") if config.birdseye.restream: - self.birdseye_buffer = frame_manager.create( + self.birdseye_buffer = SharedMemoryFrameManager().create( "birdseye", self.birdseye_manager.yuv_shape[0] * self.birdseye_manager.yuv_shape[1], ) diff --git a/frigate/output/output.py b/frigate/output/output.py index 5d564b936..bb2d73511 100644 --- a/frigate/output/output.py +++ b/frigate/output/output.py @@ -88,18 +88,17 @@ def output_frames( ( camera, + frame_name, frame_time, current_tracked_objects, motion_boxes, - regions, + _, ) = data - frame_id = f"{camera}{frame_time}" - - frame = frame_manager.get(frame_id, config.cameras[camera].frame_shape_yuv) + frame = frame_manager.get(frame_name, config.cameras[camera].frame_shape_yuv) if frame is None: - logger.debug(f"Failed to get frame {frame_id} from SHM") + logger.debug(f"Failed to get frame {frame_name} from SHM") failed_frame_requests[camera] = failed_frame_requests.get(camera, 0) + 1 if failed_frame_requests[camera] > config.cameras[camera].detect.fps: @@ -152,7 +151,7 @@ def output_frames( preview_recorders[camera].flag_offline(frame_time) preview_write_times[camera] = frame_time - frame_manager.close(frame_id) + frame_manager.close(frame_name) move_preview_frames("clips") @@ -164,15 +163,15 @@ def output_frames( ( camera, + frame_name, frame_time, current_tracked_objects, motion_boxes, regions, ) = data - frame_id = f"{camera}{frame_time}" - frame = frame_manager.get(frame_id, config.cameras[camera].frame_shape_yuv) - frame_manager.close(frame_id) + frame = frame_manager.get(frame_name, config.cameras[camera].frame_shape_yuv) + frame_manager.close(frame_name) detection_subscriber.stop() diff --git a/frigate/ptz/autotrack.py b/frigate/ptz/autotrack.py index 24b12087d..03bb3840e 100644 --- a/frigate/ptz/autotrack.py +++ b/frigate/ptz/autotrack.py @@ -59,7 +59,13 @@ class PtzMotionEstimator: self.ptz_metrics.reset.set() logger.debug(f"{config.name}: Motion estimator init") - def motion_estimator(self, detections, frame_time, camera): + def motion_estimator( + self, + detections: list[dict[str, any]], + frame_name: str, + frame_time: float, + camera: str, + ): # If we've just started up or returned to our preset, reset motion estimator for new tracking session if self.ptz_metrics.reset.is_set(): self.ptz_metrics.reset.clear() @@ -92,9 +98,8 @@ class PtzMotionEstimator: f"{camera}: Motion estimator running - frame time: {frame_time}" ) - frame_id = f"{camera}{frame_time}" yuv_frame = self.frame_manager.get( - frame_id, self.camera_config.frame_shape_yuv + frame_name, self.camera_config.frame_shape_yuv ) if yuv_frame is None: @@ -136,7 +141,7 @@ class PtzMotionEstimator: except Exception: pass - self.frame_manager.close(frame_id) + self.frame_manager.close(frame_name) return self.coord_transformations diff --git a/frigate/record/maintainer.py b/frigate/record/maintainer.py index e97fb0a44..4f976bbf6 100644 --- a/frigate/record/maintainer.py +++ b/frigate/record/maintainer.py @@ -514,6 +514,7 @@ class RecordingMaintainer(threading.Thread): if topic == DetectionTypeEnum.video: ( camera, + _, frame_time, current_tracked_objects, motion_boxes, diff --git a/frigate/review/maintainer.py b/frigate/review/maintainer.py index 38ed59294..23a42e7a7 100644 --- a/frigate/review/maintainer.py +++ b/frigate/review/maintainer.py @@ -234,6 +234,7 @@ class ReviewSegmentMaintainer(threading.Thread): def update_existing_segment( self, segment: PendingReviewSegment, + frame_name: str, frame_time: float, objects: list[TrackedObject], ) -> None: @@ -292,36 +293,34 @@ class ReviewSegmentMaintainer(threading.Thread): if should_update: try: - frame_id = f"{camera_config.name}{frame_time}" yuv_frame = self.frame_manager.get( - frame_id, camera_config.frame_shape_yuv + frame_name, camera_config.frame_shape_yuv ) if yuv_frame is None: - logger.debug(f"Failed to get frame {frame_id} from SHM") + logger.debug(f"Failed to get frame {frame_name} from SHM") return self._publish_segment_update( segment, camera_config, yuv_frame, active_objects, prev_data ) - self.frame_manager.close(frame_id) + self.frame_manager.close(frame_name) except FileNotFoundError: return if not has_activity: if not segment.has_frame: try: - frame_id = f"{camera_config.name}{frame_time}" yuv_frame = self.frame_manager.get( - frame_id, camera_config.frame_shape_yuv + frame_name, camera_config.frame_shape_yuv ) if yuv_frame is None: - logger.debug(f"Failed to get frame {frame_id} from SHM") + logger.debug(f"Failed to get frame {frame_name} from SHM") return segment.save_full_frame(camera_config, yuv_frame) - self.frame_manager.close(frame_id) + self.frame_manager.close(frame_name) self._publish_segment_update( segment, camera_config, None, [], prev_data ) @@ -338,6 +337,7 @@ class ReviewSegmentMaintainer(threading.Thread): def check_if_new_segment( self, camera: str, + frame_name: str, frame_time: float, objects: list[TrackedObject], ) -> None: @@ -414,19 +414,18 @@ class ReviewSegmentMaintainer(threading.Thread): ) try: - frame_id = f"{camera_config.name}{frame_time}" yuv_frame = self.frame_manager.get( - frame_id, camera_config.frame_shape_yuv + frame_name, camera_config.frame_shape_yuv ) if yuv_frame is None: - logger.debug(f"Failed to get frame {frame_id} from SHM") + logger.debug(f"Failed to get frame {frame_name} from SHM") return self.active_review_segments[camera].update_frame( camera_config, yuv_frame, active_objects ) - self.frame_manager.close(frame_id) + self.frame_manager.close(frame_name) self._publish_segment_start(self.active_review_segments[camera]) except FileNotFoundError: return @@ -454,16 +453,17 @@ class ReviewSegmentMaintainer(threading.Thread): if topic == DetectionTypeEnum.video: ( camera, + frame_name, frame_time, current_tracked_objects, - motion_boxes, - regions, + _, + _, ) = data elif topic == DetectionTypeEnum.audio: ( camera, frame_time, - dBFS, + _, audio_detections, ) = data elif topic == DetectionTypeEnum.api: @@ -488,6 +488,7 @@ class ReviewSegmentMaintainer(threading.Thread): if topic == DetectionTypeEnum.video: self.update_existing_segment( current_segment, + frame_name, frame_time, current_tracked_objects, ) @@ -538,6 +539,7 @@ class ReviewSegmentMaintainer(threading.Thread): if topic == DetectionTypeEnum.video: self.check_if_new_segment( camera, + frame_name, frame_time, current_tracked_objects, ) diff --git a/frigate/track/__init__.py b/frigate/track/__init__.py index 3d9e45da2..4fe51f476 100644 --- a/frigate/track/__init__.py +++ b/frigate/track/__init__.py @@ -9,5 +9,7 @@ class ObjectTracker(ABC): pass @abstractmethod - def match_and_update(self, frame_time: float, detections) -> None: + def match_and_update( + self, frame_name: str, frame_time: float, detections: list[dict[str, any]] + ) -> None: pass diff --git a/frigate/track/centroid_tracker.py b/frigate/track/centroid_tracker.py index 36d780cdf..25d4cb860 100644 --- a/frigate/track/centroid_tracker.py +++ b/frigate/track/centroid_tracker.py @@ -129,7 +129,7 @@ class CentroidTracker(ObjectTracker): self.tracked_objects[id].update(new_obj) - def update_frame_times(self, frame_time): + def update_frame_times(self, frame_name, frame_time): for id in list(self.tracked_objects.keys()): self.tracked_objects[id]["frame_time"] = frame_time self.tracked_objects[id]["motionless_count"] += 1 diff --git a/frigate/track/norfair_tracker.py b/frigate/track/norfair_tracker.py index 99085be4d..67950bd0c 100644 --- a/frigate/track/norfair_tracker.py +++ b/frigate/track/norfair_tracker.py @@ -268,7 +268,7 @@ class NorfairTracker(ObjectTracker): self.tracked_objects[id].update(obj) - def update_frame_times(self, frame_time): + def update_frame_times(self, frame_name: str, frame_time: float): # if the object was there in the last frame, assume it's still there detections = [ ( @@ -282,9 +282,11 @@ class NorfairTracker(ObjectTracker): for id, obj in self.tracked_objects.items() if self.disappeared[id] == 0 ] - self.match_and_update(frame_time, detections=detections) + self.match_and_update(frame_name, frame_time, detections=detections) - def match_and_update(self, frame_time, detections): + def match_and_update( + self, frame_name: str, frame_time: float, detections: list[dict[str, any]] + ): norfair_detections = [] for obj in detections: @@ -322,7 +324,7 @@ class NorfairTracker(ObjectTracker): ) coord_transformations = self.ptz_motion_estimator.motion_estimator( - detections, frame_time, self.camera_name + detections, frame_name, frame_time, self.camera_name ) tracked_objects = self.tracker.update( diff --git a/frigate/util/image.py b/frigate/util/image.py index cf1332752..4e3161192 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -717,19 +717,27 @@ def clipped(obj, frame_shape): class FrameManager(ABC): @abstractmethod - def create(self, name, size) -> AnyStr: + def create(self, name: str, size: int) -> AnyStr: pass @abstractmethod - def get(self, name, timeout_ms=0): + def write(self, name: str) -> memoryview: pass @abstractmethod - def close(self, name): + def get(self, name: str, timeout_ms: int = 0): pass @abstractmethod - def delete(self, name): + def close(self, name: str): + pass + + @abstractmethod + def delete(self, name: str): + pass + + @abstractmethod + def cleanup(self): pass @@ -790,6 +798,18 @@ class SharedMemoryFrameManager(FrameManager): self.shm_store[name] = shm return shm.buf + def write(self, name: str) -> memoryview: + try: + if name in self.shm_store: + shm = self.shm_store[name] + else: + shm = UntrackedSharedMemory(name=name) + self.shm_store[name] = shm + return shm.buf + except FileNotFoundError: + logger.info(f"the file {name} not found") + return None + def get(self, name: str, shape) -> Optional[np.ndarray]: try: if name in self.shm_store: @@ -824,6 +844,15 @@ class SharedMemoryFrameManager(FrameManager): except FileNotFoundError: pass + def cleanup(self) -> None: + for shm in self.shm_store.values(): + shm.close() + + try: + shm.unlink() + except FileNotFoundError: + pass + def create_mask(frame_shape, mask): mask_img = np.zeros(frame_shape, np.uint8) diff --git a/frigate/video.py b/frigate/video.py index c0341446a..4e7fe660d 100755 --- a/frigate/video.py +++ b/frigate/video.py @@ -94,7 +94,6 @@ def capture_frames( ffmpeg_process, config: CameraConfig, shm_frame_count: int, - shm_frames: list[str], frame_shape, frame_manager: FrameManager, frame_queue, @@ -109,25 +108,21 @@ def capture_frames( skipped_eps = EventsPerSecond() skipped_eps.start() + # pre-create shms + for i in range(shm_frame_count): + frame_manager.create(f"{config.name}{i}", frame_size) + + frame_index = 0 + while True: fps.value = frame_rate.eps() skipped_fps.value = skipped_eps.eps() current_frame.value = datetime.datetime.now().timestamp() - frame_name = f"{config.name}{current_frame.value}" - frame_buffer = frame_manager.create(frame_name, frame_size) + frame_name = f"{config.name}{frame_index}" + frame_buffer = frame_manager.write(frame_name) try: frame_buffer[:] = ffmpeg_process.stdout.read(frame_size) - - # update frame cache and cleanup existing frames - shm_frames.append(frame_name) - - if len(shm_frames) > shm_frame_count: - expired_frame_name = shm_frames.pop(0) - frame_manager.delete(expired_frame_name) except Exception: - # always delete the frame - frame_manager.delete(frame_name) - # shutdown has been initiated if stop_event.is_set(): break @@ -147,12 +142,16 @@ def capture_frames( # don't lock the queue to check, just try since it should rarely be full try: # add to the queue - frame_queue.put(current_frame.value, False) + frame_queue.put((frame_name, current_frame.value), False) frame_manager.close(frame_name) except queue.Full: # if the queue is full, skip this frame skipped_eps.update() + frame_index = 0 if frame_index == shm_frame_count - 1 else frame_index + 1 + + frame_manager.cleanup() + class CameraWatchdog(threading.Thread): def __init__( @@ -171,7 +170,6 @@ class CameraWatchdog(threading.Thread): self.camera_name = camera_name self.config = config self.shm_frame_count = shm_frame_count - self.shm_frames: list[str] = [] self.capture_thread = None self.ffmpeg_detect_process = None self.logpipe = LogPipe(f"ffmpeg.{self.camera_name}.detect") @@ -304,7 +302,6 @@ class CameraWatchdog(threading.Thread): self.capture_thread = CameraCapture( self.config, self.shm_frame_count, - self.shm_frames, self.ffmpeg_detect_process, self.frame_shape, self.frame_queue, @@ -345,7 +342,6 @@ class CameraCapture(threading.Thread): self, config: CameraConfig, shm_frame_count: int, - shm_frames: list[str], ffmpeg_process, frame_shape, frame_queue, @@ -357,7 +353,6 @@ class CameraCapture(threading.Thread): self.name = f"capture:{config.name}" self.config = config self.shm_frame_count = shm_frame_count - self.shm_frames = shm_frames self.frame_shape = frame_shape self.frame_queue = frame_queue self.fps = fps @@ -373,7 +368,6 @@ class CameraCapture(threading.Thread): self.ffmpeg_process, self.config, self.shm_frame_count, - self.shm_frames, self.frame_shape, self.frame_manager, self.frame_queue, @@ -479,8 +473,8 @@ def track_camera( # empty the frame queue logger.info(f"{name}: emptying frame queue") while not frame_queue.empty(): - frame_time = frame_queue.get(False) - frame_manager.delete(f"{name}{frame_time}") + (frame_name, _) = frame_queue.get(False) + frame_manager.delete(frame_name) logger.info(f"{name}: exiting subprocess") @@ -576,9 +570,9 @@ def process_frames( try: if exit_on_empty: - frame_time = frame_queue.get(False) + frame_name, frame_time = frame_queue.get(False) else: - frame_time = frame_queue.get(True, 1) + frame_name, frame_time = frame_queue.get(True, 1) except queue.Empty: if exit_on_empty: logger.info("Exiting track_objects...") @@ -588,9 +582,7 @@ def process_frames( camera_metrics.detection_frame.value = frame_time ptz_metrics.frame_time.value = frame_time - frame = frame_manager.get( - f"{camera_name}{frame_time}", (frame_shape[0] * 3 // 2, frame_shape[1]) - ) + frame = frame_manager.get(frame_name, (frame_shape[0] * 3 // 2, frame_shape[1])) if frame is None: logger.debug(f"{camera_name}: frame {frame_time} is not in memory store.") @@ -604,7 +596,7 @@ def process_frames( # if detection is disabled if not detect_config.enabled: - object_tracker.match_and_update(frame_time, []) + object_tracker.match_and_update(frame_name, frame_time, []) else: # get stationary object ids # check every Nth frame for stationary objects @@ -728,10 +720,12 @@ def process_frames( if d[0] not in model_config.all_attributes ] # now that we have refined our detections, we need to track objects - object_tracker.match_and_update(frame_time, tracked_detections) + object_tracker.match_and_update( + frame_name, frame_time, tracked_detections + ) # else, just update the frame times for the stationary objects else: - object_tracker.update_frame_times(frame_time) + object_tracker.update_frame_times(frame_name, frame_time) # group the attribute detections based on what label they apply to attribute_detections: dict[str, list[TrackedObjectAttribute]] = {} @@ -836,7 +830,7 @@ def process_frames( ) # add to the queue if not full if detected_objects_queue.full(): - frame_manager.delete(f"{camera_name}{frame_time}") + frame_manager.close(frame_name) continue else: fps_tracker.update() @@ -844,6 +838,7 @@ def process_frames( detected_objects_queue.put( ( camera_name, + frame_name, frame_time, detections, motion_boxes, @@ -851,7 +846,7 @@ def process_frames( ) ) camera_metrics.detection_fps.value = object_detector.fps.eps() - frame_manager.close(f"{camera_name}{frame_time}") + frame_manager.close(frame_name) motion_detector.stop() requestor.stop() From 5b1b6b5be082fb18ae8ac3b66f81c00ebc8a9493 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 17 Nov 2024 10:25:49 -0700 Subject: [PATCH 172/479] Fix round robin (#15035) * Move camera SHM frame creation to main process * Don't reset frame index * Don't fail if shm exists * Set more types --- frigate/app.py | 8 +++++++- frigate/util/image.py | 14 +++++++++----- frigate/video.py | 20 ++++++++++---------- 3 files changed, 26 insertions(+), 16 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index 96edfbd15..f56ed1a8b 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -68,7 +68,7 @@ from frigate.stats.util import stats_init from frigate.storage import StorageMaintainer from frigate.timeline import TimelineProcessor from frigate.util.builtin import empty_and_close_queue -from frigate.util.image import UntrackedSharedMemory +from frigate.util.image import SharedMemoryFrameManager, UntrackedSharedMemory from frigate.util.object import get_camera_regions_grid from frigate.version import VERSION from frigate.video import capture_camera, track_camera @@ -426,12 +426,18 @@ class FrigateApp: def start_camera_capture_processes(self) -> None: shm_frame_count = self.shm_frame_count() + frame_manager = SharedMemoryFrameManager() for name, config in self.config.cameras.items(): if not self.config.cameras[name].enabled: logger.info(f"Capture process not started for disabled camera {name}") continue + # pre-create shms + for i in range(shm_frame_count): + frame_size = config.frame_shape_yuv[0] * config.frame_shape_yuv[1] + frame_manager.create(f"{config.name}{i}", frame_size) + capture_process = util.Process( target=capture_camera, name=f"camera_capture:{name}", diff --git a/frigate/util/image.py b/frigate/util/image.py index 4e3161192..7b22c138e 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -790,11 +790,15 @@ class SharedMemoryFrameManager(FrameManager): self.shm_store: dict[str, UntrackedSharedMemory] = {} def create(self, name: str, size) -> AnyStr: - shm = UntrackedSharedMemory( - name=name, - create=True, - size=size, - ) + try: + shm = UntrackedSharedMemory( + name=name, + create=True, + size=size, + ) + except FileExistsError: + shm = UntrackedSharedMemory(name=name) + self.shm_store[name] = shm return shm.buf diff --git a/frigate/video.py b/frigate/video.py index 4e7fe660d..5af3e13f4 100755 --- a/frigate/video.py +++ b/frigate/video.py @@ -94,7 +94,8 @@ def capture_frames( ffmpeg_process, config: CameraConfig, shm_frame_count: int, - frame_shape, + frame_index: int, + frame_shape: tuple[int, int], frame_manager: FrameManager, frame_queue, fps: mp.Value, @@ -108,12 +109,6 @@ def capture_frames( skipped_eps = EventsPerSecond() skipped_eps.start() - # pre-create shms - for i in range(shm_frame_count): - frame_manager.create(f"{config.name}{i}", frame_size) - - frame_index = 0 - while True: fps.value = frame_rate.eps() skipped_fps.value = skipped_eps.eps() @@ -159,7 +154,7 @@ class CameraWatchdog(threading.Thread): camera_name, config: CameraConfig, shm_frame_count: int, - frame_queue, + frame_queue: mp.Queue, camera_fps, skipped_fps, ffmpeg_pid, @@ -181,6 +176,7 @@ class CameraWatchdog(threading.Thread): self.frame_shape = self.config.frame_shape_yuv self.frame_size = self.frame_shape[0] * self.frame_shape[1] self.fps_overflow_count = 0 + self.frame_index = 0 self.stop_event = stop_event self.sleeptime = self.config.ffmpeg.retry_interval @@ -302,6 +298,7 @@ class CameraWatchdog(threading.Thread): self.capture_thread = CameraCapture( self.config, self.shm_frame_count, + self.frame_index, self.ffmpeg_detect_process, self.frame_shape, self.frame_queue, @@ -342,9 +339,10 @@ class CameraCapture(threading.Thread): self, config: CameraConfig, shm_frame_count: int, + frame_index: int, ffmpeg_process, - frame_shape, - frame_queue, + frame_shape: tuple[int, int], + frame_queue: mp.Queue, fps, skipped_fps, stop_event, @@ -353,6 +351,7 @@ class CameraCapture(threading.Thread): self.name = f"capture:{config.name}" self.config = config self.shm_frame_count = shm_frame_count + self.frame_index = frame_index self.frame_shape = frame_shape self.frame_queue = frame_queue self.fps = fps @@ -368,6 +367,7 @@ class CameraCapture(threading.Thread): self.ffmpeg_process, self.config, self.shm_frame_count, + self.frame_index, self.frame_shape, self.frame_manager, self.frame_queue, From 474c248c9d7501ff7e9677578331d4ca4beff476 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 17 Nov 2024 15:57:58 -0700 Subject: [PATCH 173/479] Cleanup correctly (#15043) --- frigate/app.py | 5 +++-- frigate/video.py | 2 -- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index f56ed1a8b..6518c1ddf 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -91,6 +91,7 @@ class FrigateApp: self.processes: dict[str, int] = {} self.embeddings: Optional[EmbeddingsContext] = None self.region_grids: dict[str, list[list[dict[str, int]]]] = {} + self.frame_manager = SharedMemoryFrameManager() self.config = config def ensure_dirs(self) -> None: @@ -426,7 +427,6 @@ class FrigateApp: def start_camera_capture_processes(self) -> None: shm_frame_count = self.shm_frame_count() - frame_manager = SharedMemoryFrameManager() for name, config in self.config.cameras.items(): if not self.config.cameras[name].enabled: @@ -436,7 +436,7 @@ class FrigateApp: # pre-create shms for i in range(shm_frame_count): frame_size = config.frame_shape_yuv[0] * config.frame_shape_yuv[1] - frame_manager.create(f"{config.name}{i}", frame_size) + self.frame_manager.create(f"{config.name}{i}", frame_size) capture_process = util.Process( target=capture_camera, @@ -717,6 +717,7 @@ class FrigateApp: self.event_metadata_updater.stop() self.inter_zmq_proxy.stop() + self.frame_manager.cleanup() while len(self.detection_shms) > 0: shm = self.detection_shms.pop() shm.close() diff --git a/frigate/video.py b/frigate/video.py index 5af3e13f4..96b562e8c 100755 --- a/frigate/video.py +++ b/frigate/video.py @@ -145,8 +145,6 @@ def capture_frames( frame_index = 0 if frame_index == shm_frame_count - 1 else frame_index + 1 - frame_manager.cleanup() - class CameraWatchdog(threading.Thread): def __init__( From 26c3f9f9148ccfc46a461c6baec000b984827108 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 18 Nov 2024 08:38:58 -0700 Subject: [PATCH 174/479] Fix birdseye (#15051) --- frigate/output/birdseye.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/frigate/output/birdseye.py b/frigate/output/birdseye.py index cab155b9b..6d6391f14 100644 --- a/frigate/output/birdseye.py +++ b/frigate/output/birdseye.py @@ -388,7 +388,7 @@ class BirdsEyeFrameManager: for cam, cam_data in self.cameras.items() if self.config.cameras[cam].birdseye.enabled and cam_data["last_active_frame"] > 0 - and cam_data["current_frame"] - cam_data["last_active_frame"] + and cam_data["current_frame_time"] - cam_data["last_active_frame"] < self.inactivity_threshold ] ) @@ -405,7 +405,7 @@ class BirdsEyeFrameManager: limited_active_cameras = sorted( active_cameras, key=lambda active_camera: ( - self.cameras[active_camera]["current_frame"] + self.cameras[active_camera]["current_frame_time"] - self.cameras[active_camera]["last_active_frame"] ), ) @@ -517,7 +517,7 @@ class BirdsEyeFrameManager: self.copy_to_position( position[1], position[0], - frame, + self.cameras[position[0]]["current_frame"], ) return True @@ -689,7 +689,8 @@ class BirdsEyeFrameManager: return False # update the last active frame for the camera - self.cameras[camera]["current_frame"] = frame_time + self.cameras[camera]["current_frame"] = frame.copy() + self.cameras[camera]["current_frame_time"] = frame_time if self.camera_active(camera_config.mode, object_count, motion_count): self.cameras[camera]["last_active_frame"] = frame_time @@ -755,7 +756,7 @@ class Birdseye: current_tracked_objects: list[dict[str, any]], motion_boxes: list[list[int]], frame_time: float, - frame, + frame: np.ndarray, ) -> None: # check if there is an updated config while True: From 0b203a3673085da90f4074148b3bc657624f35ec Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 18 Nov 2024 09:14:49 -0700 Subject: [PATCH 175/479] fix writing to birdseye restream buffer (#15052) --- frigate/output/birdseye.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/frigate/output/birdseye.py b/frigate/output/birdseye.py index 6d6391f14..00f17c8f4 100644 --- a/frigate/output/birdseye.py +++ b/frigate/output/birdseye.py @@ -740,9 +740,10 @@ class Birdseye: ) self.birdseye_manager = BirdsEyeFrameManager(config, stop_event) self.config_subscriber = ConfigSubscriber("config/birdseye/") + self.frame_manager = SharedMemoryFrameManager() if config.birdseye.restream: - self.birdseye_buffer = SharedMemoryFrameManager().create( + self.birdseye_buffer = self.frame_manager.create( "birdseye", self.birdseye_manager.yuv_shape[0] * self.birdseye_manager.yuv_shape[1], ) From 66f71aecf7755771bc9a3c1dd7cf741a6e1b6955 Mon Sep 17 00:00:00 2001 From: Bazyl Ichabod Horsey <31446064+bazylhorsey@users.noreply.github.com> Date: Mon, 18 Nov 2024 12:26:36 -0600 Subject: [PATCH 176/479] fix regex for cookie_name to be general snake case (#14854) * fix regex for cookie_name to be general snake case * Update frigate/config/auth.py Co-authored-by: Blake Blackshear --------- Co-authored-by: Blake Blackshear --- frigate/config/auth.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/config/auth.py b/frigate/config/auth.py index 91a692461..a202fb1af 100644 --- a/frigate/config/auth.py +++ b/frigate/config/auth.py @@ -13,7 +13,7 @@ class AuthConfig(FrigateBaseModel): default=False, title="Reset the admin password on startup" ) cookie_name: str = Field( - default="frigate_token", title="Name for jwt token cookie", pattern=r"^[a-z]_*$" + default="frigate_token", title="Name for jwt token cookie", pattern=r"^[a-z_]+$" ) cookie_secure: bool = Field(default=False, title="Set secure flag on cookie") session_length: int = Field( From 9ae839ad723dcb2b3c89544419a5b96779a44062 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 18 Nov 2024 12:26:44 -0600 Subject: [PATCH 177/479] Tracked object metadata changes (#15055) * add enum and change topic name * frontend renaming * docs * only display sublabel score if it it exists * remove debug print --- docs/docs/integrations/mqtt.md | 12 ++++++++++++ frigate/comms/dispatcher.py | 12 +++++++++--- frigate/embeddings/maintainer.py | 7 ++++++- frigate/types.py | 4 ++++ web/src/api/ws.tsx | 4 ++-- .../components/overlay/detail/SearchDetailDialog.tsx | 2 +- web/src/pages/Explore.tsx | 8 ++++---- web/src/views/explore/ExploreView.tsx | 6 +++--- 8 files changed, 41 insertions(+), 14 deletions(-) diff --git a/docs/docs/integrations/mqtt.md b/docs/docs/integrations/mqtt.md index e606d29fc..194821cbd 100644 --- a/docs/docs/integrations/mqtt.md +++ b/docs/docs/integrations/mqtt.md @@ -94,6 +94,18 @@ Message published for each changed tracked object. The first message is publishe } ``` +### `frigate/tracked_object_update` + +Message published for updates to tracked object metadata, for example when GenAI runs and returns a tracked object description. + +```json +{ + "type": "description", + "id": "1607123955.475377-mxklsc", + "description": "The car is a red sedan moving away from the camera." +} +``` + ### `frigate/reviews` Message published for each changed review item. The first message is published when the `detection` or `alert` is initiated. When additional objects are detected or when a zone change occurs, it will publish a, `update` message with the same id. When the review activity has ended a final `end` message is published. diff --git a/frigate/comms/dispatcher.py b/frigate/comms/dispatcher.py index 1f480fa9c..2bddc97a5 100644 --- a/frigate/comms/dispatcher.py +++ b/frigate/comms/dispatcher.py @@ -22,7 +22,7 @@ from frigate.const import ( ) from frigate.models import Event, Previews, Recordings, ReviewSegment from frigate.ptz.onvif import OnvifCommandEnum, OnvifController -from frigate.types import ModelStatusTypesEnum +from frigate.types import ModelStatusTypesEnum, TrackedObjectUpdateTypesEnum from frigate.util.object import get_camera_regions_grid from frigate.util.services import restart_frigate @@ -137,8 +137,14 @@ class Dispatcher: event.data["description"] = payload["description"] event.save() self.publish( - "event_update", - json.dumps({"id": event.id, "description": event.data["description"]}), + "tracked_object_update", + json.dumps( + { + "type": TrackedObjectUpdateTypesEnum.description, + "id": event.id, + "description": event.data["description"], + } + ), ) def handle_update_model_state(): diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index 12c8bac72..dde8f8df4 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -24,6 +24,7 @@ from frigate.const import CLIPS_DIR, UPDATE_EVENT_DESCRIPTION from frigate.events.types import EventTypeEnum from frigate.genai import get_genai_client from frigate.models import Event +from frigate.types import TrackedObjectUpdateTypesEnum from frigate.util.builtin import serialize from frigate.util.image import SharedMemoryFrameManager, calculate_region @@ -287,7 +288,11 @@ class EmbeddingMaintainer(threading.Thread): # fire and forget description update self.requestor.send_data( UPDATE_EVENT_DESCRIPTION, - {"id": event.id, "description": description}, + { + "type": TrackedObjectUpdateTypesEnum.description, + "id": event.id, + "description": description, + }, ) # Embed the description diff --git a/frigate/types.py b/frigate/types.py index 3e6ad46cc..11ab31238 100644 --- a/frigate/types.py +++ b/frigate/types.py @@ -19,3 +19,7 @@ class ModelStatusTypesEnum(str, Enum): downloading = "downloading" downloaded = "downloaded" error = "error" + + +class TrackedObjectUpdateTypesEnum(str, Enum): + description = "description" diff --git a/web/src/api/ws.tsx b/web/src/api/ws.tsx index c7bb74095..9b8924d1b 100644 --- a/web/src/api/ws.tsx +++ b/web/src/api/ws.tsx @@ -407,9 +407,9 @@ export function useImproveContrast(camera: string): { return { payload: payload as ToggleableSetting, send }; } -export function useEventUpdate(): { payload: string } { +export function useTrackedObjectUpdate(): { payload: string } { const { value: { payload }, - } = useWs("event_update", ""); + } = useWs("tracked_object_update", ""); return useDeepMemo(JSON.parse(payload as string)); } diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index 5eca9a934..f7af31606 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -309,7 +309,7 @@ function ObjectDetailsTab({ return undefined; } - if (search.sub_label) { + if (search.sub_label && search.data?.sub_label_score) { return Math.round((search.data?.sub_label_score ?? 0) * 100); } else { return undefined; diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 711666807..2bf2bb022 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -1,6 +1,6 @@ import { useEmbeddingsReindexProgress, - useEventUpdate, + useTrackedObjectUpdate, useModelState, } from "@/api/ws"; import ActivityIndicator from "@/components/indicators/activity-indicator"; @@ -227,15 +227,15 @@ export default function Explore() { // mutation and revalidation - const eventUpdate = useEventUpdate(); + const trackedObjectUpdate = useTrackedObjectUpdate(); useEffect(() => { - if (eventUpdate) { + if (trackedObjectUpdate) { mutate(); } // mutate / revalidate when event description updates come in // eslint-disable-next-line react-hooks/exhaustive-deps - }, [eventUpdate]); + }, [trackedObjectUpdate]); // embeddings reindex progress diff --git a/web/src/views/explore/ExploreView.tsx b/web/src/views/explore/ExploreView.tsx index f37c37453..ea9c3cbef 100644 --- a/web/src/views/explore/ExploreView.tsx +++ b/web/src/views/explore/ExploreView.tsx @@ -15,7 +15,7 @@ import { SearchResult } from "@/types/search"; import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator"; import useImageLoaded from "@/hooks/use-image-loaded"; import ActivityIndicator from "@/components/indicators/activity-indicator"; -import { useEventUpdate } from "@/api/ws"; +import { useTrackedObjectUpdate } from "@/api/ws"; import { isEqual } from "lodash"; import TimeAgo from "@/components/dynamic/TimeAgo"; import SearchResultActions from "@/components/menu/SearchResultActions"; @@ -72,13 +72,13 @@ export default function ExploreView({ }, {}); }, [events]); - const eventUpdate = useEventUpdate(); + const trackedObjectUpdate = useTrackedObjectUpdate(); useEffect(() => { mutate(); // mutate / revalidate when event description updates come in // eslint-disable-next-line react-hooks/exhaustive-deps - }, [eventUpdate]); + }, [trackedObjectUpdate]); // update search detail when results change From a67ff3843afe5cde45a03bcaa7a374dff67e4e25 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 19 Nov 2024 09:41:16 -0600 Subject: [PATCH 178/479] Update genai docs (#15070) --- docs/docs/configuration/genai.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index da4b8afd8..67872876b 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -142,6 +142,10 @@ Frigate's thumbnail search excels at identifying specific details about tracked While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate’s default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what’s happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they’re moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation’s context. +### Using GenAI for notifications + +Frigate provides an [MQTT topic](/integrations/mqtt), `frigate/tracked_object_update`, that is updated with a JSON payload containing `event_id` and `description` when your AI provider returns a description for a tracked object. This description could be used directly in notifications, such as sending alerts to your phone or making audio announcements. If additional details from the tracked object are needed, you can query the [HTTP API](/integrations/api) using the `event_id`, eg: `http://frigate_ip:5000/api/events/`. + ## Custom Prompts Frigate sends multiple frames from the tracked object along with a prompt to your Generative AI provider asking it to generate a description. The default prompt is as follows: @@ -172,7 +176,7 @@ genai: Prompts can also be overriden at the camera level to provide a more detailed prompt to the model about your specific camera, if you desire. By default, descriptions will be generated for all tracked objects and all zones. But you can also optionally specify `objects` and `required_zones` to only generate descriptions for certain tracked objects or zones. -Optionally, you can generate the description using a snapshot (if enabled) by setting `use_snapshot` to `True`. By default, this is set to `False`, which sends the thumbnails collected over the object's lifetime to the model. Using a snapshot provides the AI with a higher-resolution image (typically downscaled by the AI itself), but the trade-off is that only a single image is used, which might limit the model's ability to determine object movement or direction. +Optionally, you can generate the description using a snapshot (if enabled) by setting `use_snapshot` to `True`. By default, this is set to `False`, which sends the uncompressed images from the `detect` stream collected over the object's lifetime to the model. Once the object lifecycle ends, only a single compressed and cropped thumbnail is saved with the tracked object. Using a snapshot might be useful when you want to _regenerate_ a tracked object's description as it will provide the AI with a higher-quality image (typically downscaled by the AI itself) than the cropped/compressed thumbnail. Using a snapshot otherwise has a trade-off in that only a single image is sent to your provider, which will limit the model's ability to determine object movement or direction. ```yaml cameras: From 66277fbb6c3085acbb0f669b0c6fc4e9eee3d5b0 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 19 Nov 2024 11:20:04 -0700 Subject: [PATCH 179/479] Fix embeddings (#15072) * Fix embeddings reading frames * Fix event update reading * Formatting * Pin AIO http to fix build failure * Pin starlette --- docker/main/requirements-wheels.txt | 2 ++ frigate/comms/events_updater.py | 2 +- frigate/embeddings/maintainer.py | 9 +++++---- frigate/events/maintainer.py | 2 +- frigate/object_processing.py | 27 +++++++++++++++------------ 5 files changed, 24 insertions(+), 18 deletions(-) diff --git a/docker/main/requirements-wheels.txt b/docker/main/requirements-wheels.txt index 795456588..4db88ccd2 100644 --- a/docker/main/requirements-wheels.txt +++ b/docker/main/requirements-wheels.txt @@ -1,5 +1,7 @@ click == 8.1.* # FastAPI +aiohttp == 3.11.2 +starlette == 0.41.2 starlette-context == 0.3.6 fastapi == 0.115.* uvicorn == 0.30.* diff --git a/frigate/comms/events_updater.py b/frigate/comms/events_updater.py index 7a5772273..98b6ccb7a 100644 --- a/frigate/comms/events_updater.py +++ b/frigate/comms/events_updater.py @@ -14,7 +14,7 @@ class EventUpdatePublisher(Publisher): super().__init__("update") def publish( - self, payload: tuple[EventTypeEnum, EventStateEnum, str, dict[str, any]] + self, payload: tuple[EventTypeEnum, EventStateEnum, str, str, dict[str, any]] ) -> None: super().publish(payload) diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index dde8f8df4..d58a7f431 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -114,7 +114,7 @@ class EmbeddingMaintainer(threading.Thread): if update is None: return - source_type, _, camera, data = update + source_type, _, camera, frame_name, data = update if not camera or source_type != EventTypeEnum.tracked_object: return @@ -134,8 +134,9 @@ class EmbeddingMaintainer(threading.Thread): # Create our own thumbnail based on the bounding box and the frame time try: - frame_id = f"{camera}{data['frame_time']}" - yuv_frame = self.frame_manager.get(frame_id, camera_config.frame_shape_yuv) + yuv_frame = self.frame_manager.get( + frame_name, camera_config.frame_shape_yuv + ) if yuv_frame is not None: data["thumbnail"] = self._create_thumbnail(yuv_frame, data["box"]) @@ -147,7 +148,7 @@ class EmbeddingMaintainer(threading.Thread): self.tracked_events[data["id"]].append(data) - self.frame_manager.close(frame_id) + self.frame_manager.close(frame_name) except FileNotFoundError: pass diff --git a/frigate/events/maintainer.py b/frigate/events/maintainer.py index b17bd5d35..3a4209ec3 100644 --- a/frigate/events/maintainer.py +++ b/frigate/events/maintainer.py @@ -75,7 +75,7 @@ class EventProcessor(threading.Thread): if update == None: continue - source_type, event_type, camera, event_data = update + source_type, event_type, camera, _, event_data = update logger.debug( f"Event received: {source_type} {event_type} {camera} {event_data['id']}" diff --git a/frigate/object_processing.py b/frigate/object_processing.py index 23c84eedd..937c935ba 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -262,7 +262,7 @@ class CameraState: # call event handlers for c in self.callbacks["start"]: - c(self.name, new_obj, frame_time) + c(self.name, new_obj, frame_name) for id in updated_ids: updated_obj = tracked_objects[id] @@ -272,7 +272,7 @@ class CameraState: if autotracker_update or significant_update: for c in self.callbacks["autotrack"]: - c(self.name, updated_obj, frame_time) + c(self.name, updated_obj, frame_name) if thumb_update and current_frame is not None: # ensure this frame is stored in the cache @@ -293,7 +293,7 @@ class CameraState: ) or significant_update: # call event handlers for c in self.callbacks["update"]: - c(self.name, updated_obj, frame_time) + c(self.name, updated_obj, frame_name) updated_obj.last_published = frame_time for id in removed_ids: @@ -302,7 +302,7 @@ class CameraState: if "end_time" not in removed_obj.obj_data: removed_obj.obj_data["end_time"] = frame_time for c in self.callbacks["end"]: - c(self.name, removed_obj, frame_time) + c(self.name, removed_obj, frame_name) # TODO: can i switch to looking this up and only changing when an event ends? # maintain best objects @@ -368,11 +368,11 @@ class CameraState: ): self.best_objects[object_type] = obj for c in self.callbacks["snapshot"]: - c(self.name, self.best_objects[object_type], frame_time) + c(self.name, self.best_objects[object_type], frame_name) else: self.best_objects[object_type] = obj for c in self.callbacks["snapshot"]: - c(self.name, self.best_objects[object_type], frame_time) + c(self.name, self.best_objects[object_type], frame_name) for c in self.callbacks["camera_activity"]: c(self.name, camera_activity) @@ -447,7 +447,7 @@ class CameraState: c(self.name, obj_name, 0) self.active_object_counts[obj_name] = 0 for c in self.callbacks["snapshot"]: - c(self.name, self.best_objects[obj_name], frame_time) + c(self.name, self.best_objects[obj_name], frame_name) # cleanup thumbnail frame cache current_thumb_frames = { @@ -518,17 +518,18 @@ class TrackedObjectProcessor(threading.Thread): self.zone_data = defaultdict(lambda: defaultdict(dict)) self.active_zone_data = defaultdict(lambda: defaultdict(dict)) - def start(camera, obj: TrackedObject, current_frame_time): + def start(camera: str, obj: TrackedObject, frame_name: str): self.event_sender.publish( ( EventTypeEnum.tracked_object, EventStateEnum.start, camera, + frame_name, obj.to_dict(), ) ) - def update(camera, obj: TrackedObject, current_frame_time): + def update(camera: str, obj: TrackedObject, frame_name: str): obj.has_snapshot = self.should_save_snapshot(camera, obj) obj.has_clip = self.should_retain_recording(camera, obj) after = obj.to_dict() @@ -544,14 +545,15 @@ class TrackedObjectProcessor(threading.Thread): EventTypeEnum.tracked_object, EventStateEnum.update, camera, + frame_name, obj.to_dict(include_thumbnail=True), ) ) - def autotrack(camera, obj: TrackedObject, current_frame_time): + def autotrack(camera: str, obj: TrackedObject, frame_name: str): self.ptz_autotracker_thread.ptz_autotracker.autotrack_object(camera, obj) - def end(camera, obj: TrackedObject, current_frame_time): + def end(camera: str, obj: TrackedObject, frame_name: str): # populate has_snapshot obj.has_snapshot = self.should_save_snapshot(camera, obj) obj.has_clip = self.should_retain_recording(camera, obj) @@ -606,11 +608,12 @@ class TrackedObjectProcessor(threading.Thread): EventTypeEnum.tracked_object, EventStateEnum.end, camera, + frame_name, obj.to_dict(include_thumbnail=True), ) ) - def snapshot(camera, obj: TrackedObject, current_frame_time): + def snapshot(camera, obj: TrackedObject, frame_name: str): mqtt_config: MqttConfig = self.config.cameras[camera].mqtt if mqtt_config.enabled and self.should_mqtt_snapshot(camera, obj): jpg_bytes = obj.get_jpg_bytes( From 0df091f38781cec0cba1ca66e0e42a720151c38a Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 19 Nov 2024 14:33:01 -0600 Subject: [PATCH 180/479] Fix link to api in genai docs (#15075) --- docs/docs/configuration/genai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index 67872876b..fac44ed03 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -144,7 +144,7 @@ While generating simple descriptions of detected objects is useful, understandin ### Using GenAI for notifications -Frigate provides an [MQTT topic](/integrations/mqtt), `frigate/tracked_object_update`, that is updated with a JSON payload containing `event_id` and `description` when your AI provider returns a description for a tracked object. This description could be used directly in notifications, such as sending alerts to your phone or making audio announcements. If additional details from the tracked object are needed, you can query the [HTTP API](/integrations/api) using the `event_id`, eg: `http://frigate_ip:5000/api/events/`. +Frigate provides an [MQTT topic](/integrations/mqtt), `frigate/tracked_object_update`, that is updated with a JSON payload containing `event_id` and `description` when your AI provider returns a description for a tracked object. This description could be used directly in notifications, such as sending alerts to your phone or making audio announcements. If additional details from the tracked object are needed, you can query the [HTTP API](/integrations/api/event-events-event-id-get) using the `event_id`, eg: `http://frigate_ip:5000/api/events/`. ## Custom Prompts From e76f4e9bd9e85e81e2f88921b70566e5ed54dca6 Mon Sep 17 00:00:00 2001 From: Rui Alves Date: Tue, 19 Nov 2024 23:35:10 +0000 Subject: [PATCH 181/479] Started unit tests for the review controller (#15077) * Started unit tests for the review controller * Revert "Started unit tests for the review controller" This reverts commit 7746eb146f813a1226544844d42eda31ed60432b. * Started unit tests for the review controller * FIrst test * Added test for review endpoint (time filter - after + before) * Assert expected event * Added more tests for review endpoint * Added test for review endpoint with all filters * Added test for review endpoint with limit * Comment * Renamed tests to increase readability --- .cspell/frigate-dictionary.txt | 3 + frigate/test/http_api/__init__.py | 0 frigate/test/http_api/base_http_test.py | 162 ++++++++++++++++++++++ frigate/test/http_api/test_http_review.py | 110 +++++++++++++++ 4 files changed, 275 insertions(+) create mode 100644 frigate/test/http_api/__init__.py create mode 100644 frigate/test/http_api/base_http_test.py create mode 100644 frigate/test/http_api/test_http_review.py diff --git a/.cspell/frigate-dictionary.txt b/.cspell/frigate-dictionary.txt index b019f8492..64fd7ca72 100644 --- a/.cspell/frigate-dictionary.txt +++ b/.cspell/frigate-dictionary.txt @@ -12,6 +12,7 @@ argmax argmin argpartition ascontiguousarray +astype authelia authentik autodetected @@ -195,6 +196,7 @@ poweroff preexec probesize protobuf +pstate psutil pubkey putenv @@ -278,6 +280,7 @@ uvicorn vaapi vainfo variations +vbios vconcat vitb vstream diff --git a/frigate/test/http_api/__init__.py b/frigate/test/http_api/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/frigate/test/http_api/base_http_test.py b/frigate/test/http_api/base_http_test.py new file mode 100644 index 000000000..013785692 --- /dev/null +++ b/frigate/test/http_api/base_http_test.py @@ -0,0 +1,162 @@ +import datetime +import logging +import os +import unittest + +from peewee_migrate import Router +from playhouse.sqlite_ext import SqliteExtDatabase +from playhouse.sqliteq import SqliteQueueDatabase + +from frigate.api.fastapi_app import create_fastapi_app +from frigate.config import FrigateConfig +from frigate.models import Event, ReviewSegment +from frigate.review.maintainer import SeverityEnum +from frigate.test.const import TEST_DB, TEST_DB_CLEANUPS + + +class BaseTestHttp(unittest.TestCase): + def setUp(self, models): + # setup clean database for each test run + migrate_db = SqliteExtDatabase("test.db") + del logging.getLogger("peewee_migrate").handlers[:] + router = Router(migrate_db) + router.run() + migrate_db.close() + self.db = SqliteQueueDatabase(TEST_DB) + self.db.bind(models) + + self.minimal_config = { + "mqtt": {"host": "mqtt"}, + "cameras": { + "front_door": { + "ffmpeg": { + "inputs": [ + {"path": "rtsp://10.0.0.1:554/video", "roles": ["detect"]} + ] + }, + "detect": { + "height": 1080, + "width": 1920, + "fps": 5, + }, + } + }, + } + self.test_stats = { + "detection_fps": 13.7, + "detectors": { + "cpu1": { + "detection_start": 0.0, + "inference_speed": 91.43, + "pid": 42, + }, + "cpu2": { + "detection_start": 0.0, + "inference_speed": 84.99, + "pid": 44, + }, + }, + "front_door": { + "camera_fps": 0.0, + "capture_pid": 53, + "detection_fps": 0.0, + "pid": 52, + "process_fps": 0.0, + "skipped_fps": 0.0, + }, + "service": { + "storage": { + "/dev/shm": { + "free": 50.5, + "mount_type": "tmpfs", + "total": 67.1, + "used": 16.6, + }, + "/media/frigate/clips": { + "free": 42429.9, + "mount_type": "ext4", + "total": 244529.7, + "used": 189607.0, + }, + "/media/frigate/recordings": { + "free": 0.2, + "mount_type": "ext4", + "total": 8.0, + "used": 7.8, + }, + "/tmp/cache": { + "free": 976.8, + "mount_type": "tmpfs", + "total": 1000.0, + "used": 23.2, + }, + }, + "uptime": 101113, + "version": "0.10.1", + "latest_version": "0.11", + }, + } + + def tearDown(self): + if not self.db.is_closed(): + self.db.close() + + try: + for file in TEST_DB_CLEANUPS: + os.remove(file) + except OSError: + pass + + def create_app(self, stats=None): + return create_fastapi_app( + FrigateConfig(**self.minimal_config), + self.db, + None, + None, + None, + None, + None, + stats, + None, + ) + + def insert_mock_event( + self, + id: str, + start_time: datetime.datetime = datetime.datetime.now().timestamp(), + ) -> Event: + """Inserts a basic event model with a given id.""" + return Event.insert( + id=id, + label="Mock", + camera="front_door", + start_time=start_time, + end_time=start_time + 20, + top_score=100, + false_positive=False, + zones=list(), + thumbnail="", + region=[], + box=[], + area=0, + has_clip=True, + has_snapshot=True, + ).execute() + + def insert_mock_review_segment( + self, + id: str, + start_time: datetime.datetime = datetime.datetime.now().timestamp(), + end_time: datetime.datetime = datetime.datetime.now().timestamp() + 20, + ) -> Event: + """Inserts a basic event model with a given id.""" + return ReviewSegment.insert( + id=id, + camera="front_door", + start_time=start_time, + end_time=end_time, + has_been_reviewed=False, + severity=SeverityEnum.alert, + thumb_path=False, + data={}, + ).execute() diff --git a/frigate/test/http_api/test_http_review.py b/frigate/test/http_api/test_http_review.py new file mode 100644 index 000000000..19e1f26f8 --- /dev/null +++ b/frigate/test/http_api/test_http_review.py @@ -0,0 +1,110 @@ +import datetime + +from fastapi.testclient import TestClient + +from frigate.models import Event, ReviewSegment +from frigate.test.http_api.base_http_test import BaseTestHttp + + +class TestHttpReview(BaseTestHttp): + def setUp(self): + super().setUp([Event, ReviewSegment]) + + # Does not return any data point since the end time (before parameter) is not passed and the review segment end_time is 2 seconds from now + def test_get_review_no_filters_no_matches(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + super().insert_mock_review_segment("123456.random", now, now + 2) + reviews_response = client.get("/review") + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 0 + + def test_get_review_no_filters(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + super().insert_mock_review_segment("123456.random", now - 2, now - 1) + reviews_response = client.get("/review") + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 1 + + def test_get_review_with_time_filter_no_matches(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + id = "123456.random" + super().insert_mock_review_segment(id, now, now + 2) + params = { + "after": now, + "before": now + 3, + } + reviews_response = client.get("/review", params=params) + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 0 + + def test_get_review_with_time_filter(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + id = "123456.random" + super().insert_mock_review_segment(id, now, now + 2) + params = { + "after": now - 1, + "before": now + 3, + } + reviews_response = client.get("/review", params=params) + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 1 + assert reviews_in_response[0]["id"] == id + + def test_get_review_with_limit_filter(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + id = "123456.random" + id2 = "654321.random" + super().insert_mock_review_segment(id, now, now + 2) + super().insert_mock_review_segment(id2, now + 1, now + 2) + params = { + "limit": 1, + "after": now, + "before": now + 3, + } + reviews_response = client.get("/review", params=params) + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 1 + assert reviews_in_response[0]["id"] == id2 + + def test_get_review_with_all_filters(self): + app = super().create_app() + now = datetime.datetime.now().timestamp() + + with TestClient(app) as client: + id = "123456.random" + super().insert_mock_review_segment(id, now, now + 2) + params = { + "cameras": "front_door", + "labels": "all", + "zones": "all", + "reviewed": 0, + "limit": 1, + "severity": "alert", + "after": now - 1, + "before": now + 3, + } + reviews_response = client.get("/review", params=params) + assert reviews_response.status_code == 200 + reviews_in_response = reviews_response.json() + assert len(reviews_in_response) == 1 + assert reviews_in_response[0]["id"] == id From 9c5a04f25fbd2d756fc787b2a50fec9519f68165 Mon Sep 17 00:00:00 2001 From: victpork <224617+victpork@users.noreply.github.com> Date: Thu, 21 Nov 2024 00:06:22 +1300 Subject: [PATCH 182/479] Added code to download weights from new host (#15087) --- notebooks/YOLO_NAS_Pretrained_Export.ipynb | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/notebooks/YOLO_NAS_Pretrained_Export.ipynb b/notebooks/YOLO_NAS_Pretrained_Export.ipynb index a3c303c01..e4e2222da 100644 --- a/notebooks/YOLO_NAS_Pretrained_Export.ipynb +++ b/notebooks/YOLO_NAS_Pretrained_Export.ipynb @@ -11,6 +11,18 @@ "! pip install -q super_gradients==3.7.1" ] }, + { + "cell_type": "code", + "source": [ + "! sed -i 's/sghub.deci.ai/sg-hub-nv.s3.amazonaws.com/' /usr/local/lib/python3.10/dist-packages/super_gradients/training/pretrained_models.py\n", + "! sed -i 's/sghub.deci.ai/sg-hub-nv.s3.amazonaws.com/' /usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py" + ], + "metadata": { + "id": "NiRCt917KKcL" + }, + "execution_count": null, + "outputs": [] + }, { "cell_type": "code", "execution_count": null, @@ -72,4 +84,4 @@ }, "nbformat": 4, "nbformat_minor": 0 -} +} \ No newline at end of file From ff92b13f35b0a73ff37cf35e4a3a5370ec0175f7 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 20 Nov 2024 09:37:33 -0700 Subject: [PATCH 183/479] Fix sending events (#15100) --- frigate/events/external.py | 1 + 1 file changed, 1 insertion(+) diff --git a/frigate/events/external.py b/frigate/events/external.py index 76b9e3208..52ff5ffb7 100644 --- a/frigate/events/external.py +++ b/frigate/events/external.py @@ -64,6 +64,7 @@ class ExternalEventProcessor: EventTypeEnum.api, EventStateEnum.start, camera, + "", { "id": event_id, "label": label, From 33957e53600afd0e1a677d6575917955d156dd4a Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 24 Nov 2024 19:07:41 -0700 Subject: [PATCH 184/479] Set hailo build library path (#15167) --- docker/hailo8l/Dockerfile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docker/hailo8l/Dockerfile b/docker/hailo8l/Dockerfile index 959e7692e..68ab09001 100644 --- a/docker/hailo8l/Dockerfile +++ b/docker/hailo8l/Dockerfile @@ -36,5 +36,8 @@ RUN pip3 install -U /deps/hailo-wheels/*.whl # Copy base files from the rootfs stage COPY --from=rootfs / / +# Set Library path for hailo driver +ENV LD_LIBRARY_PATH=/rootfs/usr/local/lib/ + # Set workdir WORKDIR /opt/frigate/ From 5cafca1be09415805cbb065a1f95ff41718dd25f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 26 Nov 2024 08:34:40 -0700 Subject: [PATCH 185/479] Add docs for go2rtc logging (#15204) --- docs/docs/configuration/advanced.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/docs/configuration/advanced.md b/docs/docs/configuration/advanced.md index 59786d212..f18e943df 100644 --- a/docs/docs/configuration/advanced.md +++ b/docs/docs/configuration/advanced.md @@ -4,7 +4,9 @@ title: Advanced Options sidebar_label: Advanced Options --- -### `logger` +### Logging + +#### Frigate `logger` Change the default log level for troubleshooting purposes. @@ -28,6 +30,18 @@ Examples of available modules are: - `watchdog.` - `ffmpeg..` NOTE: All FFmpeg logs are sent as `error` level. +#### Go2RTC Logging + +See [the go2rtc docs](for logging configuration) + +```yaml +go2rtc: + streams: + ... + log: + exec: trace +``` + ### `environment_vars` This section can be used to set environment variables for those unable to modify the environment of the container (ie. within HassOS) From 2207a91f7b6aed58983d0b0791e842228b620e38 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 27 Nov 2024 12:57:58 -0700 Subject: [PATCH 186/479] Fix ruff (#15223) --- frigate/util/builtin.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/util/builtin.py b/frigate/util/builtin.py index af8ababe9..5f573ef78 100644 --- a/frigate/util/builtin.py +++ b/frigate/util/builtin.py @@ -13,12 +13,12 @@ import urllib.parse from collections.abc import Mapping from pathlib import Path from typing import Any, Optional, Tuple, Union +from zoneinfo import ZoneInfoNotFoundError import numpy as np import pytz from ruamel.yaml import YAML from tzlocal import get_localzone -from zoneinfo import ZoneInfoNotFoundError from frigate.const import REGEX_HTTP_CAMERA_USER_PASS, REGEX_RTSP_CAMERA_USER_PASS From 2461d01329bd2d0f383ae409c2ea28f7e591ee8d Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Fri, 29 Nov 2024 07:20:33 -0600 Subject: [PATCH 187/479] Update hardware recs (#15254) --- docs/docs/frigate/hardware.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index e19ec5d0d..3d1d2aa66 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -25,7 +25,7 @@ My current favorite is the Beelink EQ13 because of the efficient N100 CPU and du | Name | Coral Inference Speed | Coral Compatibility | Notes | | ------------------------------------------------------------------------------------------------------------- | --------------------- | ------------------- | ----------------------------------------------------------------------------------------- | -| Beelink EQ13 (Amazon) | 5-10ms | USB/M.2(A+E) | Dual gigabit NICs for easy isolated camera network. Easily handles several 1080p cameras. | +| Beelink EQ13 (Amazon) | 5-10ms | USB | Dual gigabit NICs for easy isolated camera network. Easily handles several 1080p cameras. | ## Detectors From d25ffdb29276b94173fd162653a8379692767c39 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 29 Nov 2024 20:44:42 -0600 Subject: [PATCH 188/479] Fix crash when consecutive underscores are used in camera name (#15257) --- web/src/pages/Live.tsx | 1 + 1 file changed, 1 insertion(+) diff --git a/web/src/pages/Live.tsx b/web/src/pages/Live.tsx index e19dccd07..97e565ef1 100644 --- a/web/src/pages/Live.tsx +++ b/web/src/pages/Live.tsx @@ -62,6 +62,7 @@ function Live() { if (selectedCameraName) { const capitalized = selectedCameraName .split("_") + .filter((text) => text) .map((text) => text[0].toUpperCase() + text.substring(1)); document.title = `${capitalized.join(" ")} - Live - Frigate`; } else if (cameraGroup && cameraGroup != "default") { From f094c59cd0476e69b321436a8c0de8839046531a Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 30 Nov 2024 18:21:50 -0600 Subject: [PATCH 189/479] Fix formatting (#15271) --- frigate/config/config.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/frigate/config/config.py b/frigate/config/config.py index 8fbb9ec6c..8c0b52e92 100644 --- a/frigate/config/config.py +++ b/frigate/config/config.py @@ -230,12 +230,16 @@ def verify_recording_segments_setup_with_reasonable_time( try: seg_arg_index = record_args.index("-segment_time") except ValueError: - raise ValueError(f"Camera {camera_config.name} has no segment_time in \ - recording output args, segment args are required for record.") + raise ValueError( + f"Camera {camera_config.name} has no segment_time in \ + recording output args, segment args are required for record." + ) if int(record_args[seg_arg_index + 1]) > 60: - raise ValueError(f"Camera {camera_config.name} has invalid segment_time output arg, \ - segment_time must be 60 or less.") + raise ValueError( + f"Camera {camera_config.name} has invalid segment_time output arg, \ + segment_time must be 60 or less." + ) def verify_zone_objects_are_tracked(camera_config: CameraConfig) -> None: From ee816b2251bf5f9abb7105c0976743a27628c0e8 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 30 Nov 2024 18:22:36 -0600 Subject: [PATCH 190/479] Fix camera access and improve typing (#15272) * Fix camera access and improve typing: * Formatting --- frigate/api/event.py | 9 ++++++--- frigate/api/media.py | 23 +++++++++++++---------- frigate/events/external.py | 10 +++++++--- frigate/object_processing.py | 9 +++++++-- 4 files changed, 33 insertions(+), 18 deletions(-) diff --git a/frigate/api/event.py b/frigate/api/event.py index bff1edc1a..3b38ff072 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -35,8 +35,9 @@ from frigate.const import ( CLIPS_DIR, ) from frigate.embeddings import EmbeddingsContext +from frigate.events.external import ExternalEventProcessor from frigate.models import Event, ReviewSegment, Timeline -from frigate.object_processing import TrackedObject +from frigate.object_processing import TrackedObject, TrackedObjectProcessor from frigate.util.builtin import get_tz_modifiers logger = logging.getLogger(__name__) @@ -1087,9 +1088,11 @@ def create_event( ) try: - frame = request.app.detected_frames_processor.get_current_frame(camera_name) + frame_processor: TrackedObjectProcessor = request.app.detected_frames_processor + external_processor: ExternalEventProcessor = request.app.external_processor - event_id = request.app.external_processor.create_manual_event( + frame = frame_processor.get_current_frame(camera_name) + event_id = external_processor.create_manual_event( camera_name, label, body.source_type, diff --git a/frigate/api/media.py b/frigate/api/media.py index dcfc44f89..a90766899 100644 --- a/frigate/api/media.py +++ b/frigate/api/media.py @@ -36,6 +36,7 @@ from frigate.const import ( RECORD_DIR, ) from frigate.models import Event, Previews, Recordings, Regions, ReviewSegment +from frigate.object_processing import TrackedObjectProcessor from frigate.util.builtin import get_tz_modifiers from frigate.util.image import get_image_from_recording @@ -79,7 +80,11 @@ def mjpeg_feed( def imagestream( - detected_frames_processor, camera_name: str, fps: int, height: int, draw_options + detected_frames_processor: TrackedObjectProcessor, + camera_name: str, + fps: int, + height: int, + draw_options: dict[str, any], ): while True: # max out at specified FPS @@ -118,6 +123,7 @@ def latest_frame( extension: Extension, params: MediaLatestFrameQueryParams = Depends(), ): + frame_processor: TrackedObjectProcessor = request.app.detected_frames_processor draw_options = { "bounding_boxes": params.bbox, "timestamp": params.timestamp, @@ -129,17 +135,14 @@ def latest_frame( quality = params.quality if camera_name in request.app.frigate_config.cameras: - frame = request.app.detected_frames_processor.get_current_frame( - camera_name, draw_options - ) + frame = frame_processor.get_current_frame(camera_name, draw_options) retry_interval = float( request.app.frigate_config.cameras.get(camera_name).ffmpeg.retry_interval or 10 ) if frame is None or datetime.now().timestamp() > ( - request.app.detected_frames_processor.get_current_frame_time(camera_name) - + retry_interval + frame_processor.get_current_frame_time(camera_name) + retry_interval ): if request.app.camera_error_image is None: error_image = glob.glob("/opt/frigate/frigate/images/camera-error.jpg") @@ -180,7 +183,7 @@ def latest_frame( ) elif camera_name == "birdseye" and request.app.frigate_config.birdseye.restream: frame = cv2.cvtColor( - request.app.detected_frames_processor.get_current_frame(camera_name), + frame_processor.get_current_frame(camera_name), cv2.COLOR_YUV2BGR_I420, ) @@ -813,15 +816,15 @@ def grid_snapshot( ): if camera_name in request.app.frigate_config.cameras: detect = request.app.frigate_config.cameras[camera_name].detect - frame = request.app.detected_frames_processor.get_current_frame(camera_name, {}) + frame_processor: TrackedObjectProcessor = request.app.detected_frames_processor + frame = frame_processor.get_current_frame(camera_name, {}) retry_interval = float( request.app.frigate_config.cameras.get(camera_name).ffmpeg.retry_interval or 10 ) if frame is None or datetime.now().timestamp() > ( - request.app.detected_frames_processor.get_current_frame_time(camera_name) - + retry_interval + frame_processor.get_current_frame_time(camera_name) + retry_interval ): return JSONResponse( content={"success": False, "message": "Unable to get valid frame"}, diff --git a/frigate/events/external.py b/frigate/events/external.py index 52ff5ffb7..922917bb4 100644 --- a/frigate/events/external.py +++ b/frigate/events/external.py @@ -10,6 +10,7 @@ from enum import Enum from typing import Optional import cv2 +from numpy import ndarray from frigate.comms.detections_updater import DetectionPublisher, DetectionTypeEnum from frigate.comms.events_updater import EventUpdatePublisher @@ -45,7 +46,7 @@ class ExternalEventProcessor: duration: Optional[int], include_recording: bool, draw: dict[str, any], - snapshot_frame: any, + snapshot_frame: Optional[ndarray], ) -> str: now = datetime.datetime.now().timestamp() camera_config = self.config.cameras.get(camera) @@ -131,8 +132,11 @@ class ExternalEventProcessor: label: str, event_id: str, draw: dict[str, any], - img_frame: any, - ) -> str: + img_frame: Optional[ndarray], + ) -> Optional[str]: + if not img_frame: + return None + # write clean snapshot if enabled if camera_config.snapshots.clean_copy: ret, png = cv2.imencode(".png", img_frame) diff --git a/frigate/object_processing.py b/frigate/object_processing.py index 937c935ba..ef23c3de3 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -6,7 +6,7 @@ import queue import threading from collections import Counter, defaultdict from multiprocessing.synchronize import Event as MpEvent -from typing import Callable +from typing import Callable, Optional import cv2 import numpy as np @@ -784,13 +784,18 @@ class TrackedObjectProcessor(threading.Thread): else: return {} - def get_current_frame(self, camera, draw_options={}): + def get_current_frame( + self, camera: str, draw_options: dict[str, any] = {} + ) -> Optional[np.ndarray]: if camera == "birdseye": return self.frame_manager.get( "birdseye", (self.config.birdseye.height * 3 // 2, self.config.birdseye.width), ) + if camera not in self.camera_states: + return None + return self.camera_states[camera].get_current_frame(draw_options) def get_current_frame_time(self, camera) -> int: From 71e8f75a01eaedb4f4795b05b4a4145cb53bfb79 Mon Sep 17 00:00:00 2001 From: Alessandro Genova Date: Sat, 30 Nov 2024 19:27:21 -0500 Subject: [PATCH 191/479] Let the docker container spend more time to clean up and shut down (docs) (#15275) --- docs/docs/frigate/installation.md | 2 ++ docs/docs/guides/getting_started.md | 1 + 2 files changed, 3 insertions(+) diff --git a/docs/docs/frigate/installation.md b/docs/docs/frigate/installation.md index e3599e628..ede0fa897 100644 --- a/docs/docs/frigate/installation.md +++ b/docs/docs/frigate/installation.md @@ -193,6 +193,7 @@ services: container_name: frigate privileged: true # this may not be necessary for all setups restart: unless-stopped + stop_grace_period: 30s # allow enough time to shut down the various services image: ghcr.io/blakeblackshear/frigate:stable shm_size: "512mb" # update for your cameras based on calculation above devices: @@ -224,6 +225,7 @@ If you can't use docker compose, you can run the container with something simila docker run -d \ --name frigate \ --restart=unless-stopped \ + --stop-timeout 30 \ --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \ --device /dev/bus/usb:/dev/bus/usb \ --device /dev/dri/renderD128 \ diff --git a/docs/docs/guides/getting_started.md b/docs/docs/guides/getting_started.md index 829612bb0..bb880b8f0 100644 --- a/docs/docs/guides/getting_started.md +++ b/docs/docs/guides/getting_started.md @@ -115,6 +115,7 @@ services: frigate: container_name: frigate restart: unless-stopped + stop_grace_period: 30s image: ghcr.io/blakeblackshear/frigate:stable volumes: - ./config:/config From 5802a664690a662c1cae152cacde1bfea20fe528 Mon Sep 17 00:00:00 2001 From: tpjanssen <25168870+tpjanssen@users.noreply.github.com> Date: Sun, 1 Dec 2024 15:47:37 +0100 Subject: [PATCH 192/479] Fix audio events in explore section (#15286) * Fix audio events in explore section Make sure that audio events are listed in the explore section * Update audio.py * Hide other submit options Only allow submits for objects only --- docs/static/frigate-api.yaml | 4 ++-- frigate/api/defs/events_body.py | 4 ++-- frigate/events/audio.py | 4 ++++ frigate/events/external.py | 1 + web/src/components/menu/SearchResultActions.tsx | 2 ++ web/src/components/overlay/detail/ReviewDetailDialog.tsx | 1 + web/src/components/overlay/detail/SearchDetailDialog.tsx | 2 +- 7 files changed, 13 insertions(+), 5 deletions(-) diff --git a/docs/static/frigate-api.yaml b/docs/static/frigate-api.yaml index 325af2850..5e4ecdbdc 100644 --- a/docs/static/frigate-api.yaml +++ b/docs/static/frigate-api.yaml @@ -3225,7 +3225,7 @@ components: title: Sub Label score: anyOf: - - type: integer + - type: number - type: 'null' title: Score default: 0 @@ -3264,7 +3264,7 @@ components: properties: end_time: anyOf: - - type: integer + - type: number - type: 'null' title: End Time type: object diff --git a/frigate/api/defs/events_body.py b/frigate/api/defs/events_body.py index ca1256598..db2b4060b 100644 --- a/frigate/api/defs/events_body.py +++ b/frigate/api/defs/events_body.py @@ -17,14 +17,14 @@ class EventsDescriptionBody(BaseModel): class EventsCreateBody(BaseModel): source_type: Optional[str] = "api" sub_label: Optional[str] = None - score: Optional[int] = 0 + score: Optional[float] = 0 duration: Optional[int] = 30 include_recording: Optional[bool] = True draw: Optional[dict] = {} class EventsEndBody(BaseModel): - end_time: Optional[int] = None + end_time: Optional[float] = None class SubmitPlusBody(BaseModel): diff --git a/frigate/events/audio.py b/frigate/events/audio.py index 80d035894..7675f821b 100644 --- a/frigate/events/audio.py +++ b/frigate/events/audio.py @@ -216,6 +216,10 @@ class AudioEventMaintainer(threading.Thread): "label": label, "last_detection": datetime.datetime.now().timestamp(), } + else: + self.logger.warning( + f"Failed to create audio event with status code {resp.status_code}" + ) def expire_detections(self) -> None: now = datetime.datetime.now().timestamp() diff --git a/frigate/events/external.py b/frigate/events/external.py index 922917bb4..02671b207 100644 --- a/frigate/events/external.py +++ b/frigate/events/external.py @@ -108,6 +108,7 @@ class ExternalEventProcessor: EventTypeEnum.api, EventStateEnum.end, None, + "", {"id": event_id, "end_time": end_time}, ) ) diff --git a/web/src/components/menu/SearchResultActions.tsx b/web/src/components/menu/SearchResultActions.tsx index 10f0ed623..277ce2169 100644 --- a/web/src/components/menu/SearchResultActions.tsx +++ b/web/src/components/menu/SearchResultActions.tsx @@ -128,6 +128,7 @@ export default function SearchResultActions({ config?.plus?.enabled && searchResult.has_snapshot && searchResult.end_time && + searchResult.data.type == "object" && !searchResult.plus_id && ( @@ -197,6 +198,7 @@ export default function SearchResultActions({ config?.plus?.enabled && searchResult.has_snapshot && searchResult.end_time && + searchResult.data.type == "object" && !searchResult.plus_id && ( diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index 74a9950b9..c3e7ac91d 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -379,6 +379,7 @@ function EventItem({ {event.has_snapshot && event.plus_id == undefined && + event.data.type == "object" && config?.plus.enabled && ( diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index f7af31606..f63dffcc1 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -626,7 +626,7 @@ export function ObjectSnapshotTab({
)} - {search.plus_id !== "not_enabled" && search.end_time && ( + {search.data.type == "object" && search.plus_id !== "not_enabled" && search.end_time && (
From 002fdeae67d40a9d25087cb69f981e98cede2236 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 1 Dec 2024 10:39:35 -0600 Subject: [PATCH 193/479] SHM tweaks (#15274) * Use env var to control max number of frames * Handle type * Fix frame_name not being sent * Formatting --- frigate/app.py | 6 +++++- frigate/const.py | 2 ++ frigate/review/maintainer.py | 4 +++- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index 6518c1ddf..ba82757f9 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -36,6 +36,7 @@ from frigate.const import ( EXPORT_DIR, MODEL_CACHE_DIR, RECORD_DIR, + SHM_FRAMES_VAR, ) from frigate.db.sqlitevecq import SqliteVecQueueDatabase from frigate.embeddings import EmbeddingsContext, manage_embeddings @@ -523,7 +524,10 @@ class FrigateApp: if cam_total_frame_size == 0.0: return 0 - shm_frame_count = min(200, int(available_shm / (cam_total_frame_size))) + shm_frame_count = min( + int(os.environ.get(SHM_FRAMES_VAR, "50")), + int(available_shm / (cam_total_frame_size)), + ) logger.debug( f"Calculated total camera size {available_shm} / {cam_total_frame_size} :: {shm_frame_count} frames for each camera in SHM" diff --git a/frigate/const.py b/frigate/const.py index c83b10e73..5976f47b1 100644 --- a/frigate/const.py +++ b/frigate/const.py @@ -13,6 +13,8 @@ FRIGATE_LOCALHOST = "http://127.0.0.1:5000" PLUS_ENV_VAR = "PLUS_API_KEY" PLUS_API_HOST = "https://api.frigate.video" +SHM_FRAMES_VAR = "SHM_MAX_FRAMES" + # Attribute & Object constants DEFAULT_ATTRIBUTE_LABEL_MAP = { diff --git a/frigate/review/maintainer.py b/frigate/review/maintainer.py index 23a42e7a7..de137cb26 100644 --- a/frigate/review/maintainer.py +++ b/frigate/review/maintainer.py @@ -480,7 +480,9 @@ class ReviewSegmentMaintainer(threading.Thread): if not self.config.cameras[camera].record.enabled: if current_segment: - self.update_existing_segment(current_segment, frame_time, []) + self.update_existing_segment( + current_segment, frame_name, frame_time, [] + ) continue From 4a5fe4138e206fe425bfccaafa61910e297035bd Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 1 Dec 2024 12:08:03 -0600 Subject: [PATCH 194/479] Explore audio event tweaks (#15291) --- .../components/menu/SearchResultActions.tsx | 39 +++--- .../overlay/detail/SearchDetailDialog.tsx | 120 +++++++++--------- 2 files changed, 82 insertions(+), 77 deletions(-) diff --git a/web/src/components/menu/SearchResultActions.tsx b/web/src/components/menu/SearchResultActions.tsx index 277ce2169..fee12a50f 100644 --- a/web/src/components/menu/SearchResultActions.tsx +++ b/web/src/components/menu/SearchResultActions.tsx @@ -108,13 +108,15 @@ export default function SearchResultActions({ )} - - - View object lifecycle - + {searchResult.data.type == "object" && ( + + + View object lifecycle + + )} {config?.semantic_search?.enabled && isContextMenu && ( ) : ( <> - {config?.semantic_search?.enabled && ( - - - - - Find similar - - )} + {config?.semantic_search?.enabled && + searchResult.data.type == "object" && ( + + + + + Find similar + + )} {!isMobileOnly && config?.plus?.enabled && diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index f63dffcc1..b0eeac98d 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -452,7 +452,7 @@ function ObjectDetailsTab({ draggable={false} src={`${apiHost}api/events/${search.id}/thumbnail.jpg`} /> - {config?.semantic_search.enabled && ( + {config?.semantic_search.enabled && search.data.type == "object" && (
)} - {search.data.type == "object" && search.plus_id !== "not_enabled" && search.end_time && ( - - -
-
- Submit To Frigate+ -
-
- Objects in locations you want to avoid are not false - positives. Submitting them as false positives will confuse - the model. -
-
- -
- {state == "reviewing" && ( - <> - - - - )} - {state == "uploading" && } - {state == "submitted" && ( -
- - Submitted + {search.data.type == "object" && + search.plus_id !== "not_enabled" && + search.end_time && ( + + +
+
+ Submit To Frigate+
- )} -
- - - )} +
+ Objects in locations you want to avoid are not false + positives. Submitting them as false positives will + confuse the model. +
+
+ +
+ {state == "reviewing" && ( + <> + + + + )} + {state == "uploading" && } + {state == "submitted" && ( +
+ + Submitted +
+ )} +
+ + + )}
From a1fa9decaddc539e6d2748782d72248f34c746d4 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 1 Dec 2024 12:37:45 -0600 Subject: [PATCH 195/479] Fix event cleanup debug logging crash (#15293) --- frigate/events/cleanup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/events/cleanup.py b/frigate/events/cleanup.py index 8ae38b534..5400cc660 100644 --- a/frigate/events/cleanup.py +++ b/frigate/events/cleanup.py @@ -110,7 +110,7 @@ class EventCleanup(threading.Thread): .namedtuples() .iterator() ) - logger.debug(f"{len(expired_events)} events can be expired") + logger.debug(f"{len(list(expired_events))} events can be expired") # delete the media from disk for expired in expired_events: media_name = f"{expired.camera}-{expired.id}" From c95bc9fe44ba299f611033c63f98b6cdc16b2801 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 1 Dec 2024 13:33:10 -0600 Subject: [PATCH 196/479] Handle case where camera name ends in number (#15296) --- frigate/app.py | 2 +- frigate/video.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/frigate/app.py b/frigate/app.py index ba82757f9..34dcf3cd7 100644 --- a/frigate/app.py +++ b/frigate/app.py @@ -437,7 +437,7 @@ class FrigateApp: # pre-create shms for i in range(shm_frame_count): frame_size = config.frame_shape_yuv[0] * config.frame_shape_yuv[1] - self.frame_manager.create(f"{config.name}{i}", frame_size) + self.frame_manager.create(f"{config.name}_{i}", frame_size) capture_process = util.Process( target=capture_camera, diff --git a/frigate/video.py b/frigate/video.py index 96b562e8c..d8ff1a869 100755 --- a/frigate/video.py +++ b/frigate/video.py @@ -113,7 +113,7 @@ def capture_frames( fps.value = frame_rate.eps() skipped_fps.value = skipped_eps.eps() current_frame.value = datetime.datetime.now().timestamp() - frame_name = f"{config.name}{frame_index}" + frame_name = f"{config.name}_{frame_index}" frame_buffer = frame_manager.write(frame_name) try: frame_buffer[:] = ffmpeg_process.stdout.read(frame_size) From 833cdcb6d2a17a13377ef469218eec524ddc58a7 Mon Sep 17 00:00:00 2001 From: James Livulpi Date: Sun, 1 Dec 2024 21:07:44 -0500 Subject: [PATCH 197/479] fix audio event create (#15299) --- frigate/events/external.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frigate/events/external.py b/frigate/events/external.py index 02671b207..0d3408975 100644 --- a/frigate/events/external.py +++ b/frigate/events/external.py @@ -135,7 +135,7 @@ class ExternalEventProcessor: draw: dict[str, any], img_frame: Optional[ndarray], ) -> Optional[str]: - if not img_frame: + if img_frame is None: return None # write clean snapshot if enabled From 5475672a9defd1991a92ee9bcd09101af97dd975 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C4=8Cerm=C3=A1k?= Date: Mon, 2 Dec 2024 15:35:51 +0100 Subject: [PATCH 198/479] Fix extraction of Hailo userspace libs (#15187) The archive already has everything contained in a rootfs folder, extract it as-is to the root folder. This also reverts changes from 33957e53600afd0e1a677d6575917955d156dd4a which addressed the same issue in a less optimal way. --- docker/hailo8l/Dockerfile | 3 --- docker/hailo8l/install_hailort.sh | 4 +--- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/docker/hailo8l/Dockerfile b/docker/hailo8l/Dockerfile index 68ab09001..959e7692e 100644 --- a/docker/hailo8l/Dockerfile +++ b/docker/hailo8l/Dockerfile @@ -36,8 +36,5 @@ RUN pip3 install -U /deps/hailo-wheels/*.whl # Copy base files from the rootfs stage COPY --from=rootfs / / -# Set Library path for hailo driver -ENV LD_LIBRARY_PATH=/rootfs/usr/local/lib/ - # Set workdir WORKDIR /opt/frigate/ diff --git a/docker/hailo8l/install_hailort.sh b/docker/hailo8l/install_hailort.sh index 004db86c9..62eba9611 100755 --- a/docker/hailo8l/install_hailort.sh +++ b/docker/hailo8l/install_hailort.sh @@ -10,10 +10,8 @@ elif [[ "${TARGETARCH}" == "arm64" ]]; then arch="aarch64" fi -mkdir -p /rootfs - wget -qO- "https://github.com/frigate-nvr/hailort/releases/download/v${hailo_version}/hailort-${TARGETARCH}.tar.gz" | - tar -C /rootfs/ -xzf - + tar -C / -xzf - mkdir -p /hailo-wheels From 5f42caad03bd17f462836b721c7db7f7241d3b08 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 2 Dec 2024 12:12:55 -0600 Subject: [PATCH 199/479] Explore bulk actions (#15307) * use id instead of index for object details and scrolling * long press package and hook * fix long press in review * search action group * multi select in explore * add bulk deletion to backend api * clean up * mimic behavior of review * don't open dialog on left click when mutli selecting * context menu on container ref * revert long press code * clean up --- frigate/api/defs/events_body.py | 6 +- frigate/api/event.py | 59 ++++- web/package-lock.json | 10 + web/package.json | 1 + web/src/components/card/SearchThumbnail.tsx | 18 +- .../components/filter/SearchActionGroup.tsx | 132 ++++++++++ web/src/hooks/use-press.ts | 54 ++++ web/src/views/explore/ExploreView.tsx | 10 +- web/src/views/search/SearchView.tsx | 245 ++++++++++++++---- 9 files changed, 452 insertions(+), 83 deletions(-) create mode 100644 web/src/components/filter/SearchActionGroup.tsx create mode 100644 web/src/hooks/use-press.ts diff --git a/frigate/api/defs/events_body.py b/frigate/api/defs/events_body.py index db2b4060b..1c8576f02 100644 --- a/frigate/api/defs/events_body.py +++ b/frigate/api/defs/events_body.py @@ -1,4 +1,4 @@ -from typing import Optional, Union +from typing import List, Optional, Union from pydantic import BaseModel, Field @@ -27,5 +27,9 @@ class EventsEndBody(BaseModel): end_time: Optional[float] = None +class EventsDeleteBody(BaseModel): + event_ids: List[str] = Field(title="The event IDs to delete") + + class SubmitPlusBody(BaseModel): include_annotation: int = Field(default=1) diff --git a/frigate/api/event.py b/frigate/api/event.py index 3b38ff072..fafa28272 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -16,6 +16,7 @@ from playhouse.shortcuts import model_to_dict from frigate.api.defs.events_body import ( EventsCreateBody, + EventsDeleteBody, EventsDescriptionBody, EventsEndBody, EventsSubLabelBody, @@ -1036,34 +1037,64 @@ def regenerate_description( ) -@router.delete("/events/{event_id}") -def delete_event(request: Request, event_id: str): +def delete_single_event(event_id: str, request: Request) -> dict: try: event = Event.get(Event.id == event_id) except DoesNotExist: - return JSONResponse( - content=({"success": False, "message": "Event " + event_id + " not found"}), - status_code=404, - ) + return {"success": False, "message": f"Event {event_id} not found"} media_name = f"{event.camera}-{event.id}" if event.has_snapshot: - media = Path(f"{os.path.join(CLIPS_DIR, media_name)}.jpg") - media.unlink(missing_ok=True) - media = Path(f"{os.path.join(CLIPS_DIR, media_name)}-clean.png") - media.unlink(missing_ok=True) + snapshot_paths = [ + Path(f"{os.path.join(CLIPS_DIR, media_name)}.jpg"), + Path(f"{os.path.join(CLIPS_DIR, media_name)}-clean.png"), + ] + for media in snapshot_paths: + media.unlink(missing_ok=True) event.delete_instance() Timeline.delete().where(Timeline.source_id == event_id).execute() + # If semantic search is enabled, update the index if request.app.frigate_config.semantic_search.enabled: context: EmbeddingsContext = request.app.embeddings context.db.delete_embeddings_thumbnail(event_ids=[event_id]) context.db.delete_embeddings_description(event_ids=[event_id]) - return JSONResponse( - content=({"success": True, "message": "Event " + event_id + " deleted"}), - status_code=200, - ) + + return {"success": True, "message": f"Event {event_id} deleted"} + + +@router.delete("/events/{event_id}") +def delete_event(request: Request, event_id: str): + result = delete_single_event(event_id, request) + status_code = 200 if result["success"] else 404 + return JSONResponse(content=result, status_code=status_code) + + +@router.delete("/events/") +def delete_events(request: Request, body: EventsDeleteBody): + if not body.event_ids: + return JSONResponse( + content=({"success": False, "message": "No event IDs provided."}), + status_code=404, + ) + + deleted_events = [] + not_found_events = [] + + for event_id in body.event_ids: + result = delete_single_event(event_id, request) + if result["success"]: + deleted_events.append(event_id) + else: + not_found_events.append(event_id) + + response = { + "success": True, + "deleted_events": deleted_events, + "not_found_events": not_found_events, + } + return JSONResponse(content=response, status_code=200) @router.post("/events/{camera_name}/{label}/create") diff --git a/web/package-lock.json b/web/package-lock.json index a0971c361..7ce6345af 100644 --- a/web/package-lock.json +++ b/web/package-lock.json @@ -72,6 +72,7 @@ "tailwind-merge": "^2.4.0", "tailwind-scrollbar": "^3.1.0", "tailwindcss-animate": "^1.0.7", + "use-long-press": "^3.2.0", "vaul": "^0.9.1", "vite-plugin-monaco-editor": "^1.1.0", "zod": "^3.23.8" @@ -8709,6 +8710,15 @@ "scheduler": ">=0.19.0" } }, + "node_modules/use-long-press": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/use-long-press/-/use-long-press-3.2.0.tgz", + "integrity": "sha512-uq5o2qFR1VRjHn8Of7Fl344/AGvgk7C5Mcb4aSb1ZRVp6PkgdXJJLdRrlSTJQVkkQcDuqFbFc3mDX4COg7mRTA==", + "license": "MIT", + "peerDependencies": { + "react": ">=16.8.0" + } + }, "node_modules/use-sidecar": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/use-sidecar/-/use-sidecar-1.1.2.tgz", diff --git a/web/package.json b/web/package.json index 73b2ed309..d76e6ad10 100644 --- a/web/package.json +++ b/web/package.json @@ -78,6 +78,7 @@ "tailwind-merge": "^2.4.0", "tailwind-scrollbar": "^3.1.0", "tailwindcss-animate": "^1.0.7", + "use-long-press": "^3.2.0", "vaul": "^0.9.1", "vite-plugin-monaco-editor": "^1.1.0", "zod": "^3.23.8" diff --git a/web/src/components/card/SearchThumbnail.tsx b/web/src/components/card/SearchThumbnail.tsx index e96632400..7dfa7b583 100644 --- a/web/src/components/card/SearchThumbnail.tsx +++ b/web/src/components/card/SearchThumbnail.tsx @@ -1,4 +1,4 @@ -import { useCallback, useMemo } from "react"; +import { useMemo } from "react"; import { useApiHost } from "@/api"; import { getIconForLabel } from "@/utils/iconUtil"; import useSWR from "swr"; @@ -12,10 +12,11 @@ import { capitalizeFirstLetter } from "@/utils/stringUtil"; import { SearchResult } from "@/types/search"; import { cn } from "@/lib/utils"; import { TooltipPortal } from "@radix-ui/react-tooltip"; +import useContextMenu from "@/hooks/use-contextmenu"; type SearchThumbnailProps = { searchResult: SearchResult; - onClick: (searchResult: SearchResult) => void; + onClick: (searchResult: SearchResult, ctrl: boolean, detail: boolean) => void; }; export default function SearchThumbnail({ @@ -28,9 +29,9 @@ export default function SearchThumbnail({ // interactions - const handleOnClick = useCallback(() => { - onClick(searchResult); - }, [searchResult, onClick]); + useContextMenu(imgRef, () => { + onClick(searchResult, true, false); + }); const objectLabel = useMemo(() => { if ( @@ -45,7 +46,10 @@ export default function SearchThumbnail({ }, [config, searchResult]); return ( -
+
onClick(searchResult, false, true)} + > onClick(searchResult)} + onClick={() => onClick(searchResult, false, true)} > {getIconForLabel(objectLabel, "size-3 text-white")} {Math.round( diff --git a/web/src/components/filter/SearchActionGroup.tsx b/web/src/components/filter/SearchActionGroup.tsx new file mode 100644 index 000000000..aac03ad1c --- /dev/null +++ b/web/src/components/filter/SearchActionGroup.tsx @@ -0,0 +1,132 @@ +import { useCallback, useState } from "react"; +import axios from "axios"; +import { Button, buttonVariants } from "../ui/button"; +import { isDesktop } from "react-device-detect"; +import { HiTrash } from "react-icons/hi"; +import { + AlertDialog, + AlertDialogAction, + AlertDialogCancel, + AlertDialogContent, + AlertDialogDescription, + AlertDialogFooter, + AlertDialogHeader, + AlertDialogTitle, +} from "../ui/alert-dialog"; +import useKeyboardListener from "@/hooks/use-keyboard-listener"; +import { toast } from "sonner"; + +type SearchActionGroupProps = { + selectedObjects: string[]; + setSelectedObjects: (ids: string[]) => void; + pullLatestData: () => void; +}; +export default function SearchActionGroup({ + selectedObjects, + setSelectedObjects, + pullLatestData, +}: SearchActionGroupProps) { + const onClearSelected = useCallback(() => { + setSelectedObjects([]); + }, [setSelectedObjects]); + + const onDelete = useCallback(async () => { + await axios + .delete(`events/`, { + data: { event_ids: selectedObjects }, + }) + .then((resp) => { + if (resp.status == 200) { + toast.success("Tracked objects deleted successfully.", { + position: "top-center", + }); + setSelectedObjects([]); + pullLatestData(); + } + }) + .catch(() => { + toast.error("Failed to delete tracked objects.", { + position: "top-center", + }); + }); + }, [selectedObjects, setSelectedObjects, pullLatestData]); + + const [deleteDialogOpen, setDeleteDialogOpen] = useState(false); + const [bypassDialog, setBypassDialog] = useState(false); + + useKeyboardListener(["Shift"], (_, modifiers) => { + setBypassDialog(modifiers.shift); + }); + + const handleDelete = useCallback(() => { + if (bypassDialog) { + onDelete(); + } else { + setDeleteDialogOpen(true); + } + }, [bypassDialog, onDelete]); + + return ( + <> + setDeleteDialogOpen(!deleteDialogOpen)} + > + + + Confirm Delete + + + Deleting these {selectedObjects.length} tracked objects removes the + snapshot, any saved embeddings, and any associated object lifecycle + entries. Recorded footage of these tracked objects in History view + will NOT be deleted. +
+
+ Are you sure you want to proceed? +
+
+ Hold the Shift key to bypass this dialog in the future. + + + Cancel + + Delete + + + + + +
+
+
{`${selectedObjects.length} selected`}
+
{"|"}
+
+ Unselect +
+
+
+ +
+
+ + ); +} diff --git a/web/src/hooks/use-press.ts b/web/src/hooks/use-press.ts new file mode 100644 index 000000000..6e97ce11b --- /dev/null +++ b/web/src/hooks/use-press.ts @@ -0,0 +1,54 @@ +// https://gist.github.com/cpojer/641bf305e6185006ea453e7631b80f95 + +import { useCallback, useState } from "react"; +import { + LongPressCallbackMeta, + LongPressReactEvents, + useLongPress, +} from "use-long-press"; + +export default function usePress( + options: Omit[1], "onCancel" | "onStart"> & { + onLongPress: NonNullable[0]>; + onPress: (event: LongPressReactEvents) => void; + }, +) { + const { onLongPress, onPress, ...actualOptions } = options; + const [hasLongPress, setHasLongPress] = useState(false); + + const onCancel = useCallback(() => { + if (hasLongPress) { + setHasLongPress(false); + } + }, [hasLongPress]); + + const bind = useLongPress( + useCallback( + ( + event: LongPressReactEvents, + meta: LongPressCallbackMeta, + ) => { + setHasLongPress(true); + onLongPress(event, meta); + }, + [onLongPress], + ), + { + ...actualOptions, + onCancel, + onStart: onCancel, + }, + ); + + return useCallback( + () => ({ + ...bind(), + onClick: (event: LongPressReactEvents) => { + if (!hasLongPress) { + onPress(event); + } + }, + }), + [bind, hasLongPress, onPress], + ); +} diff --git a/web/src/views/explore/ExploreView.tsx b/web/src/views/explore/ExploreView.tsx index ea9c3cbef..0ec825416 100644 --- a/web/src/views/explore/ExploreView.tsx +++ b/web/src/views/explore/ExploreView.tsx @@ -26,7 +26,7 @@ type ExploreViewProps = { searchDetail: SearchResult | undefined; setSearchDetail: (search: SearchResult | undefined) => void; setSimilaritySearch: (search: SearchResult) => void; - onSelectSearch: (item: SearchResult, index: number, page?: SearchTab) => void; + onSelectSearch: (item: SearchResult, ctrl: boolean, page?: SearchTab) => void; }; export default function ExploreView({ @@ -125,7 +125,7 @@ type ThumbnailRowType = { setSearchDetail: (search: SearchResult | undefined) => void; mutate: () => void; setSimilaritySearch: (search: SearchResult) => void; - onSelectSearch: (item: SearchResult, index: number, page?: SearchTab) => void; + onSelectSearch: (item: SearchResult, ctrl: boolean, page?: SearchTab) => void; }; function ThumbnailRow({ @@ -205,7 +205,7 @@ type ExploreThumbnailImageProps = { setSearchDetail: (search: SearchResult | undefined) => void; mutate: () => void; setSimilaritySearch: (search: SearchResult) => void; - onSelectSearch: (item: SearchResult, index: number, page?: SearchTab) => void; + onSelectSearch: (item: SearchResult, ctrl: boolean, page?: SearchTab) => void; }; function ExploreThumbnailImage({ event, @@ -225,11 +225,11 @@ function ExploreThumbnailImage({ }; const handleShowObjectLifecycle = () => { - onSelectSearch(event, 0, "object lifecycle"); + onSelectSearch(event, false, "object lifecycle"); }; const handleShowSnapshot = () => { - onSelectSearch(event, 0, "snapshot"); + onSelectSearch(event, false, "snapshot"); }; return ( diff --git a/web/src/views/search/SearchView.tsx b/web/src/views/search/SearchView.tsx index 378b313e0..be430f134 100644 --- a/web/src/views/search/SearchView.tsx +++ b/web/src/views/search/SearchView.tsx @@ -30,6 +30,7 @@ import { } from "@/components/ui/tooltip"; import Chip from "@/components/indicators/Chip"; import { TooltipPortal } from "@radix-ui/react-tooltip"; +import SearchActionGroup from "@/components/filter/SearchActionGroup"; type SearchViewProps = { search: string; @@ -181,20 +182,53 @@ export default function SearchView({ // search interaction - const [selectedIndex, setSelectedIndex] = useState(null); + const [selectedObjects, setSelectedObjects] = useState([]); const itemRefs = useRef<(HTMLDivElement | null)[]>([]); const onSelectSearch = useCallback( - (item: SearchResult, index: number, page: SearchTab = "details") => { - setPage(page); - setSearchDetail(item); - setSelectedIndex(index); + (item: SearchResult, ctrl: boolean, page: SearchTab = "details") => { + if (selectedObjects.length > 1 || ctrl) { + const index = selectedObjects.indexOf(item.id); + + if (index != -1) { + if (selectedObjects.length == 1) { + setSelectedObjects([]); + } else { + const copy = [ + ...selectedObjects.slice(0, index), + ...selectedObjects.slice(index + 1), + ]; + setSelectedObjects(copy); + } + } else { + const copy = [...selectedObjects]; + copy.push(item.id); + setSelectedObjects(copy); + } + } else { + setPage(page); + setSearchDetail(item); + } }, - [], + [selectedObjects], ); + const onSelectAllObjects = useCallback(() => { + if (!uniqueResults || uniqueResults.length == 0) { + return; + } + + if (selectedObjects.length < uniqueResults.length) { + setSelectedObjects(uniqueResults.map((value) => value.id)); + } else { + setSelectedObjects([]); + } + }, [uniqueResults, selectedObjects]); + useEffect(() => { - setSelectedIndex(0); + setSelectedObjects([]); + // unselect items when search term or filter changes + // eslint-disable-next-line react-hooks/exhaustive-deps }, [searchTerm, searchFilter]); // confidence score @@ -243,23 +277,44 @@ export default function SearchView({ } switch (key) { - case "ArrowLeft": - setSelectedIndex((prevIndex) => { - const newIndex = - prevIndex === null - ? uniqueResults.length - 1 - : (prevIndex - 1 + uniqueResults.length) % uniqueResults.length; - setSearchDetail(uniqueResults[newIndex]); - return newIndex; - }); + case "a": + if (modifiers.ctrl) { + onSelectAllObjects(); + } break; - case "ArrowRight": - setSelectedIndex((prevIndex) => { + case "ArrowLeft": + if (uniqueResults.length > 0) { + const currentIndex = searchDetail + ? uniqueResults.findIndex( + (result) => result.id === searchDetail.id, + ) + : -1; + const newIndex = - prevIndex === null ? 0 : (prevIndex + 1) % uniqueResults.length; + currentIndex === -1 + ? uniqueResults.length - 1 + : (currentIndex - 1 + uniqueResults.length) % + uniqueResults.length; + setSearchDetail(uniqueResults[newIndex]); - return newIndex; - }); + } + break; + + case "ArrowRight": + if (uniqueResults.length > 0) { + const currentIndex = searchDetail + ? uniqueResults.findIndex( + (result) => result.id === searchDetail.id, + ) + : -1; + + const newIndex = + currentIndex === -1 + ? 0 + : (currentIndex + 1) % uniqueResults.length; + + setSearchDetail(uniqueResults[newIndex]); + } break; case "PageDown": contentRef.current?.scrollBy({ @@ -275,32 +330,80 @@ export default function SearchView({ break; } }, - [uniqueResults, inputFocused], + [uniqueResults, inputFocused, onSelectAllObjects, searchDetail], ); useKeyboardListener( - ["ArrowLeft", "ArrowRight", "PageDown", "PageUp"], + ["a", "ArrowLeft", "ArrowRight", "PageDown", "PageUp"], onKeyboardShortcut, !inputFocused, ); // scroll into view + const [prevSearchDetail, setPrevSearchDetail] = useState< + SearchResult | undefined + >(); + + // keep track of previous ref to outline thumbnail when dialog closes + const prevSearchDetailRef = useRef(); + useEffect(() => { - if ( - selectedIndex !== null && - uniqueResults && - itemRefs.current?.[selectedIndex] - ) { - scrollIntoView(itemRefs.current[selectedIndex], { - block: "center", - behavior: "smooth", - scrollMode: "if-needed", - }); + if (searchDetail === undefined && prevSearchDetailRef.current) { + setPrevSearchDetail(prevSearchDetailRef.current); } - // we only want to scroll when the index changes + prevSearchDetailRef.current = searchDetail; + }, [searchDetail]); + + useEffect(() => { + if (uniqueResults && itemRefs.current && prevSearchDetail) { + const selectedIndex = uniqueResults.findIndex( + (result) => result.id === prevSearchDetail.id, + ); + + const parent = itemRefs.current[selectedIndex]; + + if (selectedIndex !== -1 && parent) { + const target = parent.querySelector(".review-item-ring"); + if (target) { + scrollIntoView(target, { + block: "center", + behavior: "smooth", + scrollMode: "if-needed", + }); + target.classList.add(`outline-selected`); + target.classList.remove("outline-transparent"); + + setTimeout(() => { + target.classList.remove(`outline-selected`); + target.classList.add("outline-transparent"); + }, 3000); + } + } + } + // we only want to scroll when the dialog closes // eslint-disable-next-line react-hooks/exhaustive-deps - }, [selectedIndex]); + }, [prevSearchDetail]); + + useEffect(() => { + if (uniqueResults && itemRefs.current && searchDetail) { + const selectedIndex = uniqueResults.findIndex( + (result) => result.id === searchDetail.id, + ); + + const parent = itemRefs.current[selectedIndex]; + + if (selectedIndex !== -1 && parent) { + scrollIntoView(parent, { + block: "center", + behavior: "smooth", + scrollMode: "if-needed", + }); + } + } + // we only want to scroll when changing the detail pane + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [searchDetail]); // observer for loading more @@ -369,22 +472,39 @@ export default function SearchView({ {hasExistingSearch && (
- - - + {selectedObjects.length == 0 ? ( + <> + + + + + ) : ( +
+ +
+ )}
)} @@ -412,14 +532,14 @@ export default function SearchView({
{uniqueResults && uniqueResults.map((value, index) => { - const selected = selectedIndex === index; + const selected = selectedObjects.includes(value.id); return (
(itemRefs.current[index] = item)} data-start={value.start_time} - className="review-item relative flex flex-col rounded-lg" + className="relative flex flex-col rounded-lg" >
onSelectSearch(value, index)} + onClick={( + value: SearchResult, + ctrl: boolean, + detail: boolean, + ) => { + if (detail && selectedObjects.length == 0) { + setSearchDetail(value); + } else { + onSelectSearch( + value, + ctrl || selectedObjects.length > 0, + ); + } + }} /> {(searchTerm || searchFilter?.search_type?.includes("similarity")) && ( @@ -469,10 +602,10 @@ export default function SearchView({ }} refreshResults={refresh} showObjectLifecycle={() => - onSelectSearch(value, index, "object lifecycle") + onSelectSearch(value, false, "object lifecycle") } showSnapshot={() => - onSelectSearch(value, index, "snapshot") + onSelectSearch(value, false, "snapshot") } />
From 4dddc537350f7908a5dfdb42ecd4929c0b02ddba Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 2 Dec 2024 13:07:12 -0600 Subject: [PATCH 200/479] move label placement when overlapping small boxes (#15310) --- frigate/util/image.py | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/frigate/util/image.py b/frigate/util/image.py index 7b22c138e..301da9c6a 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -222,16 +222,25 @@ def draw_box_with_label( # set the text start position if position == "ul": text_offset_x = x_min - text_offset_y = 0 if y_min < line_height else y_min - (line_height + 8) + text_offset_y = max(0, y_min - (line_height + 8)) elif position == "ur": - text_offset_x = x_max - (text_width + 8) - text_offset_y = 0 if y_min < line_height else y_min - (line_height + 8) + text_offset_x = max(0, x_max - (text_width + 8)) + text_offset_y = max(0, y_min - (line_height + 8)) elif position == "bl": text_offset_x = x_min text_offset_y = y_max elif position == "br": - text_offset_x = x_max - (text_width + 8) + text_offset_x = max(0, x_max - (text_width + 8)) text_offset_y = y_max + + # Adjust position if it overlaps with the box + if position in {"ul", "ur"} and text_offset_y < y_min + thickness: + # Move the text below the box + text_offset_y = y_max + elif position in {"bl", "br"} and text_offset_y + line_height > y_max: + # Move the text above the box + text_offset_y = max(0, y_min - (line_height + 8)) + # make the coords of the box with a small padding of two pixels textbox_coords = ( (text_offset_x, text_offset_y), From a7294085994f819761740e8272be2de3a1efbb52 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 4 Dec 2024 06:14:53 -0600 Subject: [PATCH 201/479] preserve search query in overlay state hook (#15334) --- web/src/hooks/use-overlay-state.tsx | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/web/src/hooks/use-overlay-state.tsx b/web/src/hooks/use-overlay-state.tsx index 75d738e61..841585b25 100644 --- a/web/src/hooks/use-overlay-state.tsx +++ b/web/src/hooks/use-overlay-state.tsx @@ -15,7 +15,10 @@ export function useOverlayState( (value: S, replace: boolean = false) => { const newLocationState = { ...currentLocationState }; newLocationState[key] = value; - navigate(location.pathname, { state: newLocationState, replace }); + navigate(location.pathname + location.search, { + state: newLocationState, + replace, + }); }, // we know that these deps are correct // eslint-disable-next-line react-hooks/exhaustive-deps From a5a7cd3107d988f9aedfa92cab698c043026b4ab Mon Sep 17 00:00:00 2001 From: Rui Alves Date: Wed, 4 Dec 2024 12:52:08 +0000 Subject: [PATCH 202/479] Added more unit tests for the review controller (#15162) --- frigate/api/review.py | 20 +- frigate/test/http_api/base_http_test.py | 30 +- frigate/test/http_api/test_http_review.py | 557 ++++++++++++++++++++-- 3 files changed, 544 insertions(+), 63 deletions(-) diff --git a/frigate/api/review.py b/frigate/api/review.py index 21b468640..04e3e6dcd 100644 --- a/frigate/api/review.py +++ b/frigate/api/review.py @@ -26,6 +26,7 @@ from frigate.api.defs.review_responses import ( ) from frigate.api.defs.tags import Tags from frigate.models import Recordings, ReviewSegment +from frigate.review.maintainer import SeverityEnum from frigate.util.builtin import get_tz_modifiers logger = logging.getLogger(__name__) @@ -161,7 +162,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "alert"), + (ReviewSegment.severity == SeverityEnum.alert), ReviewSegment.has_been_reviewed, ) ], @@ -173,7 +174,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "detection"), + (ReviewSegment.severity == SeverityEnum.detection), ReviewSegment.has_been_reviewed, ) ], @@ -185,7 +186,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "alert"), + (ReviewSegment.severity == SeverityEnum.alert), 1, ) ], @@ -197,7 +198,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "detection"), + (ReviewSegment.severity == SeverityEnum.detection), 1, ) ], @@ -230,6 +231,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): label_clause = reduce(operator.or_, label_clauses) clauses.append((label_clause)) + day_in_seconds = 60 * 60 * 24 last_month = ( ReviewSegment.select( fn.strftime( @@ -246,7 +248,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "alert"), + (ReviewSegment.severity == SeverityEnum.alert), ReviewSegment.has_been_reviewed, ) ], @@ -258,7 +260,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "detection"), + (ReviewSegment.severity == SeverityEnum.detection), ReviewSegment.has_been_reviewed, ) ], @@ -270,7 +272,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "alert"), + (ReviewSegment.severity == SeverityEnum.alert), 1, ) ], @@ -282,7 +284,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == "detection"), + (ReviewSegment.severity == SeverityEnum.detection), 1, ) ], @@ -292,7 +294,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): ) .where(reduce(operator.and_, clauses)) .group_by( - (ReviewSegment.start_time + seconds_offset).cast("int") / (3600 * 24), + (ReviewSegment.start_time + seconds_offset).cast("int") / day_in_seconds, ) .order_by(ReviewSegment.start_time.desc()) ) diff --git a/frigate/test/http_api/base_http_test.py b/frigate/test/http_api/base_http_test.py index 013785692..ad1d449c5 100644 --- a/frigate/test/http_api/base_http_test.py +++ b/frigate/test/http_api/base_http_test.py @@ -9,7 +9,7 @@ from playhouse.sqliteq import SqliteQueueDatabase from frigate.api.fastapi_app import create_fastapi_app from frigate.config import FrigateConfig -from frigate.models import Event, ReviewSegment +from frigate.models import Event, Recordings, ReviewSegment from frigate.review.maintainer import SeverityEnum from frigate.test.const import TEST_DB, TEST_DB_CLEANUPS @@ -146,17 +146,35 @@ class BaseTestHttp(unittest.TestCase): def insert_mock_review_segment( self, id: str, - start_time: datetime.datetime = datetime.datetime.now().timestamp(), - end_time: datetime.datetime = datetime.datetime.now().timestamp() + 20, + start_time: float = datetime.datetime.now().timestamp(), + end_time: float = datetime.datetime.now().timestamp() + 20, + severity: SeverityEnum = SeverityEnum.alert, + has_been_reviewed: bool = False, ) -> Event: - """Inserts a basic event model with a given id.""" + """Inserts a review segment model with a given id.""" return ReviewSegment.insert( id=id, camera="front_door", start_time=start_time, end_time=end_time, - has_been_reviewed=False, - severity=SeverityEnum.alert, + has_been_reviewed=has_been_reviewed, + severity=severity, thumb_path=False, data={}, ).execute() + + def insert_mock_recording( + self, + id: str, + start_time: float = datetime.datetime.now().timestamp(), + end_time: float = datetime.datetime.now().timestamp() + 20, + ) -> Event: + """Inserts a recording model with a given id.""" + return Recordings.insert( + id=id, + path=id, + camera="front_door", + start_time=start_time, + end_time=end_time, + duration=end_time - start_time, + ).execute() diff --git a/frigate/test/http_api/test_http_review.py b/frigate/test/http_api/test_http_review.py index 19e1f26f8..3bd8779aa 100644 --- a/frigate/test/http_api/test_http_review.py +++ b/frigate/test/http_api/test_http_review.py @@ -1,76 +1,89 @@ -import datetime +from datetime import datetime, timedelta from fastapi.testclient import TestClient -from frigate.models import Event, ReviewSegment +from frigate.models import Event, Recordings, ReviewSegment +from frigate.review.maintainer import SeverityEnum from frigate.test.http_api.base_http_test import BaseTestHttp class TestHttpReview(BaseTestHttp): def setUp(self): - super().setUp([Event, ReviewSegment]) + super().setUp([Event, Recordings, ReviewSegment]) + self.app = super().create_app() + + def _get_reviews(self, ids: list[str]): + return list( + ReviewSegment.select(ReviewSegment.id) + .where(ReviewSegment.id.in_(ids)) + .execute() + ) + + def _get_recordings(self, ids: list[str]): + return list( + Recordings.select(Recordings.id).where(Recordings.id.in_(ids)).execute() + ) + + #################################################################################################################### + ################################### GET /review Endpoint ######################################################## + #################################################################################################################### # Does not return any data point since the end time (before parameter) is not passed and the review segment end_time is 2 seconds from now def test_get_review_no_filters_no_matches(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: super().insert_mock_review_segment("123456.random", now, now + 2) - reviews_response = client.get("/review") - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 0 + response = client.get("/review") + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 0 def test_get_review_no_filters(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: super().insert_mock_review_segment("123456.random", now - 2, now - 1) - reviews_response = client.get("/review") - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 1 + response = client.get("/review") + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 1 def test_get_review_with_time_filter_no_matches(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: id = "123456.random" super().insert_mock_review_segment(id, now, now + 2) params = { "after": now, "before": now + 3, } - reviews_response = client.get("/review", params=params) - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 0 + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 0 def test_get_review_with_time_filter(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: id = "123456.random" super().insert_mock_review_segment(id, now, now + 2) params = { "after": now - 1, "before": now + 3, } - reviews_response = client.get("/review", params=params) - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 1 - assert reviews_in_response[0]["id"] == id + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 1 + assert response_json[0]["id"] == id def test_get_review_with_limit_filter(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: id = "123456.random" id2 = "654321.random" super().insert_mock_review_segment(id, now, now + 2) @@ -80,17 +93,49 @@ class TestHttpReview(BaseTestHttp): "after": now, "before": now + 3, } - reviews_response = client.get("/review", params=params) - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 1 - assert reviews_in_response[0]["id"] == id2 + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 1 + assert response_json[0]["id"] == id2 + + def test_get_review_with_severity_filters_no_matches(self): + now = datetime.now().timestamp() + + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id, now, now + 2, SeverityEnum.detection) + params = { + "severity": "detection", + "after": now - 1, + "before": now + 3, + } + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 1 + assert response_json[0]["id"] == id + + def test_get_review_with_severity_filters(self): + now = datetime.now().timestamp() + + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id, now, now + 2, SeverityEnum.detection) + params = { + "severity": "alert", + "after": now - 1, + "before": now + 3, + } + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 0 def test_get_review_with_all_filters(self): - app = super().create_app() - now = datetime.datetime.now().timestamp() + now = datetime.now().timestamp() - with TestClient(app) as client: + with TestClient(self.app) as client: id = "123456.random" super().insert_mock_review_segment(id, now, now + 2) params = { @@ -103,8 +148,424 @@ class TestHttpReview(BaseTestHttp): "after": now - 1, "before": now + 3, } - reviews_response = client.get("/review", params=params) - assert reviews_response.status_code == 200 - reviews_in_response = reviews_response.json() - assert len(reviews_in_response) == 1 - assert reviews_in_response[0]["id"] == id + response = client.get("/review", params=params) + assert response.status_code == 200 + response_json = response.json() + assert len(response_json) == 1 + assert response_json[0]["id"] == id + + #################################################################################################################### + ################################### GET /review/summary Endpoint ################################################# + #################################################################################################################### + def test_get_review_summary_all_filters(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + params = { + "cameras": "front_door", + "labels": "all", + "zones": "all", + "timezone": "utc", + } + response = client.get("/review/summary", params=params) + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-24' + today_formatted = datetime.today().strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + today_formatted: { + "day": today_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + } + self.assertEqual(response_json, expected_response) + + def test_get_review_summary_no_filters(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + response = client.get("/review/summary") + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-24' + today_formatted = datetime.today().strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + today_formatted: { + "day": today_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + } + self.assertEqual(response_json, expected_response) + + def test_get_review_summary_multiple_days(self): + now = datetime.now() + five_days_ago = datetime.today() - timedelta(days=5) + + with TestClient(self.app) as client: + super().insert_mock_review_segment( + "123456.random", now.timestamp() - 2, now.timestamp() - 1 + ) + super().insert_mock_review_segment( + "654321.random", + five_days_ago.timestamp(), + five_days_ago.timestamp() + 1, + ) + response = client.get("/review/summary") + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-24' + today_formatted = now.strftime("%Y-%m-%d") + # e.g. '2024-11-19' + five_days_ago_formatted = five_days_ago.strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + today_formatted: { + "day": today_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + five_days_ago_formatted: { + "day": five_days_ago_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + } + self.assertEqual(response_json, expected_response) + + def test_get_review_summary_multiple_days_edge_cases(self): + now = datetime.now() + five_days_ago = datetime.today() - timedelta(days=5) + twenty_days_ago = datetime.today() - timedelta(days=20) + one_month_ago = datetime.today() - timedelta(days=30) + one_month_ago_ts = one_month_ago.timestamp() + + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random", now.timestamp()) + super().insert_mock_review_segment( + "123457.random", five_days_ago.timestamp() + ) + super().insert_mock_review_segment( + "123458.random", + twenty_days_ago.timestamp(), + None, + SeverityEnum.detection, + ) + # One month ago plus 5 seconds fits within the condition (review.start_time > month_ago). Assuming that the endpoint does not take more than 5 seconds to be invoked + super().insert_mock_review_segment( + "123459.random", + one_month_ago_ts + 5, + None, + SeverityEnum.detection, + ) + # This won't appear in the output since it's not within last month start_time clause (review.start_time > month_ago) + super().insert_mock_review_segment("123450.random", one_month_ago_ts) + response = client.get("/review/summary") + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-24' + today_formatted = now.strftime("%Y-%m-%d") + # e.g. '2024-11-19' + five_days_ago_formatted = five_days_ago.strftime("%Y-%m-%d") + # e.g. '2024-11-04' + twenty_days_ago_formatted = twenty_days_ago.strftime("%Y-%m-%d") + # e.g. '2024-10-24' + one_month_ago_formatted = one_month_ago.strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + today_formatted: { + "day": today_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + five_days_ago_formatted: { + "day": five_days_ago_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + twenty_days_ago_formatted: { + "day": twenty_days_ago_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 0, + "total_detection": 1, + }, + one_month_ago_formatted: { + "day": one_month_ago_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 0, + "total_detection": 1, + }, + } + self.assertEqual(response_json, expected_response) + + def test_get_review_summary_multiple_in_same_day(self): + now = datetime.now() + five_days_ago = datetime.today() - timedelta(days=5) + + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random", now.timestamp()) + five_days_ago_ts = five_days_ago.timestamp() + for i in range(20): + super().insert_mock_review_segment( + f"123456_{i}.random_alert", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.alert, + ) + for i in range(15): + super().insert_mock_review_segment( + f"123456_{i}.random_detection", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.detection, + ) + response = client.get("/review/summary") + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-24' + today_formatted = now.strftime("%Y-%m-%d") + # e.g. '2024-11-19' + five_days_ago_formatted = five_days_ago.strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + today_formatted: { + "day": today_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 1, + "total_detection": 0, + }, + five_days_ago_formatted: { + "day": five_days_ago_formatted, + "reviewed_alert": 0, + "reviewed_detection": 0, + "total_alert": 20, + "total_detection": 15, + }, + } + self.assertEqual(response_json, expected_response) + + def test_get_review_summary_multiple_in_same_day_with_reviewed(self): + five_days_ago = datetime.today() - timedelta(days=5) + + with TestClient(self.app) as client: + five_days_ago_ts = five_days_ago.timestamp() + for i in range(10): + super().insert_mock_review_segment( + f"123456_{i}.random_alert_not_reviewed", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.alert, + False, + ) + for i in range(10): + super().insert_mock_review_segment( + f"123456_{i}.random_alert_reviewed", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.alert, + True, + ) + for i in range(10): + super().insert_mock_review_segment( + f"123456_{i}.random_detection_not_reviewed", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.detection, + False, + ) + for i in range(5): + super().insert_mock_review_segment( + f"123456_{i}.random_detection_reviewed", + five_days_ago_ts, + five_days_ago_ts, + SeverityEnum.detection, + True, + ) + response = client.get("/review/summary") + assert response.status_code == 200 + response_json = response.json() + # e.g. '2024-11-19' + five_days_ago_formatted = five_days_ago.strftime("%Y-%m-%d") + expected_response = { + "last24Hours": { + "reviewed_alert": None, + "reviewed_detection": None, + "total_alert": None, + "total_detection": None, + }, + five_days_ago_formatted: { + "day": five_days_ago_formatted, + "reviewed_alert": 10, + "reviewed_detection": 5, + "total_alert": 20, + "total_detection": 15, + }, + } + self.assertEqual(response_json, expected_response) + + #################################################################################################################### + ################################### POST reviews/viewed Endpoint ################################################ + #################################################################################################################### + def test_post_reviews_viewed_no_body(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + response = client.post("/reviews/viewed") + # Missing ids + assert response.status_code == 422 + + def test_post_reviews_viewed_no_body_ids(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + body = {"ids": [""]} + response = client.post("/reviews/viewed", json=body) + # Missing ids + assert response.status_code == 422 + + def test_post_reviews_viewed_non_existent_id(self): + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id) + body = {"ids": ["1"]} + response = client.post("/reviews/viewed", json=body) + assert response.status_code == 200 + response = response.json() + assert response["success"] == True + assert response["message"] == "Reviewed multiple items" + # Verify that in DB the review segment was not changed + review_segment_in_db = ( + ReviewSegment.select(ReviewSegment.has_been_reviewed) + .where(ReviewSegment.id == id) + .get() + ) + assert review_segment_in_db.has_been_reviewed == False + + def test_post_reviews_viewed(self): + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id) + body = {"ids": [id]} + response = client.post("/reviews/viewed", json=body) + assert response.status_code == 200 + response = response.json() + assert response["success"] == True + assert response["message"] == "Reviewed multiple items" + # Verify that in DB the review segment was changed + review_segment_in_db = ( + ReviewSegment.select(ReviewSegment.has_been_reviewed) + .where(ReviewSegment.id == id) + .get() + ) + assert review_segment_in_db.has_been_reviewed == True + + #################################################################################################################### + ################################### POST reviews/delete Endpoint ################################################ + #################################################################################################################### + def test_post_reviews_delete_no_body(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + response = client.post("/reviews/delete") + # Missing ids + assert response.status_code == 422 + + def test_post_reviews_delete_no_body_ids(self): + with TestClient(self.app) as client: + super().insert_mock_review_segment("123456.random") + body = {"ids": [""]} + response = client.post("/reviews/delete", json=body) + # Missing ids + assert response.status_code == 422 + + def test_post_reviews_delete_non_existent_id(self): + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id) + body = {"ids": ["1"]} + response = client.post("/reviews/delete", json=body) + assert response.status_code == 200 + response_json = response.json() + assert response_json["success"] == True + assert response_json["message"] == "Delete reviews" + # Verify that in DB the review segment was not deleted + review_ids_in_db_after = self._get_reviews([id]) + assert len(review_ids_in_db_after) == 1 + assert review_ids_in_db_after[0].id == id + + def test_post_reviews_delete(self): + with TestClient(self.app) as client: + id = "123456.random" + super().insert_mock_review_segment(id) + body = {"ids": [id]} + response = client.post("/reviews/delete", json=body) + assert response.status_code == 200 + response_json = response.json() + assert response_json["success"] == True + assert response_json["message"] == "Delete reviews" + # Verify that in DB the review segment was deleted + review_ids_in_db_after = self._get_reviews([id]) + assert len(review_ids_in_db_after) == 0 + + def test_post_reviews_delete_many(self): + with TestClient(self.app) as client: + ids = ["123456.random", "654321.random"] + for id in ids: + super().insert_mock_review_segment(id) + super().insert_mock_recording(id) + + review_ids_in_db_before = self._get_reviews(ids) + recordings_ids_in_db_before = self._get_recordings(ids) + assert len(review_ids_in_db_before) == 2 + assert len(recordings_ids_in_db_before) == 2 + + body = {"ids": ids} + response = client.post("/reviews/delete", json=body) + assert response.status_code == 200 + response_json = response.json() + assert response_json["success"] == True + assert response_json["message"] == "Delete reviews" + + # Verify that in DB all review segments and recordings that were passed were deleted + review_ids_in_db_after = self._get_reviews(ids) + recording_ids_in_db_after = self._get_recordings(ids) + assert len(review_ids_in_db_after) == 0 + assert len(recording_ids_in_db_after) == 0 From c0ba98e26fa5fc103a84235d870a9ee68a4fa4a1 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 4 Dec 2024 09:54:10 -0600 Subject: [PATCH 203/479] Explore sorting (#15342) * backend * add type and params * radio group in ui * ensure search_type is cleared on reset --- frigate/api/event.py | 22 +- .../components/filter/SearchFilterGroup.tsx | 206 +++++++++++++++++- web/src/components/input/InputWithTags.tsx | 4 + .../overlay/dialog/SearchFilterDialog.tsx | 2 +- web/src/pages/Explore.tsx | 11 +- web/src/types/search.ts | 11 + 6 files changed, 241 insertions(+), 15 deletions(-) diff --git a/frigate/api/event.py b/frigate/api/event.py index fafa28272..dc98d094e 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -248,6 +248,8 @@ def events(params: EventsQueryParams = Depends()): order_by = Event.start_time.asc() elif sort == "date_desc": order_by = Event.start_time.desc() + else: + order_by = Event.start_time.desc() else: order_by = Event.start_time.desc() @@ -582,19 +584,17 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends()) processed_events.append(processed_event) - # Sort by search distance if search_results are available, otherwise by start_time as default - if search_results: + if (sort is None or sort == "relevance") and search_results: processed_events.sort(key=lambda x: x.get("search_distance", float("inf"))) + elif min_score is not None and max_score is not None and sort == "score_asc": + processed_events.sort(key=lambda x: x["score"]) + elif min_score is not None and max_score is not None and sort == "score_desc": + processed_events.sort(key=lambda x: x["score"], reverse=True) + elif sort == "date_asc": + processed_events.sort(key=lambda x: x["start_time"]) else: - if sort == "score_asc": - processed_events.sort(key=lambda x: x["score"]) - elif sort == "score_desc": - processed_events.sort(key=lambda x: x["score"], reverse=True) - elif sort == "date_asc": - processed_events.sort(key=lambda x: x["start_time"]) - else: - # "date_desc" default - processed_events.sort(key=lambda x: x["start_time"], reverse=True) + # "date_desc" default + processed_events.sort(key=lambda x: x["start_time"], reverse=True) # Limit the number of events returned processed_events = processed_events[:limit] diff --git a/web/src/components/filter/SearchFilterGroup.tsx b/web/src/components/filter/SearchFilterGroup.tsx index 5f3755e15..e8599895d 100644 --- a/web/src/components/filter/SearchFilterGroup.tsx +++ b/web/src/components/filter/SearchFilterGroup.tsx @@ -15,13 +15,15 @@ import { SearchFilter, SearchFilters, SearchSource, + SearchSortType, } from "@/types/search"; import { DateRange } from "react-day-picker"; import { cn } from "@/lib/utils"; -import { MdLabel } from "react-icons/md"; +import { MdLabel, MdSort } from "react-icons/md"; import PlatformAwareDialog from "../overlay/dialog/PlatformAwareDialog"; import SearchFilterDialog from "../overlay/dialog/SearchFilterDialog"; import { CalendarRangeFilterButton } from "./CalendarFilterButton"; +import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group"; type SearchFilterGroupProps = { className: string; @@ -107,6 +109,25 @@ export default function SearchFilterGroup({ [config, allLabels, allZones], ); + const availableSortTypes = useMemo(() => { + const sortTypes = ["date_asc", "date_desc"]; + if (filter?.min_score || filter?.max_score) { + sortTypes.push("score_desc", "score_asc"); + } + if (filter?.event_id || filter?.query) { + sortTypes.push("relevance"); + } + return sortTypes as SearchSortType[]; + }, [filter]); + + const defaultSortType = useMemo(() => { + if (filter?.query || filter?.event_id) { + return "relevance"; + } else { + return "date_desc"; + } + }, [filter]); + const groups = useMemo(() => { if (!config) { return []; @@ -179,6 +200,16 @@ export default function SearchFilterGroup({ filterValues={filterValues} onUpdateFilter={onUpdateFilter} /> + {filters.includes("sort") && Object.keys(filter ?? {}).length > 0 && ( + { + onUpdateFilter({ ...filter, sort: newSort }); + }} + /> + )}
); } @@ -362,3 +393,176 @@ export function GeneralFilterContent({ ); } + +type SortTypeButtonProps = { + availableSortTypes: SearchSortType[]; + defaultSortType: SearchSortType; + selectedSortType: SearchSortType | undefined; + updateSortType: (sortType: SearchSortType | undefined) => void; +}; +function SortTypeButton({ + availableSortTypes, + defaultSortType, + selectedSortType, + updateSortType, +}: SortTypeButtonProps) { + const [open, setOpen] = useState(false); + const [currentSortType, setCurrentSortType] = useState< + SearchSortType | undefined + >(selectedSortType as SearchSortType); + + // ui + + useEffect(() => { + setCurrentSortType(selectedSortType); + // only refresh when state changes + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [selectedSortType]); + + const trigger = ( + + ); + const content = ( + setOpen(false)} + /> + ); + + return ( + { + if (!open) { + setCurrentSortType(selectedSortType); + } + + setOpen(open); + }} + /> + ); +} + +type SortTypeContentProps = { + availableSortTypes: SearchSortType[]; + defaultSortType: SearchSortType; + selectedSortType: SearchSortType | undefined; + currentSortType: SearchSortType | undefined; + updateSortType: (sort_type: SearchSortType | undefined) => void; + setCurrentSortType: (sort_type: SearchSortType | undefined) => void; + onClose: () => void; +}; +export function SortTypeContent({ + availableSortTypes, + defaultSortType, + selectedSortType, + currentSortType, + updateSortType, + setCurrentSortType, + onClose, +}: SortTypeContentProps) { + const sortLabels = { + date_asc: "Date (Ascending)", + date_desc: "Date (Descending)", + score_asc: "Object Score (Ascending)", + score_desc: "Object Score (Descending)", + relevance: "Relevance", + }; + + return ( + <> +
+
+ + setCurrentSortType(value as SearchSortType) + } + className="w-full space-y-1" + > + {availableSortTypes.map((value) => ( +
+ + +
+ ))} + +
+
+ +
+ + +
+ + ); +} diff --git a/web/src/components/input/InputWithTags.tsx b/web/src/components/input/InputWithTags.tsx index 8f60bb73e..d5904b2a5 100644 --- a/web/src/components/input/InputWithTags.tsx +++ b/web/src/components/input/InputWithTags.tsx @@ -18,6 +18,7 @@ import { FilterType, SavedSearchQuery, SearchFilter, + SearchSortType, SearchSource, } from "@/types/search"; import useSuggestions from "@/hooks/use-suggestions"; @@ -323,6 +324,9 @@ export default function InputWithTags({ case "event_id": newFilters.event_id = value; break; + case "sort": + newFilters.sort = value as SearchSortType; + break; default: // Handle array types (cameras, labels, subLabels, zones) if (!newFilters[type]) newFilters[type] = []; diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx index 845c3bc1a..65109591b 100644 --- a/web/src/components/overlay/dialog/SearchFilterDialog.tsx +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -175,7 +175,7 @@ export default function SearchFilterDialog({ time_range: undefined, zones: undefined, sub_labels: undefined, - search_type: ["thumbnail", "description"], + search_type: undefined, min_score: undefined, max_score: undefined, has_snapshot: undefined, diff --git a/web/src/pages/Explore.tsx b/web/src/pages/Explore.tsx index 2bf2bb022..ce2560868 100644 --- a/web/src/pages/Explore.tsx +++ b/web/src/pages/Explore.tsx @@ -116,6 +116,7 @@ export default function Explore() { is_submitted: searchSearchParams["is_submitted"], has_clip: searchSearchParams["has_clip"], event_id: searchSearchParams["event_id"], + sort: searchSearchParams["sort"], limit: Object.keys(searchSearchParams).length == 0 ? API_LIMIT : undefined, timezone, @@ -148,6 +149,7 @@ export default function Explore() { is_submitted: searchSearchParams["is_submitted"], has_clip: searchSearchParams["has_clip"], event_id: searchSearchParams["event_id"], + sort: searchSearchParams["sort"], timezone, include_thumbnails: 0, }, @@ -165,12 +167,17 @@ export default function Explore() { const [url, params] = searchQuery; - // If it's not the first page, use the last item's start_time as the 'before' parameter + const isAscending = params.sort?.includes("date_asc"); + if (pageIndex > 0 && previousPageData) { const lastDate = previousPageData[previousPageData.length - 1].start_time; return [ url, - { ...params, before: lastDate.toString(), limit: API_LIMIT }, + { + ...params, + [isAscending ? "after" : "before"]: lastDate.toString(), + limit: API_LIMIT, + }, ]; } diff --git a/web/src/types/search.ts b/web/src/types/search.ts index fafedad10..1d8de1611 100644 --- a/web/src/types/search.ts +++ b/web/src/types/search.ts @@ -6,6 +6,7 @@ const SEARCH_FILTERS = [ "zone", "sub", "source", + "sort", ] as const; export type SearchFilters = (typeof SEARCH_FILTERS)[number]; export const DEFAULT_SEARCH_FILTERS: SearchFilters[] = [ @@ -16,10 +17,18 @@ export const DEFAULT_SEARCH_FILTERS: SearchFilters[] = [ "zone", "sub", "source", + "sort", ]; export type SearchSource = "similarity" | "thumbnail" | "description"; +export type SearchSortType = + | "date_asc" + | "date_desc" + | "score_asc" + | "score_desc" + | "relevance"; + export type SearchResult = { id: string; camera: string; @@ -65,6 +74,7 @@ export type SearchFilter = { time_range?: string; search_type?: SearchSource[]; event_id?: string; + sort?: SearchSortType; }; export const DEFAULT_TIME_RANGE_AFTER = "00:00"; @@ -86,6 +96,7 @@ export type SearchQueryParams = { query?: string; page?: number; time_range?: string; + sort?: SearchSortType; }; export type SearchQuery = [string, SearchQueryParams] | null; From 32322b23b24f0ed5ea8d9c284d4e494a3c773019 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 4 Dec 2024 16:43:10 -0600 Subject: [PATCH 204/479] Update nvidia docs to reflect preset (#15347) --- .../configuration/hardware_acceleration.md | 21 ++----------------- 1 file changed, 2 insertions(+), 19 deletions(-) diff --git a/docs/docs/configuration/hardware_acceleration.md b/docs/docs/configuration/hardware_acceleration.md index c6acdea14..e70e57497 100644 --- a/docs/docs/configuration/hardware_acceleration.md +++ b/docs/docs/configuration/hardware_acceleration.md @@ -231,28 +231,11 @@ docker run -d \ ### Setup Decoder -The decoder you need to pass in the `hwaccel_args` will depend on the input video. - -A list of supported codecs (you can use `ffmpeg -decoders | grep cuvid` in the container to get the ones your card supports) - -``` - V..... h263_cuvid Nvidia CUVID H263 decoder (codec h263) - V..... h264_cuvid Nvidia CUVID H264 decoder (codec h264) - V..... hevc_cuvid Nvidia CUVID HEVC decoder (codec hevc) - V..... mjpeg_cuvid Nvidia CUVID MJPEG decoder (codec mjpeg) - V..... mpeg1_cuvid Nvidia CUVID MPEG1VIDEO decoder (codec mpeg1video) - V..... mpeg2_cuvid Nvidia CUVID MPEG2VIDEO decoder (codec mpeg2video) - V..... mpeg4_cuvid Nvidia CUVID MPEG4 decoder (codec mpeg4) - V..... vc1_cuvid Nvidia CUVID VC1 decoder (codec vc1) - V..... vp8_cuvid Nvidia CUVID VP8 decoder (codec vp8) - V..... vp9_cuvid Nvidia CUVID VP9 decoder (codec vp9) -``` - -For example, for H264 video, you'll select `preset-nvidia-h264`. +Using `preset-nvidia` ffmpeg will automatically select the necessary profile for the incoming video, and will log an error if the profile is not supported by your GPU. ```yaml ffmpeg: - hwaccel_args: preset-nvidia-h264 + hwaccel_args: preset-nvidia ``` If everything is working correctly, you should see a significant improvement in performance. From 47d495fc012c0d897db1f6aa5c54e40225d795ed Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 4 Dec 2024 16:54:57 -0600 Subject: [PATCH 205/479] Make note of go2rtc encoded URLs (#15348) * Make note of go2rtc encoded URLs * clarify --- docs/docs/configuration/restream.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/docs/configuration/restream.md b/docs/docs/configuration/restream.md index 1ad09cc8d..0db4ded80 100644 --- a/docs/docs/configuration/restream.md +++ b/docs/docs/configuration/restream.md @@ -132,6 +132,28 @@ cameras: - detect ``` +## Handling Complex Passwords + +go2rtc expects URL-encoded passwords in the config, [urlencoder.org](https://urlencoder.org) can be used for this purpose. + +For example: + +```yaml +go2rtc: + streams: + my_camera: rtsp://username:$@foo%@192.168.1.100 +``` + +becomes + +```yaml +go2rtc: + streams: + my_camera: rtsp://username:$%40foo%25@192.168.1.100 +``` + +See [this comment(https://github.com/AlexxIT/go2rtc/issues/1217#issuecomment-2242296489) for more information. + ## Advanced Restream Configurations The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.2#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below: From d3b631a9522cab9d357ac098b902ca272ee850c3 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 6 Dec 2024 08:04:02 -0600 Subject: [PATCH 206/479] Api improvements (#15327) * Organize api files * Add more API definitions for events * Add export select by ID * Typing fixes * Update openapi spec * Change type * Fix test * Fix message * Fix tests --- docs/static/frigate-api.yaml | 1199 ++++++++++------- frigate/api/app.py | 4 +- frigate/api/auth.py | 2 +- .../defs/{ => query}/app_query_parameters.py | 0 .../{ => query}/events_query_parameters.py | 0 .../{ => query}/media_query_parameters.py | 0 .../regenerate_query_parameters.py | 0 .../{ => query}/review_query_parameters.py | 0 frigate/api/defs/{ => request}/app_body.py | 0 frigate/api/defs/{ => request}/events_body.py | 0 frigate/api/defs/{ => request}/review_body.py | 0 frigate/api/defs/response/event_response.py | 40 + .../defs/{ => response}/generic_response.py | 0 .../review_response.py} | 0 frigate/api/event.py | 62 +- frigate/api/export.py | 12 + frigate/api/media.py | 2 +- frigate/api/review.py | 10 +- frigate/test/http_api/test_http_review.py | 6 +- frigate/test/test_http.py | 2 +- 20 files changed, 813 insertions(+), 526 deletions(-) rename frigate/api/defs/{ => query}/app_query_parameters.py (100%) rename frigate/api/defs/{ => query}/events_query_parameters.py (100%) rename frigate/api/defs/{ => query}/media_query_parameters.py (100%) rename frigate/api/defs/{ => query}/regenerate_query_parameters.py (100%) rename frigate/api/defs/{ => query}/review_query_parameters.py (100%) rename frigate/api/defs/{ => request}/app_body.py (100%) rename frigate/api/defs/{ => request}/events_body.py (100%) rename frigate/api/defs/{ => request}/review_body.py (100%) create mode 100644 frigate/api/defs/response/event_response.py rename frigate/api/defs/{ => response}/generic_response.py (100%) rename frigate/api/defs/{review_responses.py => response/review_response.py} (100%) diff --git a/docs/static/frigate-api.yaml b/docs/static/frigate-api.yaml index 5e4ecdbdc..e05330a9d 100644 --- a/docs/static/frigate-api.yaml +++ b/docs/static/frigate-api.yaml @@ -2,7 +2,7 @@ openapi: 3.1.0 info: # To avoid the introduction page we set the title to empty string # https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/blob/4e771d309f6defe395449b26cc3c65814d72cbcc/packages/docusaurus-plugin-openapi-docs/src/openapi/openapi.ts#L92-L129 - title: '' + title: "" version: 0.1.0 servers: @@ -17,11 +17,11 @@ paths: summary: Auth operationId: auth_auth_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /profile: get: tags: @@ -29,11 +29,11 @@ paths: summary: Profile operationId: profile_profile_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /logout: get: tags: @@ -41,11 +41,11 @@ paths: summary: Logout operationId: logout_logout_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /login: post: tags: @@ -57,19 +57,19 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/AppPostLoginBody' + $ref: "#/components/schemas/AppPostLoginBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /users: get: tags: @@ -77,11 +77,11 @@ paths: summary: Get Users operationId: get_users_users_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} post: tags: - Auth @@ -92,20 +92,20 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/AppPostUsersBody' + $ref: "#/components/schemas/AppPostUsersBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /users/{username}: + $ref: "#/components/schemas/HTTPValidationError" + "/users/{username}": delete: tags: - Auth @@ -119,18 +119,18 @@ paths: type: string title: Username responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /users/{username}/password: + $ref: "#/components/schemas/HTTPValidationError" + "/users/{username}/password": put: tags: - Auth @@ -148,19 +148,19 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/AppPutPasswordBody' + $ref: "#/components/schemas/AppPutPasswordBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /review: get: tags: @@ -207,7 +207,7 @@ paths: required: false schema: allOf: - - $ref: '#/components/schemas/SeverityEnum' + - $ref: "#/components/schemas/SeverityEnum" title: Severity - name: before in: query @@ -222,21 +222,21 @@ paths: type: number title: After responses: - '200': + "200": description: Successful Response content: application/json: schema: type: array items: - $ref: '#/components/schemas/ReviewSegmentResponse' + $ref: "#/components/schemas/ReviewSegmentResponse" title: Response Review Review Get - '422': + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /review/summary: get: tags: @@ -273,18 +273,18 @@ paths: default: utc title: Timezone responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/ReviewSummaryResponse' - '422': + $ref: "#/components/schemas/ReviewSummaryResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /reviews/viewed: post: tags: @@ -296,20 +296,20 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/ReviewModifyMultipleBody' + $ref: "#/components/schemas/ReviewModifyMultipleBody" responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/GenericResponse' - '422': + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /reviews/delete: post: tags: @@ -321,20 +321,20 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/ReviewModifyMultipleBody' + $ref: "#/components/schemas/ReviewModifyMultipleBody" responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/GenericResponse' - '422': + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /review/activity/motion: get: tags: @@ -370,22 +370,22 @@ paths: default: 30 title: Scale responses: - '200': + "200": description: Successful Response content: application/json: schema: type: array items: - $ref: '#/components/schemas/ReviewActivityMotionResponse' + $ref: "#/components/schemas/ReviewActivityMotionResponse" title: Response Motion Activity Review Activity Motion Get - '422': + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /review/event/{event_id}: + $ref: "#/components/schemas/HTTPValidationError" + "/review/event/{event_id}": get: tags: - Review @@ -399,19 +399,19 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/ReviewSegmentResponse' - '422': + $ref: "#/components/schemas/ReviewSegmentResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /review/{review_id}: + $ref: "#/components/schemas/HTTPValidationError" + "/review/{review_id}": get: tags: - Review @@ -425,19 +425,19 @@ paths: type: string title: Review Id responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/ReviewSegmentResponse' - '422': + $ref: "#/components/schemas/ReviewSegmentResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /review/{review_id}/viewed: + $ref: "#/components/schemas/HTTPValidationError" + "/review/{review_id}/viewed": delete: tags: - Review @@ -451,18 +451,18 @@ paths: type: string title: Review Id responses: - '200': + "200": description: Successful Response content: application/json: schema: - $ref: '#/components/schemas/GenericResponse' - '422': + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /: get: tags: @@ -470,7 +470,7 @@ paths: summary: Is Healthy operationId: is_healthy__get responses: - '200': + "200": description: Successful Response content: text/plain: @@ -483,11 +483,11 @@ paths: summary: Config Schema operationId: config_schema_config_schema_json_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /go2rtc/streams: get: tags: @@ -495,12 +495,12 @@ paths: summary: Go2Rtc Streams operationId: go2rtc_streams_go2rtc_streams_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - /go2rtc/streams/{camera_name}: + schema: {} + "/go2rtc/streams/{camera_name}": get: tags: - App @@ -514,17 +514,17 @@ paths: type: string title: Camera Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /version: get: tags: @@ -532,7 +532,7 @@ paths: summary: Version operationId: version_version_get responses: - '200': + "200": description: Successful Response content: text/plain: @@ -545,11 +545,11 @@ paths: summary: Stats operationId: stats_stats_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /stats/history: get: tags: @@ -564,17 +564,17 @@ paths: type: string title: Keys responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /config: get: tags: @@ -582,11 +582,11 @@ paths: summary: Config operationId: config_config_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /config/raw: get: tags: @@ -594,11 +594,11 @@ paths: summary: Config Raw operationId: config_raw_config_raw_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /config/save: post: tags: @@ -619,17 +619,17 @@ paths: schema: title: Body responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /config/set: put: tags: @@ -641,19 +641,19 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/AppConfigSetBody' + $ref: "#/components/schemas/AppConfigSetBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /ffprobe: get: tags: @@ -666,20 +666,20 @@ paths: required: false schema: type: string - default: '' + default: "" title: Paths responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /vainfo: get: tags: @@ -687,11 +687,11 @@ paths: summary: Vainfo operationId: vainfo_vainfo_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /nvinfo: get: tags: @@ -699,12 +699,12 @@ paths: summary: Nvinfo operationId: nvinfo_nvinfo_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - /logs/{service}: + schema: {} + "/logs/{service}": get: tags: - App @@ -729,7 +729,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Download - name: start in: query @@ -737,7 +737,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 0 title: Start - name: end @@ -746,20 +746,20 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: End responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /restart: post: tags: @@ -767,11 +767,11 @@ paths: summary: Restart operationId: restart_restart_post responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /labels: get: tags: @@ -784,20 +784,20 @@ paths: required: false schema: type: string - default: '' + default: "" title: Camera responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /sub_labels: get: tags: @@ -811,20 +811,20 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Split Joined responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /timeline: get: tags: @@ -852,20 +852,20 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Source Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /timeline/hourly: get: tags: @@ -880,7 +880,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Cameras - name: labels @@ -889,7 +889,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Labels - name: after @@ -898,7 +898,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: After - name: before in: query @@ -906,7 +906,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Before - name: limit in: query @@ -914,7 +914,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 200 title: Limit - name: timezone @@ -923,22 +923,22 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: utc title: Timezone responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /preview/{camera_name}/start/{start_ts}/end/{end_ts}: + $ref: "#/components/schemas/HTTPValidationError" + "/preview/{camera_name}/start/{start_ts}/end/{end_ts}": get: tags: - Preview @@ -965,18 +965,18 @@ paths: type: number title: End Ts responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /preview/{year_month}/{day}/{hour}/{camera_name}/{tz_name}: + $ref: "#/components/schemas/HTTPValidationError" + "/preview/{year_month}/{day}/{hour}/{camera_name}/{tz_name}": get: tags: - Preview @@ -1016,18 +1016,18 @@ paths: type: string title: Tz Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /preview/{camera_name}/start/{start_ts}/end/{end_ts}/frames: + $ref: "#/components/schemas/HTTPValidationError" + "/preview/{camera_name}/start/{start_ts}/end/{end_ts}/frames": get: tags: - Preview @@ -1055,17 +1055,17 @@ paths: type: number title: End Ts responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /notifications/pubkey: get: tags: @@ -1073,11 +1073,11 @@ paths: summary: Get Vapid Pub Key operationId: get_vapid_pub_key_notifications_pubkey_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } + schema: {} /notifications/register: post: tags: @@ -1091,17 +1091,17 @@ paths: type: object title: Body responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /exports: get: tags: @@ -1109,12 +1109,12 @@ paths: summary: Get Exports operationId: get_exports_exports_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - /export/{camera_name}/start/{start_time}/end/{end_time}: + schema: {} + "/export/{camera_name}/start/{start_time}/end/{end_time}": post: tags: - Export @@ -1145,20 +1145,20 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/ExportRecordingsBody' + $ref: "#/components/schemas/ExportRecordingsBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /export/{event_id}/{new_name}: + $ref: "#/components/schemas/HTTPValidationError" + "/export/{event_id}/{new_name}": patch: tags: - Export @@ -1178,18 +1178,18 @@ paths: type: string title: New Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /export/{event_id}: + $ref: "#/components/schemas/HTTPValidationError" + "/export/{event_id}": delete: tags: - Export @@ -1203,17 +1203,42 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" + "/exports/{export_id}": + get: + tags: + - Export + summary: Get Export + operationId: get_export_exports__export_id__get + parameters: + - name: export_id + in: path + required: true + schema: + type: string + title: Export Id + responses: + "200": + description: Successful Response + content: + application/json: + schema: {} + "422": + description: Validation Error + content: + application/json: + schema: + $ref: "#/components/schemas/HTTPValidationError" /events: get: tags: @@ -1227,7 +1252,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Camera - name: cameras @@ -1236,7 +1261,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Cameras - name: label @@ -1245,7 +1270,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Label - name: labels @@ -1254,7 +1279,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Labels - name: sub_label @@ -1263,7 +1288,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Sub Label - name: sub_labels @@ -1272,7 +1297,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Sub Labels - name: zone @@ -1281,7 +1306,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Zone - name: zones @@ -1290,7 +1315,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Zones - name: limit @@ -1299,7 +1324,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 100 title: Limit - name: after @@ -1308,7 +1333,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: After - name: before in: query @@ -1316,7 +1341,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Before - name: time_range in: query @@ -1324,8 +1349,8 @@ paths: schema: anyOf: - type: string - - type: 'null' - default: 00:00,24:00 + - type: "null" + default: "00:00,24:00" title: Time Range - name: has_clip in: query @@ -1333,7 +1358,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Has Clip - name: has_snapshot in: query @@ -1341,7 +1366,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Has Snapshot - name: in_progress in: query @@ -1349,7 +1374,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: In Progress - name: include_thumbnails in: query @@ -1357,7 +1382,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 1 title: Include Thumbnails - name: favorites @@ -1366,7 +1391,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Favorites - name: min_score in: query @@ -1374,7 +1399,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Min Score - name: max_score in: query @@ -1382,7 +1407,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Max Score - name: is_submitted in: query @@ -1390,7 +1415,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Is Submitted - name: min_length in: query @@ -1398,7 +1423,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Min Length - name: max_length in: query @@ -1406,7 +1431,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Max Length - name: event_id in: query @@ -1414,7 +1439,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Event Id - name: sort in: query @@ -1422,7 +1447,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Sort - name: timezone in: query @@ -1430,21 +1455,25 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: utc title: Timezone responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + type: array + items: + $ref: "#/components/schemas/EventResponse" + title: Response Events Events Get + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /events/explore: get: tags: @@ -1460,17 +1489,21 @@ paths: default: 10 title: Limit responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + type: array + items: + $ref: "#/components/schemas/EventResponse" + title: Response Events Explore Events Explore Get + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /event_ids: get: tags: @@ -1485,17 +1518,21 @@ paths: type: string title: Ids responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + type: array + items: + $ref: "#/components/schemas/EventResponse" + title: Response Event Ids Event Ids Get + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /events/search: get: tags: @@ -1509,7 +1546,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Query - name: event_id in: query @@ -1517,7 +1554,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Event Id - name: search_type in: query @@ -1525,7 +1562,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: thumbnail title: Search Type - name: include_thumbnails @@ -1534,7 +1571,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 1 title: Include Thumbnails - name: limit @@ -1543,7 +1580,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 50 title: Limit - name: cameras @@ -1552,7 +1589,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Cameras - name: labels @@ -1561,7 +1598,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Labels - name: zones @@ -1570,7 +1607,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: all title: Zones - name: after @@ -1579,7 +1616,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: After - name: before in: query @@ -1587,7 +1624,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Before - name: time_range in: query @@ -1595,8 +1632,8 @@ paths: schema: anyOf: - type: string - - type: 'null' - default: 00:00,24:00 + - type: "null" + default: "00:00,24:00" title: Time Range - name: has_clip in: query @@ -1604,7 +1641,7 @@ paths: schema: anyOf: - type: boolean - - type: 'null' + - type: "null" title: Has Clip - name: has_snapshot in: query @@ -1612,15 +1649,23 @@ paths: schema: anyOf: - type: boolean - - type: 'null' + - type: "null" title: Has Snapshot + - name: is_submitted + in: query + required: false + schema: + anyOf: + - type: boolean + - type: "null" + title: Is Submitted - name: timezone in: query required: false schema: anyOf: - type: string - - type: 'null' + - type: "null" default: utc title: Timezone - name: min_score @@ -1629,7 +1674,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Min Score - name: max_score in: query @@ -1637,7 +1682,7 @@ paths: schema: anyOf: - type: number - - type: 'null' + - type: "null" title: Max Score - name: sort in: query @@ -1645,20 +1690,20 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" title: Sort responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /events/summary: get: tags: @@ -1672,7 +1717,7 @@ paths: schema: anyOf: - type: string - - type: 'null' + - type: "null" default: utc title: Timezone - name: has_clip @@ -1681,7 +1726,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Has Clip - name: has_snapshot in: query @@ -1689,21 +1734,21 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Has Snapshot responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}": get: tags: - Events @@ -1717,17 +1762,18 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/EventResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" delete: tags: - Events @@ -1741,18 +1787,19 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/retain: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/retain": post: tags: - Events @@ -1766,17 +1813,18 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" delete: tags: - Events @@ -1790,18 +1838,19 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/plus: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/plus": post: tags: - Events @@ -1819,21 +1868,22 @@ paths: application/json: schema: allOf: - - $ref: '#/components/schemas/SubmitPlusBody' + - $ref: "#/components/schemas/SubmitPlusBody" title: Body responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/EventUploadPlusResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/false_positive: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/false_positive": put: tags: - Events @@ -1847,18 +1897,19 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/EventUploadPlusResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/sub_label: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/sub_label": post: tags: - Events @@ -1876,20 +1927,21 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/EventsSubLabelBody' + $ref: "#/components/schemas/EventsSubLabelBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/description: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/description": post: tags: - Events @@ -1907,20 +1959,21 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/EventsDescriptionBody' + $ref: "#/components/schemas/EventsDescriptionBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/description/regenerate: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/description/regenerate": put: tags: - Events @@ -1938,23 +1991,49 @@ paths: required: false schema: anyOf: - - $ref: '#/components/schemas/RegenerateDescriptionEnum' - - type: 'null' + - $ref: "#/components/schemas/RegenerateDescriptionEnum" + - type: "null" default: thumbnails title: Source responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{camera_name}/{label}/create: + $ref: "#/components/schemas/HTTPValidationError" + /events/: + delete: + tags: + - Events + summary: Delete Events + operationId: delete_events_events__delete + requestBody: + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/EventsDeleteBody" + responses: + "200": + description: Successful Response + content: + application/json: + schema: + $ref: "#/components/schemas/EventMultiDeleteResponse" + "422": + description: Validation Error + content: + application/json: + schema: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{camera_name}/{label}/create": post: tags: - Events @@ -1978,27 +2057,28 @@ paths: application/json: schema: allOf: - - $ref: '#/components/schemas/EventsCreateBody' + - $ref: "#/components/schemas/EventsCreateBody" default: source_type: api score: 0 duration: 30 include_recording: true - draw: { } + draw: {} title: Body responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/EventCreateResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/end: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/end": put: tags: - Events @@ -2016,20 +2096,21 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/EventsEndBody' + $ref: "#/components/schemas/EventsEndBody" responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: + $ref: "#/components/schemas/GenericResponse" + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}": get: tags: - Media @@ -2062,7 +2143,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Bbox - name: timestamp in: query @@ -2070,7 +2151,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Timestamp - name: zones in: query @@ -2078,7 +2159,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Zones - name: mask in: query @@ -2086,7 +2167,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Mask - name: motion in: query @@ -2094,7 +2175,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Motion - name: regions in: query @@ -2102,21 +2183,21 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Regions responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/ptz/info: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/ptz/info": get: tags: - Media @@ -2130,18 +2211,18 @@ paths: type: string title: Camera Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/latest.{extension}: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/latest.{extension}": get: tags: - Media @@ -2158,14 +2239,14 @@ paths: in: path required: true schema: - $ref: '#/components/schemas/Extension' + $ref: "#/components/schemas/Extension" - name: bbox in: query required: false schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Bbox - name: timestamp in: query @@ -2173,7 +2254,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Timestamp - name: zones in: query @@ -2181,7 +2262,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Zones - name: mask in: query @@ -2189,7 +2270,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Mask - name: motion in: query @@ -2197,7 +2278,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Motion - name: regions in: query @@ -2205,7 +2286,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Regions - name: quality in: query @@ -2213,7 +2294,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 70 title: Quality - name: height @@ -2222,21 +2303,21 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Height responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/recordings/{frame_time}/snapshot.{format}: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/recordings/{frame_time}/snapshot.{format}": get: tags: - Media @@ -2272,18 +2353,18 @@ paths: type: integer title: Height responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/plus/{frame_time}: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/plus/{frame_time}": post: tags: - Media @@ -2303,17 +2384,17 @@ paths: type: string title: Frame Time responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" /recordings/storage: get: tags: @@ -2321,12 +2402,12 @@ paths: summary: Get Recordings Storage Usage operationId: get_recordings_storage_usage_recordings_storage_get responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - /{camera_name}/recordings/summary: + schema: {} + "/{camera_name}/recordings/summary": get: tags: - Media @@ -2348,18 +2429,18 @@ paths: default: utc title: Timezone responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/recordings: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/recordings": get: tags: - Media @@ -2380,28 +2461,28 @@ paths: required: false schema: type: number - default: 1731275308.238304 + default: 1733228876.15567 title: After - name: before in: query required: false schema: type: number - default: 1731278908.238313 + default: 1733232476.15567 title: Before responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/start/{start_ts}/end/{end_ts}/clip.mp4: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/start/{start_ts}/end/{end_ts}/clip.mp4": get: tags: - Media @@ -2427,18 +2508,18 @@ paths: type: number title: End Ts responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /vod/{camera_name}/start/{start_ts}/end/{end_ts}: + $ref: "#/components/schemas/HTTPValidationError" + "/vod/{camera_name}/start/{start_ts}/end/{end_ts}": get: tags: - Media @@ -2464,18 +2545,18 @@ paths: type: number title: End Ts responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /vod/{year_month}/{day}/{hour}/{camera_name}: + $ref: "#/components/schemas/HTTPValidationError" + "/vod/{year_month}/{day}/{hour}/{camera_name}": get: tags: - Media @@ -2508,18 +2589,18 @@ paths: type: string title: Camera Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /vod/{year_month}/{day}/{hour}/{camera_name}/{tz_name}: + $ref: "#/components/schemas/HTTPValidationError" + "/vod/{year_month}/{day}/{hour}/{camera_name}/{tz_name}": get: tags: - Media @@ -2557,18 +2638,18 @@ paths: type: string title: Tz Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /vod/event/{event_id}: + $ref: "#/components/schemas/HTTPValidationError" + "/vod/event/{event_id}": get: tags: - Media @@ -2582,18 +2663,18 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/snapshot.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/snapshot.jpg": get: tags: - Media @@ -2612,7 +2693,7 @@ paths: schema: anyOf: - type: boolean - - type: 'null' + - type: "null" default: false title: Download - name: timestamp @@ -2621,7 +2702,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Timestamp - name: bbox in: query @@ -2629,7 +2710,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Bbox - name: crop in: query @@ -2637,7 +2718,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Crop - name: height in: query @@ -2645,7 +2726,7 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" title: Height - name: quality in: query @@ -2653,22 +2734,22 @@ paths: schema: anyOf: - type: integer - - type: 'null' + - type: "null" default: 70 title: Quality responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/thumbnail.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/thumbnail.jpg": get: tags: - Media @@ -2701,18 +2782,18 @@ paths: default: ios title: Format responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/grid.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/grid.jpg": get: tags: - Media @@ -2740,18 +2821,18 @@ paths: default: 0.5 title: Font Scale responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/snapshot-clean.png: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/snapshot-clean.png": get: tags: - Media @@ -2772,18 +2853,18 @@ paths: default: false title: Download responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/clip.mp4: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/clip.mp4": get: tags: - Media @@ -2797,18 +2878,18 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /events/{event_id}/preview.gif: + $ref: "#/components/schemas/HTTPValidationError" + "/events/{event_id}/preview.gif": get: tags: - Media @@ -2822,18 +2903,18 @@ paths: type: string title: Event Id responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/start/{start_ts}/end/{end_ts}/preview.gif: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/start/{start_ts}/end/{end_ts}/preview.gif": get: tags: - Media @@ -2868,18 +2949,18 @@ paths: title: Max Cache Age description: Max cache age in seconds. Default 30 days in seconds. responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/start/{start_ts}/end/{end_ts}/preview.mp4: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/start/{start_ts}/end/{end_ts}/preview.mp4": get: tags: - Media @@ -2914,18 +2995,18 @@ paths: title: Max Cache Age description: Max cache age in seconds. Default 7 days in seconds. responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /review/{event_id}/preview: + $ref: "#/components/schemas/HTTPValidationError" + "/review/{event_id}/preview": get: tags: - Media @@ -2949,18 +3030,18 @@ paths: default: gif title: Format responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /preview/{file_name}/thumbnail.webp: + $ref: "#/components/schemas/HTTPValidationError" + "/preview/{file_name}/thumbnail.webp": get: tags: - Media @@ -2975,18 +3056,18 @@ paths: type: string title: File Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /preview/{file_name}/thumbnail.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/preview/{file_name}/thumbnail.jpg": get: tags: - Media @@ -3001,18 +3082,18 @@ paths: type: string title: File Name responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/{label}/thumbnail.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/{label}/thumbnail.jpg": get: tags: - Media @@ -3032,18 +3113,18 @@ paths: type: string title: Label responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/{label}/best.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/{label}/best.jpg": get: tags: - Media @@ -3063,18 +3144,18 @@ paths: type: string title: Label responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/{label}/clip.mp4: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/{label}/clip.mp4": get: tags: - Media @@ -3094,18 +3175,18 @@ paths: type: string title: Label responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' - /{camera_name}/{label}/snapshot.jpg: + $ref: "#/components/schemas/HTTPValidationError" + "/{camera_name}/{label}/snapshot.jpg": get: tags: - Media @@ -3128,17 +3209,17 @@ paths: type: string title: Label responses: - '200': + "200": description: Successful Response content: application/json: - schema: { } - '422': + schema: {} + "422": description: Validation Error content: application/json: schema: - $ref: '#/components/schemas/HTTPValidationError' + $ref: "#/components/schemas/HTTPValidationError" components: schemas: AppConfigSetBody: @@ -3210,51 +3291,199 @@ components: - total_alert - total_detection title: DayReview + EventCreateResponse: + properties: + success: + type: boolean + title: Success + message: + type: string + title: Message + event_id: + type: string + title: Event Id + type: object + required: + - success + - message + - event_id + title: EventCreateResponse + EventMultiDeleteResponse: + properties: + success: + type: boolean + title: Success + deleted_events: + items: + type: string + type: array + title: Deleted Events + not_found_events: + items: + type: string + type: array + title: Not Found Events + type: object + required: + - success + - deleted_events + - not_found_events + title: EventMultiDeleteResponse + EventResponse: + properties: + id: + type: string + title: Id + label: + type: string + title: Label + sub_label: + anyOf: + - type: string + - type: "null" + title: Sub Label + camera: + type: string + title: Camera + start_time: + type: number + title: Start Time + end_time: + anyOf: + - type: number + - type: "null" + title: End Time + false_positive: + type: boolean + title: False Positive + zones: + items: + type: string + type: array + title: Zones + thumbnail: + type: string + title: Thumbnail + has_clip: + type: boolean + title: Has Clip + has_snapshot: + type: boolean + title: Has Snapshot + retain_indefinitely: + type: boolean + title: Retain Indefinitely + plus_id: + anyOf: + - type: string + - type: "null" + title: Plus Id + model_hash: + anyOf: + - type: string + - type: "null" + title: Model Hash + detector_type: + anyOf: + - type: string + - type: "null" + title: Detector Type + model_type: + anyOf: + - type: string + - type: "null" + title: Model Type + data: + title: Data + type: object + required: + - id + - label + - sub_label + - camera + - start_time + - end_time + - false_positive + - zones + - thumbnail + - has_clip + - has_snapshot + - retain_indefinitely + - plus_id + - model_hash + - detector_type + - model_type + - data + title: EventResponse + EventUploadPlusResponse: + properties: + success: + type: boolean + title: Success + plus_id: + type: string + title: Plus Id + type: object + required: + - success + - plus_id + title: EventUploadPlusResponse EventsCreateBody: properties: source_type: anyOf: - type: string - - type: 'null' + - type: "null" title: Source Type default: api sub_label: anyOf: - type: string - - type: 'null' + - type: "null" title: Sub Label score: anyOf: - type: number - - type: 'null' + - type: "null" title: Score default: 0 duration: anyOf: - type: integer - - type: 'null' + - type: "null" title: Duration default: 30 include_recording: anyOf: - type: boolean - - type: 'null' + - type: "null" title: Include Recording default: true draw: anyOf: - type: object - - type: 'null' + - type: "null" title: Draw - default: { } + default: {} type: object title: EventsCreateBody + EventsDeleteBody: + properties: + event_ids: + items: + type: string + type: array + title: The event IDs to delete + type: object + required: + - event_ids + title: EventsDeleteBody EventsDescriptionBody: properties: description: anyOf: - type: string - - type: 'null' + - type: "null" title: The description of the event type: object required: @@ -3265,7 +3494,7 @@ components: end_time: anyOf: - type: number - - type: 'null' + - type: "null" title: End Time type: object title: EventsEndBody @@ -3280,7 +3509,7 @@ components: - type: number maximum: 1 exclusiveMinimum: 0 - - type: 'null' + - type: "null" title: Score for sub label type: object required: @@ -3290,12 +3519,12 @@ components: properties: playback: allOf: - - $ref: '#/components/schemas/PlaybackFactorEnum' + - $ref: "#/components/schemas/PlaybackFactorEnum" title: Playback factor default: realtime source: allOf: - - $ref: '#/components/schemas/PlaybackSourceEnum' + - $ref: "#/components/schemas/PlaybackSourceEnum" title: Playback source default: recordings name: @@ -3332,7 +3561,7 @@ components: properties: detail: items: - $ref: '#/components/schemas/ValidationError' + $ref: "#/components/schemas/ValidationError" type: array title: Detail type: object @@ -3426,7 +3655,7 @@ components: type: boolean title: Has Been Reviewed severity: - $ref: '#/components/schemas/SeverityEnum' + $ref: "#/components/schemas/SeverityEnum" thumb_path: type: string title: Thumb Path @@ -3446,10 +3675,10 @@ components: ReviewSummaryResponse: properties: last24Hours: - $ref: '#/components/schemas/Last24HoursReview' + $ref: "#/components/schemas/Last24HoursReview" root: additionalProperties: - $ref: '#/components/schemas/DayReview' + $ref: "#/components/schemas/DayReview" type: object title: Root type: object diff --git a/frigate/api/app.py b/frigate/api/app.py index d4c5cdba3..53855efca 100644 --- a/frigate/api/app.py +++ b/frigate/api/app.py @@ -17,8 +17,8 @@ from fastapi.responses import JSONResponse, PlainTextResponse from markupsafe import escape from peewee import operator -from frigate.api.defs.app_body import AppConfigSetBody -from frigate.api.defs.app_query_parameters import AppTimelineHourlyQueryParameters +from frigate.api.defs.query.app_query_parameters import AppTimelineHourlyQueryParameters +from frigate.api.defs.request.app_body import AppConfigSetBody from frigate.api.defs.tags import Tags from frigate.config import FrigateConfig from frigate.const import CONFIG_DIR diff --git a/frigate/api/auth.py b/frigate/api/auth.py index 8976469f5..cb5497f98 100644 --- a/frigate/api/auth.py +++ b/frigate/api/auth.py @@ -18,7 +18,7 @@ from joserfc import jwt from peewee import DoesNotExist from slowapi import Limiter -from frigate.api.defs.app_body import ( +from frigate.api.defs.request.app_body import ( AppPostLoginBody, AppPostUsersBody, AppPutPasswordBody, diff --git a/frigate/api/defs/app_query_parameters.py b/frigate/api/defs/query/app_query_parameters.py similarity index 100% rename from frigate/api/defs/app_query_parameters.py rename to frigate/api/defs/query/app_query_parameters.py diff --git a/frigate/api/defs/events_query_parameters.py b/frigate/api/defs/query/events_query_parameters.py similarity index 100% rename from frigate/api/defs/events_query_parameters.py rename to frigate/api/defs/query/events_query_parameters.py diff --git a/frigate/api/defs/media_query_parameters.py b/frigate/api/defs/query/media_query_parameters.py similarity index 100% rename from frigate/api/defs/media_query_parameters.py rename to frigate/api/defs/query/media_query_parameters.py diff --git a/frigate/api/defs/regenerate_query_parameters.py b/frigate/api/defs/query/regenerate_query_parameters.py similarity index 100% rename from frigate/api/defs/regenerate_query_parameters.py rename to frigate/api/defs/query/regenerate_query_parameters.py diff --git a/frigate/api/defs/review_query_parameters.py b/frigate/api/defs/query/review_query_parameters.py similarity index 100% rename from frigate/api/defs/review_query_parameters.py rename to frigate/api/defs/query/review_query_parameters.py diff --git a/frigate/api/defs/app_body.py b/frigate/api/defs/request/app_body.py similarity index 100% rename from frigate/api/defs/app_body.py rename to frigate/api/defs/request/app_body.py diff --git a/frigate/api/defs/events_body.py b/frigate/api/defs/request/events_body.py similarity index 100% rename from frigate/api/defs/events_body.py rename to frigate/api/defs/request/events_body.py diff --git a/frigate/api/defs/review_body.py b/frigate/api/defs/request/review_body.py similarity index 100% rename from frigate/api/defs/review_body.py rename to frigate/api/defs/request/review_body.py diff --git a/frigate/api/defs/response/event_response.py b/frigate/api/defs/response/event_response.py new file mode 100644 index 000000000..75cf670dc --- /dev/null +++ b/frigate/api/defs/response/event_response.py @@ -0,0 +1,40 @@ +from typing import Any, Optional + +from pydantic import BaseModel + + +class EventResponse(BaseModel): + id: str + label: str + sub_label: Optional[str] + camera: str + start_time: float + end_time: Optional[float] + false_positive: bool + zones: list[str] + thumbnail: str + has_clip: bool + has_snapshot: bool + retain_indefinitely: bool + plus_id: Optional[str] + model_hash: Optional[str] + detector_type: Optional[str] + model_type: Optional[str] + data: dict[str, Any] + + +class EventCreateResponse(BaseModel): + success: bool + message: str + event_id: str + + +class EventMultiDeleteResponse(BaseModel): + success: bool + deleted_events: list[str] + not_found_events: list[str] + + +class EventUploadPlusResponse(BaseModel): + success: bool + plus_id: str diff --git a/frigate/api/defs/generic_response.py b/frigate/api/defs/response/generic_response.py similarity index 100% rename from frigate/api/defs/generic_response.py rename to frigate/api/defs/response/generic_response.py diff --git a/frigate/api/defs/review_responses.py b/frigate/api/defs/response/review_response.py similarity index 100% rename from frigate/api/defs/review_responses.py rename to frigate/api/defs/response/review_response.py diff --git a/frigate/api/event.py b/frigate/api/event.py index dc98d094e..3ba4ae426 100644 --- a/frigate/api/event.py +++ b/frigate/api/event.py @@ -14,7 +14,16 @@ from fastapi.responses import JSONResponse from peewee import JOIN, DoesNotExist, fn, operator from playhouse.shortcuts import model_to_dict -from frigate.api.defs.events_body import ( +from frigate.api.defs.query.events_query_parameters import ( + DEFAULT_TIME_RANGE, + EventsQueryParams, + EventsSearchQueryParams, + EventsSummaryQueryParams, +) +from frigate.api.defs.query.regenerate_query_parameters import ( + RegenerateQueryParameters, +) +from frigate.api.defs.request.events_body import ( EventsCreateBody, EventsDeleteBody, EventsDescriptionBody, @@ -22,19 +31,15 @@ from frigate.api.defs.events_body import ( EventsSubLabelBody, SubmitPlusBody, ) -from frigate.api.defs.events_query_parameters import ( - DEFAULT_TIME_RANGE, - EventsQueryParams, - EventsSearchQueryParams, - EventsSummaryQueryParams, -) -from frigate.api.defs.regenerate_query_parameters import ( - RegenerateQueryParameters, +from frigate.api.defs.response.event_response import ( + EventCreateResponse, + EventMultiDeleteResponse, + EventResponse, + EventUploadPlusResponse, ) +from frigate.api.defs.response.generic_response import GenericResponse from frigate.api.defs.tags import Tags -from frigate.const import ( - CLIPS_DIR, -) +from frigate.const import CLIPS_DIR from frigate.embeddings import EmbeddingsContext from frigate.events.external import ExternalEventProcessor from frigate.models import Event, ReviewSegment, Timeline @@ -46,7 +51,7 @@ logger = logging.getLogger(__name__) router = APIRouter(tags=[Tags.events]) -@router.get("/events") +@router.get("/events", response_model=list[EventResponse]) def events(params: EventsQueryParams = Depends()): camera = params.camera cameras = params.cameras @@ -265,7 +270,7 @@ def events(params: EventsQueryParams = Depends()): return JSONResponse(content=list(events)) -@router.get("/events/explore") +@router.get("/events/explore", response_model=list[EventResponse]) def events_explore(limit: int = 10): # get distinct labels for all events distinct_labels = Event.select(Event.label).distinct().order_by(Event.label) @@ -310,7 +315,8 @@ def events_explore(limit: int = 10): "data": { k: v for k, v in event.data.items() - if k in ["type", "score", "top_score", "description"] + if k + in ["type", "score", "top_score", "description", "sub_label_score"] }, "event_count": label_counts[event.label], } @@ -326,7 +332,7 @@ def events_explore(limit: int = 10): return JSONResponse(content=processed_events) -@router.get("/event_ids") +@router.get("/event_ids", response_model=list[EventResponse]) def event_ids(ids: str): ids = ids.split(",") @@ -647,7 +653,7 @@ def events_summary(params: EventsSummaryQueryParams = Depends()): return JSONResponse(content=[e for e in groups.dicts()]) -@router.get("/events/{event_id}") +@router.get("/events/{event_id}", response_model=EventResponse) def event(event_id: str): try: return model_to_dict(Event.get(Event.id == event_id)) @@ -655,7 +661,7 @@ def event(event_id: str): return JSONResponse(content="Event not found", status_code=404) -@router.post("/events/{event_id}/retain") +@router.post("/events/{event_id}/retain", response_model=GenericResponse) def set_retain(event_id: str): try: event = Event.get(Event.id == event_id) @@ -674,7 +680,7 @@ def set_retain(event_id: str): ) -@router.post("/events/{event_id}/plus") +@router.post("/events/{event_id}/plus", response_model=EventUploadPlusResponse) def send_to_plus(request: Request, event_id: str, body: SubmitPlusBody = None): if not request.app.frigate_config.plus_api.is_active(): message = "PLUS_API_KEY environment variable is not set" @@ -786,7 +792,7 @@ def send_to_plus(request: Request, event_id: str, body: SubmitPlusBody = None): ) -@router.put("/events/{event_id}/false_positive") +@router.put("/events/{event_id}/false_positive", response_model=EventUploadPlusResponse) def false_positive(request: Request, event_id: str): if not request.app.frigate_config.plus_api.is_active(): message = "PLUS_API_KEY environment variable is not set" @@ -875,7 +881,7 @@ def false_positive(request: Request, event_id: str): ) -@router.delete("/events/{event_id}/retain") +@router.delete("/events/{event_id}/retain", response_model=GenericResponse) def delete_retain(event_id: str): try: event = Event.get(Event.id == event_id) @@ -894,7 +900,7 @@ def delete_retain(event_id: str): ) -@router.post("/events/{event_id}/sub_label") +@router.post("/events/{event_id}/sub_label", response_model=GenericResponse) def set_sub_label( request: Request, event_id: str, @@ -946,7 +952,7 @@ def set_sub_label( ) -@router.post("/events/{event_id}/description") +@router.post("/events/{event_id}/description", response_model=GenericResponse) def set_description( request: Request, event_id: str, @@ -993,7 +999,7 @@ def set_description( ) -@router.put("/events/{event_id}/description/regenerate") +@router.put("/events/{event_id}/description/regenerate", response_model=GenericResponse) def regenerate_description( request: Request, event_id: str, params: RegenerateQueryParameters = Depends() ): @@ -1064,14 +1070,14 @@ def delete_single_event(event_id: str, request: Request) -> dict: return {"success": True, "message": f"Event {event_id} deleted"} -@router.delete("/events/{event_id}") +@router.delete("/events/{event_id}", response_model=GenericResponse) def delete_event(request: Request, event_id: str): result = delete_single_event(event_id, request) status_code = 200 if result["success"] else 404 return JSONResponse(content=result, status_code=status_code) -@router.delete("/events/") +@router.delete("/events/", response_model=EventMultiDeleteResponse) def delete_events(request: Request, body: EventsDeleteBody): if not body.event_ids: return JSONResponse( @@ -1097,7 +1103,7 @@ def delete_events(request: Request, body: EventsDeleteBody): return JSONResponse(content=response, status_code=200) -@router.post("/events/{camera_name}/{label}/create") +@router.post("/events/{camera_name}/{label}/create", response_model=EventCreateResponse) def create_event( request: Request, camera_name: str, @@ -1153,7 +1159,7 @@ def create_event( ) -@router.put("/events/{event_id}/end") +@router.put("/events/{event_id}/end", response_model=GenericResponse) def end_event(request: Request, event_id: str, body: EventsEndBody): try: end_time = body.end_time or datetime.datetime.now().timestamp() diff --git a/frigate/api/export.py b/frigate/api/export.py index 71f612371..ba0f8be28 100644 --- a/frigate/api/export.py +++ b/frigate/api/export.py @@ -9,6 +9,7 @@ import psutil from fastapi import APIRouter, Request from fastapi.responses import JSONResponse from peewee import DoesNotExist +from playhouse.shortcuts import model_to_dict from frigate.api.defs.request.export_recordings_body import ExportRecordingsBody from frigate.api.defs.tags import Tags @@ -207,3 +208,14 @@ def export_delete(event_id: str): ), status_code=200, ) + + +@router.get("/exports/{export_id}") +def get_export(export_id: str): + try: + return JSONResponse(content=model_to_dict(Export.get(Export.id == export_id))) + except DoesNotExist: + return JSONResponse( + content={"success": False, "message": "Export not found"}, + status_code=404, + ) diff --git a/frigate/api/media.py b/frigate/api/media.py index a90766899..e19fe547f 100644 --- a/frigate/api/media.py +++ b/frigate/api/media.py @@ -20,7 +20,7 @@ from pathvalidate import sanitize_filename from peewee import DoesNotExist, fn from tzlocal import get_localzone_name -from frigate.api.defs.media_query_parameters import ( +from frigate.api.defs.query.media_query_parameters import ( Extension, MediaEventsSnapshotQueryParams, MediaLatestFrameQueryParams, diff --git a/frigate/api/review.py b/frigate/api/review.py index 04e3e6dcd..56bd937bc 100644 --- a/frigate/api/review.py +++ b/frigate/api/review.py @@ -12,14 +12,14 @@ from fastapi.responses import JSONResponse from peewee import Case, DoesNotExist, fn, operator from playhouse.shortcuts import model_to_dict -from frigate.api.defs.generic_response import GenericResponse -from frigate.api.defs.review_body import ReviewModifyMultipleBody -from frigate.api.defs.review_query_parameters import ( +from frigate.api.defs.query.review_query_parameters import ( ReviewActivityMotionQueryParams, ReviewQueryParams, ReviewSummaryQueryParams, ) -from frigate.api.defs.review_responses import ( +from frigate.api.defs.request.review_body import ReviewModifyMultipleBody +from frigate.api.defs.response.generic_response import GenericResponse +from frigate.api.defs.response.review_response import ( ReviewActivityMotionResponse, ReviewSegmentResponse, ReviewSummaryResponse, @@ -364,7 +364,7 @@ def delete_reviews(body: ReviewModifyMultipleBody): ReviewSegment.delete().where(ReviewSegment.id << list_of_ids).execute() return JSONResponse( - content=({"success": True, "message": "Delete reviews"}), status_code=200 + content=({"success": True, "message": "Deleted review items."}), status_code=200 ) diff --git a/frigate/test/http_api/test_http_review.py b/frigate/test/http_api/test_http_review.py index 3bd8779aa..352b79c7a 100644 --- a/frigate/test/http_api/test_http_review.py +++ b/frigate/test/http_api/test_http_review.py @@ -525,7 +525,7 @@ class TestHttpReview(BaseTestHttp): assert response.status_code == 200 response_json = response.json() assert response_json["success"] == True - assert response_json["message"] == "Delete reviews" + assert response_json["message"] == "Deleted review items." # Verify that in DB the review segment was not deleted review_ids_in_db_after = self._get_reviews([id]) assert len(review_ids_in_db_after) == 1 @@ -540,7 +540,7 @@ class TestHttpReview(BaseTestHttp): assert response.status_code == 200 response_json = response.json() assert response_json["success"] == True - assert response_json["message"] == "Delete reviews" + assert response_json["message"] == "Deleted review items." # Verify that in DB the review segment was deleted review_ids_in_db_after = self._get_reviews([id]) assert len(review_ids_in_db_after) == 0 @@ -562,7 +562,7 @@ class TestHttpReview(BaseTestHttp): assert response.status_code == 200 response_json = response.json() assert response_json["success"] == True - assert response_json["message"] == "Delete reviews" + assert response_json["message"] == "Deleted review items." # Verify that in DB all review segments and recordings that were passed were deleted review_ids_in_db_after = self._get_reviews(ids) diff --git a/frigate/test/test_http.py b/frigate/test/test_http.py index d5927ad2b..213794259 100644 --- a/frigate/test/test_http.py +++ b/frigate/test/test_http.py @@ -168,7 +168,7 @@ class TestHttp(unittest.TestCase): assert event assert event["id"] == id - assert event == model_to_dict(Event.get(Event.id == id)) + assert event["id"] == model_to_dict(Event.get(Event.id == id))["id"] def test_get_bad_event(self): app = create_fastapi_app( From 8aa6297308cae24d9082dd9a8c2a8d8d79ccaa00 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 6 Dec 2024 09:32:16 -0600 Subject: [PATCH 207/479] Ensure label does not overlap with box or go out of frame (#15376) --- frigate/util/image.py | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/frigate/util/image.py b/frigate/util/image.py index 301da9c6a..24f523f1e 100644 --- a/frigate/util/image.py +++ b/frigate/util/image.py @@ -219,6 +219,8 @@ def draw_box_with_label( text_width = size[0][0] text_height = size[0][1] line_height = text_height + size[1] + # get frame height + frame_height = frame.shape[0] # set the text start position if position == "ul": text_offset_x = x_min @@ -228,18 +230,23 @@ def draw_box_with_label( text_offset_y = max(0, y_min - (line_height + 8)) elif position == "bl": text_offset_x = x_min - text_offset_y = y_max + text_offset_y = min(frame_height - line_height, y_max) elif position == "br": text_offset_x = max(0, x_max - (text_width + 8)) - text_offset_y = y_max - - # Adjust position if it overlaps with the box - if position in {"ul", "ur"} and text_offset_y < y_min + thickness: - # Move the text below the box - text_offset_y = y_max - elif position in {"bl", "br"} and text_offset_y + line_height > y_max: - # Move the text above the box - text_offset_y = max(0, y_min - (line_height + 8)) + text_offset_y = min(frame_height - line_height, y_max) + # Adjust position if it overlaps with the box or goes out of frame + if position in {"ul", "ur"}: + if text_offset_y < y_min + thickness: # Label overlaps with the box + if y_min - (line_height + 8) < 0 and y_max + line_height <= frame_height: + # Not enough space above, and there is space below + text_offset_y = y_max + elif y_min - (line_height + 8) >= 0: + # Enough space above, keep the label at the top + text_offset_y = max(0, y_min - (line_height + 8)) + elif position in {"bl", "br"}: + if text_offset_y + line_height > frame_height: + # If there's not enough space below, try above the box + text_offset_y = max(0, y_min - (line_height + 8)) # make the coords of the box with a small padding of two pixels textbox_coords = ( From bb86e71e6559963a8f562abedef650d9e4265390 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 6 Dec 2024 10:25:43 -0600 Subject: [PATCH 208/479] fix auth remote addr access (#15378) --- frigate/api/auth.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/frigate/api/auth.py b/frigate/api/auth.py index cb5497f98..8f0fead85 100644 --- a/frigate/api/auth.py +++ b/frigate/api/auth.py @@ -85,7 +85,12 @@ def get_remote_addr(request: Request): return str(ip) # if there wasn't anything in the route, just return the default - return request.remote_addr or "127.0.0.1" + remote_addr = None + + if hasattr(request, "remote_addr"): + remote_addr = request.remote_addr + + return remote_addr or "127.0.0.1" def get_jwt_secret() -> str: From d0cc8cb64b1c8bc1d2a66a83e41079681bee76cc Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Fri, 6 Dec 2024 20:07:43 -0600 Subject: [PATCH 209/479] API response cleanup (#15389) * API response cleanup * Remove extra field definition --- frigate/api/defs/response/event_response.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/frigate/api/defs/response/event_response.py b/frigate/api/defs/response/event_response.py index 75cf670dc..17b9b166f 100644 --- a/frigate/api/defs/response/event_response.py +++ b/frigate/api/defs/response/event_response.py @@ -1,6 +1,6 @@ from typing import Any, Optional -from pydantic import BaseModel +from pydantic import BaseModel, ConfigDict class EventResponse(BaseModel): @@ -10,7 +10,7 @@ class EventResponse(BaseModel): camera: str start_time: float end_time: Optional[float] - false_positive: bool + false_positive: Optional[bool] zones: list[str] thumbnail: str has_clip: bool @@ -22,6 +22,8 @@ class EventResponse(BaseModel): model_type: Optional[str] data: dict[str, Any] + model_config = ConfigDict(protected_namespaces=()) + class EventCreateResponse(BaseModel): success: bool From 0b9c4c18dd96197c6f6c2dc80170aecfada72595 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 9 Dec 2024 09:25:45 -0600 Subject: [PATCH 210/479] Refactor event cleanup to consider review severity (#15415) * Keep track of objects max review severity * Refactor cleanup to split snapshots and clips * Cleanup events based on review severity * Cleanup review imports * Don't catch detections --- .../api/defs/query/review_query_parameters.py | 2 +- frigate/api/defs/response/review_response.py | 2 +- frigate/api/review.py | 2 +- frigate/events/cleanup.py | 199 +++++++++++++----- frigate/events/maintainer.py | 1 + frigate/object_processing.py | 25 +-- frigate/review/maintainer.py | 7 +- frigate/review/types.py | 6 + frigate/test/http_api/base_http_test.py | 2 +- frigate/test/http_api/test_http_review.py | 2 +- frigate/track/tracked_object.py | 23 ++ 11 files changed, 187 insertions(+), 84 deletions(-) create mode 100644 frigate/review/types.py diff --git a/frigate/api/defs/query/review_query_parameters.py b/frigate/api/defs/query/review_query_parameters.py index 4361d313c..ee9af740e 100644 --- a/frigate/api/defs/query/review_query_parameters.py +++ b/frigate/api/defs/query/review_query_parameters.py @@ -3,7 +3,7 @@ from typing import Union from pydantic import BaseModel from pydantic.json_schema import SkipJsonSchema -from frigate.review.maintainer import SeverityEnum +from frigate.review.types import SeverityEnum class ReviewQueryParams(BaseModel): diff --git a/frigate/api/defs/response/review_response.py b/frigate/api/defs/response/review_response.py index 39e078b21..b2fed3b1a 100644 --- a/frigate/api/defs/response/review_response.py +++ b/frigate/api/defs/response/review_response.py @@ -3,7 +3,7 @@ from typing import Dict from pydantic import BaseModel, Json -from frigate.review.maintainer import SeverityEnum +from frigate.review.types import SeverityEnum class ReviewSegmentResponse(BaseModel): diff --git a/frigate/api/review.py b/frigate/api/review.py index 56bd937bc..e5692f009 100644 --- a/frigate/api/review.py +++ b/frigate/api/review.py @@ -26,7 +26,7 @@ from frigate.api.defs.response.review_response import ( ) from frigate.api.defs.tags import Tags from frigate.models import Recordings, ReviewSegment -from frigate.review.maintainer import SeverityEnum +from frigate.review.types import SeverityEnum from frigate.util.builtin import get_tz_modifiers logger = logging.getLogger(__name__) diff --git a/frigate/events/cleanup.py b/frigate/events/cleanup.py index 5400cc660..b1b485c3d 100644 --- a/frigate/events/cleanup.py +++ b/frigate/events/cleanup.py @@ -4,7 +4,6 @@ import datetime import logging import os import threading -from enum import Enum from multiprocessing.synchronize import Event as MpEvent from pathlib import Path @@ -16,11 +15,6 @@ from frigate.models import Event, Timeline logger = logging.getLogger(__name__) -class EventCleanupType(str, Enum): - clips = "clips" - snapshots = "snapshots" - - CHUNK_SIZE = 50 @@ -67,19 +61,11 @@ class EventCleanup(threading.Thread): return self.camera_labels[camera]["labels"] - def expire(self, media_type: EventCleanupType) -> list[str]: + def expire_snapshots(self) -> list[str]: ## Expire events from unlisted cameras based on the global config - if media_type == EventCleanupType.clips: - expire_days = max( - self.config.record.alerts.retain.days, - self.config.record.detections.retain.days, - ) - file_extension = None # mp4 clips are no longer stored in /clips - update_params = {"has_clip": False} - else: - retain_config = self.config.snapshots.retain - file_extension = "jpg" - update_params = {"has_snapshot": False} + retain_config = self.config.snapshots.retain + file_extension = "jpg" + update_params = {"has_snapshot": False} distinct_labels = self.get_removed_camera_labels() @@ -87,10 +73,7 @@ class EventCleanup(threading.Thread): # loop over object types in db for event in distinct_labels: # get expiration time for this label - if media_type == EventCleanupType.snapshots: - expire_days = retain_config.objects.get( - event.label, retain_config.default - ) + expire_days = retain_config.objects.get(event.label, retain_config.default) expire_after = ( datetime.datetime.now() - datetime.timedelta(days=expire_days) @@ -162,13 +145,7 @@ class EventCleanup(threading.Thread): ## Expire events from cameras based on the camera config for name, camera in self.config.cameras.items(): - if media_type == EventCleanupType.clips: - expire_days = max( - camera.record.alerts.retain.days, - camera.record.detections.retain.days, - ) - else: - retain_config = camera.snapshots.retain + retain_config = camera.snapshots.retain # get distinct objects in database for this camera distinct_labels = self.get_camera_labels(name) @@ -176,10 +153,9 @@ class EventCleanup(threading.Thread): # loop over object types in db for event in distinct_labels: # get expiration time for this label - if media_type == EventCleanupType.snapshots: - expire_days = retain_config.objects.get( - event.label, retain_config.default - ) + expire_days = retain_config.objects.get( + event.label, retain_config.default + ) expire_after = ( datetime.datetime.now() - datetime.timedelta(days=expire_days) @@ -206,19 +182,143 @@ class EventCleanup(threading.Thread): for event in expired_events: events_to_update.append(event.id) - if media_type == EventCleanupType.snapshots: - try: - media_name = f"{event.camera}-{event.id}" - media_path = Path( - f"{os.path.join(CLIPS_DIR, media_name)}.{file_extension}" - ) - media_path.unlink(missing_ok=True) - media_path = Path( - f"{os.path.join(CLIPS_DIR, media_name)}-clean.png" - ) - media_path.unlink(missing_ok=True) - except OSError as e: - logger.warning(f"Unable to delete event images: {e}") + try: + media_name = f"{event.camera}-{event.id}" + media_path = Path( + f"{os.path.join(CLIPS_DIR, media_name)}.{file_extension}" + ) + media_path.unlink(missing_ok=True) + media_path = Path( + f"{os.path.join(CLIPS_DIR, media_name)}-clean.png" + ) + media_path.unlink(missing_ok=True) + except OSError as e: + logger.warning(f"Unable to delete event images: {e}") + + # update the clips attribute for the db entry + for i in range(0, len(events_to_update), CHUNK_SIZE): + batch = events_to_update[i : i + CHUNK_SIZE] + logger.debug(f"Updating {update_params} for {len(batch)} events") + Event.update(update_params).where(Event.id << batch).execute() + + return events_to_update + + def expire_clips(self) -> list[str]: + ## Expire events from unlisted cameras based on the global config + expire_days = max( + self.config.record.alerts.retain.days, + self.config.record.detections.retain.days, + ) + file_extension = None # mp4 clips are no longer stored in /clips + update_params = {"has_clip": False} + + # get expiration time for this label + + expire_after = ( + datetime.datetime.now() - datetime.timedelta(days=expire_days) + ).timestamp() + # grab all events after specific time + expired_events: list[Event] = ( + Event.select( + Event.id, + Event.camera, + ) + .where( + Event.camera.not_in(self.camera_keys), + Event.start_time < expire_after, + Event.retain_indefinitely == False, + ) + .namedtuples() + .iterator() + ) + logger.debug(f"{len(list(expired_events))} events can be expired") + # delete the media from disk + for expired in expired_events: + media_name = f"{expired.camera}-{expired.id}" + media_path = Path(f"{os.path.join(CLIPS_DIR, media_name)}.{file_extension}") + + try: + media_path.unlink(missing_ok=True) + if file_extension == "jpg": + media_path = Path( + f"{os.path.join(CLIPS_DIR, media_name)}-clean.png" + ) + media_path.unlink(missing_ok=True) + except OSError as e: + logger.warning(f"Unable to delete event images: {e}") + + # update the clips attribute for the db entry + query = Event.select(Event.id).where( + Event.camera.not_in(self.camera_keys), + Event.start_time < expire_after, + Event.retain_indefinitely == False, + ) + + events_to_update = [] + + for batch in query.iterator(): + events_to_update.extend([event.id for event in batch]) + if len(events_to_update) >= CHUNK_SIZE: + logger.debug( + f"Updating {update_params} for {len(events_to_update)} events" + ) + Event.update(update_params).where( + Event.id << events_to_update + ).execute() + events_to_update = [] + + # Update any remaining events + if events_to_update: + logger.debug( + f"Updating clips/snapshots attribute for {len(events_to_update)} events" + ) + Event.update(update_params).where(Event.id << events_to_update).execute() + + events_to_update = [] + now = datetime.datetime.now() + + ## Expire events from cameras based on the camera config + for name, camera in self.config.cameras.items(): + expire_days = max( + camera.record.alerts.retain.days, + camera.record.detections.retain.days, + ) + alert_expire_date = ( + now - datetime.timedelta(days=camera.record.alerts.retain.days) + ).timestamp() + detection_expire_date = ( + now - datetime.timedelta(days=camera.record.detections.retain.days) + ).timestamp() + # grab all events after specific time + expired_events = ( + Event.select( + Event.id, + Event.camera, + ) + .where( + Event.camera == name, + Event.retain_indefinitely == False, + ( + ( + (Event.data["max_severity"] != "detection") + | (Event.data["max_severity"].is_null()) + ) + & (Event.end_time < alert_expire_date) + ) + | ( + (Event.data["max_severity"] == "detection") + & (Event.end_time < detection_expire_date) + ), + ) + .namedtuples() + .iterator() + ) + + # delete the grabbed clips from disk + # only snapshots are stored in /clips + # so no need to delete mp4 files + for event in expired_events: + events_to_update.append(event.id) # update the clips attribute for the db entry for i in range(0, len(events_to_update), CHUNK_SIZE): @@ -230,8 +330,9 @@ class EventCleanup(threading.Thread): def run(self) -> None: # only expire events every 5 minutes - while not self.stop_event.wait(300): - events_with_expired_clips = self.expire(EventCleanupType.clips) + while not self.stop_event.wait(1): + events_with_expired_clips = self.expire_clips() + return # delete timeline entries for events that have expired recordings # delete up to 100,000 at a time @@ -242,7 +343,7 @@ class EventCleanup(threading.Thread): Timeline.source_id << deleted_events_list[i : i + max_deletes] ).execute() - self.expire(EventCleanupType.snapshots) + self.expire_snapshots() # drop events from db where has_clip and has_snapshot are false events = ( diff --git a/frigate/events/maintainer.py b/frigate/events/maintainer.py index 3a4209ec3..e2b9245d6 100644 --- a/frigate/events/maintainer.py +++ b/frigate/events/maintainer.py @@ -210,6 +210,7 @@ class EventProcessor(threading.Thread): "top_score": event_data["top_score"], "attributes": attributes, "type": "object", + "max_severity": event_data.get("max_severity"), }, } diff --git a/frigate/object_processing.py b/frigate/object_processing.py index ef23c3de3..b5196e686 100644 --- a/frigate/object_processing.py +++ b/frigate/object_processing.py @@ -702,30 +702,7 @@ class TrackedObjectProcessor(threading.Thread): return False # If the object is not considered an alert or detection - review_config = self.config.cameras[camera].review - if not ( - ( - obj.obj_data["label"] in review_config.alerts.labels - and ( - not review_config.alerts.required_zones - or set(obj.entered_zones) & set(review_config.alerts.required_zones) - ) - ) - or ( - ( - not review_config.detections.labels - or obj.obj_data["label"] in review_config.detections.labels - ) - and ( - not review_config.detections.required_zones - or set(obj.entered_zones) - & set(review_config.detections.required_zones) - ) - ) - ): - logger.debug( - f"Not creating clip for {obj.obj_data['id']} because it did not qualify as an alert or detection" - ) + if obj.max_severity is None: return False return True diff --git a/frigate/review/maintainer.py b/frigate/review/maintainer.py index de137cb26..8aa0f65e0 100644 --- a/frigate/review/maintainer.py +++ b/frigate/review/maintainer.py @@ -7,7 +7,6 @@ import random import string import sys import threading -from enum import Enum from multiprocessing.synchronize import Event as MpEvent from pathlib import Path from typing import Optional @@ -27,6 +26,7 @@ from frigate.const import ( from frigate.events.external import ManualEventState from frigate.models import ReviewSegment from frigate.object_processing import TrackedObject +from frigate.review.types import SeverityEnum from frigate.util.image import SharedMemoryFrameManager, calculate_16_9_crop logger = logging.getLogger(__name__) @@ -39,11 +39,6 @@ THRESHOLD_ALERT_ACTIVITY = 120 THRESHOLD_DETECTION_ACTIVITY = 30 -class SeverityEnum(str, Enum): - alert = "alert" - detection = "detection" - - class PendingReviewSegment: def __init__( self, diff --git a/frigate/review/types.py b/frigate/review/types.py new file mode 100644 index 000000000..0046f9b69 --- /dev/null +++ b/frigate/review/types.py @@ -0,0 +1,6 @@ +from enum import Enum + + +class SeverityEnum(str, Enum): + alert = "alert" + detection = "detection" diff --git a/frigate/test/http_api/base_http_test.py b/frigate/test/http_api/base_http_test.py index ad1d449c5..e7a1d03e8 100644 --- a/frigate/test/http_api/base_http_test.py +++ b/frigate/test/http_api/base_http_test.py @@ -10,7 +10,7 @@ from playhouse.sqliteq import SqliteQueueDatabase from frigate.api.fastapi_app import create_fastapi_app from frigate.config import FrigateConfig from frigate.models import Event, Recordings, ReviewSegment -from frigate.review.maintainer import SeverityEnum +from frigate.review.types import SeverityEnum from frigate.test.const import TEST_DB, TEST_DB_CLEANUPS diff --git a/frigate/test/http_api/test_http_review.py b/frigate/test/http_api/test_http_review.py index 352b79c7a..11bd33495 100644 --- a/frigate/test/http_api/test_http_review.py +++ b/frigate/test/http_api/test_http_review.py @@ -3,7 +3,7 @@ from datetime import datetime, timedelta from fastapi.testclient import TestClient from frigate.models import Event, Recordings, ReviewSegment -from frigate.review.maintainer import SeverityEnum +from frigate.review.types import SeverityEnum from frigate.test.http_api.base_http_test import BaseTestHttp diff --git a/frigate/track/tracked_object.py b/frigate/track/tracked_object.py index 65e7a2ed5..3280965da 100644 --- a/frigate/track/tracked_object.py +++ b/frigate/track/tracked_object.py @@ -13,6 +13,7 @@ from frigate.config import ( CameraConfig, ModelConfig, ) +from frigate.review.types import SeverityEnum from frigate.util.image import ( area, calculate_region, @@ -59,6 +60,27 @@ class TrackedObject: self.pending_loitering = False self.previous = self.to_dict() + @property + def max_severity(self) -> Optional[str]: + review_config = self.camera_config.review + + if self.obj_data["label"] in review_config.alerts.labels and ( + not review_config.alerts.required_zones + or set(self.entered_zones) & set(review_config.alerts.required_zones) + ): + return SeverityEnum.alert + + if ( + not review_config.detections.labels + or self.obj_data["label"] in review_config.detections.labels + ) and ( + not review_config.detections.required_zones + or set(self.entered_zones) & set(review_config.detections.required_zones) + ): + return SeverityEnum.detection + + return None + def _is_false_positive(self): # once a true positive, always a true positive if not self.false_positive: @@ -232,6 +254,7 @@ class TrackedObject: "attributes": self.attributes, "current_attributes": self.obj_data["attributes"], "pending_loitering": self.pending_loitering, + "max_severity": self.max_severity, } if include_thumbnail: From 6b12a45a958b6eb2175d4071156d9039399c8d7a Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Tue, 10 Dec 2024 07:42:55 -0600 Subject: [PATCH 211/479] return 401 for login failures (#15432) * return 401 for login failures * only setup the rate limiter when configured --- frigate/api/auth.py | 4 ++-- frigate/api/fastapi_app.py | 6 +++++- web/src/api/index.tsx | 7 +++++-- web/src/components/auth/AuthForm.tsx | 2 +- 4 files changed, 13 insertions(+), 6 deletions(-) diff --git a/frigate/api/auth.py b/frigate/api/auth.py index 8f0fead85..be5917450 100644 --- a/frigate/api/auth.py +++ b/frigate/api/auth.py @@ -329,7 +329,7 @@ def login(request: Request, body: AppPostLoginBody): try: db_user: User = User.get_by_id(user) except DoesNotExist: - return JSONResponse(content={"message": "Login failed"}, status_code=400) + return JSONResponse(content={"message": "Login failed"}, status_code=401) password_hash = db_user.password_hash if verify_password(password, password_hash): @@ -340,7 +340,7 @@ def login(request: Request, body: AppPostLoginBody): response, JWT_COOKIE_NAME, encoded_jwt, expiration, JWT_COOKIE_SECURE ) return response - return JSONResponse(content={"message": "Login failed"}, status_code=400) + return JSONResponse(content={"message": "Login failed"}, status_code=401) @router.get("/users") diff --git a/frigate/api/fastapi_app.py b/frigate/api/fastapi_app.py index e3542458e..168404ea6 100644 --- a/frigate/api/fastapi_app.py +++ b/frigate/api/fastapi_app.py @@ -87,7 +87,11 @@ def create_fastapi_app( logger.info("FastAPI started") # Rate limiter (used for login endpoint) - auth.rateLimiter.set_limit(frigate_config.auth.failed_login_rate_limit or "") + if frigate_config.auth.failed_login_rate_limit is None: + limiter.enabled = False + else: + auth.rateLimiter.set_limit(frigate_config.auth.failed_login_rate_limit) + app.state.limiter = limiter app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler) app.add_middleware(SlowAPIMiddleware) diff --git a/web/src/api/index.tsx b/web/src/api/index.tsx index 3ac8806c7..a9044a6d7 100644 --- a/web/src/api/index.tsx +++ b/web/src/api/index.tsx @@ -29,8 +29,11 @@ export function ApiProvider({ children, options }: ApiProviderType) { error.response && [401, 302, 307].includes(error.response.status) ) { - window.location.href = - error.response.headers.get("location") ?? "login"; + // redirect to the login page if not already there + const loginPage = error.response.headers.get("location") ?? "login"; + if (window.location.href !== loginPage) { + window.location.href = loginPage; + } } }, ...options, diff --git a/web/src/components/auth/AuthForm.tsx b/web/src/components/auth/AuthForm.tsx index 9daa92966..99ce37283 100644 --- a/web/src/components/auth/AuthForm.tsx +++ b/web/src/components/auth/AuthForm.tsx @@ -63,7 +63,7 @@ export function UserAuthForm({ className, ...props }: UserAuthFormProps) { toast.error("Exceeded rate limit. Try again later.", { position: "top-center", }); - } else if (err.response?.status === 400) { + } else if (err.response?.status === 401) { toast.error("Login failed", { position: "top-center", }); From 0e3fb6cbddd3bafc1135b3d298bf5514c309299f Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Wed, 11 Dec 2024 18:46:42 -0600 Subject: [PATCH 212/479] Standardize handling of config files (#15451) * Standardize handling of config files * Formatting * Remove unused --- frigate/api/app.py | 26 ++++---------------------- frigate/config/config.py | 9 +++------ frigate/detectors/plugins/rknn.py | 6 +++--- frigate/ptz/autotrack.py | 11 ++--------- frigate/util/config.py | 10 ++++++++++ 5 files changed, 22 insertions(+), 40 deletions(-) diff --git a/frigate/api/app.py b/frigate/api/app.py index 53855efca..a94c6415c 100644 --- a/frigate/api/app.py +++ b/frigate/api/app.py @@ -21,13 +21,13 @@ from frigate.api.defs.query.app_query_parameters import AppTimelineHourlyQueryPa from frigate.api.defs.request.app_body import AppConfigSetBody from frigate.api.defs.tags import Tags from frigate.config import FrigateConfig -from frigate.const import CONFIG_DIR from frigate.models import Event, Timeline from frigate.util.builtin import ( clean_camera_user_pass, get_tz_modifiers, update_yaml_from_url, ) +from frigate.util.config import find_config_file from frigate.util.services import ( ffprobe_stream, get_nvidia_driver_info, @@ -147,13 +147,7 @@ def config(request: Request): @router.get("/config/raw") def config_raw(): - config_file = os.environ.get("CONFIG_FILE", "/config/config.yml") - - # Check if we can use .yaml instead of .yml - config_file_yaml = config_file.replace(".yml", ".yaml") - - if os.path.isfile(config_file_yaml): - config_file = config_file_yaml + config_file = find_config_file() if not os.path.isfile(config_file): return JSONResponse( @@ -198,13 +192,7 @@ def config_save(save_option: str, body: Any = Body(media_type="text/plain")): # Save the config to file try: - config_file = os.environ.get("CONFIG_FILE", "/config/config.yml") - - # Check if we can use .yaml instead of .yml - config_file_yaml = config_file.replace(".yml", ".yaml") - - if os.path.isfile(config_file_yaml): - config_file = config_file_yaml + config_file = find_config_file() with open(config_file, "w") as f: f.write(new_config) @@ -253,13 +241,7 @@ def config_save(save_option: str, body: Any = Body(media_type="text/plain")): @router.put("/config/set") def config_set(request: Request, body: AppConfigSetBody): - config_file = os.environ.get("CONFIG_FILE", f"{CONFIG_DIR}/config.yml") - - # Check if we can use .yaml instead of .yml - config_file_yaml = config_file.replace(".yml", ".yaml") - - if os.path.isfile(config_file_yaml): - config_file = config_file_yaml + config_file = find_config_file() with open(config_file, "r") as f: old_raw_config = f.read() diff --git a/frigate/config/config.py b/frigate/config/config.py index 8c0b52e92..770588b93 100644 --- a/frigate/config/config.py +++ b/frigate/config/config.py @@ -29,6 +29,7 @@ from frigate.util.builtin import ( ) from frigate.util.config import ( StreamInfoRetriever, + find_config_file, get_relative_coordinates, migrate_frigate_config, ) @@ -67,7 +68,6 @@ logger = logging.getLogger(__name__) yaml = YAML() -DEFAULT_CONFIG_FILE = "/config/config.yml" DEFAULT_CONFIG = """ mqtt: enabled: False @@ -638,16 +638,13 @@ class FrigateConfig(FrigateBaseModel): @classmethod def load(cls, **kwargs): - config_path = os.environ.get("CONFIG_FILE", DEFAULT_CONFIG_FILE) - - if not os.path.isfile(config_path): - config_path = config_path.replace("yml", "yaml") + config_path = find_config_file() # No configuration file found, create one. new_config = False if not os.path.isfile(config_path): logger.info("No config file found, saving default config") - config_path = DEFAULT_CONFIG_FILE + config_path = config_path new_config = True else: # Check if the config file needs to be migrated. diff --git a/frigate/detectors/plugins/rknn.py b/frigate/detectors/plugins/rknn.py index dae5cc057..df94d7b62 100644 --- a/frigate/detectors/plugins/rknn.py +++ b/frigate/detectors/plugins/rknn.py @@ -136,17 +136,17 @@ class Rknn(DetectionApi): def check_config(self, config): if (config.model.width != 320) or (config.model.height != 320): raise Exception( - "Make sure to set the model width and height to 320 in your config.yml." + "Make sure to set the model width and height to 320 in your config." ) if config.model.input_pixel_format != "bgr": raise Exception( - 'Make sure to set the model input_pixel_format to "bgr" in your config.yml.' + 'Make sure to set the model input_pixel_format to "bgr" in your config.' ) if config.model.input_tensor != "nhwc": raise Exception( - 'Make sure to set the model input_tensor to "nhwc" in your config.yml.' + 'Make sure to set the model input_tensor to "nhwc" in your config.' ) def detect_raw(self, tensor_input): diff --git a/frigate/ptz/autotrack.py b/frigate/ptz/autotrack.py index 03bb3840e..9f7f5f1b8 100644 --- a/frigate/ptz/autotrack.py +++ b/frigate/ptz/autotrack.py @@ -2,7 +2,6 @@ import copy import logging -import os import queue import threading import time @@ -29,11 +28,11 @@ from frigate.const import ( AUTOTRACKING_ZOOM_EDGE_THRESHOLD, AUTOTRACKING_ZOOM_IN_HYSTERESIS, AUTOTRACKING_ZOOM_OUT_HYSTERESIS, - CONFIG_DIR, ) from frigate.ptz.onvif import OnvifController from frigate.track.tracked_object import TrackedObject from frigate.util.builtin import update_yaml_file +from frigate.util.config import find_config_file from frigate.util.image import SharedMemoryFrameManager, intersection_over_union logger = logging.getLogger(__name__) @@ -328,13 +327,7 @@ class PtzAutoTracker: self.autotracker_init[camera] = True def _write_config(self, camera): - config_file = os.environ.get("CONFIG_FILE", f"{CONFIG_DIR}/config.yml") - - # Check if we can use .yaml instead of .yml - config_file_yaml = config_file.replace(".yml", ".yaml") - - if os.path.isfile(config_file_yaml): - config_file = config_file_yaml + config_file = find_config_file() logger.debug( f"{camera}: Writing new config with autotracker motion coefficients: {self.config.cameras[camera].onvif.autotracking.movement_weights}" diff --git a/frigate/util/config.py b/frigate/util/config.py index 3f3c45aa6..6bdbc0430 100644 --- a/frigate/util/config.py +++ b/frigate/util/config.py @@ -14,6 +14,16 @@ from frigate.util.services import get_video_properties logger = logging.getLogger(__name__) CURRENT_CONFIG_VERSION = "0.15-0" +DEFAULT_CONFIG_FILE = "/config/config.yml" + + +def find_config_file() -> str: + config_path = os.environ.get("CONFIG_FILE", DEFAULT_CONFIG_FILE) + + if not os.path.isfile(config_path): + config_path = config_path.replace("yml", "yaml") + + return config_path def migrate_frigate_config(config_file: str): From 53b96dfb897fba2e3453c5d5b5fbc0a122a2efbf Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Wed, 11 Dec 2024 20:19:08 -0600 Subject: [PATCH 213/479] Improve semantic search docs (#15453) --- docs/docs/configuration/semantic_search.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/docs/configuration/semantic_search.md b/docs/docs/configuration/semantic_search.md index 7b0a0bc91..ab3937c53 100644 --- a/docs/docs/configuration/semantic_search.md +++ b/docs/docs/configuration/semantic_search.md @@ -5,7 +5,7 @@ title: Using Semantic Search Semantic Search in Frigate allows you to find tracked objects within your review items using either the image itself, a user-defined text description, or an automatically generated one. This feature works by creating _embeddings_ — numerical vector representations — for both the images and text descriptions of your tracked objects. By comparing these embeddings, Frigate assesses their similarities to deliver relevant search results. -Frigate has support for [Jina AI's CLIP model](https://huggingface.co/jinaai/jina-clip-v1) to create embeddings, which runs locally. Embeddings are then saved to Frigate's database. +Frigate uses [Jina AI's CLIP model](https://huggingface.co/jinaai/jina-clip-v1) to create and save embeddings to Frigate's database. All of this runs locally. Semantic Search is accessed via the _Explore_ view in the Frigate UI. @@ -19,7 +19,7 @@ For best performance, 16GB or more of RAM and a dedicated GPU are recommended. ## Configuration -Semantic Search is disabled by default, and must be enabled in your config file before it can be used. Semantic Search is a global configuration setting. +Semantic Search is disabled by default, and must be enabled in your config file or in the UI's Settings page before it can be used. Semantic Search is a global configuration setting. ```yaml semantic_search: @@ -29,9 +29,9 @@ semantic_search: :::tip -The embeddings database can be re-indexed from the existing tracked objects in your database by adding `reindex: True` to your `semantic_search` configuration. Depending on the number of tracked objects you have, it can take a long while to complete and may max out your CPU while indexing. Make sure to set the config back to `False` before restarting Frigate again. +The embeddings database can be re-indexed from the existing tracked objects in your database by adding `reindex: True` to your `semantic_search` configuration or by toggling the switch on the Search Settings page in the UI and restarting Frigate. Depending on the number of tracked objects you have, it can take a long while to complete and may max out your CPU while indexing. Make sure to turn the UI's switch off or set the config back to `False` before restarting Frigate again. -If you are enabling the Search feature for the first time, be advised that Frigate does not automatically index older tracked objects. You will need to enable the `reindex` feature in order to do that. +If you are enabling Semantic Search for the first time, be advised that Frigate does not automatically index older tracked objects. You will need to enable the `reindex` feature in order to do that. ::: @@ -39,9 +39,9 @@ If you are enabling the Search feature for the first time, be advised that Friga The vision model is able to embed both images and text into the same vector space, which allows `image -> image` and `text -> image` similarity searches. Frigate uses this model on tracked objects to encode the thumbnail image and store it in the database. When searching for tracked objects via text in the search box, Frigate will perform a `text -> image` similarity search against this embedding. When clicking "Find Similar" in the tracked object detail pane, Frigate will perform an `image -> image` similarity search to retrieve the closest matching thumbnails. -The text model is used to embed tracked object descriptions and perform searches against them. Descriptions can be created, viewed, and modified on the Search page when clicking on the gray tracked object chip at the top left of each review item. See [the Generative AI docs](/configuration/genai.md) for more information on how to automatically generate tracked object descriptions. +The text model is used to embed tracked object descriptions and perform searches against them. Descriptions can be created, viewed, and modified on the Explore page when clicking on thumbnail of a tracked object. See [the Generative AI docs](/configuration/genai.md) for more information on how to automatically generate tracked object descriptions. -Differently weighted CLIP models are available and can be selected by setting the `model_size` config option as `small` or `large`: +Differently weighted versions of the Jina model are available and can be selected by setting the `model_size` config option as `small` or `large`: ```yaml semantic_search: @@ -50,7 +50,7 @@ semantic_search: ``` - Configuring the `large` model employs the full Jina model and will automatically run on the GPU if applicable. -- Configuring the `small` model employs a quantized version of the model that uses less RAM and runs on CPU with a very negligible difference in embedding quality. +- Configuring the `small` model employs a quantized version of the Jina model that uses less RAM and runs on CPU with a very negligible difference in embedding quality. ### GPU Acceleration @@ -84,7 +84,7 @@ If the correct build is used for your GPU and the `large` model is configured, t ## Usage and Best Practices -1. Semantic Search is used in conjunction with the other filters available on the Search page. Use a combination of traditional filtering and Semantic Search for the best results. +1. Semantic Search is used in conjunction with the other filters available on the Explore page. Use a combination of traditional filtering and Semantic Search for the best results. 2. Use the thumbnail search type when searching for particular objects in the scene. Use the description search type when attempting to discern the intent of your object. 3. Because of how the AI models Frigate uses have been trained, the comparison between text and image embedding distances generally means that with multi-modal (`thumbnail` and `description`) searches, results matching `description` will appear first, even if a `thumbnail` embedding may be a better match. Play with the "Search Type" setting to help find what you are looking for. Note that if you are generating descriptions for specific objects or zones only, this may cause search results to prioritize the objects with descriptions even if the the ones without them are more relevant. 4. Make your search language and tone closely match exactly what you're looking for. If you are using thumbnail search, **phrase your query as an image caption**. Searching for "red car" may not work as well as "red sedan driving down a residential street on a sunny day". From b4d82084a98cc245e73a1be922270994275ce128 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 12 Dec 2024 08:22:30 -0600 Subject: [PATCH 214/479] Fixes (#15465) * Fix single event return * Allow customizing if search is preserved for overlay state * Remove timeout * Cleanup * Cleanup naming --- frigate/events/cleanup.py | 5 +++-- web/src/hooks/use-overlay-state.tsx | 3 ++- web/src/pages/Events.tsx | 7 +++++-- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/frigate/events/cleanup.py b/frigate/events/cleanup.py index b1b485c3d..741f0884c 100644 --- a/frigate/events/cleanup.py +++ b/frigate/events/cleanup.py @@ -256,8 +256,9 @@ class EventCleanup(threading.Thread): events_to_update = [] - for batch in query.iterator(): - events_to_update.extend([event.id for event in batch]) + for event in query.iterator(): + events_to_update.append(event) + if len(events_to_update) >= CHUNK_SIZE: logger.debug( f"Updating {update_params} for {len(events_to_update)} events" diff --git a/web/src/hooks/use-overlay-state.tsx b/web/src/hooks/use-overlay-state.tsx index 841585b25..7a43383d4 100644 --- a/web/src/hooks/use-overlay-state.tsx +++ b/web/src/hooks/use-overlay-state.tsx @@ -5,6 +5,7 @@ import { usePersistence } from "./use-persistence"; export function useOverlayState( key: string, defaultValue: S | undefined = undefined, + preserveSearch: boolean = true, ): [S | undefined, (value: S, replace?: boolean) => void] { const location = useLocation(); const navigate = useNavigate(); @@ -15,7 +16,7 @@ export function useOverlayState( (value: S, replace: boolean = false) => { const newLocationState = { ...currentLocationState }; newLocationState[key] = value; - navigate(location.pathname + location.search, { + navigate(location.pathname + (preserveSearch ? location.search : ""), { state: newLocationState, replace, }); diff --git a/web/src/pages/Events.tsx b/web/src/pages/Events.tsx index 8c6f3cd38..28625bbd8 100644 --- a/web/src/pages/Events.tsx +++ b/web/src/pages/Events.tsx @@ -39,8 +39,11 @@ export default function Events() { const [showReviewed, setShowReviewed] = usePersistence("showReviewed", false); - const [recording, setRecording] = - useOverlayState("recording"); + const [recording, setRecording] = useOverlayState( + "recording", + undefined, + false, + ); useSearchEffect("id", (reviewId: string) => { axios From ed2e1f3f72315b3cbbaa6a24c2b6f12d8b918ccd Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 12 Dec 2024 08:46:06 -0600 Subject: [PATCH 215/479] Remove debug cleanup change (#15468) --- frigate/events/cleanup.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/frigate/events/cleanup.py b/frigate/events/cleanup.py index 741f0884c..713e33630 100644 --- a/frigate/events/cleanup.py +++ b/frigate/events/cleanup.py @@ -331,9 +331,8 @@ class EventCleanup(threading.Thread): def run(self) -> None: # only expire events every 5 minutes - while not self.stop_event.wait(1): + while not self.stop_event.wait(300): events_with_expired_clips = self.expire_clips() - return # delete timeline entries for events that have expired recordings # delete up to 100,000 at a time From d302b6e1988c2e5bd333b1efc3dad1ce38b48f73 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 12 Dec 2024 14:46:00 -0600 Subject: [PATCH 216/479] Cap storage bandwidth (#15473) --- frigate/storage.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/frigate/storage.py b/frigate/storage.py index 2dbd07a51..1c4650271 100644 --- a/frigate/storage.py +++ b/frigate/storage.py @@ -17,6 +17,8 @@ bandwidth_equation = Recordings.segment_size / ( Recordings.end_time - Recordings.start_time ) +MAX_CALCULATED_BANDWIDTH = 10000 # 10Gb/hr + class StorageMaintainer(threading.Thread): """Maintain frigates recording storage.""" @@ -52,6 +54,12 @@ class StorageMaintainer(threading.Thread): * 3600, 2, ) + + if bandwidth > MAX_CALCULATED_BANDWIDTH: + logger.warning( + f"{camera} has a bandwidth of {bandwidth} MB/hr which exceeds the expected maximum. This typically indicates an issue with the cameras recordings." + ) + bandwidth = MAX_CALCULATED_BANDWIDTH except TypeError: bandwidth = 0 From f336a91fee407022694a59eecc947ada2f4f5cb4 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Thu, 12 Dec 2024 21:22:47 -0600 Subject: [PATCH 217/479] Cleanup handling of first object message (#15480) --- frigate/events/maintainer.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/frigate/events/maintainer.py b/frigate/events/maintainer.py index e2b9245d6..68e7432ab 100644 --- a/frigate/events/maintainer.py +++ b/frigate/events/maintainer.py @@ -82,18 +82,23 @@ class EventProcessor(threading.Thread): ) if source_type == EventTypeEnum.tracked_object: + id = event_data["id"] self.timeline_queue.put( ( camera, source_type, event_type, - self.events_in_process.get(event_data["id"]), + self.events_in_process.get(id), event_data, ) ) - if event_type == EventStateEnum.start: - self.events_in_process[event_data["id"]] = event_data + # if this is the first message, just store it and continue, its not time to insert it in the db + if ( + event_type == EventStateEnum.start + or id not in self.events_in_process + ): + self.events_in_process[id] = event_data continue self.handle_object_detection(event_type, camera, event_data) @@ -123,10 +128,6 @@ class EventProcessor(threading.Thread): """handle tracked object event updates.""" updated_db = False - # if this is the first message, just store it and continue, its not time to insert it in the db - if event_type == EventStateEnum.start: - self.events_in_process[event_data["id"]] = event_data - if should_update_db(self.events_in_process[event_data["id"]], event_data): updated_db = True camera_config = self.config.cameras[camera] From 869fa2631ef765b761ea9f8d39c643869d82c755 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Fri, 13 Dec 2024 07:34:09 -0600 Subject: [PATCH 218/479] apply zizmor recommendations (#15490) --- .github/workflows/ci.yml | 16 ++++++++++++- .github/workflows/dependabot-auto-merge.yaml | 24 -------------------- .github/workflows/pull_request.yml | 12 +++++++++- .github/workflows/release.yml | 9 ++++++-- .github/workflows/stale.yml | 5 ++-- 5 files changed, 36 insertions(+), 30 deletions(-) delete mode 100644 .github/workflows/dependabot-auto-merge.yaml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3b06f0dc8..996b1e8e7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,7 +7,7 @@ on: - dev - master paths-ignore: - - 'docs/**' + - "docs/**" # only run the latest commit to avoid cache overwrites concurrency: @@ -24,6 +24,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -45,6 +47,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -86,6 +90,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -112,6 +118,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -140,6 +148,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -165,6 +175,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup @@ -188,6 +200,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up QEMU and Buildx id: setup uses: ./.github/actions/setup diff --git a/.github/workflows/dependabot-auto-merge.yaml b/.github/workflows/dependabot-auto-merge.yaml deleted file mode 100644 index 1c047c346..000000000 --- a/.github/workflows/dependabot-auto-merge.yaml +++ /dev/null @@ -1,24 +0,0 @@ -name: dependabot-auto-merge -on: pull_request - -permissions: - contents: write - -jobs: - dependabot-auto-merge: - runs-on: ubuntu-latest - if: github.actor == 'dependabot[bot]' - steps: - - name: Get Dependabot metadata - id: metadata - uses: dependabot/fetch-metadata@v2 - with: - github-token: ${{ secrets.GITHUB_TOKEN }} - - name: Enable auto-merge for Dependabot PRs - if: steps.metadata.outputs.dependency-type == 'direct:development' && (steps.metadata.outputs.update-type == 'version-update:semver-minor' || steps.metadata.outputs.update-type == 'version-update:semver-patch') - run: | - gh pr review --approve "$PR_URL" - gh pr merge --auto --squash "$PR_URL" - env: - PR_URL: ${{ github.event.pull_request.html_url }} - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml index bce97a07e..39c76e350 100644 --- a/.github/workflows/pull_request.yml +++ b/.github/workflows/pull_request.yml @@ -3,7 +3,7 @@ name: On pull request on: pull_request: paths-ignore: - - 'docs/**' + - "docs/**" env: DEFAULT_PYTHON: 3.9 @@ -19,6 +19,8 @@ jobs: DOCKER_BUILDKIT: "1" steps: - uses: actions/checkout@v4 + with: + persist-credentials: false - uses: actions/setup-node@master with: node-version: 16.x @@ -38,6 +40,8 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + with: + persist-credentials: false - uses: actions/setup-node@master with: node-version: 16.x @@ -52,6 +56,8 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + with: + persist-credentials: false - uses: actions/setup-node@master with: node-version: 20.x @@ -67,6 +73,8 @@ jobs: steps: - name: Check out the repository uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up Python ${{ env.DEFAULT_PYTHON }} uses: actions/setup-python@v5.1.0 with: @@ -88,6 +96,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v4 + with: + persist-credentials: false - uses: actions/setup-node@master with: node-version: 16.x diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 6e90b9c78..ace4c3b3f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -11,6 +11,8 @@ jobs: steps: - uses: actions/checkout@v4 + with: + persist-credentials: false - id: lowercaseRepo uses: ASzc/change-string-case-action@v6 with: @@ -22,10 +24,13 @@ jobs: username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Create tag variables + env: + TAG: ${{ github.ref_name }} + LOWERCASE_REPO: ${{ steps.lowercaseRepo.outputs.lowercase }} run: | - BUILD_TYPE=$([[ "${{ github.ref_name }}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] && echo "stable" || echo "beta") + BUILD_TYPE=$([[ "${TAG}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] && echo "stable" || echo "beta") echo "BUILD_TYPE=${BUILD_TYPE}" >> $GITHUB_ENV - echo "BASE=ghcr.io/${{ steps.lowercaseRepo.outputs.lowercase }}" >> $GITHUB_ENV + echo "BASE=ghcr.io/${LOWERCASE_REPO}" >> $GITHUB_ENV echo "BUILD_TAG=${GITHUB_SHA::7}" >> $GITHUB_ENV echo "CLEAN_VERSION=$(echo ${GITHUB_REF##*/} | tr '[:upper:]' '[:lower:]' | sed 's/^[v]//')" >> $GITHUB_ENV - name: Tag and push the main image diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index 8e7e3223c..011f70afd 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -23,7 +23,9 @@ jobs: exempt-pr-labels: "pinned,security,dependencies" operations-per-run: 120 - name: Print outputs - run: echo ${{ join(steps.stale.outputs.*, ',') }} + env: + STALE_OUTPUT: ${{ join(steps.stale.outputs.*, ',') }} + run: echo "$STALE_OUTPUT" # clean_ghcr: # name: Delete outdated dev container images @@ -38,4 +40,3 @@ jobs: # account-type: personal # token: ${{ secrets.GITHUB_TOKEN }} # token-type: github-token - From 1ea282fba88960524ff1847197a9fb34a2dc5908 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 13 Dec 2024 13:02:41 -0600 Subject: [PATCH 219/479] Improve the message for missing objects in review items (#15500) --- .../overlay/detail/ReviewDetailDialog.tsx | 26 +++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index c3e7ac91d..d3c8864b7 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -74,6 +74,23 @@ export default function ReviewDetailDialog({ return events.length != review?.data.detections.length; }, [review, events]); + const missingObjects = useMemo(() => { + if (!review || !events) { + return []; + } + + const detectedIds = review.data.detections; + const missing = Array.from( + new Set( + events + .filter((event) => !detectedIds.includes(event.id)) + .map((event) => event.label), + ), + ); + + return missing; + }, [review, events]); + const formattedDate = useFormattedTimestamp( review?.start_time ?? 0, config?.ui.time_format == "24hour" @@ -263,8 +280,13 @@ export default function ReviewDetailDialog({
{hasMismatch && (
- Some objects that were detected are not included in this list - because the object does not have a snapshot + Some objects may have been detected in this review item that + did not qualify as an alert or detection. Adjust your + configuration if you want Frigate to save tracked objects for + any missing labels. + {missingObjects.length > 0 && ( +
{missingObjects.join(", ")}
+ )}
)}
From 0763f560478949953b400260b6c471ddf88ecccb Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Fri, 13 Dec 2024 13:52:56 -0600 Subject: [PATCH 220/479] Update iframe interval recommendation (#15501) * Update iframe interval recommendation * clarify * tweaks * wording --- docs/docs/configuration/live.md | 2 +- docs/docs/frigate/camera_setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/configuration/live.md b/docs/docs/configuration/live.md index 31e720031..25bf7537c 100644 --- a/docs/docs/configuration/live.md +++ b/docs/docs/configuration/live.md @@ -23,7 +23,7 @@ If you are using go2rtc, you should adjust the following settings in your camera - Video codec: **H.264** - provides the most compatible video codec with all Live view technologies and browsers. Avoid any kind of "smart codec" or "+" codec like _H.264+_ or _H.265+_. as these non-standard codecs remove keyframes (see below). - Audio codec: **AAC** - provides the most compatible audio codec with all Live view technologies and browsers that support audio. -- I-frame interval (sometimes called the keyframe interval, the interframe space, or the GOP length): match your camera's frame rate, or choose "1x" (for interframe space on Reolink cameras). For example, if your stream outputs 20fps, your i-frame interval should be 20 (or 1x on Reolink). Values higher than the frame rate will cause the stream to take longer to begin playback. See [this page](https://gardinal.net/understanding-the-keyframe-interval/) for more on keyframes. +- I-frame interval (sometimes called the keyframe interval, the interframe space, or the GOP length): match your camera's frame rate, or choose "1x" (for interframe space on Reolink cameras). For example, if your stream outputs 20fps, your i-frame interval should be 20 (or 1x on Reolink). Values higher than the frame rate will cause the stream to take longer to begin playback. See [this page](https://gardinal.net/understanding-the-keyframe-interval/) for more on keyframes. For many users this may not be an issue, but it should be noted that that a 1x i-frame interval will cause more storage utilization if you are using the stream for the `record` role as well. The default video and audio codec on your camera may not always be compatible with your browser, which is why setting them to H.264 and AAC is recommended. See the [go2rtc docs](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness) for codec support information. diff --git a/docs/docs/frigate/camera_setup.md b/docs/docs/frigate/camera_setup.md index 33ae24cab..421046dd7 100644 --- a/docs/docs/frigate/camera_setup.md +++ b/docs/docs/frigate/camera_setup.md @@ -28,7 +28,7 @@ For the Dahua/Loryta 5442 camera, I use the following settings: - Encode Mode: H.264 - Resolution: 2688\*1520 - Frame Rate(FPS): 15 -- I Frame Interval: 30 +- I Frame Interval: 30 (15 can also be used to prioritize streaming performance - see the [camera settings recommendations](../configuration/live) for more info) **Sub Stream (Detection)** From 1b7fe9523df83d6ade94903e132233cc80abf31c Mon Sep 17 00:00:00 2001 From: FL42 <46161216+fl42@users.noreply.github.com> Date: Sat, 14 Dec 2024 15:54:13 +0100 Subject: [PATCH 221/479] fix: use requests.Session() for DeepStack API (#15505) --- frigate/detectors/plugins/deepstack.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/frigate/detectors/plugins/deepstack.py b/frigate/detectors/plugins/deepstack.py index 20d37fa8e..e00a4e70d 100644 --- a/frigate/detectors/plugins/deepstack.py +++ b/frigate/detectors/plugins/deepstack.py @@ -32,6 +32,7 @@ class DeepStack(DetectionApi): self.api_timeout = detector_config.api_timeout self.api_key = detector_config.api_key self.labels = detector_config.model.merged_labelmap + self.session = requests.Session() def get_label_index(self, label_value): if label_value.lower() == "truck": @@ -51,7 +52,7 @@ class DeepStack(DetectionApi): data = {"api_key": self.api_key} try: - response = requests.post( + response = self.session.post( self.api_url, data=data, files={"image": image_bytes}, From 17f8939f9783f67a0f29ee861a299c78573eea0d Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sat, 14 Dec 2024 13:58:39 -0600 Subject: [PATCH 222/479] Add FAQ to explain why streams might work in VLC but not in Frigate (#15513) * Add faq to explain why streams might work in VLC but not in Frigate * fix go2rtc version number * wording * mention udp input args and preset --- docs/docs/configuration/advanced.md | 2 +- docs/docs/troubleshooting/faqs.md | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/docs/configuration/advanced.md b/docs/docs/configuration/advanced.md index da5383886..4007154b1 100644 --- a/docs/docs/configuration/advanced.md +++ b/docs/docs/configuration/advanced.md @@ -174,7 +174,7 @@ NOTE: The folder that is set for the config needs to be the folder that contains ### Custom go2rtc version -Frigate currently includes go2rtc v1.9.4, there may be certain cases where you want to run a different version of go2rtc. +Frigate currently includes go2rtc v1.9.2, there may be certain cases where you want to run a different version of go2rtc. To do this: diff --git a/docs/docs/troubleshooting/faqs.md b/docs/docs/troubleshooting/faqs.md index 7356fecec..889db3370 100644 --- a/docs/docs/troubleshooting/faqs.md +++ b/docs/docs/troubleshooting/faqs.md @@ -98,3 +98,11 @@ docker run -d \ -p 8555:8555/udp \ ghcr.io/blakeblackshear/frigate:stable ``` + +### My RTSP stream works fine in VLC, but it does not work when I put the same URL in my Frigate config. Is this a bug? + +No. Frigate uses the TCP protocol to connect to your camera's RTSP URL. VLC automatically switches between UDP and TCP depending on network conditions and stream availability. So a stream that works in VLC but not in Frigate is likely due to VLC selecting UDP as the transfer protocol. + +TCP ensures that all data packets arrive in the correct order. This is crucial for video recording, decoding, and stream processing, which is why Frigate enforces a TCP connection. UDP is faster but less reliable, as it does not guarantee packet delivery or order, and VLC does not have the same requirements as Frigate. + +You can still configure Frigate to use UDP by using ffmpeg input args or the preset `preset-rtsp-udp`. See the [ffmpeg presets](/configuration/ffmpeg_presets) documentation. From 33ee32865f2ba5de3dded425b2caa6a2753124f9 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sun, 15 Dec 2024 16:56:24 -0600 Subject: [PATCH 223/479] Ensure that go2rtc streams are cleaned (#15524) * Ensure that go2rtc streams are cleaned * Formatting * Handle go2rtc config correctly * Set type --- frigate/api/app.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/frigate/api/app.py b/frigate/api/app.py index a94c6415c..57efdeb15 100644 --- a/frigate/api/app.py +++ b/frigate/api/app.py @@ -134,9 +134,25 @@ def config(request: Request): for zone_name, zone in config_obj.cameras[camera_name].zones.items(): camera_dict["zones"][zone_name]["color"] = zone.color + # remove go2rtc stream passwords + go2rtc: dict[str, any] = config_obj.go2rtc.model_dump( + mode="json", warnings="none", exclude_none=True + ) + for stream_name, stream in go2rtc.get("streams", {}).items(): + if isinstance(stream, str): + cleaned = clean_camera_user_pass(stream) + else: + cleaned = [] + + for item in stream: + cleaned.append(clean_camera_user_pass(item)) + + config["go2rtc"]["streams"][stream_name] = cleaned + config["plus"] = {"enabled": request.app.frigate_config.plus_api.is_active()} config["model"]["colormap"] = config_obj.model.colormap + # use merged labelamp for detector_config in config["detectors"].values(): detector_config["model"]["labelmap"] = ( request.app.frigate_config.model.merged_labelmap From d49f958d4d4dafe77718358d41cce7530ed9f217 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 15 Dec 2024 17:03:19 -0600 Subject: [PATCH 224/479] Don't crop by region for genai snapshot for manual events (#15525) --- frigate/embeddings/maintainer.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index d58a7f431..1bc872736 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -335,8 +335,10 @@ class EmbeddingMaintainer(threading.Thread): ) # crop snapshot based on region before sending off to genai + # provide full image if region doesn't exist (manual events) + region = event.data.get("region", [0, 0, 1, 1]) height, width = img.shape[:2] - x1_rel, y1_rel, width_rel, height_rel = event.data["region"] + x1_rel, y1_rel, width_rel, height_rel = region x1, y1 = int(x1_rel * width), int(y1_rel * height) cropped_image = img[ From 717493e66821bcb26239fe81e8651ef9a64d1ca4 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Sun, 15 Dec 2024 20:51:23 -0600 Subject: [PATCH 225/479] Improve handling of error conditions with ollama and snapshot regeneration (#15527) --- frigate/embeddings/maintainer.py | 13 +++++++++---- frigate/genai/ollama.py | 5 +++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index 1bc872736..2540db4f8 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -325,10 +325,15 @@ class EmbeddingMaintainer(threading.Thread): ) if event.has_snapshot and source == "snapshot": - with open( - os.path.join(CLIPS_DIR, f"{event.camera}-{event.id}.jpg"), - "rb", - ) as image_file: + snapshot_file = os.path.join(CLIPS_DIR, f"{event.camera}-{event.id}.jpg") + + if not os.path.isfile(snapshot_file): + logger.error( + f"Cannot regenerate description for {event.id}, snapshot file not found: {snapshot_file}" + ) + return + + with open(snapshot_file, "rb") as image_file: snapshot_image = image_file.read() img = cv2.imdecode( np.frombuffer(snapshot_image, dtype=np.int8), cv2.IMREAD_COLOR diff --git a/frigate/genai/ollama.py b/frigate/genai/ollama.py index e61441eba..e67d532f0 100644 --- a/frigate/genai/ollama.py +++ b/frigate/genai/ollama.py @@ -38,6 +38,11 @@ class OllamaClient(GenAIClient): def _send(self, prompt: str, images: list[bytes]) -> Optional[str]: """Submit a request to Ollama""" + if self.provider is None: + logger.warning( + "Ollama provider has not been initialized, a description will not be generated. Check your Ollama configuration." + ) + return None try: result = self.provider.generate( self.genai_config.model, From 292499aebc934f4e08168d82ae6d4665af44ac28 Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Mon, 16 Dec 2024 10:18:34 -0600 Subject: [PATCH 226/479] Improve review message again (#15538) --- .../overlay/detail/ReviewDetailDialog.tsx | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/web/src/components/overlay/detail/ReviewDetailDialog.tsx b/web/src/components/overlay/detail/ReviewDetailDialog.tsx index d3c8864b7..8d2f13d89 100644 --- a/web/src/components/overlay/detail/ReviewDetailDialog.tsx +++ b/web/src/components/overlay/detail/ReviewDetailDialog.tsx @@ -280,12 +280,24 @@ export default function ReviewDetailDialog({
{hasMismatch && (
- Some objects may have been detected in this review item that - did not qualify as an alert or detection. Adjust your - configuration if you want Frigate to save tracked objects for - any missing labels. + {(() => { + const detectedCount = Math.abs( + (events?.length ?? 0) - + (review?.data.detections.length ?? 0), + ); + const objectLabel = + detectedCount === 1 ? "object was" : "objects were"; + + return `${detectedCount} unavailable ${objectLabel} detected and included in this review item.`; + })()}{" "} + Those objects either did not qualify as an alert or detection + or have already been cleaned up/deleted. {missingObjects.length > 0 && ( -
{missingObjects.join(", ")}
+
+ Adjust your configuration if you want Frigate to save + tracked objects for the following labels:{" "} + {missingObjects.join(", ")} +
)}
)} From d9ef8fa20616fb5fa9df6539fc193903ce537f71 Mon Sep 17 00:00:00 2001 From: Giorgio Ughini Date: Tue, 17 Dec 2024 14:44:00 +0100 Subject: [PATCH 227/479] Fix always the same image is sent to GenAI (#15550) * Fix always the same image is sent to GenAI * Fix typo for bug where identical images are sent to GenAI * Correct formatting --- frigate/embeddings/maintainer.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/frigate/embeddings/maintainer.py b/frigate/embeddings/maintainer.py index 2540db4f8..4f81ec2d6 100644 --- a/frigate/embeddings/maintainer.py +++ b/frigate/embeddings/maintainer.py @@ -221,7 +221,10 @@ class EmbeddingMaintainer(threading.Thread): [snapshot_image] if event.has_snapshot and camera_config.genai.use_snapshot else ( - [thumbnail for data in self.tracked_events[event_id]] + [ + data["thumbnail"] + for data in self.tracked_events[event_id] + ] if len(self.tracked_events.get(event_id, [])) > 0 else [thumbnail] ) @@ -357,7 +360,7 @@ class EmbeddingMaintainer(threading.Thread): [snapshot_image] if event.has_snapshot and source == "snapshot" else ( - [thumbnail for data in self.tracked_events[event_id]] + [data["thumbnail"] for data in self.tracked_events[event_id]] if len(self.tracked_events.get(event_id, [])) > 0 else [thumbnail] ) From 5acbe37e6ffabb038983d316e08c2666b2d47d8a Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 17 Dec 2024 11:31:59 -0600 Subject: [PATCH 228/479] Update camera specific settings to make note of hikvision authentication (#15552) --- docs/docs/configuration/camera_specific.md | 23 ++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/docs/configuration/camera_specific.md b/docs/docs/configuration/camera_specific.md index 74f2a22b0..89df93a1d 100644 --- a/docs/docs/configuration/camera_specific.md +++ b/docs/docs/configuration/camera_specific.md @@ -95,6 +95,29 @@ ffmpeg: input_args: preset-rtsp-blue-iris ``` +### Hikvision Cameras + +Hikvision cameras should be connected to via RTSP using the following format: + +``` +rtsp://username:password@192.168.50.155/streaming/channels/101 # this is the main stream +rtsp://username:password@192.168.50.155/streaming/channels/102 # this is the sub stream +rtsp://username:password@192.168.50.155/streaming/channels/103 # some cameras support a third or fourth stream +``` + +:::note + +[Some users have reported](https://www.reddit.com/r/frigate_nvr/comments/1hg4ze7/hikvision_security_settings) that newer Hikvision cameras require adjustments to the security settings: + +``` +RTSP Authentication - digest/basic +RTSP Digest Algorithm - MD5 +WEB Authentication - digest/basic +WEB Digest Algorithm - MD5 +``` + +::: + ### Reolink Cameras Reolink has older cameras (ex: 410 & 520) as well as newer camera (ex: 520a & 511wa) which support different subsets of options. In both cases using the http stream is recommended. From 3dc26e78ef974efacea22271c3b3f074b5a977df Mon Sep 17 00:00:00 2001 From: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Date: Tue, 17 Dec 2024 17:33:04 -0600 Subject: [PATCH 229/479] Genai descriptions are not generated until tracked objects end (#15561) --- docs/docs/configuration/genai.md | 2 + .../overlay/detail/SearchDetailDialog.tsx | 134 +++++++++++------- 2 files changed, 84 insertions(+), 52 deletions(-) diff --git a/docs/docs/configuration/genai.md b/docs/docs/configuration/genai.md index fac44ed03..9d5f62b8c 100644 --- a/docs/docs/configuration/genai.md +++ b/docs/docs/configuration/genai.md @@ -5,6 +5,8 @@ title: Generative AI Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail. +Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle. Descriptions can also be regenerated manually via the Frigate UI. + :::info Semantic Search must be enabled to use Generative AI. diff --git a/web/src/components/overlay/detail/SearchDetailDialog.tsx b/web/src/components/overlay/detail/SearchDetailDialog.tsx index b0eeac98d..ba32eec2b 100644 --- a/web/src/components/overlay/detail/SearchDetailDialog.tsx +++ b/web/src/components/overlay/detail/SearchDetailDialog.tsx @@ -469,60 +469,90 @@ function ObjectDetailsTab({
-
Description
-