Compare commits

...

4 Commits

Author SHA1 Message Date
lucaszhu-hue
8c5e85d978
Merge a2b92caab0 into d036061e3f 2026-06-20 18:18:48 -05:00
Josh Hawkins
d036061e3f
cache the preview_frames directory listing so concurrent per-camera frame requests share one scan instead of each re-listing the whole directory (#23526)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
2026-06-20 14:56:05 -05:00
Josh Hawkins
5003ab895c
add camera search, select-all/clear, and group selection to the multi-camera export dialog (#23516)
Some checks are pending
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
2026-06-19 15:50:19 -06:00
Lucas Zhu
a2b92caab0 feat(genai): add Atlas Cloud as an OpenAI-compatible GenAI provider
Add an `atlas` GenAI provider backed by Atlas Cloud, an OpenAI-compatible
inference platform serving vision-capable models. The provider subclasses
the existing OpenAIClient and only defaults the base_url to the Atlas
endpoint, reusing all vision, streaming, reasoning, and tool-calling logic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 00:59:49 +08:00
9 changed files with 422 additions and 25 deletions

View File

@ -12,6 +12,44 @@
\[English\] | [简体中文](https://github.com/blakeblackshear/frigate/blob/dev/README_CN.md)
---
<p align="center">
<a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate">
<img src="docs/static/img/branding/atlas-cloud-logo.png" alt="Atlas Cloud" width="200">
</a>
</p>
<p align="center">
<b><a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate">Atlas Cloud</a></b> is an OpenAI-compatible inference platform that can power Frigate's
<a href="https://docs.frigate.video/configuration/genai/">Generative AI</a> features as a drop-in multimodal LLM backend.
Point the <code>atlas</code> provider at Atlas Cloud and use a vision-capable model
(such as <code>qwen/qwen3-vl-235b-a22b-thinking</code> or <code>Qwen/Qwen3-VL-235B-A22B-Instruct</code>)
to generate natural-language object and review descriptions from detection frames —
no local GPU required. See the <a href="https://docs.frigate.video/configuration/genai/">GenAI configuration docs</a>
to get started, or grab a <a href="https://www.atlascloud.ai/console/coding-plan">coding plan</a>.
</p>
<details>
<summary>Vision-capable Atlas Cloud models for GenAI descriptions</summary>
Frigate's GenAI features require a **vision-capable** model. Good multimodal choices on Atlas Cloud include:
- `qwen/qwen3-vl-235b-a22b-thinking`
- `Qwen/Qwen3-VL-235B-A22B-Instruct`
- `qwen/qwen3-vl-30b-a3b-instruct`
- `qwen/qwen3-vl-30b-a3b-thinking`
- `qwen/qwen3-vl-8b-instruct`
- `google/gemini-3.5-flash`
- `google/gemini-3.1-pro-preview`
The full, always-current model catalog is available at the
[Atlas Cloud console](https://www.atlascloud.ai/console).
</details>
---
A complete and local NVR designed for [Home Assistant](https://www.home-assistant.io) with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras.
Use of a GPU or AI accelerator is highly recommended. AI accelerators will outperform even the best CPUs with very little overhead. See Frigate's supported [object detectors](https://docs.frigate.video/configuration/object_detectors/).

View File

@ -12,6 +12,43 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
---
<p align="center">
<a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate">
<img src="docs/static/img/branding/atlas-cloud-logo.png" alt="Atlas Cloud" width="200">
</a>
</p>
<p align="center">
<b><a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate">Atlas Cloud</a></b> 是一个兼容 OpenAI 接口的推理平台,可作为即插即用的多模态 LLM 后端,
为 Frigate 的<a href="https://docs.frigate.video/configuration/genai/">生成式 AIGenerative AI</a>功能提供算力支持。
只需将 <code>atlas</code> provider 指向 Atlas Cloud并选用一个支持视觉的模型
(例如 <code>qwen/qwen3-vl-235b-a22b-thinking</code><code>Qwen/Qwen3-VL-235B-A22B-Instruct</code>
即可基于检测帧画面生成自然语言的物体描述与审查摘要,无需本地 GPU。
请参阅 <a href="https://docs.frigate.video/configuration/genai/">GenAI 配置文档</a>开始使用,
或了解 <a href="https://www.atlascloud.ai/console/coding-plan">coding plan</a>
</p>
<details>
<summary>适合做 GenAI 描述的 Atlas Cloud 多模态模型</summary>
Frigate 的 GenAI 功能要求使用**支持视觉**的模型。Atlas Cloud 上推荐的多模态模型包括:
- `qwen/qwen3-vl-235b-a22b-thinking`
- `Qwen/Qwen3-VL-235B-A22B-Instruct`
- `qwen/qwen3-vl-30b-a3b-instruct`
- `qwen/qwen3-vl-30b-a3b-thinking`
- `qwen/qwen3-vl-8b-instruct`
- `google/gemini-3.5-flash`
- `google/gemini-3.1-pro-preview`
完整且实时更新的模型列表请见 [Atlas Cloud 控制台](https://www.atlascloud.ai/console)。
</details>
---
一个完整的本地网络视频录像机NVR专为[Home Assistant](https://www.home-assistant.io)设计,具备 AI 目标/物体检测功能。使用 OpenCV 和 TensorFlow 在本地为 IP 摄像头执行实时物体检测。
强烈推荐使用 GPU 或者 AI 加速器(例如[Google Coral 加速器](https://coral.ai/products/) 或者 [Hailo](https://hailo.ai/)等)。它们的运行效率远远高于现在的顶级 CPU并且功耗也极低。

View File

@ -386,3 +386,44 @@ genai:
</TabItem>
</ConfigTabs>
### Atlas Cloud
[Atlas Cloud](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate) is an OpenAI-compatible inference platform that serves a range of vision-capable models, so it can act as a drop-in multimodal backend for Frigate's Generative AI features. The `atlas` provider defaults its base URL to the Atlas Cloud endpoint, so a minimal config only needs your API key and a model.
#### Supported Models
You must use a vision capable model with Frigate. Recommended multimodal models on Atlas Cloud include `qwen/qwen3-vl-235b-a22b-thinking`, `Qwen/Qwen3-VL-235B-A22B-Instruct`, `qwen/qwen3-vl-30b-a3b-instruct`, and `google/gemini-3.5-flash`. The full, always-current catalog is available in the [Atlas Cloud console](https://www.atlascloud.ai/console).
#### Get API Key
To start using Atlas Cloud, create an API key from the [Atlas Cloud console](https://www.atlascloud.ai/console/api-keys).
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `atlas`
- Set **API key** to your Atlas Cloud API key (or use an environment variable such as `{FRIGATE_ATLAS_API_KEY}`)
- Set **Model** to a vision-capable model (e.g., `qwen/qwen3-vl-235b-a22b-thinking`)
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: atlas
api_key: "{FRIGATE_ATLAS_API_KEY}"
model: qwen/qwen3-vl-235b-a22b-thinking
```
</TabItem>
</ConfigTabs>
:::note
The `atlas` provider points to `https://api.atlascloud.ai/v1` by default. To target a different OpenAI-compatible endpoint, set `base_url` explicitly.
:::

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

View File

@ -1,7 +1,9 @@
"""Preview apis."""
import bisect
import logging
import os
import threading
from datetime import datetime, timedelta, timezone
import pytz
@ -133,6 +135,32 @@ def preview_hour(
return preview_ts(camera_name, start_ts, end_ts, allowed_cameras)
# cache one sorted listing of the shared preview_frames dir
_preview_listing_lock = threading.Lock()
_preview_listing_cache: tuple[float, list[str]] = (-1.0, [])
def _get_preview_frame_listing(preview_dir: str) -> list[str]:
"""Return the sorted preview_frames listing, cached until the dir changes."""
global _preview_listing_cache
# mtime bumps when a frame is added or removed, invalidating the cache
mtime = os.stat(preview_dir).st_mtime
cached_mtime, files = _preview_listing_cache
if mtime == cached_mtime:
return files
with _preview_listing_lock:
# another thread may have refreshed the cache while we waited
cached_mtime, files = _preview_listing_cache
if mtime == cached_mtime:
return files
files = sorted(entry.name for entry in os.scandir(preview_dir))
_preview_listing_cache = (mtime, files)
return files
@router.get(
"/preview/{camera_name}/start/{start_ts}/end/{end_ts}/frames",
response_model=PreviewFramesResponse,
@ -149,23 +177,15 @@ def get_preview_frames_from_cache(camera_name: str, start_ts: float, end_ts: flo
start_file = f"{file_start}{start_ts}.{PREVIEW_FRAME_TYPE}"
end_file = f"{file_start}{end_ts}.{PREVIEW_FRAME_TYPE}"
camera_files = [
entry.name
for entry in os.scandir(preview_dir)
if entry.name.startswith(file_start)
files = _get_preview_frame_listing(preview_dir)
# a camera's frames form a contiguous slice of the sorted listing;
# bisect locates it without scanning the whole directory
left = bisect.bisect_left(files, start_file)
right = bisect.bisect_right(files, end_file)
selected_previews = [
file for file in files[left:right] if file.startswith(file_start)
]
camera_files.sort()
selected_previews = []
for file in camera_files:
if file < start_file:
continue
if file > end_file:
break
selected_previews.append(file)
return JSONResponse(
content=selected_previews,

View File

@ -12,6 +12,7 @@ __all__ = ["GenAIConfig", "GenAIProviderEnum", "GenAIRoleEnum"]
class GenAIProviderEnum(str, Enum):
openai = "openai"
azure_openai = "azure_openai"
atlas = "atlas"
gemini = "gemini"
ollama = "ollama"
llamacpp = "llamacpp"

View File

@ -0,0 +1,71 @@
"""Atlas Cloud Provider for Frigate AI.
Atlas Cloud (https://www.atlascloud.ai) is an OpenAI-compatible inference
platform that serves a range of vision-capable models. Because its chat
completions API follows the OpenAI standard, this provider inherits all
transport, vision, streaming, reasoning, and tool-calling logic from
:class:`OpenAIClient` and only overrides what is Atlas-specific:
- Client construction: defaults ``base_url`` to the Atlas Cloud endpoint
when the user has not set one explicitly, so a minimal config (provider +
api_key + model) works out of the box. A user-supplied ``base_url`` still
takes precedence.
- Context size: the Atlas ``/models`` endpoint does not reliably surface a
per-model context window, so we fall back to a conservative default rather
than the model-name heuristic used by OpenAI. It can be overridden via
``provider_options.context_size``.
"""
import logging
from typing import Optional
from openai import OpenAI
from frigate.config import GenAIProviderEnum
from frigate.genai import register_genai_provider
from frigate.genai.plugins.openai import OpenAIClient
logger = logging.getLogger(__name__)
DEFAULT_BASE_URL = "https://api.atlascloud.ai/v1"
# Atlas serves large-context models, but its model listing does not expose a
# per-model context window; default conservatively and let users override via
# provider_options.context_size when they know their model's window.
DEFAULT_CONTEXT_SIZE = 32000
@register_genai_provider(GenAIProviderEnum.atlas)
class AtlasClient(OpenAIClient):
"""Generative AI client for Frigate using Atlas Cloud."""
def _init_provider(self) -> OpenAI:
"""Initialize the OpenAI client pointed at Atlas Cloud.
Defaults ``base_url`` to the Atlas endpoint when the user has not set
one, then defers to the OpenAI implementation for everything else.
"""
if not self.genai_config.base_url:
self.genai_config.base_url = DEFAULT_BASE_URL
return super()._init_provider()
def get_context_size(self) -> int:
"""Return the context window for Atlas models.
A manually specified ``context_size`` in ``provider_options`` always
wins; otherwise fall back to a conservative default since Atlas does
not reliably surface per-model context windows.
"""
if self.context_size is not None:
return self.context_size
provider_context_size: Optional[int] = self.genai_config.provider_options.get(
"context_size"
)
if provider_context_size is not None:
self.context_size = provider_context_size
return self.context_size
self.context_size = DEFAULT_CONTEXT_SIZE
return self.context_size

View File

@ -70,6 +70,13 @@
"selectFromTimeline": "Select from Timeline",
"cameraSelection": "Cameras",
"cameraSelectionHelp": "Cameras with tracked objects in this time range are pre-selected",
"searchOrSelectGroup": "Search, or select a camera group...",
"selectAll": "Select all cameras",
"clearSelection": "Clear selection",
"selectWithActivity": "Cameras with tracked objects",
"selectGroup": "Select group",
"noMatchingCameras": "No cameras match your search",
"selectedCount": "{{selected}} / {{total}} selected",
"checkingActivity": "Checking camera activity...",
"noCameras": "No cameras available",
"detectionCount_one": "1 tracked object",

View File

@ -39,6 +39,16 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import {
Command,
CommandGroup,
CommandInput,
CommandItem,
CommandList,
CommandSeparator,
} from "../ui/command";
import { IconRenderer } from "../icons/IconPicker";
import * as LuIcons from "react-icons/lu";
import { isDesktop, isMobile } from "react-device-detect";
import { Drawer, DrawerContent, DrawerTrigger } from "../ui/drawer";
import SaveExportOverlay from "./SaveExportOverlay";
@ -376,6 +386,9 @@ export function ExportContent({
const [newCaseName, setNewCaseName] = useState("");
const [newCaseDescription, setNewCaseDescription] = useState("");
const [isStartingBatchExport, setIsStartingBatchExport] = useState(false);
const [cameraSearch, setCameraSearch] = useState("");
const [cameraMenuOpen, setCameraMenuOpen] = useState(false);
const cameraMenuRef = useRef<HTMLDivElement>(null);
const multiRangeKey = useMemo(() => {
if (activeTab !== "multi" || !range) {
return undefined;
@ -577,6 +590,75 @@ export function ExportContent({
);
}, []);
const availableCameraIds = useMemo(
() => cameraActivities.map((activity) => activity.camera),
[cameraActivities],
);
const activeCameraIds = useMemo(
() =>
cameraActivities
.filter((activity) => activity.hasDetections)
.map((activity) => activity.camera),
[cameraActivities],
);
const cameraGroups = useMemo(
() =>
Object.entries(config?.camera_groups ?? {})
.map(([name, group]) => ({
name,
icon: group.icon,
order: group.order,
cameras: group.cameras.filter((cameraId) =>
availableCameraIds.includes(cameraId),
),
}))
.filter((group) => group.cameras.length > 0)
.sort((a, b) => a.order - b.order),
[config?.camera_groups, availableCameraIds],
);
// Filter the rendered camera cards by the search query
const filteredCameraActivities = useMemo(() => {
const query = cameraSearch.trim().toLowerCase();
if (!query) {
return cameraActivities;
}
return cameraActivities.filter((activity) => {
const friendlyName = resolveCameraName(config, activity.camera);
return (
activity.camera.toLowerCase().includes(query) ||
friendlyName.toLowerCase().includes(query)
);
});
}, [cameraActivities, cameraSearch, config]);
// Group/all/activity selection replaces the current selection
const applyCameraSelection = useCallback((cameraIds: string[]) => {
setHasManualCameraSelection(true);
setSelectedCameraIds(cameraIds);
setCameraMenuOpen(false);
}, []);
// Close the dropdown when focus leaves the camera selection control entirely
const handleCameraInputBlur = useCallback((event: React.FocusEvent) => {
if (
cameraMenuRef.current &&
!cameraMenuRef.current.contains(event.relatedTarget as Node)
) {
setCameraMenuOpen(false);
}
}, []);
// Reset the search and dropdown when leaving the multi-camera tab
useEffect(() => {
if (activeTab !== "multi") {
setCameraSearch("");
setCameraMenuOpen(false);
}
}, [activeTab]);
const startBatchExport = useCallback(async () => {
if (isStartingBatchExport) {
return;
@ -802,7 +884,7 @@ export function ExportContent({
{isAdmin && (
<div className="space-y-2">
<Label className="text-sm text-secondary-foreground">
<Label className="text-sm text-primary">
{t("export.case.label")}
</Label>
<Select
@ -859,7 +941,7 @@ export function ExportContent({
)}
>
<div className="space-y-2">
<Label className="text-sm text-secondary-foreground">
<Label className="text-sm text-primary">
{t("export.multiCamera.timeRange")}
</Label>
<div className="flex items-center gap-2">
@ -902,16 +984,109 @@ export function ExportContent({
</div>
<div className="space-y-2">
<Label className="text-sm text-secondary-foreground">
{t("export.multiCamera.cameraSelection")}
</Label>
<div className="flex items-center justify-between gap-2">
<Label className="text-sm text-primary">
{t("export.multiCamera.cameraSelection")}
</Label>
{availableCameraIds.length > 0 && (
<span className="text-xs text-muted-foreground">
{t("export.multiCamera.selectedCount", {
selected: selectedCameraCount,
total: availableCameraIds.length,
})}
</span>
)}
</div>
<div className="text-xs text-muted-foreground">
{t("export.multiCamera.cameraSelectionHelp")}
</div>
{!isEventsLoading && availableCameraIds.length > 0 && (
<div className="relative" ref={cameraMenuRef}>
<Command
shouldFilter={false}
className="overflow-visible rounded-md border bg-secondary/40"
>
<CommandInput
value={cameraSearch}
onValueChange={setCameraSearch}
onFocus={() => setCameraMenuOpen(true)}
onBlur={handleCameraInputBlur}
placeholder={t("export.multiCamera.searchOrSelectGroup")}
/>
{/* Hide the actions/groups menu while a search query is
active so it doesn't cover the filtered camera cards. */}
{cameraMenuOpen && cameraSearch.trim().length === 0 && (
<CommandList className="absolute top-full z-10 mt-1 max-h-72 w-full rounded-md border bg-background shadow-md">
<CommandGroup>
<CommandItem
value="action:select-all"
className="cursor-pointer"
onSelect={() =>
applyCameraSelection(availableCameraIds)
}
>
<span>{t("export.multiCamera.selectAll")}</span>
<span className="ml-auto text-xs text-muted-foreground">
{availableCameraIds.length}
</span>
</CommandItem>
<CommandItem
value="action:clear"
className="cursor-pointer"
onSelect={() => applyCameraSelection([])}
>
{t("export.multiCamera.clearSelection")}
</CommandItem>
<CommandItem
value="action:activity"
className="cursor-pointer"
onSelect={() => applyCameraSelection(activeCameraIds)}
>
<span>
{t("export.multiCamera.selectWithActivity")}
</span>
<span className="ml-auto text-xs text-muted-foreground">
{activeCameraIds.length}
</span>
</CommandItem>
</CommandGroup>
{cameraGroups.length > 0 && (
<>
<CommandSeparator />
<CommandGroup
heading={t("export.multiCamera.selectGroup")}
>
{cameraGroups.map((group) => (
<CommandItem
key={group.name}
value={`group:${group.name}`}
className="cursor-pointer"
onSelect={() =>
applyCameraSelection(group.cameras)
}
>
<IconRenderer
icon={LuIcons[group.icon]}
className="mr-2 size-4 text-secondary-foreground"
/>
<span className="truncate">{group.name}</span>
<span className="ml-auto text-xs text-muted-foreground">
{group.cameras.length}
</span>
</CommandItem>
))}
</CommandGroup>
</>
)}
</CommandList>
)}
</Command>
</div>
)}
<div
className={cn(
"scrollbar-container space-y-2",
isDesktop && "max-h-64 overflow-y-auto pr-1",
isDesktop && "max-h-64 overflow-y-auto p-0.5 pr-1",
)}
>
{isEventsLoading && (
@ -924,7 +1099,14 @@ export function ExportContent({
{t("export.multiCamera.noCameras")}
</div>
)}
{cameraActivities.map((activity) => {
{!isEventsLoading &&
cameraActivities.length > 0 &&
filteredCameraActivities.length === 0 && (
<div className="px-2 py-4 text-sm text-muted-foreground">
{t("export.multiCamera.noMatchingCameras")}
</div>
)}
{filteredCameraActivities.map((activity) => {
const isSelected = selectedCameraIds.includes(activity.camera);
return (
@ -981,7 +1163,7 @@ export function ExportContent({
</div>
<div className="space-y-2">
<Label className="text-sm text-secondary-foreground">
<Label className="text-sm text-primary">
{t("export.multiCamera.nameLabel")}
</Label>
<Input
@ -994,7 +1176,7 @@ export function ExportContent({
{isAdmin && (
<div className="space-y-2">
<Label className="text-sm text-secondary-foreground">
<Label className="text-sm text-primary">
{t("export.case.label")}
</Label>
<Select