diff --git a/README.md b/README.md index 1fb158b191..d45e98f3e7 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,44 @@ \[English\] | [简体中文](https://github.com/blakeblackshear/frigate/blob/dev/README_CN.md) +--- + +

+ + Atlas Cloud + +

+ +

+ Atlas Cloud is an OpenAI-compatible inference platform that can power Frigate's + Generative AI features as a drop-in multimodal LLM backend. + Point the atlas provider at Atlas Cloud and use a vision-capable model + (such as qwen/qwen3-vl-235b-a22b-thinking or Qwen/Qwen3-VL-235B-A22B-Instruct) + to generate natural-language object and review descriptions from detection frames — + no local GPU required. See the GenAI configuration docs + to get started, or grab a coding plan. +

+ +
+Vision-capable Atlas Cloud models for GenAI descriptions + +Frigate's GenAI features require a **vision-capable** model. Good multimodal choices on Atlas Cloud include: + +- `qwen/qwen3-vl-235b-a22b-thinking` +- `Qwen/Qwen3-VL-235B-A22B-Instruct` +- `qwen/qwen3-vl-30b-a3b-instruct` +- `qwen/qwen3-vl-30b-a3b-thinking` +- `qwen/qwen3-vl-8b-instruct` +- `google/gemini-3.5-flash` +- `google/gemini-3.1-pro-preview` + +The full, always-current model catalog is available at the +[Atlas Cloud console](https://www.atlascloud.ai/console). + +
+ +--- + A complete and local NVR designed for [Home Assistant](https://www.home-assistant.io) with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras. Use of a GPU or AI accelerator is highly recommended. AI accelerators will outperform even the best CPUs with very little overhead. See Frigate's supported [object detectors](https://docs.frigate.video/configuration/object_detectors/). diff --git a/README_CN.md b/README_CN.md index 62df77b5c9..669839659a 100644 --- a/README_CN.md +++ b/README_CN.md @@ -12,6 +12,43 @@ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +--- + +

+ + Atlas Cloud + +

+ +

+ Atlas Cloud 是一个兼容 OpenAI 接口的推理平台,可作为即插即用的多模态 LLM 后端, + 为 Frigate 的生成式 AI(Generative AI)功能提供算力支持。 + 只需将 atlas provider 指向 Atlas Cloud,并选用一个支持视觉的模型 + (例如 qwen/qwen3-vl-235b-a22b-thinkingQwen/Qwen3-VL-235B-A22B-Instruct), + 即可基于检测帧画面生成自然语言的物体描述与审查摘要,无需本地 GPU。 + 请参阅 GenAI 配置文档开始使用, + 或了解 coding plan。 +

+ +
+适合做 GenAI 描述的 Atlas Cloud 多模态模型 + +Frigate 的 GenAI 功能要求使用**支持视觉**的模型。Atlas Cloud 上推荐的多模态模型包括: + +- `qwen/qwen3-vl-235b-a22b-thinking` +- `Qwen/Qwen3-VL-235B-A22B-Instruct` +- `qwen/qwen3-vl-30b-a3b-instruct` +- `qwen/qwen3-vl-30b-a3b-thinking` +- `qwen/qwen3-vl-8b-instruct` +- `google/gemini-3.5-flash` +- `google/gemini-3.1-pro-preview` + +完整且实时更新的模型列表请见 [Atlas Cloud 控制台](https://www.atlascloud.ai/console)。 + +
+ +--- + 一个完整的本地网络视频录像机(NVR),专为[Home Assistant](https://www.home-assistant.io)设计,具备 AI 目标/物体检测功能。使用 OpenCV 和 TensorFlow 在本地为 IP 摄像头执行实时物体检测。 强烈推荐使用 GPU 或者 AI 加速器(例如[Google Coral 加速器](https://coral.ai/products/) 或者 [Hailo](https://hailo.ai/)等)。它们的运行效率远远高于现在的顶级 CPU,并且功耗也极低。 diff --git a/docs/docs/configuration/genai/config.md b/docs/docs/configuration/genai/config.md index 9f396d3ccc..2f44962f04 100644 --- a/docs/docs/configuration/genai/config.md +++ b/docs/docs/configuration/genai/config.md @@ -386,3 +386,44 @@ genai: + +### Atlas Cloud + +[Atlas Cloud](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=frigate) is an OpenAI-compatible inference platform that serves a range of vision-capable models, so it can act as a drop-in multimodal backend for Frigate's Generative AI features. The `atlas` provider defaults its base URL to the Atlas Cloud endpoint, so a minimal config only needs your API key and a model. + +#### Supported Models + +You must use a vision capable model with Frigate. Recommended multimodal models on Atlas Cloud include `qwen/qwen3-vl-235b-a22b-thinking`, `Qwen/Qwen3-VL-235B-A22B-Instruct`, `qwen/qwen3-vl-30b-a3b-instruct`, and `google/gemini-3.5-flash`. The full, always-current catalog is available in the [Atlas Cloud console](https://www.atlascloud.ai/console). + +#### Get API Key + +To start using Atlas Cloud, create an API key from the [Atlas Cloud console](https://www.atlascloud.ai/console/api-keys). + +#### Configuration + + + + +1. Navigate to . + - Set **Provider** to `atlas` + - Set **API key** to your Atlas Cloud API key (or use an environment variable such as `{FRIGATE_ATLAS_API_KEY}`) + - Set **Model** to a vision-capable model (e.g., `qwen/qwen3-vl-235b-a22b-thinking`) + + + + +```yaml +genai: + provider: atlas + api_key: "{FRIGATE_ATLAS_API_KEY}" + model: qwen/qwen3-vl-235b-a22b-thinking +``` + + + + +:::note + +The `atlas` provider points to `https://api.atlascloud.ai/v1` by default. To target a different OpenAI-compatible endpoint, set `base_url` explicitly. + +::: diff --git a/docs/static/img/branding/atlas-cloud-logo.png b/docs/static/img/branding/atlas-cloud-logo.png new file mode 100644 index 0000000000..780a47698c Binary files /dev/null and b/docs/static/img/branding/atlas-cloud-logo.png differ diff --git a/frigate/config/camera/genai.py b/frigate/config/camera/genai.py index 5b94755723..71911b0550 100644 --- a/frigate/config/camera/genai.py +++ b/frigate/config/camera/genai.py @@ -12,6 +12,7 @@ __all__ = ["GenAIConfig", "GenAIProviderEnum", "GenAIRoleEnum"] class GenAIProviderEnum(str, Enum): openai = "openai" azure_openai = "azure_openai" + atlas = "atlas" gemini = "gemini" ollama = "ollama" llamacpp = "llamacpp" diff --git a/frigate/genai/plugins/atlas.py b/frigate/genai/plugins/atlas.py new file mode 100644 index 0000000000..dc1d78bb31 --- /dev/null +++ b/frigate/genai/plugins/atlas.py @@ -0,0 +1,71 @@ +"""Atlas Cloud Provider for Frigate AI. + +Atlas Cloud (https://www.atlascloud.ai) is an OpenAI-compatible inference +platform that serves a range of vision-capable models. Because its chat +completions API follows the OpenAI standard, this provider inherits all +transport, vision, streaming, reasoning, and tool-calling logic from +:class:`OpenAIClient` and only overrides what is Atlas-specific: + +- Client construction: defaults ``base_url`` to the Atlas Cloud endpoint + when the user has not set one explicitly, so a minimal config (provider + + api_key + model) works out of the box. A user-supplied ``base_url`` still + takes precedence. +- Context size: the Atlas ``/models`` endpoint does not reliably surface a + per-model context window, so we fall back to a conservative default rather + than the model-name heuristic used by OpenAI. It can be overridden via + ``provider_options.context_size``. +""" + +import logging +from typing import Optional + +from openai import OpenAI + +from frigate.config import GenAIProviderEnum +from frigate.genai import register_genai_provider +from frigate.genai.plugins.openai import OpenAIClient + +logger = logging.getLogger(__name__) + +DEFAULT_BASE_URL = "https://api.atlascloud.ai/v1" + +# Atlas serves large-context models, but its model listing does not expose a +# per-model context window; default conservatively and let users override via +# provider_options.context_size when they know their model's window. +DEFAULT_CONTEXT_SIZE = 32000 + + +@register_genai_provider(GenAIProviderEnum.atlas) +class AtlasClient(OpenAIClient): + """Generative AI client for Frigate using Atlas Cloud.""" + + def _init_provider(self) -> OpenAI: + """Initialize the OpenAI client pointed at Atlas Cloud. + + Defaults ``base_url`` to the Atlas endpoint when the user has not set + one, then defers to the OpenAI implementation for everything else. + """ + if not self.genai_config.base_url: + self.genai_config.base_url = DEFAULT_BASE_URL + + return super()._init_provider() + + def get_context_size(self) -> int: + """Return the context window for Atlas models. + + A manually specified ``context_size`` in ``provider_options`` always + wins; otherwise fall back to a conservative default since Atlas does + not reliably surface per-model context windows. + """ + if self.context_size is not None: + return self.context_size + + provider_context_size: Optional[int] = self.genai_config.provider_options.get( + "context_size" + ) + if provider_context_size is not None: + self.context_size = provider_context_size + return self.context_size + + self.context_size = DEFAULT_CONTEXT_SIZE + return self.context_size