mirror of https://github.com/blakeblackshear/frigate.git synced 2026-06-21 03:41:55 +03:00

Lucas Zhu a2b92caab0 feat(genai): add Atlas Cloud as an OpenAI-compatible GenAI provider

Add an `atlas` GenAI provider backed by Atlas Cloud, an OpenAI-compatible
inference platform serving vision-capable models. The provider subclasses
the existing OpenAIClient and only defaults the base_url to the Atlas
endpoint, reusing all vision, streaming, reasoning, and tool-calling logic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-18 00:59:49 +08:00

5.2 KiB

Raw Blame History

Frigate NVR™ - Realtime Object Detection for IP Cameras

English

Atlas Cloud is an OpenAI-compatible inference platform that can power Frigate's Generative AI features as a drop-in multimodal LLM backend. Point the atlas provider at Atlas Cloud and use a vision-capable model (such as qwen/qwen3-vl-235b-a22b-thinking or Qwen/Qwen3-VL-235B-A22B-Instruct) to generate natural-language object and review descriptions from detection frames — no local GPU required. See the GenAI configuration docs to get started, or grab a coding plan.

Vision-capable Atlas Cloud models for GenAI descriptions

Frigate's GenAI features require a vision-capable model. Good multimodal choices on Atlas Cloud include:

qwen/qwen3-vl-235b-a22b-thinking
Qwen/Qwen3-VL-235B-A22B-Instruct
qwen/qwen3-vl-30b-a3b-instruct
qwen/qwen3-vl-30b-a3b-thinking
qwen/qwen3-vl-8b-instruct
google/gemini-3.5-flash
google/gemini-3.1-pro-preview

The full, always-current model catalog is available at the Atlas Cloud console.

A complete and local NVR designed for Home Assistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras.

Use of a GPU or AI accelerator is highly recommended. AI accelerators will outperform even the best CPUs with very little overhead. See Frigate's supported object detectors.

Tight integration with Home Assistant via a custom component
Designed to minimize resource use and maximize performance by only looking for objects when and where it is necessary
Leverages multiprocessing heavily with an emphasis on realtime over processing every frame
Uses a very low overhead motion detection to determine where to run object detection
Object detection with TensorFlow runs in separate processes for maximum FPS
Communicates over MQTT for easy integration into other systems
Records video with retention settings based on detected objects
24/7 recording
Re-streaming via RTSP to reduce the number of connections to your camera
WebRTC & MSE support for low-latency live view

Documentation

View the documentation at https://docs.frigate.video

Donations

If you would like to make a donation to support development, please use Github Sponsors.

License

This project is licensed under the MIT License.

Code: The source code, configuration files, and documentation in this repository are available under the MIT License. You are free to use, modify, and distribute the code as long as you include the original copyright notice.
Trademarks: The "Frigate" name, the "Frigate NVR" brand, and the Frigate logo are trademarks of Frigate, Inc. and are not covered by the MIT License.

Please see our Trademark Policy for details on acceptable use of our brand assets.