Compare commits

...
2 Commits
Author SHA1 Message Date
Nicolas MowenandGitHub ff2cd6a417 Add LFM to recommended local models (#24812) 2026-10-27 21:58:47 -05:00
Gleb NikiforovandGitHub 4f03480afd Fix audio worker startup after enabling a disabled camera (#24806)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* Fix audio worker startup after enabling a disabled camera

* Document audio lifecycle test helpers

* Focus audio startup coverage on the enabled update regression
2026-10-26 17:44:48 -05:00
3 changed files with 45 additions and 7 deletions
+7 -6
View File
@@ -75,11 +75,12 @@ Running Generative AI models on CPU is not recommended, as high inference times
You must use a vision-capable model with Frigate. The following models are recommended for local deployment of the `descriptions` and `chat` roles:
| Model | Review [frame mode](/configuration/genai/genai_review#frame-mode) | Notes |
| ------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-vl` | `frames` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. Follows a sequence of frames on its own. |
| `qwen3.6`/`qwen3.8` | `frames` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | `annotated_frames` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. Loses track of activity that repeats or reverses, so it benefits from annotated frames. |
| Model | Review [frame mode](/configuration/genai/genai_review#frame-mode) | Review [response style](/configuration/genai/genai_review#response-style) | Notes |
| ------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `qwen3-vl` | `frames` | `natural` | Strong visual and situational understanding, enhanced ability to identify smaller objects and interactions with object. Follows a sequence of frames on its own. |
| `qwen3.6`/`qwen3.8` | `frames` | `natural` | Strong situational understanding, but missing DeepStack from qwen3-vl leading to worse performance for identifying objects in people's hand and other small details. |
| `gemma4` | `annotated_frames` | `default` | Strong situational understanding, sometimes resorts to more vague terms like 'interacts' instead of assigning a specific action. Loses track of activity that repeats or reverses, so it benefits from annotated frames. |
| `lfm3-vl` | `annotated_frames` | `concise` | Very efficient with fast inference, making it a good choice for less powerful hardware or high volumes of review items. |
#### Embedding models
@@ -95,7 +96,7 @@ The `embeddings` role needs a different kind of model. Text queries are matched
The `transcribe` role needs a model that accepts audio input. A text-only or vision-only model cannot serve this role. The following are recommended for local deployment of the `transcribe` role:
| Model | Notes |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qwen3-asr` | Dedicated speech recognition model covering 30 languages, and the better choice for transcription quality. It only transcribes, so it cannot be shared with the `descriptions` or `chat` roles. |
| `gemma4` | General multimodal model that accepts audio as well as images, so one served model can cover `transcribe` alongside the other roles. Transcript quality is below `qwen3-asr`, particularly on noisy audio. |
+1
View File
@@ -178,6 +178,7 @@ class AudioProcessor(FrigateProcess):
[
CameraConfigUpdateEnum.add,
CameraConfigUpdateEnum.audio,
CameraConfigUpdateEnum.enabled,
CameraConfigUpdateEnum.ffmpeg,
CameraConfigUpdateEnum.remove,
],
+37 -1
View File
@@ -1,4 +1,4 @@
"""Tests for audio maintainer spawning on runtime camera adds."""
"""Tests for audio maintainer spawning on runtime camera updates."""
import threading
import unittest
@@ -42,6 +42,42 @@ class TestAudioSpawnIfNeeded(unittest.TestCase):
processor.logger = MagicMock()
return processor
def test_initially_disabled_camera_spawns_after_enabled_update(self):
"""Consume an enabled update through the manager's real subscriber."""
config = _build_config()
config.cameras["front_door"].enabled = False
stop = MagicMock()
stop.wait.side_effect = [False, True]
processor = AudioProcessor(
config, {"front_door": MagicMock()}, MagicMock(), stop
)
processor.logger = MagicMock()
updates = iter([("config/cameras/front_door/enabled", True), (None, None)])
self.addCleanup(
setattr, threading.current_thread(), "name", threading.current_thread().name
)
with (
patch.object(processor, "pre_run_setup"),
patch("frigate.config.camera.updater.ConfigSubscriber") as transport,
patch("frigate.events.audio.AudioEventMaintainer") as maintainer,
):
def receive_update():
self.assertEqual(processor.audio_threads, {})
maintainer.assert_not_called()
return next(updates)
transport.return_value.check_for_update.side_effect = receive_update
maintainer.return_value.is_alive.return_value = False
processor.run()
self.assertEqual(
processor.audio_threads, {"front_door": maintainer.return_value}
)
maintainer.assert_called_once()
maintainer.return_value.start.assert_called_once()
def test_skips_camera_whose_metrics_have_not_been_created_yet(self):
"""The camera maintainer creates metrics on its own poll of the add
update, so the audio processor can see the camera first."""