Compare commits

...
5 Commits
Author SHA1 Message Date
mohitandGitHub c28f307d26 Merge ee823f8488 into c007661a71 2026-07-03 20:49:59 +01:00
Blake BlackshearandGitHub c007661a71 Enhancements (#23611)
CI / AMD64 Build (push) Has been cancelled
CI / ARM Build (push) Has been cancelled
CI / Jetson Jetpack 6 (push) Has been cancelled
CI / AMD64 Extra Build (push) Has been cancelled
CI / ARM Extra Build (push) Has been cancelled
CI / Synaptics Build (push) Has been cancelled
CI / Assemble and push default build (push) Has been cancelled
* better default handling if no proxy chain is present

* add note about SameSite for jwt cookie
2026-07-02 07:05:00 -05:00
9b02c7318d Miscellaneous fixes (#23610)
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
* Handle back seeking going to previous clip

* scope /recordings/unavailable query to the caller's allowed cameras

* listen for config updates in activity manager

* don't set search after awaited request

Intentionally do NOT setSearch() to mark the open event submitted. This runs after the awaited request, by which point the user may have closed the dialog; re-setting the parent's selected event would resurrect it and the force-open effect would reopen it (see #23599). The local "submitted" state covers the open card, and mutate() updates the events cache so the grid and any future open reflect the result.

* fix ruff

#23201 removed pathlib import but for some reason it's just now causing ruff to fail

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2026-07-01 15:03:34 -05:00
Mohit Varikuti ee823f8488 fix(genai): drop twelvelabs SDK dep, call Marengo via REST 2026-06-25 16:45:21 -07:00
Mohit Varikuti 85e16eb31b feat(genai): add TwelveLabs Marengo embeddings provider
Adds an opt-in 'twelvelabs' GenAI provider implementing the embeddings
role using TwelveLabs' Marengo multimodal model. Text and image inputs
are embedded into a shared vector space, powering semantic search over
event thumbnails.

The provider only implements the embeddings role; descriptions/chat are
left to other providers since Marengo is an embedding model. Marengo
returns 512-dim vectors, which the existing GenAIEmbedding adapter pads
to Frigate's 768-dim search schema.

Includes no-network unit tests and a live test gated on
TWELVELABS_API_KEY.
2026-06-25 00:43:04 -07:00
14 changed files with 463 additions and 24 deletions
@@ -3,6 +3,7 @@
import json
import os
import sys
from pathlib import Path
from typing import Any
from ruamel.yaml import YAML
+35 -1
View File
@@ -9,7 +9,7 @@ import NavPath from "@site/src/components/NavPath";
## Configuration
A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently 4 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI-Compatible section below.
A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently several native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI-Compatible section below.
To use Generative AI, you must define a single provider at the global level of your Frigate configuration. If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
@@ -386,3 +386,37 @@ genai:
</TabItem>
</ConfigTabs>
### TwelveLabs (Marengo embeddings)
[TwelveLabs](https://twelvelabs.io) provides the Marengo multimodal model, which embeds text and images into a shared vector space. This makes it usable as the `embeddings` provider for Frigate's [Semantic Search](/configuration/semantic_search), letting a natural-language query match stored event thumbnails directly.
This provider implements only the `embeddings` role. Use a separate provider (such as Ollama, Gemini, or OpenAI) for the `descriptions` and `chat` roles if you want generated descriptions as well.
#### Get API Key
Create an API key from the [TwelveLabs dashboard](https://playground.twelvelabs.io/dashboard/api-key). There is a generous free tier.
#### Configuration
<ConfigTabs>
<TabItem value="ui">
1. Navigate to <NavPath path="Settings > Enrichments > Generative AI" />.
- Set **Provider** to `twelvelabs`
- Set **API key** to your TwelveLabs API key (or use an environment variable such as `{FRIGATE_TWELVELABS_API_KEY}`)
- Optionally set **Model** to override the default Marengo model (`marengo3.0`)
</TabItem>
<TabItem value="yaml">
```yaml
genai:
provider: twelvelabs
api_key: "{FRIGATE_TWELVELABS_API_KEY}"
roles:
- embeddings
```
</TabItem>
</ConfigTabs>
+16 -8
View File
@@ -254,7 +254,14 @@ rateLimiter = RateLimiter()
def get_remote_addr(request: Request):
route = list(reversed(request.headers.get("x-forwarded-for").split(",")))
# fall back to the direct TCP peer when no proxy chain is present
direct_addr = request.client.host if request.client else None
forwarded_for = request.headers.get("x-forwarded-for")
if not forwarded_for:
return direct_addr or "127.0.0.1"
route = list(reversed(forwarded_for.split(",")))
logger.debug(f"IP Route: {[r for r in route]}")
trusted_proxies = []
for proxy in request.app.frigate_config.auth.trusted_proxies:
@@ -291,13 +298,8 @@ def get_remote_addr(request: Request):
logger.debug(f"First untrusted IP: {str(ip)}")
return str(ip)
# if there wasn't anything in the route, just return the default
remote_addr = None
if hasattr(request, "remote_addr"):
remote_addr = request.remote_addr
return remote_addr or "127.0.0.1"
# every hop in the route was trusted, so fall back to the direct peer
return direct_addr or "127.0.0.1"
def _cleanup_first_load_seen() -> None:
@@ -416,6 +418,12 @@ def create_encoded_jwt(user, role, expiration, secret):
def set_jwt_cookie(response: Response, cookie_name, encoded_jwt, expiration, secure):
# TODO: ideally this would set secure as well, but that requires TLS
# SameSite is intentionally left unset (browsers default to Lax). Setting
# SameSite=Lax/Strict would stop the cookie from being sent in cross-origin
# iframes, breaking embedded views such as the Home Assistant Frigate card.
# CSRF is instead mitigated by requiring a custom X-CSRF-TOKEN header, which
# cross-origin pages cannot set without a CORS preflight that Frigate never
# grants (see check_csrf in api/fastapi_app.py).
response.set_cookie(
key=cookie_name,
value=encoded_jwt,
+9 -11
View File
@@ -269,12 +269,12 @@ async def no_recordings(
cameras = params.cameras
if cameras != "all":
requested = set(unquote(cameras).split(","))
filtered = requested.intersection(allowed_cameras)
if not filtered:
return JSONResponse(content=[])
cameras = ",".join(filtered)
camera_list = list(requested.intersection(allowed_cameras))
else:
cameras = allowed_cameras
camera_list = list(allowed_cameras)
if not camera_list:
return JSONResponse(content=[])
before = params.before or datetime.datetime.now().timestamp()
after = (
@@ -283,12 +283,10 @@ async def no_recordings(
)
scale = params.scale
clauses = [(Recordings.end_time >= after) & (Recordings.start_time <= before)]
if cameras != "all":
camera_list = cameras.split(",")
clauses.append((Recordings.camera << camera_list))
else:
camera_list = allowed_cameras
clauses = [
(Recordings.end_time >= after) & (Recordings.start_time <= before),
(Recordings.camera << camera_list),
]
# Get recording start times
data: list[Recordings] = (
+45
View File
@@ -13,6 +13,10 @@ from frigate.comms.event_metadata_updater import (
EventMetadataTypeEnum,
)
from frigate.config import CameraConfig, FrigateConfig
from frigate.config.camera.updater import (
CameraConfigUpdateEnum,
CameraConfigUpdateSubscriber,
)
logger = logging.getLogger(__name__)
@@ -29,6 +33,11 @@ class CameraActivityManager:
self.zone_all_object_counts: dict[str, Counter] = {}
self.zone_active_object_counts: dict[str, Counter] = {}
self.all_zone_labels: dict[str, set[str]] = {}
self.config_subscriber = CameraConfigUpdateSubscriber(
config,
config.cameras,
[CameraConfigUpdateEnum.zones, CameraConfigUpdateEnum.objects],
)
for camera_config in config.cameras.values():
if not camera_config.enabled_in_config:
@@ -56,7 +65,40 @@ class CameraActivityManager:
else camera_config.objects.track
)
def __rebuild_zone_labels(self) -> None:
"""Rebuild zone label tracking after a runtime zones/objects change."""
new_zone_labels: dict[str, set[str]] = {}
for camera_config in self.config.cameras.values():
if not camera_config.enabled_in_config or camera_config.name is None:
continue
for zone, zone_config in camera_config.zones.items():
new_zone_labels.setdefault(zone, set()).update(
zone_config.objects
if zone_config.objects
else camera_config.objects.track
)
# drop counters for zones that no longer exist
for zone in list(self.zone_all_object_counts.keys()):
if zone not in new_zone_labels:
self.zone_all_object_counts.pop(zone, None)
self.zone_active_object_counts.pop(zone, None)
# ensure counters exist for new zones so the first count is published
for zone in new_zone_labels:
self.zone_all_object_counts.setdefault(zone, Counter())
self.zone_active_object_counts.setdefault(zone, Counter())
self.all_zone_labels = new_zone_labels
def update_activity(self, new_activity: dict[str, dict[str, Any]]) -> None:
updated_topics = self.config_subscriber.check_for_updates()
if "zones" in updated_topics or "objects" in updated_topics:
self.__rebuild_zone_labels()
all_objects: list[dict[str, Any]] = []
for camera in new_activity.keys():
@@ -161,6 +203,9 @@ class CameraActivityManager:
self.publish(f"{camera}/all", sum(list(all_objects.values())))
self.publish(f"{camera}/all/active", sum(list(active_objects.values())))
def stop(self) -> None:
self.config_subscriber.stop()
class AudioActivityManager:
def __init__(
+2
View File
@@ -397,6 +397,8 @@ class Dispatcher:
comm.publish(topic, payload, retain)
def stop(self) -> None:
self.camera_activity.stop()
for comm in self.comms:
comm.stop()
+1
View File
@@ -15,6 +15,7 @@ class GenAIProviderEnum(str, Enum):
gemini = "gemini"
ollama = "ollama"
llamacpp = "llamacpp"
twelvelabs = "twelvelabs"
class GenAIRoleEnum(str, Enum):
+139
View File
@@ -0,0 +1,139 @@
"""TwelveLabs provider for Frigate AI.
Provides multimodal embeddings for Frigate's semantic search via TwelveLabs'
Marengo model. Marengo produces text and image embeddings in a shared vector
space, so a natural-language query and a stored event thumbnail can be matched
directly — which is exactly what the ``embeddings`` GenAI role drives.
This provider is opt-in: it is only used when a ``genai`` config entry sets
``provider: twelvelabs`` and includes the ``embeddings`` role. It does not
implement the ``descriptions``/``chat`` roles — Marengo is an embedding model,
and TwelveLabs' Pegasus description model operates on whole video clips rather
than the per-frame thumbnails Frigate hands to ``_send``.
Marengo embeddings are 512-dimensional; Frigate's semantic search schema
expects 768 dimensions. The shared :class:`~frigate.embeddings.genai_embedding.GenAIEmbedding`
adapter zero-pads shorter vectors, so the dimension difference is handled
upstream and is consistent for both text and image inputs.
"""
import logging
import numpy as np
import requests
from frigate.config import GenAIProviderEnum
from frigate.genai import GenAIClient, register_genai_provider
logger = logging.getLogger(__name__)
# Default Marengo model. Overridable via the `model` config field.
DEFAULT_MODEL = "marengo3.0"
# Marengo embed REST endpoint. No SDK is needed — this is a plain multipart POST
# made through Frigate's existing `requests` dependency.
EMBED_URL = "https://api.twelvelabs.io/v1.3/embed"
@register_genai_provider(GenAIProviderEnum.twelvelabs)
class TwelveLabsClient(GenAIClient):
"""GenAI client for Frigate using TwelveLabs Marengo embeddings."""
def _init_provider(self):
"""Validate config for the TwelveLabs REST provider.
The provider is just an HTTPS API, so there is no client object to
build — the API key is the only thing required. A non-None sentinel is
returned so the shared ``ensure_provider``/initialization machinery
treats the provider as available.
"""
if not self.genai_config.api_key:
logger.error("TwelveLabs provider requires an api_key.")
return None
return self.genai_config.api_key
@property
def _model(self) -> str:
return self.genai_config.model or DEFAULT_MODEL
def list_models(self) -> list[str]:
"""Marengo is the embedding model exposed by this provider."""
return [DEFAULT_MODEL]
def embed(
self,
texts: list[str] | None = None,
images: list[bytes] | None = None,
) -> list[np.ndarray]:
"""Generate Marengo embeddings for text and/or images.
The TwelveLabs embed API embeds a single input per call, so inputs are
sent one at a time. Returns one 512-dim float32 vector per input, in
order (texts first, then images). The shared GenAIEmbedding adapter
pads these to Frigate's 768-dim search schema.
Calls the Marengo REST endpoint directly via ``requests`` — no SDK.
"""
if self.provider is None:
logger.warning(
"TwelveLabs provider has not been initialized. Check your configuration."
)
return []
results: list[np.ndarray] = []
for text in texts or []:
vector = self._embed_one(text=text)
if vector is not None:
results.append(vector)
for image in images or []:
vector = self._embed_one(image=image)
if vector is not None:
results.append(vector)
return results
def _embed_one(
self, text: str | None = None, image: bytes | None = None
) -> np.ndarray | None:
"""Embed a single text or image input, returning a float32 vector.
Posts a multipart form to the Marengo embed endpoint (``model_name`` plus
either a ``text`` or an ``image_file`` part). The endpoint requires
multipart/form-data, so every field — including text — is passed via
``files`` (the ``(None, value)`` form makes requests emit a multipart
text part). ``self.provider`` holds the validated API key. The 512-dim
vector is at ``<text|image>_embedding.segments[0].float`` in the JSON
response.
"""
headers = {"x-api-key": self.provider}
files: dict = {"model_name": (None, self._model)}
if text is not None:
files["text"] = (None, text)
result_key = "text_embedding"
else:
files["image_file"] = ("image.jpg", image, "image/jpeg")
result_key = "image_embedding"
try:
response = requests.post(
EMBED_URL,
headers=headers,
files=files,
timeout=self.timeout,
)
response.raise_for_status()
result = response.json().get(result_key) or {}
segments = result.get("segments") or []
if not segments:
logger.warning("TwelveLabs returned no embedding for input.")
return None
return np.array(segments[0]["float"], dtype=np.float32)
except Exception as e:
logger.warning("TwelveLabs returned an error: %s", e)
return None
+52
View File
@@ -475,3 +475,55 @@ class TestHttpMedia(BaseTestHttp):
assert response.status_code == 200
assert response.json() == []
def test_recordings_unavailable_cameras_all_scopes_to_allowed_cameras(self):
"""cameras=all must not error and must only consider allowed cameras.
allowed_cameras is mocked to ["front_door"]. A back_door recording that
would otherwise fill the gap must be ignored, and the request must not
500 the way it did when cameras was reassigned to a list.
"""
with AuthTestClient(self.app) as client:
# front_door has a 20s gap (1010-1030).
Recordings.insert(
id="front_a",
path="/media/recordings/front_a.mp4",
camera="front_door",
start_time=1000,
end_time=1010,
duration=10,
motion=0,
).execute()
Recordings.insert(
id="front_b",
path="/media/recordings/front_b.mp4",
camera="front_door",
start_time=1030,
end_time=1040,
duration=10,
motion=0,
).execute()
# back_door is not in allowed_cameras; its full-window coverage must
# not mask the front_door gap.
Recordings.insert(
id="back_a",
path="/media/recordings/back_a.mp4",
camera="back_door",
start_time=1000,
end_time=1040,
duration=40,
motion=0,
).execute()
response = client.get(
"/recordings/unavailable",
params={
"after": 1000,
"before": 1040,
"scale": 5,
"cameras": "all",
},
)
assert response.status_code == 200
assert response.json() == [{"start_time": 1010, "end_time": 1030}]
+138
View File
@@ -0,0 +1,138 @@
"""Tests for the TwelveLabs GenAI provider (Marengo embeddings)."""
import io
import os
import unittest
from unittest.mock import MagicMock, patch
import numpy as np
from frigate.config.camera.genai import (
GenAIConfig,
GenAIProviderEnum,
GenAIRoleEnum,
)
from frigate.genai.plugins.twelvelabs import DEFAULT_MODEL, TwelveLabsClient
def _make_config(model: str = "") -> GenAIConfig:
return GenAIConfig(
provider=GenAIProviderEnum.twelvelabs,
api_key="test-key",
model=model,
roles=[GenAIRoleEnum.embeddings],
)
def _response(key: str, values):
"""Mimic the Marengo REST JSON: ``{<key>: {segments: [{float: [...]}]}}``."""
resp = MagicMock()
resp.raise_for_status.return_value = None
resp.json.return_value = {key: {"segments": [{"float": values}]}}
return resp
class TestTwelveLabsEmbedNoNetwork(unittest.TestCase):
"""Unit tests with ``requests`` mocked — no network access, no SDK."""
def _client(self) -> TwelveLabsClient:
client = TwelveLabsClient.__new__(TwelveLabsClient)
client.genai_config = _make_config()
client.timeout = 120
client.provider = "test-key"
return client
@patch("frigate.genai.plugins.twelvelabs.requests.post")
def test_text_embedding_returns_vector(self, post):
post.return_value = _response("text_embedding", [0.1, 0.2, 0.3])
out = self._client().embed(texts=["a person walking a dog"])
self.assertEqual(len(out), 1)
self.assertIsInstance(out[0], np.ndarray)
self.assertEqual(out[0].dtype, np.float32)
np.testing.assert_allclose(out[0], [0.1, 0.2, 0.3], rtol=1e-6)
_, kwargs = post.call_args
self.assertEqual(kwargs["files"]["model_name"][1], DEFAULT_MODEL)
self.assertEqual(kwargs["files"]["text"][1], "a person walking a dog")
self.assertEqual(kwargs["headers"]["x-api-key"], "test-key")
self.assertNotIn("image_file", kwargs["files"])
@patch("frigate.genai.plugins.twelvelabs.requests.post")
def test_image_embedding_uses_image_file(self, post):
post.return_value = _response("image_embedding", [1.0, 2.0])
out = self._client().embed(images=[b"\xff\xd8\xff jpeg bytes"])
self.assertEqual(len(out), 1)
_, kwargs = post.call_args
self.assertEqual(kwargs["files"]["image_file"][1], b"\xff\xd8\xff jpeg bytes")
self.assertNotIn("text", kwargs["files"])
@patch("frigate.genai.plugins.twelvelabs.requests.post")
def test_custom_model_name_is_used(self, post):
post.return_value = _response("text_embedding", [0.0])
client = self._client()
client.genai_config = _make_config(model="marengo-custom")
client.embed(texts=["x"])
_, kwargs = post.call_args
self.assertEqual(kwargs["files"]["model_name"][1], "marengo-custom")
@patch("frigate.genai.plugins.twelvelabs.requests.post")
def test_empty_segments_are_skipped(self, post):
resp = MagicMock()
resp.raise_for_status.return_value = None
resp.json.return_value = {"text_embedding": {"segments": []}}
post.return_value = resp
self.assertEqual(self._client().embed(texts=["x"]), [])
@patch("frigate.genai.plugins.twelvelabs.requests.post")
def test_api_error_is_swallowed(self, post):
post.side_effect = RuntimeError("boom")
self.assertEqual(self._client().embed(texts=["x"]), [])
def test_no_provider_returns_empty(self):
client = self._client()
client.provider = None
self.assertEqual(client.embed(texts=["x"]), [])
def test_no_inputs_returns_empty(self):
self.assertEqual(self._client().embed(), [])
@unittest.skipUnless(
os.environ.get("TWELVELABS_API_KEY"),
"TWELVELABS_API_KEY not set; skipping live TwelveLabs API test",
)
class TestTwelveLabsEmbedLive(unittest.TestCase):
"""Live smoke test against the real TwelveLabs API (Marengo)."""
def _client(self) -> TwelveLabsClient:
config = _make_config()
config.api_key = os.environ["TWELVELABS_API_KEY"]
return TwelveLabsClient(config)
def test_text_embedding_dim(self):
out = self._client().embed(texts=["a delivery person at the front door"])
self.assertEqual(len(out), 1)
self.assertEqual(out[0].shape, (512,))
def test_image_embedding_dim(self):
from PIL import Image
arr = (np.random.rand(224, 224, 3) * 255).astype("uint8")
buf = io.BytesIO()
Image.fromarray(arr, "RGB").save(buf, format="JPEG")
out = self._client().embed(images=[buf.getvalue()])
self.assertEqual(len(out), 1)
self.assertEqual(out[0].shape, (512,))
if __name__ == "__main__":
unittest.main()
@@ -1263,7 +1263,6 @@ function ObjectDetailsTab({
}
setState("submitted");
setSearch({ ...search, plus_id: "new_upload" });
mutate(
(key) => isEventsKey(key),
(currentData: SearchResult[][] | SearchResult[] | undefined) =>
@@ -1286,7 +1285,7 @@ function ObjectDetailsTab({
);
}
},
[search, mutate, mapSearchResults, setSearch, isEventsKey, t],
[search, mutate, mapSearchResults, isEventsKey, t],
);
const popoverContainerRef = useRef<HTMLDivElement | null>(null);
+10 -2
View File
@@ -47,6 +47,7 @@ type HlsVideoPlayerProps = {
frigateControls?: boolean;
inpointOffset?: number;
onClipEnded?: (currentTime: number) => void;
onClipPrevious?: (diff: number) => void;
onPlayerLoaded?: () => void;
onTimeUpdate?: (time: number) => void;
onPlaying?: () => void;
@@ -74,6 +75,7 @@ export default function HlsVideoPlayer({
frigateControls = true,
inpointOffset = 0,
onClipEnded,
onClipPrevious,
onPlayerLoaded,
onTimeUpdate,
onPlaying,
@@ -339,11 +341,17 @@ export default function HlsVideoPlayer({
onSeek={(diff) => {
const currentTime = videoRef.current?.currentTime;
if (!videoRef.current || !currentTime) {
if (!videoRef.current || currentTime == undefined) {
return;
}
videoRef.current.currentTime = Math.max(0, currentTime + diff);
const newTime = currentTime + diff;
if (newTime < 0 && onClipPrevious) {
onClipPrevious(diff);
} else {
videoRef.current.currentTime = Math.max(0, newTime);
}
}}
onSetPlaybackRate={(rate) => {
setPlaybackRate(rate, true);
@@ -49,6 +49,7 @@ type DynamicVideoPlayerProps = {
onControllerReady: (controller: DynamicVideoController) => void;
onTimestampUpdate?: (timestamp: number) => void;
onClipEnded?: () => void;
onClipPrevious?: (diff: number) => void;
onSeekToTime?: (timestamp: number, play?: boolean) => void;
setFullResolution: React.Dispatch<React.SetStateAction<VideoResolutionType>>;
toggleFullscreen: () => void;
@@ -68,6 +69,7 @@ export default function DynamicVideoPlayer({
onControllerReady,
onTimestampUpdate,
onClipEnded,
onClipPrevious,
onSeekToTime,
setFullResolution,
toggleFullscreen,
@@ -343,6 +345,7 @@ export default function DynamicVideoPlayer({
onTimeUpdate={onTimeUpdate}
onPlayerLoaded={onPlayerLoaded}
onClipEnded={onValidateClipEnd}
onClipPrevious={onClipPrevious}
onSeekToTime={(timestamp, play) => {
if (onSeekToTime) {
onSeekToTime(timestamp, play);
+11
View File
@@ -336,6 +336,16 @@ export function RecordingView({
[currentTimeRange, updateSelectedSegment],
);
const onClipPrevious = useCallback(
(diff: number) => {
manuallySetCurrentTime(
currentTime + diff,
mainControllerRef.current?.isPlaying() ?? false,
);
},
[currentTime, manuallySetCurrentTime],
);
const onShareReviewLink = useCallback(
(timestamp: number) => {
const reviewUrl = createRecordingReviewUrl(location.pathname, {
@@ -902,6 +912,7 @@ export function RecordingView({
);
}}
onClipEnded={onClipEnded}
onClipPrevious={onClipPrevious}
onSeekToTime={manuallySetCurrentTime}
onControllerReady={(controller) => {
mainControllerRef.current = controller;