Early 0.18 work (#22138)

* Update version * Create scaffolding for case management (#21293) * implement case management for export apis (#21295) * refactor vainfo to search for first GPU (#21296) use existing LibvaGpuSelector to pick appropritate libva device * Case management UI (#21299) * Refactor export cards to match existing cards in other UI pages * Show cases separately from exports * Add proper filtering and display of cases * Add ability to edit and select cases for exports * Cleanup typing * Hide if no unassigned * Cleanup hiding logic * fix scrolling * Improve layout * Camera connection quality indicator (#21297) * add camera connection quality metrics and indicator * formatting * move stall calcs to watchdog * clean up * change watchdog to 1s and separately track time for ffmpeg retry_interval * implement status caching to reduce message volume * Export filter UI (#21322) * Get started on export filters * implement basic filter * Implement filtering and adjust api * Improve filter handling * Improve navigation * Cleanup * handle scrolling * Refactor temperature reporting for detectors and implement Hailo temp reading (#21395) * Add Hailo temperature retrieval * Refactor `get_hailo_temps()` to use ctxmanager * Show Hailo temps in system UI * Move hailo_platform import to get_hailo_temps * Refactor temperatures calculations to use within detector block * Adjust webUI to handle new location --------- Co-authored-by: tigattack <10629864+tigattack@users.noreply.github.com> * Camera-specific hwaccel settings for timelapse exports (correct base) (#21386) * added hwaccel_args to camera.record.export config struct * populate camera.record.export.hwaccel_args with a cascade up to camera then global if 'auto' * use new hwaccel args in export * added documentation for camera-specific hwaccel export * fix c/p error * missed an import * fleshed out the docs and comments a bit * ruff lint * separated out the tips in the doc * fix documentation * fix and simplify reference config doc * Add support for GPU and NPU temperatures (#21495) * Add rockchip temps * Add support for GPU and NPU temperatures in the frontend * Add support for Nvidia temperature * Improve separation * Adjust graph scaling * Exports Improvements (#21521) * Add images to case folder view * Add ability to select case in export dialog * Add to mobile review too * Add API to handle deleting recordings (#21520) * Add recording delete API * Re-organize recordings apis * Fix import * Consolidate query types * Add media sync API endpoint (#21526) * add media cleanup functions * add endpoint * remove scheduled sync recordings from cleanup * move to utils dir * tweak import * remove sync_recordings and add config migrator * remove sync_recordings * docs * remove key * clean up docs * docs fix * docs tweak * Media sync API refactor and UI (#21542) * generic job infrastructure * types and dispatcher changes for jobs * save data in memory only for completed jobs * implement media sync job and endpoints * change logs to debug * websocket hook and types * frontend * i18n * docs tweaks * endpoint descriptions * tweak docs * use same logging pattern in sync_recordings as the other sync functions (#21625) * Fix incorrect counting in sync_recordings (#21626) * Update go2rtc to v1.9.13 (#21648) Co-authored-by: Eugeny Tulupov <eugeny.tulupov@spirent.com> * Refactor Time-Lapse Export (#21668) * refactor time lapse creation to be a separate API call with ability to pass arbitrary ffmpeg args * Add CPU fallback * Optimize empty directory cleanup for recordings (#21695) The previous empty directory cleanup did a full recursive directory walk, which can be extremely slow. This new implementation only removes directories which have a chance of being empty due to a recent file deletion. * Implement llama.cpp GenAI Provider (#21690) * Implement llama.cpp GenAI Provider * Add docs * Update links * Fix broken mqtt links * Fix more broken anchors * Remove parents in remove_empty_directories (#21726) The original implementation did a full directory tree walk to find and remove empty directories, so this implementation should remove the parents as well, like the original did. * Implement LLM Chat API with tool calling support (#21731) * Implement initial tools definiton APIs * Add initial chat completion API with tool support * Implement other providers * Cleanup * Offline preview image (#21752) * use latest preview frame for latest image when camera is offline * remove frame extraction logic * tests * frontend * add description to api endpoint * Update to ROCm 7.2.0 (#21753) * Update to ROCm 7.2.0 * ROCm now works properly with JinaV1 * Arcface has compilation error * Add live context tool to LLM (#21754) * Add live context tool * Improve handling of images in request * Improve prompt caching * Add networking options for configuring listening ports (#21779) * feat: add X-Frame-Time when returning snapshot (#21932) Co-authored-by: Florent MORICONI <170678386+fmcloudconsulting@users.noreply.github.com> * Improve jsmpeg player websocket handling (#21943) * improve jsmpeg player websocket handling prevent websocket console messages from appearing when player is destroyed * reformat files after ruff upgrade * Allow API Events to be Detections or Alerts, depending on the Event Label (#21923) * - API created events will be alerts OR detections, depending on the event label, defaulting to alerts - Indefinite API events will extend the recording segment until those events are ended - API event start time is the actual start time, instead of having a pre-buffer of record.event_pre_capture * Instead of checking for indefinite events on a camera before deciding if we should end the segment, only update last_detection_time and last_alert_time if frame_time is greater, which should have the same effect * Add the ability to set a pre_capture number of seconds when creating a manual event via the API. Default behavior unchanged * Remove unnecessary _publish_segment_start() call * Formatting * handle last_alert_time or last_detection_time being None when checking them against the frame_time * comment manual_info["label"].split(": ")[0] for clarity * ffmpeg Preview Segment Optimization for "high" and "very_high" (#21996) * Introduce qmax parameter for ffmpeg preview encoding Added PREVIEW_QMAX_PARAM to control ffmpeg encoding quality. * formatting * Fix spacing in qmax parameters for preview quality * Adapt to new Gemini format * Fix frame time access * Remove exceptions * Cleanup --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> Co-authored-by: tigattack <10629864+tigattack@users.noreply.github.com> Co-authored-by: Andrew Roberts <adroberts@gmail.com> Co-authored-by: Eugeny Tulupov <zhekka3@gmail.com> Co-authored-by: Eugeny Tulupov <eugeny.tulupov@spirent.com> Co-authored-by: John Shaw <1753078+johnshaw@users.noreply.github.com> Co-authored-by: Eric Work <work.eric@gmail.com> Co-authored-by: FL42 <46161216+fl42@users.noreply.github.com> Co-authored-by: Florent MORICONI <170678386+fmcloudconsulting@users.noreply.github.com> Co-authored-by: nulledy <254504350+nulledy@users.noreply.github.com>
2026-10-15 11:12:49 +03:00 · 2026-02-26 21:16:10 -07:00
parent 7df3622243
commit d24b96d3bb
107 changed files with 6766 additions and 1050 deletions
@@ -0,0 +1,21 @@
+"""Generic base class for long-running background jobs."""
+
+from dataclasses import asdict, dataclass, field
+from typing import Any, Optional
+
+
+@dataclass
+class Job:
+    """Base class for long-running background jobs."""
+
+    id: str = field(default_factory=lambda: __import__("uuid").uuid4().__str__()[:12])
+    job_type: str = ""  # Must be set by subclasses
+    status: str = "queued"  # queued, running, success, failed, cancelled
+    results: Optional[dict[str, Any]] = None
+    start_time: Optional[float] = None
+    end_time: Optional[float] = None
+    error_message: Optional[str] = None
+
+    def to_dict(self) -> dict[str, Any]:
+        """Convert to dictionary for WebSocket transmission."""
+        return asdict(self)
@@ -0,0 +1,70 @@
+"""Generic job management for long-running background tasks."""
+
+import threading
+from typing import Optional
+
+from frigate.jobs.job import Job
+from frigate.types import JobStatusTypesEnum
+
+# Global state and locks for enforcing single concurrent job per job type
+_job_locks: dict[str, threading.Lock] = {}
+_current_jobs: dict[str, Optional[Job]] = {}
+# Keep completed jobs for retrieval, keyed by (job_type, job_id)
+_completed_jobs: dict[tuple[str, str], Job] = {}
+
+
+def _get_lock(job_type: str) -> threading.Lock:
+    """Get or create a lock for the specified job type."""
+    if job_type not in _job_locks:
+        _job_locks[job_type] = threading.Lock()
+    return _job_locks[job_type]
+
+
+def set_current_job(job: Job) -> None:
+    """Set the current job for a given job type."""
+    lock = _get_lock(job.job_type)
+    with lock:
+        # Store the previous job if it was completed
+        old_job = _current_jobs.get(job.job_type)
+        if old_job and old_job.status in (
+            JobStatusTypesEnum.success,
+            JobStatusTypesEnum.failed,
+            JobStatusTypesEnum.cancelled,
+        ):
+            _completed_jobs[(job.job_type, old_job.id)] = old_job
+        _current_jobs[job.job_type] = job
+
+
+def clear_current_job(job_type: str, job_id: Optional[str] = None) -> None:
+    """Clear the current job for a given job type, optionally checking the ID."""
+    lock = _get_lock(job_type)
+    with lock:
+        if job_type in _current_jobs:
+            current = _current_jobs[job_type]
+            if current is None or (job_id is None or current.id == job_id):
+                _current_jobs[job_type] = None
+
+
+def get_current_job(job_type: str) -> Optional[Job]:
+    """Get the current running/queued job for a given job type, if any."""
+    lock = _get_lock(job_type)
+    with lock:
+        return _current_jobs.get(job_type)
+
+
+def get_job_by_id(job_type: str, job_id: str) -> Optional[Job]:
+    """Get job by ID. Checks current job first, then completed jobs."""
+    lock = _get_lock(job_type)
+    with lock:
+        # Check if it's the current job
+        current = _current_jobs.get(job_type)
+        if current and current.id == job_id:
+            return current
+        # Check if it's a completed job
+        return _completed_jobs.get((job_type, job_id))
+
+
+def job_is_running(job_type: str) -> bool:
+    """Check if a job of the given type is currently running or queued."""
+    job = get_current_job(job_type)
+    return job is not None and job.status in ("queued", "running")
@@ -0,0 +1,135 @@
+"""Media sync job management with background execution."""
+
+import logging
+import threading
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Optional
+
+from frigate.comms.inter_process import InterProcessRequestor
+from frigate.const import UPDATE_JOB_STATE
+from frigate.jobs.job import Job
+from frigate.jobs.manager import (
+    get_current_job,
+    get_job_by_id,
+    job_is_running,
+    set_current_job,
+)
+from frigate.types import JobStatusTypesEnum
+from frigate.util.media import sync_all_media
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class MediaSyncJob(Job):
+    """In-memory job state for media sync operations."""
+
+    job_type: str = "media_sync"
+    dry_run: bool = False
+    media_types: list[str] = field(default_factory=lambda: ["all"])
+    force: bool = False
+
+
+class MediaSyncRunner(threading.Thread):
+    """Thread-based runner for media sync jobs."""
+
+    def __init__(self, job: MediaSyncJob) -> None:
+        super().__init__(daemon=True, name="media_sync")
+        self.job = job
+        self.requestor = InterProcessRequestor()
+
+    def run(self) -> None:
+        """Execute the media sync job and broadcast status updates."""
+        try:
+            # Update job status to running
+            self.job.status = JobStatusTypesEnum.running
+            self.job.start_time = datetime.now().timestamp()
+            self._broadcast_status()
+
+            # Execute sync with provided parameters
+            logger.debug(
+                f"Starting media sync job {self.job.id}: "
+                f"media_types={self.job.media_types}, "
+                f"dry_run={self.job.dry_run}, "
+                f"force={self.job.force}"
+            )
+
+            results = sync_all_media(
+                dry_run=self.job.dry_run,
+                media_types=self.job.media_types,
+                force=self.job.force,
+            )
+
+            # Store results and mark as complete
+            self.job.results = results.to_dict()
+            self.job.status = JobStatusTypesEnum.success
+            self.job.end_time = datetime.now().timestamp()
+
+            logger.debug(f"Media sync job {self.job.id} completed successfully")
+            self._broadcast_status()
+
+        except Exception as e:
+            logger.error(f"Media sync job {self.job.id} failed: {e}", exc_info=True)
+            self.job.status = JobStatusTypesEnum.failed
+            self.job.error_message = str(e)
+            self.job.end_time = datetime.now().timestamp()
+            self._broadcast_status()
+
+        finally:
+            if self.requestor:
+                self.requestor.stop()
+
+    def _broadcast_status(self) -> None:
+        """Broadcast job status update via IPC to all WebSocket subscribers."""
+        try:
+            self.requestor.send_data(
+                UPDATE_JOB_STATE,
+                self.job.to_dict(),
+            )
+        except Exception as e:
+            logger.warning(f"Failed to broadcast media sync status: {e}")
+
+
+def start_media_sync_job(
+    dry_run: bool = False,
+    media_types: Optional[list[str]] = None,
+    force: bool = False,
+) -> Optional[str]:
+    """Start a new media sync job if none is currently running.
+
+    Returns job ID on success, None if job already running.
+    """
+    # Check if a job is already running
+    if job_is_running("media_sync"):
+        current = get_current_job("media_sync")
+        logger.warning(
+            f"Media sync job {current.id} is already running. Rejecting new request."
+        )
+        return None
+
+    # Create and start new job
+    job = MediaSyncJob(
+        dry_run=dry_run,
+        media_types=media_types or ["all"],
+        force=force,
+    )
+
+    logger.debug(f"Creating new media sync job: {job.id}")
+    set_current_job(job)
+
+    # Start the background runner
+    runner = MediaSyncRunner(job)
+    runner.start()
+
+    return job.id
+
+
+def get_current_media_sync_job() -> Optional[MediaSyncJob]:
+    """Get the current running/queued media sync job, if any."""
+    return get_current_job("media_sync")
+
+
+def get_media_sync_job_by_id(job_id: str) -> Optional[MediaSyncJob]:
+    """Get media sync job by ID. Currently only tracks the current job."""
+    return get_job_by_id("media_sync", job_id)