Compare commits

..
3 Commits
Author SHA1 Message Date
Josh HawkinsandGitHub 4f2a297745 remove all references to degirum in frigate (#23882)
CI / AMD64 Build (push) Waiting to run
CI / ARM Build (push) Waiting to run
CI / Jetson Jetpack 6 (push) Waiting to run
CI / AMD64 Extra Build (push) Blocked by required conditions
CI / ARM Extra Build (push) Blocked by required conditions
CI / Synaptics Build (push) Blocked by required conditions
CI / Assemble and push default build (push) Blocked by required conditions
the company ceased operations on 1 Aug 2026
2026-08-01 08:00:36 -06:00
Josh HawkinsandGitHub b848c90f02 Fix wrong box format passed to cv2.dnn.NMSBoxes (#23876)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
2026-07-31 08:57:23 -05:00
Josh HawkinsandGitHub f1cc0e49d4 Miscellaneous fixes (0.18 beta) (#23873)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s
* improve display of gpu graphs in system metrics

* docs tweaks

* Only hide cameras with ui.dashboard disabled from the All Cameras dashboard

The settings camera selector and zone editor also filtered on ui.dashboard, so hiding a camera from the dashboard made its zones and masks uneditable in the UI (GH 23870). Drop those filters and correct the field title, help text, and reference docs to describe what the option actually does

* hide cameras with ui.review disabled from the Motion tab and the review summaries

The Motion tab built its own camera list that never checked ui.review, so a hidden camera still got a preview tile, and its motion and overlap queries fell back to every allowed camera. The review and recordings summaries had the same gap: they are aggregate day counts that can't be filtered client side, so a hidden camera kept contributing to the severity tab counts and calendar indicators while its items were absent from the list. Filter the motion camera list on ui.review and query all four endpoints with the visible camera list instead of letting the backend default to all, and skip the summary queries until the config resolves so the counts don't briefly render as zero.

* Scope every review page query to the cameras visible in review

The segments and the summary counts were derived from different camera sets: the list was fetched for all cameras and filtered client side, while the summaries were fetched for the visible cameras only when no explicit camera filter was set. A ?cameras= link can name a camera hidden from review, which left the count above zero with an empty list, pinning the new items to review popover open and making the auto refresh effect loop. Intersect an explicit camera selection with the visible list rather than trusting it, pass that to the segment and summary queries alike, and drop the now redundant client side filter, which the raw segments handed to the history view were bypassing anyway.
2026-07-30 17:20:41 -05:00
33 changed files with 503 additions and 1055 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
default_target: local
COMMIT_HASH := $(shell git log -1 --pretty=format:"%h"|tail -1)
VERSION = 0.19.0
VERSION = 0.18.0
IMAGE_REPO ?= ghcr.io/blakeblackshear/frigate
GITHUB_REF_NAME ?= $(shell git rev-parse --abbrev-ref HEAD)
BOARDS= #Initialized empty
-2
View File
@@ -79,7 +79,5 @@ sherpa-onnx==1.12.*
faster-whisper==1.1.*
librosa==0.11.*
soundfile==0.13.*
# DeGirum detector
degirum == 0.16.*
# Memory profiling
memray == 1.15.*
-75
View File
@@ -1269,78 +1269,3 @@ axengine:
input_dtype: int
input_pixel_format: bgr
labelmap_path: /labelmap/coco-80.txt
degirumAiServer:
title: DeGirum AI Server
models:
- key: ai-server-inference
label: AI Server Inference
recommended: true
download: |-
Launch a DeGirum AI server as a Docker container, then point the detector at it. Add this to your `docker-compose.yml`:
```yaml
degirum_detector:
container_name: degirum
image: degirum/aiserver:latest
privileged: true
ports:
- "8778:8778"
```
Set `location` to the server's service name, container name, or `host:port`.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `degirum` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
degirum_detector:
type: degirum
location: degirum
zoo: degirum/public
token: dg_example_token
degirumLocal:
title: DeGirum Local
models:
- key: local-inference
label: Local Inference
recommended: true
download: Run hardware directly inside the Frigate container with `@local`, removing the AI server hop. The matching device runtime (e.g. the Hailo runtime) must be installed in the container; confirm it with `degirum sys-info`.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `@local` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
degirum_detector:
type: degirum
location: @local
zoo: degirum/public
token: dg_example_token
degirumCloud:
title: DeGirum AI Hub Cloud
models:
- key: ai-hub-cloud-inference
label: AI Hub Cloud Inference
recommended: true
download: Run inferences on DeGirum's [AI Hub](https://hub.degirum.com) cloud with `@cloud`. Sign up, create an access token, and set it as `token`. Network latency may require lowering your detection fps.
ui: |
Navigate to **Settings > System > Detectors and model** and select **DeGirum** from the detector type dropdown and click **Add**.
| Field | Value |
| --- | --- |
| **Location** | `@cloud` |
| **Zoo** | `degirum/public` |
| **Token** | your AI Hub token (optional for the public zoo) |
yaml: |
degirum_detector:
type: degirum
location: @cloud
zoo: degirum/public
token: dg_example_token
@@ -981,7 +981,9 @@ cameras:
# Optional: Adjust sort order of cameras in the UI. Larger numbers come later (default: shown below)
# By default the cameras are sorted alphabetically.
order: 0
# Optional: Whether or not to show the camera in the Frigate UI (default: shown below)
# Optional: Whether or not to show the camera on the default All Cameras live dashboard.
# The camera is still available everywhere else, including camera groups and settings
# (default: shown below)
dashboard: True
# Optional: Whether this camera is visible in review (the review page and its camera
# filter, motion review, and the history view) (default: shown below)
+1 -1
View File
@@ -334,7 +334,7 @@ When your browser runs into problems playing back your camera streams, it will l
- **stalled**
- What it means: Playback has stalled because the player has fallen too far behind live (extended buffering or no data arriving).
- What to try: This is usually indicative of the browser struggling to decode too many high-resolution streams at once. Try selecting a lower-bandwidth stream (substream), reduce the number of live streams open, improve the network connection, or lower the camera resolution. Also check your camera's keyframe (I-frame) interval: shorter intervals make playback start and recover faster. You can also try increasing the timeout value in the UI pane of Frigate's settings.
- What to try: This is usually indicative of the browser struggling to decode too many high-resolution streams at once. Try selecting a lower-bandwidth stream (substream), reduce the number of live streams open, improve the network connection, or lower the camera resolution. Also check your camera's keyframe (I-frame) interval: shorter intervals make playback start and recover faster. You can also try increasing the timeout value in <NavPath path="Settings > UI" /> .
- Possible console messages from the player code:
- `Buffer time (10 seconds) exceeded, browser may not be playing media correctly.`
@@ -24,7 +24,6 @@ Frigate supports multiple different detectors that work on different types of ha
- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB, Mini PCIe, and m.2 formats allowing for a wide range of compatibility with devices.
- [Hailo](#hailo-8): The Hailo8 and Hailo8L AI Acceleration module is available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
- <CommunityBadge /> [MemryX](#memryx-mx3): The MX3 Acceleration module is available in m.2 format, offering broad compatibility across various platforms.
- <CommunityBadge /> [DeGirum](#degirum): Service for using hardware devices in the cloud or locally. Hardware and models provided on the cloud on [their website](https://hub.degirum.com).
**AMD**
@@ -755,87 +754,6 @@ Explanation of the parameters:
- **example**: Specifying `output_name = "frigate-{quant}-{input_basename}-{soc}-v{tk_version}"` could result in a model called `frigate-i8-my_model-rk3588-v2.3.0.rknn`.
- `config`: Configuration passed to `rknn-toolkit2` for model conversion. For an explanation of all available parameters have a look at section "2.2. Model configuration" of [this manual](https://github.com/MarcA711/rknn-toolkit2/releases/download/v2.3.2/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.3.2_EN.pdf).
## DeGirum
DeGirum is a detector that can use any type of hardware listed on [their website](https://hub.degirum.com). DeGirum can be used with local hardware through a DeGirum AI Server, or through the use of `@local`. You can also connect directly to DeGirum's AI Hub to run inferences. **Please Note:** This detector _cannot_ be used for commercial purposes.
### Configuration {#configuration-degirum}
#### AI Server Inference
Before starting with the config file for this section, you must first launch an AI server. DeGirum has an AI server ready to use as a docker container. Add this to your `docker-compose.yml` to get started:
```yaml
degirum_detector:
container_name: degirum
image: degirum/aiserver:latest
privileged: true
ports:
- "8778:8778"
```
All supported hardware will automatically be found on your AI server host as long as relevant runtimes and drivers are properly installed on your machine. Refer to [DeGirum's docs site](https://docs.degirum.com/pysdk/runtimes-and-drivers) if you have any trouble.
Once completed, configure the detector as follows:
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumAiServer.models} />
Setting up a model in the `config.yml` is similar to setting up an AI server.
You can set it to:
- A model listed on the [AI Hub](https://hub.degirum.com), given that the correct zoo name is listed in your detector
- If this is what you choose to do, the correct model will be downloaded onto your machine before running.
- A local directory acting as a zoo. See DeGirum's docs site [for more information](https://docs.degirum.com/pysdk/user-guide-pysdk/organizing-models#model-zoo-directory-structure).
- A path to some model.json.
```yaml
model:
path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1 # directory to model .json and file
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
#### Local Inference
It is also possible to eliminate the need for an AI server and run the hardware directly. The benefit of this approach is that you eliminate any bottlenecks that occur when transferring prediction results from the AI server docker container to the frigate one. However, the method of implementing local inference is different for every device and hardware combination, so it's usually more trouble than it's worth. A general guideline to achieve this would be:
1. Ensuring that the frigate docker container has the runtime you want to use. So for instance, running `@local` for Hailo means making sure the container you're using has the Hailo runtime installed.
2. To double check the runtime is detected by the DeGirum detector, make sure the `degirum sys-info` command properly shows whatever runtimes you mean to install.
3. Create a DeGirum detector in your configuration.
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumLocal.models} />
Once `degirum_detector` is setup, you can choose a model through 'model' section in the `config.yml` file.
```yaml
model:
path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
#### AI Hub Cloud Inference
If you do not possess whatever hardware you want to run, there's also the option to run cloud inferences. Do note that your detection fps might need to be lowered as network latency does significantly slow down this method of detection. For use with Frigate, we highly recommend using a local AI server as described above. To set up cloud inferences,
1. Sign up at [DeGirum's AI Hub](https://hub.degirum.com).
2. Get an access token.
3. Create a DeGirum detector in your configuration.
<ModelConfigDropdown detectorTitle="DeGirum" models={objectDetectorsModels.degirumCloud.models} />
Once `degirum_detector` is setup, you can choose a model through 'model' section in the `config.yml` file.
```yaml
model:
path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
width: 300 # width is in the model name as the first number in the "int"x"int" section
height: 300 # height is in the model name as the second number in the "int"x"int" section
input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
```
## AXERA
Hardware accelerated object detection is supported on the following SoCs:
+1 -1
View File
@@ -39,7 +39,7 @@ The per-clip variation is typically quite low and is mostly an artifact of keyfr
Debug Replay lets you re-run Frigate's detection pipeline against a section of recorded video without manually configuring a dummy camera. It automatically extracts the recording, creates a temporary camera with the same detection settings as the original, and loops the clip through the pipeline so you can observe detections in real time.
The replay camera behaves like a live camera feed rather than History's video player: it loops the clip continuously as Frigate analyzes it and has no playback controls, so you cannot pause, scrub, or step through it frame by frame.
The replay camera behaves like a live camera feed rather than History's video player: it loops the clip continuously as Frigate analyzes it and has no playback controls, so you cannot pause, scrub, or step through it frame by frame. The Debug Replay camera does not save recordings or snapshots or surface anything in Explore, but it otherwise behaves like a regular camera, including running enrichments such as Face Recognition, LPR, and custom classification.
Debug Replay isn't intended to be a one-stop pane for all Frigate diagnostics or a comprehensive debugging environment for every Frigate feature. It merely makes it easier to spin up a "dummy camera" and perform some common adjustments in real time. You'll still need to use the normal tools (logs, an MQTT client, etc) to debug your feature.
-38
View File
@@ -2872,44 +2872,6 @@ paths:
- frigateUserAuth: []
x-required-role: any
description: '**Access:** Any authenticated user.'
/categorized_object_names:
get:
tags:
- App
summary: Get known object names by object type
description: |-
**Access:** Any authenticated user.
Returns the sub labels and attributes this install can attach,
grouped by object type. Unlike /sub_labels, which reflects what has already been
detected, this reads the config and model files, so it covers recognized face
names, named license plates, custom object classification categories, and the
detector attributes of tracked objects.
operationId: categorized_object_names_categorized_object_names_get
parameters:
- name: object_type
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
title: Object Type
responses:
'200':
description: Successful Response
content:
application/json:
schema: {}
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- frigateUserAuth: []
x-required-role: any
/audio_labels:
get:
tags:
-23
View File
@@ -71,7 +71,6 @@ from frigate.util.config import (
find_config_file,
redact_credential,
)
from frigate.util.object_names import get_categorized_object_names
from frigate.util.schema import get_config_schema
from frigate.util.services import (
get_nvidia_driver_info,
@@ -1314,28 +1313,6 @@ def get_sub_labels(
return JSONResponse(content=sub_labels)
@router.get(
"/categorized_object_names",
dependencies=[Depends(allow_any_authenticated())],
summary="Get known object names by object type",
description="""Returns the sub labels and attributes this install can attach,
grouped by object type. Unlike /sub_labels, which reflects what has already been
detected, this reads the config and model files, so it covers recognized face
names, named license plates, custom object classification categories, and the
detector attributes of tracked objects.""",
)
def categorized_object_names(
request: Request,
object_type: str | None = None,
allowed_cameras: list[str] = Depends(get_allowed_cameras_for_filter),
):
return JSONResponse(
content=get_categorized_object_names(
request.app.frigate_config, allowed_cameras, object_type
)
)
@router.get("/audio_labels", dependencies=[Depends(allow_any_authenticated())])
def get_audio_labels():
labels = load_labels("/audio-labelmap.txt", prefill=521)
-1
View File
@@ -83,7 +83,6 @@ def require_admin_by_default():
"/nvinfo",
"/labels",
"/sub_labels",
"/categorized_object_names",
"/plus/models",
"/recognized_license_plates",
"/timeline",
+2 -37
View File
@@ -50,7 +50,6 @@ from frigate.jobs.vlm_watch import (
stop_vlm_watch_job,
)
from frigate.models import Event
from frigate.util.object_names import get_categorized_object_names
logger = logging.getLogger(__name__)
@@ -540,13 +539,6 @@ async def execute_tool(
if tool_name == "search_objects":
return await _execute_search_objects(request, arguments, allowed_cameras)
if tool_name == "get_categorized_object_names":
return JSONResponse(
content=_execute_get_categorized_object_names(
request, arguments, allowed_cameras
)
)
if tool_name == "find_similar_objects":
result = await _execute_find_similar_objects(
request, arguments, allowed_cameras
@@ -725,29 +717,6 @@ async def _execute_set_camera_state(
return {"success": True, "camera": camera, "feature": feature, "value": value}
def _execute_get_categorized_object_names(
request: Request,
arguments: dict[str, Any],
allowed_cameras: list[str],
) -> dict[str, Any]:
object_type = arguments.get("object_type") or None
names = get_categorized_object_names(
request.app.frigate_config, allowed_cameras, object_type
)
if not names:
return {
"names": {},
"message": (
f"No names configured for '{object_type}'."
if object_type
else "No names configured; search by label or semantic_query."
),
}
return {"names": names}
async def _execute_tool_internal(
tool_name: str,
arguments: dict[str, Any],
@@ -772,10 +741,6 @@ async def _execute_tool_internal(
except (json.JSONDecodeError, AttributeError) as e:
logger.warning(f"Failed to extract tool result: {e}")
return {"error": "Failed to parse tool result"}
elif tool_name == "get_categorized_object_names":
return _execute_get_categorized_object_names(
request, arguments, allowed_cameras
)
elif tool_name == "find_similar_objects":
return await _execute_find_similar_objects(request, arguments, allowed_cameras)
elif tool_name == "set_camera_state":
@@ -808,8 +773,8 @@ async def _execute_tool_internal(
else:
logger.error(
"Tool call failed: unknown tool %r. Expected one of: search_objects, find_similar_objects, "
"get_categorized_object_names, get_live_context, start_camera_watch, stop_camera_watch, "
"get_profile_status, get_recap. Arguments received: %s",
"get_live_context, start_camera_watch, stop_camera_watch, get_profile_status, get_recap. "
"Arguments received: %s",
tool_name,
json.dumps(arguments),
)
+4 -4
View File
@@ -574,11 +574,11 @@ async def vod_ts(
Recordings.start_time,
)
.where(
Recordings.camera == camera_name,
Recordings.start_time >= start_ts - MAX_SEGMENT_DURATION,
Recordings.start_time <= end_ts,
Recordings.end_time >= start_ts,
Recordings.start_time.between(start_ts, end_ts)
| Recordings.end_time.between(start_ts, end_ts)
| ((start_ts > Recordings.start_time) & (end_ts < Recordings.end_time))
)
.where(Recordings.camera == camera_name)
.order_by(Recordings.start_time.asc())
.iterator()
)
+1 -2
View File
@@ -25,7 +25,7 @@ from frigate.api.defs.query.recordings_query_parameters import (
)
from frigate.api.defs.response.generic_response import GenericResponse
from frigate.api.defs.tags import Tags
from frigate.const import MAX_SEGMENT_DURATION, RECORD_DIR
from frigate.const import RECORD_DIR
from frigate.models import Event, Recordings
from frigate.util.time import get_dst_transitions
@@ -243,7 +243,6 @@ async def recordings(
)
.where(
Recordings.camera == camera_name,
Recordings.start_time >= after - MAX_SEGMENT_DURATION,
Recordings.end_time >= after,
Recordings.start_time <= before,
)
+2 -2
View File
@@ -13,8 +13,8 @@ class CameraUiConfig(FrigateBaseModel):
)
dashboard: bool = Field(
default=True,
title="Show in UI",
description="Toggle whether this camera is visible everywhere in the Frigate UI. Disabling this will require manually editing the config to view this camera in the UI again.",
title="Show on Live dashboard",
description="Toggle whether this camera is visible on the default All Cameras live dashboard. The camera remains available everywhere else in the UI, including camera groups and settings.",
)
review: bool = Field(
default=True,
-157
View File
@@ -1,157 +0,0 @@
import logging
import queue
from typing import Literal
import numpy as np
from pydantic import ConfigDict, Field
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detector_config import BaseDetectorConfig
logger = logging.getLogger(__name__)
DETECTOR_KEY = "degirum"
### DETECTOR CONFIG ###
class DGDetectorConfig(BaseDetectorConfig):
"""DeGirum detector for running models via DeGirum cloud or local inference services."""
model_config = ConfigDict(
title="DeGirum",
)
type: Literal[DETECTOR_KEY]
location: str = Field(
default=None,
title="Inference Location",
description="Location of the DeGirim inference engine (e.g. '@cloud', '127.0.0.1').",
)
zoo: str = Field(
default=None,
title="Model Zoo",
description="Path or URL to the DeGirum model zoo.",
)
token: str = Field(
default=None,
title="DeGirum Cloud Token",
description="Token for DeGirum Cloud access.",
)
### ACTUAL DETECTOR ###
class DGDetector(DetectionApi):
type_key = DETECTOR_KEY
def __init__(self, detector_config: DGDetectorConfig):
try:
import degirum as dg
except ModuleNotFoundError:
raise ImportError("Unable to import DeGirum detector.") from None
self._queue = queue.Queue()
self._zoo = dg.connect(
detector_config.location, detector_config.zoo, detector_config.token
)
logger.debug(f"Models in zoo: {self._zoo.list_models()}")
self.dg_model = self._zoo.load_model(
detector_config.model.path,
)
# Setting input image format to raw reduces preprocessing time
self.dg_model.input_image_format = "RAW"
# Prioritize the most powerful hardware available
self.select_best_device_type()
# Frigate handles pre processing as long as these are all set
input_shape = self.dg_model.input_shape[0]
self.model_height = input_shape[1]
self.model_width = input_shape[2]
# Passing in dummy frame so initial connection latency happens in
# init function and not during actual prediction
frame = np.zeros(
(detector_config.model.width, detector_config.model.height, 3),
dtype=np.uint8,
)
# Pass in frame to overcome first frame latency
self.dg_model(frame)
self.prediction = self.prediction_generator()
def select_best_device_type(self):
"""
Helper function that selects fastest hardware available per model runtime
"""
types = self.dg_model.supported_device_types
device_map = {
"OPENVINO": ["GPU", "NPU", "CPU"],
"HAILORT": ["HAILO8L", "HAILO8"],
"N2X": ["ORCA1", "CPU"],
"ONNX": ["VITIS_NPU", "CPU"],
"RKNN": ["RK3566", "RK3568", "RK3588"],
"TENSORRT": ["DLA", "GPU", "DLA_ONLY"],
"TFLITE": ["ARMNN", "EDGETPU", "CPU"],
}
runtime = types[0].split("/")[0]
# Just create an array of format {runtime}/{hardware} for every hardware
# in the value for appropriate key in device_map
self.dg_model.device_type = [
f"{runtime}/{hardware}" for hardware in device_map[runtime]
]
def prediction_generator(self):
"""
Generator for all incoming frames. By using this generator, we don't have to keep
reconnecting our websocket on every "predict" call.
"""
logger.debug("Prediction generator was called")
with self.dg_model as model:
while 1:
logger.info(f"q size before calling get: {self._queue.qsize()}")
data = self._queue.get(block=True)
logger.info(f"q size after calling get: {self._queue.qsize()}")
logger.debug(
f"Data we're passing into model predict: {data}, shape of data: {data.shape}"
)
result = model.predict(data)
logger.debug(f"Prediction result: {result}")
yield result
def detect_raw(self, tensor_input):
# Reshaping tensor to work with pysdk
truncated_input = tensor_input.reshape(tensor_input.shape[1:])
logger.debug(f"Detect raw was called for tensor input: {tensor_input}")
# add tensor_input to input queue
self._queue.put(truncated_input)
logger.debug(f"Queue size after adding truncated input: {self._queue.qsize()}")
# define empty detection result
detections = np.zeros((20, 6), np.float32)
# grab prediction
res = next(self.prediction)
# If we have an empty prediction, return immediately
if len(res.results) == 0 or len(res.results[0]) == 0:
return detections
i = 0
for result in res.results:
if i >= 20:
break
detections[i] = [
result["category_id"],
float(result["score"]),
result["bbox"][1] / self.model_height,
result["bbox"][0] / self.model_width,
result["bbox"][3] / self.model_height,
result["bbox"][2] / self.model_width,
]
i += 1
logger.debug(f"Detections output: {detections}")
return detections
+2 -1
View File
@@ -9,6 +9,7 @@ from pydantic import ConfigDict, Field
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detector_config import BaseDetectorConfig, ModelTypeEnum
from frigate.util.model import xyxy_to_xywh_for_nms
try:
from tflite_runtime.interpreter import Interpreter, load_delegate
@@ -297,7 +298,7 @@ class EdgeTpuTfl(DetectionApi):
# until after filtering out redundant boxes
# Shift the logit scores to be non-negative (required by cv2)
indices = cv2.dnn.NMSBoxes(
bboxes=boxes_filtered_decoded,
bboxes=xyxy_to_xywh_for_nms(boxes_filtered_decoded),
scores=max_scores_filtered_shiftedpositive,
score_threshold=(
self.min_logit_value + self.logit_shift_to_positive_values
+3 -2
View File
@@ -17,6 +17,7 @@ from frigate.detectors.detector_config import (
ModelTypeEnum,
)
from frigate.util.file import FileLock
from frigate.util.model import xyxy_to_xywh_for_nms
logger = logging.getLogger(__name__)
@@ -581,7 +582,7 @@ class MemryXDetector(DetectionApi):
# Convert coordinates to integers
x_min, y_min, x_max, y_max = map(int, [x_min, y_min, x_max, y_max])
# Append valid detections [class_id, confidence, x, y, width, height]
# Append valid detections [class_id, confidence, x_min, y_min, x_max, y_max]
detections.append([class_id, confidence, x_min, y_min, x_max, y_max])
final_detections = np.zeros((20, 6), np.float32)
@@ -595,7 +596,7 @@ class MemryXDetector(DetectionApi):
detections = np.array(detections, dtype=np.float32)
# Apply Non-Maximum Suppression (NMS)
bboxes = detections[:, 2:6].tolist() # (x_min, y_min, width, height)
bboxes = xyxy_to_xywh_for_nms(detections[:, 2:6])
scores = detections[:, 1].tolist() # Confidence scores
indices = cv2.dnn.NMSBoxes(bboxes, scores, 0.45, 0.5)
+2 -2
View File
@@ -12,7 +12,7 @@ from frigate.const import MODEL_CACHE_DIR, SUPPORTED_RK_SOCS
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detection_runners import RKNNModelRunner
from frigate.detectors.detector_config import BaseDetectorConfig, ModelTypeEnum
from frigate.util.model import post_process_yolo
from frigate.util.model import post_process_yolo, xyxy_to_xywh_for_nms
from frigate.util.rknn_converter import auto_convert_model
logger = logging.getLogger(__name__)
@@ -285,7 +285,7 @@ class Rknn(DetectionApi):
# run nms
indices = cv2.dnn.NMSBoxes(
bboxes=boxes,
bboxes=xyxy_to_xywh_for_nms(boxes),
scores=scores,
score_threshold=0.4,
nms_threshold=0.4,
+118 -66
View File
@@ -262,10 +262,6 @@ def get_tool_definitions(
`attribute` parameter is exposed for filtering by their labels. When the
embeddings model only understands English (JinaV1), the `semantic_query`
description instructs the model to write the query in English.
Descriptions here stay mechanical: which tool to reach for, and how the
filters relate to each other, is stated once in the system prompt so the
guidance is not paid for twice on every request.
"""
search_objects_properties: dict[str, Any] = {
"camera": {
@@ -274,13 +270,26 @@ def get_tool_definitions(
},
"label": {
"type": "string",
"description": "Tracked object class to filter by.",
"description": (
"Generic object class to filter by — one of the tracked detector "
"labels such as 'person', 'package', 'car', 'dog', 'bird'. Use "
"this for broad queries like 'show me all cars today'. Combine "
"with semantic_query when the user also describes appearance or "
"behavior (e.g. label='person', semantic_query='riding a lawn "
"mower')."
),
},
"sub_label": {
"type": "string",
"description": (
"Name recognized in the detection: a person, delivery company, "
"animal species or breed, or license plate."
"Filter by a DISCRETE NAMED entity recognized in the detection. "
"Use this for: a known person's name ('John'), a delivery "
"company ('Amazon', 'UPS'), a recognized animal species or "
"breed ('blue jay', 'cardinal', 'golden retriever'), or a "
"license plate string. When filtering by a specific name, set "
"only sub_label and leave label unset. Do NOT use sub_label "
"for descriptions of appearance, clothing, or actions — those "
"belong in semantic_query."
),
},
"after": {
@@ -304,11 +313,20 @@ def get_tool_definitions(
}
if attribute_classifications:
model_outline = "; ".join(
f"{m['name']} (applies to {', '.join(m['objects']) or 'any object'})"
for m in attribute_classifications
)
search_objects_properties["attribute"] = {
"type": "string",
"description": (
"Attribute label produced by a configured classification model "
"(case-sensitive)."
"Filter by a classification attribute label produced by a "
"configured attribute classification model. Use this INSTEAD "
"of semantic_query when the user's request matches one of "
"these classifications. Configured models: "
f"{model_outline}. "
"Set the value to the attribute label that matches the user's "
"phrasing (case-sensitive)."
),
}
@@ -316,12 +334,29 @@ def get_tool_definitions(
search_objects_properties["semantic_query"] = {
"type": "string",
"description": (
"Description of an appearance or activity, used to semantically "
"narrow results."
"Optional natural-language description of a PHYSICAL "
"CHARACTERISTIC, APPEARANCE, or ACTIVITY the user mentioned, "
"used to semantically narrow results. Only set this when the "
"user describes something beyond what label and sub_label can "
"express on their own.\n"
"USE for descriptive phrases like: 'riding a lawn mower', "
"'wearing a red jacket', 'carrying a package', 'walking a "
"dog', 'on a bicycle', 'holding an umbrella'.\n"
"DO NOT USE for:\n"
"- specific named people, pets, or delivery companies → use sub_label\n"
"- animal species or breed names like 'blue jay', 'cardinal', "
"'golden retriever' → use sub_label\n"
"- license plate strings → use sub_label\n"
"- generic object queries like 'all cars today' or 'every "
"person' → use label alone with no semantic_query\n"
"When set, combine with label/time/camera/zone filters as "
"usual (e.g. label='person', semantic_query='riding a lawn "
"mower', after='2024-05-01T00:00:00Z')."
+ (
" The configured embeddings model only understands English, so "
"always write this in English, translating the user's "
"description if they phrased it in another language."
" The configured embeddings model only understands "
"English, so always write semantic_query in English, "
"translating the user's description if they phrased it "
"in another language."
if embeddings_language == "english"
else ""
)
@@ -329,9 +364,25 @@ def get_tool_definitions(
}
search_objects_description = (
"Search the historical record of tracked detections. Use this ONLY for "
"questions about the PAST, e.g. 'did anyone come by today?', 'when was the "
"last car?'. For alerting on future events use start_camera_watch instead."
"Search the historical record of detected objects in Frigate. "
"Use this ONLY for questions about the PAST e.g. 'did anyone come by today?', "
"'when was the last car?', 'show me detections from yesterday'. "
"Do NOT use this for monitoring or alerting requests about future events — "
"use start_camera_watch instead for those. "
"An 'object' in Frigate represents a tracked detection (e.g., a person, package, car).\n\n"
"Choose filters based on what the user is asking for:\n"
"- Generic class query ('show me all cars today'): set `label` only.\n"
"- Specific NAMED entity (known person, delivery company, animal "
"species/breed like 'blue jay' or 'golden retriever', license "
"plate): set `sub_label` only and leave `label` unset.\n"
)
if semantic_search_enabled:
search_objects_description += (
"- Physical CHARACTERISTIC, APPEARANCE, or ACTIVITY that is not a "
"discrete name ('person riding a lawn mower', 'someone in a red "
"jacket', 'person carrying a package'): set `semantic_query` with "
"the descriptive phrase, optionally alongside `label` for the "
"object class. Do NOT put descriptive phrases in sub_label."
)
return [
@@ -347,34 +398,20 @@ def get_tool_definitions(
"required": [],
},
},
{
"type": "function",
"function": {
"name": "get_categorized_object_names",
"description": (
"Every name that can be attached as a sub_label, grouped by object "
"type: recognized faces, named license plates, classification "
"categories, and delivery logos."
),
"parameters": {
"type": "object",
"properties": {
"object_type": {
"type": "string",
"description": "Optional object label (e.g. 'person', 'car'). Omit for all.",
},
},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "find_similar_objects",
"description": (
"Find tracked objects visually and semantically similar to a "
"specific past event. Requires semantic search to be enabled."
"Find tracked objects that are visually and semantically similar "
"to a specific past event. Use this when the user references a "
"particular object they have seen and wants to find other "
"sightings of the same or similar one ('that green car', 'the "
"person in the red jacket', 'the package that was delivered'). "
"Prefer this over search_objects whenever the user's intent is "
"'find more like this specific one.' Use search_objects first "
"only if you need to locate the anchor event. Requires semantic "
"search to be enabled."
),
"parameters": {
"type": "object",
@@ -436,8 +473,9 @@ def get_tool_definitions(
"function": {
"name": "set_camera_state",
"description": (
"Change a camera's feature state, e.g. turn detection on or off. "
"Only call this when the user explicitly asks to change a setting. "
"Change a camera's feature state (e.g., turn detection on/off, enable/disable recordings). "
"Use camera='*' to apply to all cameras at once. "
"Only call this tool when the user explicitly asks to change a camera setting. "
"Requires admin privileges."
),
"parameters": {
@@ -479,7 +517,7 @@ def get_tool_definitions(
},
"value": {
"type": "string",
"description": "The value to set, as accepted by the chosen feature.",
"description": "The value to set. ON or OFF for toggles, a number for thresholds, a profile name or 'none' for profile.",
},
},
"required": ["camera", "feature", "value"],
@@ -491,9 +529,11 @@ def get_tool_definitions(
"function": {
"name": "get_live_context",
"description": (
"Current live image and detections (tracked objects, zones, "
"timestamps) for one camera. Use this for questions about what is "
"happening right now. Call it again for each additional camera."
"Get the current live image and detection information for a single camera: objects being tracked, "
"zones, timestamps. Use this to understand what is visible in the live view. "
"Call this when answering questions about what is happening right now on a specific camera. "
"Operates on one camera at a time; call the tool again for each additional camera. "
"Wildcards and empty values are not accepted."
),
"parameters": {
"type": "object",
@@ -501,8 +541,8 @@ def get_tool_definitions(
"camera": {
"type": "string",
"description": (
"Exact name of a single camera. Wildcards (e.g. '*', "
"'all') and empty strings are not accepted."
"Exact name of a single camera to get live context for. "
"Wildcards (e.g. '*', 'all') and empty strings are not accepted."
),
},
},
@@ -515,9 +555,10 @@ def get_tool_definitions(
"function": {
"name": "start_camera_watch",
"description": (
"Start a continuous watch job that monitors a camera and notifies "
"the user when a condition is met, e.g. 'tell me when guests "
"arrive'. Only one watch job can run at a time. Returns a job ID."
"Start a continuous VLM watch job that monitors a camera and sends a notification "
"when a specified condition is met. Use this when the user wants to be alerted about "
"a future event, e.g. 'tell me when guests arrive' or 'notify me when the package is picked up'. "
"Only one watch job can run at a time. Returns a job ID."
),
"parameters": {
"type": "object",
@@ -557,7 +598,10 @@ def get_tool_definitions(
"type": "function",
"function": {
"name": "stop_camera_watch",
"description": "Cancel the currently running watch job.",
"description": (
"Cancel the currently running VLM watch job. Use this when the user wants to "
"stop a previously started watch, e.g. 'stop watching the front door'."
),
"parameters": {
"type": "object",
"properties": {},
@@ -570,9 +614,11 @@ def get_tool_definitions(
"function": {
"name": "get_profile_status",
"description": (
"Get the active profile and when each profile was last activated. "
"Call this before get_recap to derive the time window for requests "
"like 'what happened while I was away?'."
"Get the current profile status including the active profile and "
"timestamps of when each profile was last activated. Use this to "
"determine time periods for recap requests — e.g. when the user asks "
"'what happened while I was away?', call this first to find the relevant "
"time window based on profile activation history."
),
"parameters": {
"type": "object",
@@ -586,9 +632,11 @@ def get_tool_definitions(
"function": {
"name": "get_recap",
"description": (
"Get all activity (alerts and detections) for a time period, as a "
"chronological list with camera, objects, zones, and descriptions "
"when available. Summarize the results for the user."
"Get a recap of all activity (alerts and detections) for a given time period. "
"Use this after calling get_profile_status to retrieve what happened during "
"a specific window — e.g. 'what happened while I was away?'. Returns a "
"chronological list of activity with camera, objects, zones, and GenAI-generated "
"descriptions when available. Summarize the results for the user."
),
"parameters": {
"type": "object",
@@ -675,13 +723,14 @@ def build_chat_system_prompt(
)
speed_units_section = f"\n\nReport object speeds to the user in {speed_unit}."
filter_routing_section = (
semantic_search_section = ""
if semantic_search_enabled:
semantic_search_section = (
"\n\nWhen routing a search_objects call, pick filters by the shape of the user's request:\n"
"- Generic class ('show me all cars today'): set `label` only.\n"
"- Specific named entity — a known person ('John'), delivery company ('Amazon'), animal species/breed ('blue jay', 'golden retriever'), or license plate: set `sub_label` only and leave `label` unset. Call get_categorized_object_names first and use the exact spelling it returns; a guessed spelling matches nothing. If the name is absent, say it is not configured rather than searching for it."
"- Specific named entity — a known person ('John'), delivery company ('Amazon'), animal species/breed ('blue jay', 'cardinal', 'golden retriever'), or license plate: set `sub_label` only and leave `label` unset.\n"
"- Physical characteristic, appearance, or activity that is NOT a discrete name ('find me people riding a lawn mower', 'someone in a red jacket', 'a person carrying a package'): set `semantic_query` with the descriptive phrase, optionally combined with `label` for the object class. Never put descriptive phrases in `sub_label`."
)
if semantic_search_enabled:
filter_routing_section += "\n- Physical characteristic, appearance, or activity that is NOT a discrete name ('riding a lawn mower', 'someone in a red jacket'): set `semantic_query` with the descriptive phrase, optionally combined with `label`. Never put descriptive phrases in `sub_label`."
attribute_classification_section = ""
if attribute_classifications:
@@ -690,9 +739,9 @@ def build_chat_system_prompt(
for m in attribute_classifications
)
attribute_classification_section = (
"\n\nConfigured attribute classification models:\n"
"\n\nAttribute classification models are configured for the following object types:\n"
f"{model_lines}\n"
"When the user's request matches one of these classifications, set the search_objects `attribute` field to the matching label (case-sensitive) rather than using `semantic_query`. Reserve `semantic_query` for descriptive phrases outside the configured attribute labels."
"When the user's request matches one of these classifications, set the search_objects `attribute` field to the matching label rather than using `semantic_query`. Reserve `semantic_query` for descriptive phrases that fall outside the configured attribute labels."
)
return f"""You are a helpful assistant for Frigate, a security camera NVR system. You help users answer questions about their cameras, detected objects, and events.
@@ -701,6 +750,9 @@ Current server local date and time: {current_date_str} at {current_time_str}
Do not start your response with phrases like "I will check...", "Let me see...", or "Let me look...". Answer directly.
Always present times in the server's local timezone. When tool results include start_time_local and end_time_local, quote those strings exactly; never convert or invent timestamps, and fall back to UTC or ISO format only when a result has no local time fields. Resolve relative dates like "today" or "this week" against the current date above, and pass dates to tools in ISO 8601 (e.g. {current_date_str}T00:00:00Z for the start of today).
Always present times to the user in the server's local timezone. When tool results include start_time_local and end_time_local, use those exact strings when listing or describing detection times—do not convert or invent timestamps. Do not use UTC or ISO format with Z for the user-facing answer unless the tool result only provides Unix timestamps without local time fields.
When users ask about "today", "yesterday", "this week", etc., use the current date above as reference.
When searching for objects or events, use ISO 8601 format for dates (e.g., {current_date_str}T00:00:00Z for the start of today).
Always be accurate with time calculations based on the current date provided.
When the user refers to a specific object they have seen ("that green car", "the person in the red jacket", "a package left today"), prefer find_similar_objects over search_objects, using search_objects only to locate the anchor event and passing its id along. Keep search_objects for generic queries like "show me all cars today". If a user message begins with [attached_event:<id>], treat that id as the anchor for any similarity or "tell me more" request in the same message.{filter_routing_section}{attribute_classification_section}{cameras_section}{speed_units_section}"""
When a user refers to a specific object they have seen or describe with identifying details ("that green car", "the person in the red jacket", "a package left today"), prefer the find_similar_objects tool over search_objects. Use search_objects first only to locate the anchor event, then pass its id to find_similar_objects. For generic queries like "show me all cars today", keep using search_objects. If a user message begins with [attached_event:<id>], treat that event id as the anchor for any similarity or "tell me more" request in the same message and call find_similar_objects with that id.{semantic_search_section}{attribute_classification_section}{cameras_section}{speed_units_section}"""
-174
View File
@@ -1,73 +1,15 @@
"""Unit tests for recordings/media API endpoints."""
from dataclasses import dataclass
from datetime import UTC, datetime
from unittest.mock import patch
import pytz
from fastapi import Request
from frigate.api.auth import get_allowed_cameras_for_filter, get_current_user
from frigate.const import MAX_SEGMENT_DURATION
from frigate.models import Recordings
from frigate.test.http_api.base_http_test import AuthTestClient, BaseTestHttp
@dataclass(frozen=True)
class RangeCase:
"""Expected behavior for one segment relative to the requested range.
Offsets are seconds from REQUEST_START; the request ends at +100 seconds.
"""
name: str
start_offset: float
end_offset: float
included_in_recordings: bool
vod_clip_from_ms: int | None = None
vod_duration_ms: int | None = None
REQUEST_START = 1000
REQUEST_END = 1100
RANGE_CASES = (
RangeCase("before", -MAX_SEGMENT_DURATION + 1, -1, False),
RangeCase("meets_start", -10, 0, True),
RangeCase(
"overlaps_start",
-MAX_SEGMENT_DURATION + 0.5,
0.25,
True,
vod_clip_from_ms=599500,
vod_duration_ms=250,
),
RangeCase("starts_at_start", 0, 10, True, vod_duration_ms=10000),
RangeCase("inside", 20, 80, True, vod_duration_ms=60000),
RangeCase("ends_at_end", 90, 100, True, vod_duration_ms=10000),
RangeCase("matches_range", 0, 100, True, vod_duration_ms=100000),
RangeCase("starts_with_range", 0, 110, True, vod_duration_ms=100000),
RangeCase(
"covers_range",
-20,
120,
True,
vod_clip_from_ms=20000,
vod_duration_ms=100000,
),
RangeCase(
"ends_with_range",
-10,
100,
True,
vod_clip_from_ms=10000,
vod_duration_ms=100000,
),
RangeCase("overlaps_end", 95, 105, True, vod_duration_ms=5000),
RangeCase("starts_at_end", 100, 110, True),
RangeCase("after", 101, 110, False),
)
class TestHttpMedia(BaseTestHttp):
"""Test media API endpoints, particularly recordings with DST handling."""
@@ -102,26 +44,6 @@ class TestHttpMedia(BaseTestHttp):
self.app.dependency_overrides.clear()
super().tearDown()
def _assert_vod_response(
self,
response,
expected_clips: list[tuple[str, int | None, int]],
) -> None:
"""Assert VOD clip metadata and its derived duration fields."""
assert response.status_code == 200
vod = response.json()
assert [
(
clip["path"],
clip.get("clipFrom"),
clip["keyFrameDurations"][0],
)
for clip in vod["sequences"][0]["clips"]
] == expected_clips
expected_durations = [clip[2] for clip in expected_clips]
assert vod["durations"] == expected_durations
assert vod["segment_duration"] == max(expected_durations)
def test_recordings_summary_across_dst_spring_forward(self):
"""
Test recordings summary across spring DST transition (spring forward).
@@ -482,102 +404,6 @@ class TestHttpMedia(BaseTestHttp):
assert "2024-03-10" in summary
assert summary["2024-03-10"] is True
def test_recordings_handles_all_range_relations(self):
"""Recordings return every interval relation that touches the range."""
with AuthTestClient(self.app) as client:
for case in RANGE_CASES:
with self.subTest(case=case.name):
Recordings.delete().execute()
super().insert_mock_recording(
case.name,
REQUEST_START + case.start_offset,
REQUEST_START + case.end_offset,
)
response = client.get(
"/front_door/recordings",
params={"after": REQUEST_START, "before": REQUEST_END},
)
assert response.status_code == 200
expected_ids = [case.name] if case.included_in_recordings else []
assert [
recording["id"] for recording in response.json()
] == expected_ids
def test_vod_handles_all_range_relations(self):
"""VOD clips every interval relation with positive playback duration."""
with (
AuthTestClient(self.app) as client,
patch(
"frigate.api.media.get_keyframe_before",
side_effect=lambda _path, offset: offset,
),
):
for case in RANGE_CASES:
with self.subTest(case=case.name):
Recordings.delete().execute()
super().insert_mock_recording(
case.name,
REQUEST_START + case.start_offset,
REQUEST_START + case.end_offset,
)
response = client.get(
f"/vod/front_door/start/{REQUEST_START}/end/{REQUEST_END}"
)
if case.vod_duration_ms is None:
assert response.status_code == 404
continue
self._assert_vod_response(
response,
[
(
case.name,
case.vod_clip_from_ms,
case.vod_duration_ms,
)
],
)
def test_vod_handles_segment_ending_at_start_with_keyframe_fallbacks(self):
"""VOD keeps a boundary segment when keyframe lookup extends it."""
def keyframe_before(path: str, offset: int) -> int | None:
return offset - 1000 if path == "previous_keyframe" else None
with (
AuthTestClient(self.app) as client,
patch(
"frigate.api.media.get_keyframe_before",
side_effect=keyframe_before,
),
):
super().insert_mock_recording(
"previous_keyframe",
REQUEST_START - 10,
REQUEST_START,
)
super().insert_mock_recording(
"missing_keyframe",
REQUEST_START - 5,
REQUEST_START,
)
response = client.get(
f"/vod/front_door/start/{REQUEST_START}/end/{REQUEST_END}"
)
self._assert_vod_response(
response,
[
("previous_keyframe", 9000, 1000),
("missing_keyframe", None, 5000),
],
)
def test_recordings_unavailable_reports_gap_between_recordings(self):
"""A gap between two recordings is reported as an unavailable segment."""
with AuthTestClient(self.app) as client:
+1 -88
View File
@@ -1,12 +1,7 @@
import unittest
from io import StringIO
from unittest.mock import MagicMock, patch
from frigate.util.services import (
get_amd_gpu_stats,
get_intel_gpu_stats,
get_openvino_npu_stats,
)
from frigate.util.services import get_amd_gpu_stats, get_intel_gpu_stats
class TestGpuStats(unittest.TestCase):
@@ -22,88 +17,6 @@ class TestGpuStats(unittest.TestCase):
amd_stats = get_amd_gpu_stats()
assert amd_stats == {"gpu": "4.17%", "mem": "60.37%"}
@patch("frigate.util.services.time.sleep")
@patch("frigate.util.services.time.time", side_effect=[0.0, 1.0])
@patch(
"frigate.util.services.os.readlink",
return_value="/sys/bus/pci/drivers/intel_vpu",
)
@patch(
"frigate.util.services.glob.glob",
return_value=["/sys/class/accel/accel0"],
)
@patch(
"builtins.open",
side_effect=[StringIO("1000"), StringIO("1250")],
)
def test_openvino_npu_stats_discovers_accel0(
self, open_file, glob, readlink, time, sleep
):
assert get_openvino_npu_stats() == {"npu": "25.0", "mem": "-%"}
open_file.assert_any_call(
"/sys/class/accel/accel0/device/power/runtime_active_time"
)
@patch("frigate.util.services.time.sleep")
@patch("frigate.util.services.time.time", side_effect=[0.0, 1.0])
@patch(
"frigate.util.services.os.readlink",
side_effect=[
"/sys/bus/pci/drivers/other",
"/sys/bus/pci/drivers/intel_vpu",
],
)
@patch(
"frigate.util.services.glob.glob",
return_value=[
"/sys/class/accel/accel0",
"/sys/class/accel/accel1",
],
)
@patch(
"builtins.open",
side_effect=[StringIO("1000"), StringIO("1250")],
)
def test_openvino_npu_stats_skips_non_intel_accelerator(
self, open_file, glob, readlink, time, sleep
):
assert get_openvino_npu_stats() == {"npu": "25.0", "mem": "-%"}
open_file.assert_any_call(
"/sys/class/accel/accel1/device/power/runtime_active_time"
)
@patch(
"frigate.util.services.os.readlink",
return_value="/sys/bus/pci/drivers/other",
)
@patch(
"frigate.util.services.glob.glob",
return_value=["/sys/class/accel/accel0"],
)
@patch("builtins.open")
def test_openvino_npu_stats_no_intel_accelerator(self, open_file, glob, readlink):
assert get_openvino_npu_stats() is None
open_file.assert_not_called()
@patch(
"frigate.util.services.os.readlink",
return_value="/sys/bus/pci/drivers/intel_vpu",
)
@patch(
"frigate.util.services.glob.glob",
return_value=["/sys/class/accel/accel0"],
)
@patch("builtins.open", side_effect=FileNotFoundError)
def test_openvino_npu_stats_runtime_counter_unavailable(
self, open_file, glob, readlink
):
assert get_openvino_npu_stats() is None
open_file.assert_called_once_with(
"/sys/class/accel/accel0/device/power/runtime_active_time"
)
@patch("frigate.stats.intel_gpu_info.intel_gpu_name_resolver.get_names")
@patch("frigate.util.services.time.sleep")
@patch("frigate.util.services.time.monotonic")
+226
View File
@@ -0,0 +1,226 @@
"""Tests for detector post-processing NMS box format handling.
cv2.dnn.NMSBoxes expects boxes as [x, y, width, height]. Passing corner
coordinates [x1, y1, x2, y2] makes OpenCV treat x2/y2 as width/height,
inflating every box toward the bottom-right by its distance from the origin.
Two well separated objects far from the origin then appear to overlap and the
lower scoring one is silently suppressed.
The regression geometry used throughout: two boxes with zero true overlap,
A = (393, 499, 484, 620) and B = (527, 499, 618, 620) in a 640x640 input
(43 px gap). Misread as [x, y, w, h] their IoU is 0.465, above the 0.4 NMS
threshold, so the buggy format drops the lower scoring box while correct
conversion keeps both.
"""
import math
import unittest
from queue import Queue
import numpy as np
from frigate.detectors.plugins.memryx import MemryXDetector
from frigate.util.model import (
post_process_dfine,
post_process_rfdetr,
post_process_yolo,
post_process_yolox,
)
WIDTH = 640
HEIGHT = 640
# box A: xyxy (393, 499, 484, 620) as center format
A_CX, A_CY, A_W, A_H = 438.5, 559.5, 91.0, 121.0
# box B: xyxy (527, 499, 618, 620) as center format
B_CX, B_CY, B_W, B_H = 572.5, 559.5, 91.0, 121.0
# expected normalized output rows: [class_id, conf, y1, x1, y2, x2]
A_ROW = [499 / 640, 393 / 640, 620 / 640, 484 / 640]
B_ROW = [499 / 640, 527 / 640, 620 / 640, 618 / 640]
def kept(detections: np.ndarray) -> np.ndarray:
"""Rows of the padded (20, 6) output that hold real detections."""
return detections[detections[:, 1] > 0]
class TestYoloNmsPostProcess(unittest.TestCase):
def _single_output(self, rows: list[list[float]]) -> list[np.ndarray]:
"""Build a single-tensor YOLO output (1, attrs, anchors) from
[cx, cy, w, h, class scores...] rows, padded with empty anchors."""
anchors = np.zeros((10, len(rows[0])), dtype=np.float32)
anchors[: len(rows)] = np.array(rows, dtype=np.float32)
return [anchors.T[np.newaxis, ...]]
def test_keeps_separated_objects_far_from_origin(self):
output = self._single_output(
[
[A_CX, A_CY, A_W, A_H, 0.90, 0.0],
[B_CX, B_CY, B_W, B_H, 0.0, 0.85],
]
)
detections = kept(post_process_yolo(output, WIDTH, HEIGHT))
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(detections[0], [0, 0.90, *A_ROW], atol=2e-3)
np.testing.assert_allclose(detections[1], [1, 0.85, *B_ROW], atol=2e-3)
def test_still_suppresses_true_duplicates(self):
# same object twice, shifted 4 px: true IoU 0.92, must dedupe to one
output = self._single_output(
[
[A_CX, A_CY, A_W, A_H, 0.90, 0.0],
[A_CX + 4, A_CY, A_W, A_H, 0.85, 0.0],
]
)
detections = kept(post_process_yolo(output, WIDTH, HEIGHT))
self.assertEqual(len(detections), 1)
np.testing.assert_allclose(detections[0], [0, 0.90, *A_ROW], atol=2e-3)
class TestMultipartYoloPostProcess(unittest.TestCase):
def _multipart_output(self) -> list[np.ndarray]:
"""Build a 3-scale anchor-based YOLO output containing boxes A and B,
both decoded through anchor 0 of the stride-32 scale."""
outputs = [
np.zeros((1, 255, 80, 80), dtype=np.float32),
np.zeros((1, 255, 40, 40), dtype=np.float32),
np.zeros((1, 255, 20, 20), dtype=np.float32),
]
stride, (anchor_w, anchor_h) = 32, (142, 110)
for cx, cy, w, h, conf, class_channel in [
(A_CX, A_CY, A_W, A_H, 0.95, 5), # class 0
(B_CX, B_CY, B_W, B_H, 0.90, 6), # class 1
]:
cell_x, cell_y = int(cx // stride), int(cy // stride)
dx = (cx / stride - cell_x + 0.5) / 2
dy = (cy / stride - cell_y + 0.5) / 2
dw = math.sqrt(w / anchor_w) / 2
dh = math.sqrt(h / anchor_h) / 2
# anchor 0 occupies channels 0-84 of the 255 channel tensor
outputs[2][0, 0:4, cell_y, cell_x] = [dx, dy, dw, dh]
outputs[2][0, 4, cell_y, cell_x] = conf
outputs[2][0, class_channel, cell_y, cell_x] = 1.0
return outputs
def test_keeps_separated_objects_far_from_origin(self):
detections = kept(post_process_yolo(self._multipart_output(), WIDTH, HEIGHT))
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(detections[0], [0, 0.95, *A_ROW], atol=2e-3)
np.testing.assert_allclose(detections[1], [1, 0.90, *B_ROW], atol=2e-3)
def test_empty_output_returns_no_detections(self):
outputs = [
np.zeros((1, 255, 80, 80), dtype=np.float32),
np.zeros((1, 255, 40, 40), dtype=np.float32),
np.zeros((1, 255, 20, 20), dtype=np.float32),
]
detections = kept(post_process_yolo(outputs, WIDTH, HEIGHT))
self.assertEqual(len(detections), 0)
class TestYoloxPostProcess(unittest.TestCase):
def test_keeps_separated_objects_far_from_origin(self):
# with zero grids and unit strides the decode reduces to
# cx = raw cx and w = exp(raw w)
rows = np.zeros((10, 7), dtype=np.float32)
rows[0] = [A_CX, A_CY, math.log(A_W), math.log(A_H), 1.0, 0.90, 0.0]
rows[1] = [B_CX, B_CY, math.log(B_W), math.log(B_H), 1.0, 0.0, 0.85]
predictions = rows[np.newaxis, ...]
grids = np.zeros((1, 10, 2), dtype=np.float32)
expanded_strides = np.ones((1, 10, 1), dtype=np.float32)
detections = kept(
post_process_yolox(predictions, WIDTH, HEIGHT, grids, expanded_strides)
)
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(detections[0], [0, 0.90, *A_ROW], atol=2e-3)
np.testing.assert_allclose(detections[1], [1, 0.85, *B_ROW], atol=2e-3)
class TestDfinePostProcess(unittest.TestCase):
def test_keeps_separated_objects_far_from_origin(self):
# D-FINE emits absolute pixel xyxy boxes alongside labels and scores
labels = np.zeros((1, 10), dtype=np.int64)
labels[0, 1] = 1
boxes = np.zeros((1, 10, 4), dtype=np.float32)
boxes[0, 0] = [393, 499, 484, 620]
boxes[0, 1] = [527, 499, 618, 620]
scores = np.zeros((1, 10), dtype=np.float32)
scores[0, 0] = 0.90
scores[0, 1] = 0.85
detections = kept(post_process_dfine([labels, boxes, scores], WIDTH, HEIGHT))
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(detections[0], [0, 0.90, *A_ROW], atol=2e-3)
np.testing.assert_allclose(detections[1], [1, 0.85, *B_ROW], atol=2e-3)
class TestRfdetrPostProcess(unittest.TestCase):
def test_keeps_separated_objects_far_from_origin(self):
# RF-DETR emits normalized center format boxes and class logits where
# logit index 0 is the background class
boxes = np.zeros((1, 10, 4), dtype=np.float32)
boxes[0, 0] = [A_CX / WIDTH, A_CY / HEIGHT, A_W / WIDTH, A_H / HEIGHT]
boxes[0, 1] = [B_CX / WIDTH, B_CY / HEIGHT, B_W / WIDTH, B_H / HEIGHT]
# background heavy logits everywhere, then two confident objects
logits = np.tile(np.array([10.0, 0.0, 0.0], dtype=np.float32), (1, 10, 1))
logits[0, 0] = [0.0, 4.0, 0.0] # class 0 after background offset
logits[0, 1] = [0.0, 0.0, 3.5] # class 1 after background offset
detections = kept(post_process_rfdetr([boxes, logits]))
conf_a = math.exp(4.0) / (math.exp(4.0) + 2)
conf_b = math.exp(3.5) / (math.exp(3.5) + 2)
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(detections[0], [0, conf_a, *A_ROW], atol=2e-3)
np.testing.assert_allclose(detections[1], [1, conf_b, *B_ROW], atol=2e-3)
class TestMemryxSsdlitePostProcess(unittest.TestCase):
def test_keeps_separated_objects_far_from_origin(self):
# the NMS math runs on the host CPU, so the real method is testable
# without MemryX hardware; it only needs the model dimensions and
# the output queue
detector = object.__new__(MemryXDetector)
detector.memx_model_width = WIDTH
detector.memx_model_height = HEIGHT
detector.output_queue = Queue()
# this path uses a 0.5 NMS threshold, so use a tighter pair: zero
# true overlap (10 px gap), IoU 0.69 when misread as [x, y, w, h]
dets = np.zeros((1, 10, 5), dtype=np.float32)
dets[0, 0] = [480, 500, 540, 620, 0.90]
dets[0, 1] = [550, 500, 610, 620, 0.85]
labels = np.zeros((1, 10), dtype=np.float32)
labels[0, 1] = 1
detector.post_process_ssdlite([dets, labels])
detections = kept(detector.output_queue.get())
self.assertEqual(len(detections), 2)
np.testing.assert_allclose(
detections[0],
[0, 0.90, 500 / 640, 480 / 640, 620 / 640, 540 / 640],
atol=2e-3,
)
np.testing.assert_allclose(
detections[1],
[1, 0.85, 500 / 640, 550 / 640, 620 / 640, 610 / 640],
atol=2e-3,
)
if __name__ == "__main__":
unittest.main()
+37 -5
View File
@@ -16,6 +16,31 @@ logger = logging.getLogger(__name__)
### Post Processing
def xyxy_to_xywh_for_nms(boxes: np.ndarray | list) -> np.ndarray:
"""Convert [x1, y1, x2, y2] boxes to the [x, y, width, height] format
that cv2.dnn.NMSBoxes expects.
Passing corner coordinates directly makes OpenCV treat x2/y2 as the box
size, inflating every box toward the bottom-right by its distance from
the origin, which suppresses valid detections near other objects.
Args:
boxes: Array-like of shape (N, 4) in corner format.
Returns:
Float32 array of shape (N, 4) in top-left plus size format.
"""
boxes = np.asarray(boxes, dtype=np.float32)
if boxes.size == 0:
return np.zeros((0, 4), dtype=np.float32)
xywh = boxes.copy()
xywh[:, 2] -= xywh[:, 0]
xywh[:, 3] -= xywh[:, 1]
return xywh
def post_process_dfine(
tensor_output: np.ndarray, width: int, height: int
) -> np.ndarray:
@@ -25,7 +50,9 @@ def post_process_dfine(
input_shape = np.array([height, width, height, width])
boxes = np.divide(boxes, input_shape, dtype=np.float32)
indices = cv2.dnn.NMSBoxes(boxes, scores, score_threshold=0.4, nms_threshold=0.4)
indices = cv2.dnn.NMSBoxes(
xyxy_to_xywh_for_nms(boxes), scores, score_threshold=0.4, nms_threshold=0.4
)
detections = np.zeros((20, 6), np.float32)
for i, (bbox, confidence, class_id) in enumerate(
@@ -78,7 +105,10 @@ def post_process_rfdetr(tensor_output: list[np.ndarray, np.ndarray]) -> np.ndarr
# apply nms
indices = cv2.dnn.NMSBoxes(
filtered_boxes, filtered_scores, score_threshold=0.4, nms_threshold=0.4
xyxy_to_xywh_for_nms(filtered_boxes),
filtered_scores,
score_threshold=0.4,
nms_threshold=0.4,
)
detections = np.zeros((20, 6), np.float32)
@@ -159,7 +189,7 @@ def __post_process_multipart_yolo(
all_class_ids.append(class_id)
indices = cv2.dnn.NMSBoxes(
bboxes=all_boxes,
bboxes=xyxy_to_xywh_for_nms(all_boxes),
scores=all_scores,
score_threshold=0.4,
nms_threshold=0.4,
@@ -206,7 +236,9 @@ def __post_process_nms_yolo(predictions: np.ndarray, width, height) -> np.ndarra
boxes = boxes_xyxy
# run NMS
indices = cv2.dnn.NMSBoxes(boxes, scores, score_threshold=0.4, nms_threshold=0.4)
indices = cv2.dnn.NMSBoxes(
xyxy_to_xywh_for_nms(boxes), scores, score_threshold=0.4, nms_threshold=0.4
)
detections = np.zeros((20, 6), np.float32)
for i, (bbox, confidence, class_id) in enumerate(
zip(boxes[indices], scores[indices], class_ids[indices])
@@ -258,7 +290,7 @@ def post_process_yolox(
scores = scores[np.arange(len(cls_inds)), cls_inds]
indices = cv2.dnn.NMSBoxes(
boxes_xyxy, scores, score_threshold=0.4, nms_threshold=0.4
xyxy_to_xywh_for_nms(boxes_xyxy), scores, score_threshold=0.4, nms_threshold=0.4
)
detections = np.zeros((20, 6), np.float32)
-209
View File
@@ -1,209 +0,0 @@
"""Aggregation of the known sub label names an object can be tagged with."""
import logging
import os
from pathvalidate import sanitize_filename
from frigate.config import FrigateConfig
from frigate.config.classification import ObjectClassificationType
from frigate.const import CLIPS_DIR, FACE_DIR, MODEL_CACHE_DIR
from frigate.util.builtin import load_labels
logger = logging.getLogger(__name__)
# subdirectory of FACE_DIR holding unassigned training images, not a face name
FACE_TRAIN_DIR = "train"
# category used by classification models for "no match", never attached to an object
CLASSIFICATION_NONE_CATEGORY = "none"
def get_categorized_object_names(
config: FrigateConfig,
allowed_cameras: list[str],
object_type: str | None = None,
) -> dict[str, list[str]]:
"""Collect every sub label name this install can attach, by object type.
Unlike the database-backed /sub_labels endpoint, this reads the config and
model files, so it also covers names that are configured but have not been
detected yet. Names come from the detector's logo attributes (limited to
objects the allowed cameras actually track), LPR known plate names,
registered face names, and custom object classification categories.
Structural attributes such as `face` and `license_plate` are excluded: they
describe a part of an object rather than naming it, and are never attached
as a sub label.
Args:
config: The running Frigate config
allowed_cameras: Cameras the requesting user may see
object_type: Optional object label to restrict the result to
Returns:
Mapping of object label to its known sub label names, sorted and
deduplicated. Object types with no known names are omitted.
"""
tracked_objects = _get_tracked_objects(config, allowed_cameras)
names: dict[str, set[str]] = {}
logos = set(config.model.all_attribute_logos)
# 1. detector logo attributes, only for objects that are actually tracked
for label, label_attributes in config.model.attributes_map.items():
if label not in tracked_objects:
continue
label_logos = logos.intersection(label_attributes)
if label_logos:
names.setdefault(label, set()).update(label_logos)
# 2. LPR known plate names, for objects that can carry a plate
if config.lpr.known_plates and _lpr_enabled(config, allowed_cameras):
known_plates = set(config.lpr.known_plates)
for label in _objects_with_attribute(config, tracked_objects, "license_plate"):
names.setdefault(label, set()).update(known_plates)
# 3. registered face names, for objects that can carry a face
if _face_recognition_enabled(config, allowed_cameras):
face_names = _get_face_names()
if face_names:
for label in _objects_with_attribute(config, tracked_objects, "face"):
names.setdefault(label, set()).update(face_names)
# 4. custom object classification categories
for model_key, model_config in config.classification.custom.items():
if not model_config.enabled or model_config.object_config is None:
continue
if (
model_config.object_config.classification_type
!= ObjectClassificationType.sub_label
):
continue
categories = _get_classification_categories(model_key)
if not categories:
continue
for label in model_config.object_config.objects:
names.setdefault(label, set()).update(categories)
return {
label: sorted(label_names)
for label, label_names in sorted(names.items())
if label_names and (object_type is None or label == object_type)
}
def _get_tracked_objects(config: FrigateConfig, allowed_cameras: list[str]) -> set[str]:
"""Get the union of objects tracked by the cameras the user can see."""
tracked: set[str] = set()
for camera_name in allowed_cameras:
camera_config = config.cameras.get(camera_name)
if camera_config is None:
continue
tracked.update(camera_config.objects.track)
return tracked
def _objects_with_attribute(
config: FrigateConfig, tracked_objects: set[str], attribute: str
) -> set[str]:
"""Get the tracked objects that a given attribute can be recognized on.
The attribute may also be tracked as an object in its own right, as
`license_plate` is on a dedicated LPR camera, in which case the name is
attached to that object directly.
"""
objects = {
label
for label, label_attributes in config.model.attributes_map.items()
if attribute in label_attributes and label in tracked_objects
}
if attribute in tracked_objects:
objects.add(attribute)
return objects
def _lpr_enabled(config: FrigateConfig, allowed_cameras: list[str]) -> bool:
return any(
config.cameras[camera_name].lpr.enabled
for camera_name in allowed_cameras
if camera_name in config.cameras
)
def _face_recognition_enabled(
config: FrigateConfig, allowed_cameras: list[str]
) -> bool:
return any(
config.cameras[camera_name].face_recognition.enabled
for camera_name in allowed_cameras
if camera_name in config.cameras
)
def _get_face_names() -> set[str]:
"""Get the names of every registered face collection."""
if not os.path.exists(FACE_DIR):
return set()
try:
entries = os.listdir(FACE_DIR)
except OSError:
logger.debug("Failed to read face directory %s", FACE_DIR)
return set()
return {
name
for name in entries
if name != FACE_TRAIN_DIR and os.path.isdir(os.path.join(FACE_DIR, name))
}
def _get_classification_categories(model_key: str) -> set[str]:
"""Get the categories a custom classification model can output.
The trained labelmap is authoritative, but it only exists once the model
has been trained, so fall back to the dataset directories that will become
the labelmap on the next training run.
"""
safe_key = sanitize_filename(model_key)
categories: set[str] = set()
labelmap_path = os.path.join(MODEL_CACHE_DIR, safe_key, "labelmap.txt")
if os.path.exists(labelmap_path):
try:
labelmap = load_labels(labelmap_path, prefill=0, indexed=False)
except OSError:
logger.debug("Failed to read labelmap %s", labelmap_path)
labelmap = {}
categories.update(label for label in labelmap.values() if label)
dataset_dir = os.path.join(CLIPS_DIR, safe_key, "dataset")
if os.path.exists(dataset_dir):
try:
entries = os.listdir(dataset_dir)
except OSError:
logger.debug("Failed to read dataset directory %s", dataset_dir)
entries = []
categories.update(
name for name in entries if os.path.isdir(os.path.join(dataset_dir, name))
)
categories.discard(CLASSIFICATION_NONE_CATEGORY)
return categories
+3 -18
View File
@@ -1,7 +1,6 @@
"""Utilities for services."""
import asyncio
import glob
import json
import logging
import os
@@ -671,33 +670,19 @@ def get_intel_gpu_stats(
def get_openvino_npu_stats() -> dict[str, str] | None:
"""Get NPU stats using openvino."""
for accel_path in sorted(glob.glob("/sys/class/accel/accel*")):
try:
driver = os.path.basename(os.readlink(f"{accel_path}/device/driver"))
except OSError:
continue
if driver != "intel_vpu":
continue
NPU_RUNTIME_PATH = "/sys/devices/pci0000:00/0000:00:0b.0/power/runtime_active_time"
try:
runtime_path = f"{accel_path}/device/power/runtime_active_time"
with open(runtime_path) as f:
with open(NPU_RUNTIME_PATH) as f:
initial_runtime = float(f.read().strip())
break
except (FileNotFoundError, PermissionError, ValueError):
continue
else:
return None
try:
initial_time = time.time()
# Sleep for 1 second to get an accurate reading
time.sleep(1.0)
# Read runtime value again
with open(runtime_path) as f:
with open(NPU_RUNTIME_PATH) as f:
current_runtime = float(f.read().strip())
current_time = time.time()
+2 -2
View File
@@ -859,8 +859,8 @@
"description": "Numeric order used to sort the camera in the UI (default dashboard and lists); larger numbers appear later."
},
"dashboard": {
"label": "Show in UI",
"description": "Toggle whether this camera is visible everywhere in the Frigate UI. Disabling this will require manually editing the config to view this camera in the UI again."
"label": "Show on Live dashboard",
"description": "Toggle whether this camera is visible on the default All Cameras live dashboard. The camera remains available everywhere else in the UI, including camera groups and settings."
},
"review": {
"label": "Show in review",
+2 -18
View File
@@ -357,22 +357,6 @@
"description": "Optional API key for authenticated DeepStack services."
}
},
"degirum": {
"label": "DeGirum",
"description": "DeGirum detector for running models via DeGirum cloud or local inference services.",
"location": {
"label": "Inference Location",
"description": "Location of the DeGirim inference engine (e.g. '@cloud', '127.0.0.1')."
},
"zoo": {
"label": "Model Zoo",
"description": "Path or URL to the DeGirum model zoo."
},
"token": {
"label": "DeGirum Cloud Token",
"description": "Token for DeGirum Cloud access."
}
},
"edgetpu": {
"label": "EdgeTPU",
"description": "EdgeTPU detector that runs TensorFlow Lite models compiled for Coral EdgeTPU using the EdgeTPU delegate.",
@@ -1543,8 +1527,8 @@
"description": "Numeric order used to sort the camera in the UI (default dashboard and lists); larger numbers appear later."
},
"dashboard": {
"label": "Show in UI",
"description": "Toggle whether this camera is visible everywhere in the Frigate UI. Disabling this will require manually editing the config to view this camera in the UI again."
"label": "Show on Live dashboard",
"description": "Toggle whether this camera is visible on the default All Cameras live dashboard. The camera remains available everywhere else in the UI, including camera groups and settings."
},
"review": {
"label": "Show in review",
+1 -1
View File
@@ -499,7 +499,7 @@
"webuiUrlHelp": "URL to visit the camera's web UI directly from the Debug view. Leave blank to disable the link.",
"webuiUrlInvalid": "Must be a valid URL (e.g., https://example.com).",
"dashboardLabel": "Show on Live dashboard",
"dashboardHelp": "Show this camera on the Live dashboard.",
"dashboardHelp": "Show this camera on the default All Cameras live dashboard. It remains available everywhere else, including camera groups.",
"reviewLabel": "Show in Review",
"reviewHelp": "Show this camera in Review, including the camera filter, motion review, and the history view."
}
+1 -1
View File
@@ -78,7 +78,7 @@ export default function ZoneEditPane({
}
return Object.values(config.cameras)
.filter((conf) => conf.ui.dashboard && conf.enabled_in_config)
.filter((conf) => conf.enabled_in_config)
.sort((aConf, bConf) => aConf.ui.order - bConf.ui.order);
}, [config]);
+32 -13
View File
@@ -198,6 +198,19 @@ export default function Events() {
return true;
});
const reviewCamerasParam = useMemo(() => {
const selected: string | undefined = reviewSearchParams["cameras"];
if (!selected) {
return reviewCameras.join(",");
}
const selectedCameras = new Set(selected.split(","));
return reviewCameras
.filter((camera) => selectedCameras.has(camera))
.join(",");
}, [reviewCameras, reviewSearchParams]);
useSearchEffect("labels", (labels: string) => {
setReviewFilter({
...reviewFilter,
@@ -330,8 +343,12 @@ export default function Events() {
}, []);
const getKey = useCallback(() => {
if (!timezone) {
return null;
}
const params = {
cameras: reviewSearchParams["cameras"],
cameras: reviewCamerasParam,
labels: reviewSearchParams["labels"],
zones: reviewSearchParams["zones"],
reviewed: null, // We want both reviewed and unreviewed items as we filter in the UI
@@ -339,7 +356,7 @@ export default function Events() {
after: reviewSearchParams["after"] || last24Hours.after,
};
return ["review", params];
}, [reviewSearchParams, last24Hours]);
}, [reviewSearchParams, reviewCamerasParam, last24Hours, timezone]);
const { data: reviews, mutate: updateSegments } = useSWR<ReviewSegment[]>(
getKey,
@@ -361,10 +378,6 @@ export default function Events() {
const motion: ReviewSegment[] = [];
reviews?.forEach((segment) => {
if (config?.cameras[segment.camera]?.ui?.review === false) {
return;
}
all.push(segment);
switch (segment.severity) {
@@ -386,7 +399,7 @@ export default function Events() {
detection: detections,
significant_motion: motion,
};
}, [reviews, config?.cameras]);
}, [reviews]);
// update review items in place when a review segment ends
const reviewUpdate = useFrigateReviews();
@@ -450,15 +463,17 @@ export default function Events() {
// review summary
const { data: reviewSummary, mutate: updateSummary } = useSWR<ReviewSummary>(
[
timezone
? [
"review/summary",
{
timezone: timezone,
cameras: reviewSearchParams["cameras"] ?? null,
cameras: reviewCamerasParam,
labels: reviewSearchParams["labels"] ?? null,
zones: reviewSearchParams["zones"] ?? null,
},
],
]
: null,
{
revalidateOnFocus: true,
refreshInterval: 30000,
@@ -473,13 +488,17 @@ export default function Events() {
// recordings summary
const { data: recordingsSummary } = useSWR<RecordingsSummary>([
const { data: recordingsSummary } = useSWR<RecordingsSummary>(
timezone
? [
"recordings/summary",
{
timezone: timezone,
cameras: reviewSearchParams["cameras"] ?? null,
cameras: reviewCamerasParam,
},
]);
]
: null,
);
// preview videos
const previewTimes = useMemo(() => {
+1 -6
View File
@@ -706,12 +706,7 @@ export default function Settings() {
}
return Object.values(config.cameras)
.filter(
(conf) =>
conf.ui.dashboard &&
conf.enabled_in_config &&
!isReplayCamera(conf.name),
)
.filter((conf) => conf.enabled_in_config && !isReplayCamera(conf.name))
.sort((aConf, bConf) => aConf.ui.order - bConf.ui.order);
}, [config]);
+10 -2
View File
@@ -1024,6 +1024,9 @@ function MotionReview({
if (!allowedCameras.includes(cam.name)) {
return false;
}
if (cam.ui?.review === false) {
return false;
}
if (selectedCams && !selectedCams.includes(cam.name)) {
return false;
}
@@ -1033,6 +1036,11 @@ function MotionReview({
return cameras.sort((a, b) => a.ui.order - b.ui.order);
}, [config, filter, allowedCameras]);
const reviewCamerasParam = useMemo(
() => reviewCameras.map((cam) => cam.name).join(","),
[reviewCameras],
);
const videoPlayersRef = useRef<{ [camera: string]: PreviewController }>({});
// motion data
@@ -1052,7 +1060,7 @@ function MotionReview({
before: alignedBefore,
after: alignedAfter,
scale: segmentDuration / 2,
cameras: filter?.cameras?.join(",") ?? null,
cameras: reviewCamerasParam,
},
]);
@@ -1061,7 +1069,7 @@ function MotionReview({
{
before: alignedBefore,
after: alignedAfter,
cameras: filter?.cameras?.join(",") ?? null,
cameras: reviewCamerasParam,
},
]);
+33 -6
View File
@@ -540,6 +540,36 @@ export default function GeneralMetrics({
return Object.keys(series).length > 0 ? Object.values(series) : undefined;
}, [statsHistory]);
// Number of cards the hardware grid renders. Which ones appear depends on
// the vendor, so the column count follows the count rather than assuming a
// fixed set is present.
const hardwareCardCount = useMemo(() => {
if (!statsHistory[0]?.gpu_usages) {
return 0;
}
const hasNpu = statsHistory[0].npu_usages != undefined;
return (
1 + // gpu usage always renders alongside gpu_usages
(gpuMemSeries ? 1 : 0) +
(gpuEncSeries?.length ? 1 : 0) +
(gpuComputeSeries?.length ? 1 : 0) +
(gpuDecSeries?.length ? 1 : 0) +
(gpuTempSeries?.length ? 1 : 0) +
(hasNpu ? 1 : 0) +
(hasNpu && npuTempSeries?.length ? 1 : 0)
);
}, [
statsHistory,
gpuMemSeries,
gpuEncSeries,
gpuComputeSeries,
gpuDecSeries,
gpuTempSeries,
npuTempSeries,
]);
// other processes stats
const hardwareType = useMemo(() => {
@@ -763,12 +793,9 @@ export default function GeneralMetrics({
<div
className={cn(
"mt-4 grid grid-cols-1 gap-2 sm:grid-cols-2",
gpuTempSeries?.length && "md:grid-cols-3",
(gpuEncSeries?.length || gpuComputeSeries?.length) &&
"xl:grid-cols-4",
(gpuEncSeries?.length || gpuComputeSeries?.length) &&
gpuTempSeries?.length &&
"3xl:grid-cols-5",
hardwareCardCount >= 3 && "lg:grid-cols-3",
hardwareCardCount >= 4 && "xl:grid-cols-4",
hardwareCardCount >= 5 && "3xl:grid-cols-5",
)}
>
{statsHistory[0]?.gpu_usages && (