diff --git a/docs/docs/configuration/face_recognition.md b/docs/docs/configuration/face_recognition.md index 4d934afce..aac1be9b5 100644 --- a/docs/docs/configuration/face_recognition.md +++ b/docs/docs/configuration/face_recognition.md @@ -3,19 +3,55 @@ id: face_recognition title: Face Recognition --- -Face recognition allows people to be assigned names and when their face is recognized Frigate will assign the person's name as a sub label. This information is included in the UI, filters, as well as in notifications. +Face recognition identifies known individuals by matching detected faces with previously learned facial data. When a known person is recognized, their name will be added as a `sub_label`. This information is included in the UI, filters, as well as in notifications. + +## Model Requirements Frigate has support for CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally. A lightweight face landmark detection model is also used to align faces before running them through the face recognizer. +Users running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient. + +Users without a model that detects faces can still run face recognition. Frigate uses a lightweight DNN face detection model that runs on the CPU. In this case, you should _not_ define `face` in your list of objects to track. + +:::note + +Frigate needs to first detect a `face` before it can recognize a face. + +::: + +## Minimum System Requirements + +Face recognition is lightweight and runs on the CPU, there are no significantly different system requirements than running Frigate itself. + ## Configuration -Face recognition is disabled by default, face recognition must be enabled in your config file before it can be used. Face recognition is a global configuration setting. +Face recognition is disabled by default, face recognition must be enabled in the UI or in your config file before it can be used. Face recognition is a global configuration setting. ```yaml face_recognition: enabled: true ``` +## Advanced Configuration + +Fine-tune face recognition with these optional parameters: + +### Detection + +- `detection_threshold`: Face detection confidence score required before recognition runs: + - Default: `0.7` + - Note: This is field only applies to the standalone face detection model, `min_score` should be used to filter for models that have face detection built in. +- `min_area`: Defines the minimum size (in pixels) a face must be before recognition runs. + - Default: `500` pixels. + - Depending on the resolution of your camera's `detect` stream, you can increase this value to ignore small or distant faces. + +### Recognition + +- `recognition_threshold`: Recognition confidence score required to add the face to the object as a sub label. + - Default: `0.9`. +- `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this. + - Default: `True`. + ## Dataset The number of images needed for a sufficient training set for face recognition varies depending on several factors: diff --git a/docs/docs/configuration/license_plate_recognition.md b/docs/docs/configuration/license_plate_recognition.md index 776f30cf9..ee490a7a6 100644 --- a/docs/docs/configuration/license_plate_recognition.md +++ b/docs/docs/configuration/license_plate_recognition.md @@ -51,16 +51,16 @@ Fine-tune the LPR feature using these optional parameters: - **`detection_threshold`**: License plate object detection confidence score required before recognition runs. - Default: `0.7` - - Note: If you are using a Frigate+ model and you set the `threshold` in your objects config for `license_plate` higher than this value, recognition will never run. It's best to ensure these values match, or this `detection_threshold` is lower than your object config `threshold`. + - Note: This is field only applies to the standalone license plate detection model, `min_score` should be used to filter for models that have license plate detection built in. - **`min_area`**: Defines the minimum size (in pixels) a license plate must be before recognition runs. - Default: `1000` pixels. - Depending on the resolution of your camera's `detect` stream, you can increase this value to ignore small or distant plates. ### Recognition -- **`recognition_threshold`**: Recognition confidence score required to add the plate to the object as a sub label. +- **`recognition_threshold`**: Recognition confidence score required to add the plate to the object as a `recognized_license_plate` and/or `sub_label`. - Default: `0.9`. -- **`min_plate_length`**: Specifies the minimum number of characters a detected license plate must have to be added as a sub label to an object. +- **`min_plate_length`**: Specifies the minimum number of characters a detected license plate must have to be added as a `recognized_license_plate` and/or `sub_label` to an object. - Use this to filter out short, incomplete, or incorrect detections. - **`format`**: A regular expression defining the expected format of detected plates. Plates that do not match this format will be discarded. - `"^[A-Z]{1,3} [A-Z]{1,2} [0-9]{1,4}$"` matches plates like "B AB 1234" or "M X 7" diff --git a/frigate/api/auth.py b/frigate/api/auth.py index c0ed94d5c..f806a0c30 100644 --- a/frigate/api/auth.py +++ b/frigate/api/auth.py @@ -189,21 +189,15 @@ def set_jwt_cookie(response: Response, cookie_name, encoded_jwt, expiration, sec async def get_current_user(request: Request): - JWT_COOKIE_NAME = request.app.frigate_config.auth.cookie_name - encoded_token = request.cookies.get(JWT_COOKIE_NAME) - if not encoded_token: - return JSONResponse(content={"message": "No JWT token found"}, status_code=401) + username = request.headers.get("remote-user") + role = request.headers.get("remote-role") - try: - token = jwt.decode(encoded_token, request.app.jwt_token) - if "sub" not in token.claims or "role" not in token.claims: - return JSONResponse( - content={"message": "Invalid JWT token"}, status_code=401 - ) - return {"username": token.claims["sub"], "role": token.claims["role"]} - except Exception as e: - logger.error(f"Error parsing JWT: {e}") - return JSONResponse(content={"message": "Invalid JWT token"}, status_code=401) + if not username or not role: + return JSONResponse( + content={"message": "No authorization headers."}, status_code=401 + ) + + return {"username": username, "role": role} def require_role(required_roles: List[str]): diff --git a/frigate/api/media.py b/frigate/api/media.py index b74ec93d1..83307a15c 100644 --- a/frigate/api/media.py +++ b/frigate/api/media.py @@ -27,6 +27,7 @@ from frigate.api.defs.query.media_query_parameters import ( MediaRecordingsSummaryQueryParams, ) from frigate.api.defs.tags import Tags +from frigate.camera.state import CameraState from frigate.config import FrigateConfig from frigate.const import ( CACHE_DIR, @@ -106,10 +107,10 @@ def imagestream( @router.get("/{camera_name}/ptz/info") -def camera_ptz_info(request: Request, camera_name: str): +async def camera_ptz_info(request: Request, camera_name: str): if camera_name in request.app.frigate_config.cameras: return JSONResponse( - content=request.app.onvif.get_camera_info(camera_name), + content=await request.app.onvif.get_camera_info(camera_name), ) else: return JSONResponse( @@ -765,12 +766,15 @@ def event_snapshot( except DoesNotExist: # see if the object is currently being tracked try: - camera_states = request.app.detected_frames_processor.camera_states.values() + camera_states: list[CameraState] = ( + request.app.detected_frames_processor.camera_states.values() + ) for camera_state in camera_states: if event_id in camera_state.tracked_objects: tracked_obj = camera_state.tracked_objects.get(event_id) if tracked_obj is not None: - jpg_bytes = tracked_obj.get_jpg_bytes( + jpg_bytes = tracked_obj.get_img_bytes( + ext="jpg", timestamp=params.timestamp, bounding_box=params.bbox, crop=params.crop, @@ -779,17 +783,19 @@ def event_snapshot( ) except Exception: return JSONResponse( - content={"success": False, "message": "Event not found"}, + content={"success": False, "message": "Ongoing event not found"}, status_code=404, ) except Exception: return JSONResponse( - content={"success": False, "message": "Event not found"}, status_code=404 + content={"success": False, "message": "Unknown error occurred"}, + status_code=404, ) if jpg_bytes is None: return JSONResponse( - content={"success": False, "message": "Event not found"}, status_code=404 + content={"success": False, "message": "Live frame not available"}, + status_code=404, ) headers = { diff --git a/frigate/api/review.py b/frigate/api/review.py index 4788356f3..b04c8353a 100644 --- a/frigate/api/review.py +++ b/frigate/api/review.py @@ -9,10 +9,10 @@ import pandas as pd from fastapi import APIRouter from fastapi.params import Depends from fastapi.responses import JSONResponse -from peewee import Case, DoesNotExist, fn, operator +from peewee import Case, DoesNotExist, IntegrityError, fn, operator from playhouse.shortcuts import model_to_dict -from frigate.api.auth import require_role +from frigate.api.auth import get_current_user, require_role from frigate.api.defs.query.review_query_parameters import ( ReviewActivityMotionQueryParams, ReviewQueryParams, @@ -26,7 +26,7 @@ from frigate.api.defs.response.review_response import ( ReviewSummaryResponse, ) from frigate.api.defs.tags import Tags -from frigate.models import Recordings, ReviewSegment +from frigate.models import Recordings, ReviewSegment, UserReviewStatus from frigate.review.types import SeverityEnum from frigate.util.builtin import get_tz_modifiers @@ -36,7 +36,15 @@ router = APIRouter(tags=[Tags.review]) @router.get("/review", response_model=list[ReviewSegmentResponse]) -def review(params: ReviewQueryParams = Depends()): +async def review( + params: ReviewQueryParams = Depends(), + current_user: dict = Depends(get_current_user), +): + if isinstance(current_user, JSONResponse): + return current_user + + user_id = current_user["username"] + cameras = params.cameras labels = params.labels zones = params.zones @@ -74,9 +82,7 @@ def review(params: ReviewQueryParams = Depends()): (ReviewSegment.data["objects"].cast("text") % f'*"{label}"*') | (ReviewSegment.data["audio"].cast("text") % f'*"{label}"*') ) - - label_clause = reduce(operator.or_, label_clauses) - clauses.append((label_clause)) + clauses.append(reduce(operator.or_, label_clauses)) if zones != "all": # use matching so segments with multiple zones @@ -88,27 +94,52 @@ def review(params: ReviewQueryParams = Depends()): zone_clauses.append( (ReviewSegment.data["zones"].cast("text") % f'*"{zone}"*') ) - - zone_clause = reduce(operator.or_, zone_clauses) - clauses.append((zone_clause)) - - if reviewed == 0: - clauses.append((ReviewSegment.has_been_reviewed == False)) + clauses.append(reduce(operator.or_, zone_clauses)) if severity: clauses.append((ReviewSegment.severity == severity)) - review = ( - ReviewSegment.select() + # Join with UserReviewStatus to get per-user review status + review_query = ( + ReviewSegment.select( + ReviewSegment.id, + ReviewSegment.camera, + ReviewSegment.start_time, + ReviewSegment.end_time, + ReviewSegment.severity, + ReviewSegment.thumb_path, + ReviewSegment.data, + fn.COALESCE(UserReviewStatus.has_been_reviewed, False).alias( + "has_been_reviewed" + ), + ) + .left_outer_join( + UserReviewStatus, + on=( + (ReviewSegment.id == UserReviewStatus.review_segment) + & (UserReviewStatus.user_id == user_id) + ), + ) .where(reduce(operator.and_, clauses)) - .order_by(ReviewSegment.severity.asc()) + ) + + # Filter unreviewed items without subquery + if reviewed == 0: + review_query = review_query.where( + (UserReviewStatus.has_been_reviewed == False) + | (UserReviewStatus.has_been_reviewed.is_null()) + ) + + # Apply ordering and limit + review_query = ( + review_query.order_by(ReviewSegment.severity.asc()) .order_by(ReviewSegment.start_time.desc()) .limit(limit) .dicts() .iterator() ) - return JSONResponse(content=[r for r in review]) + return JSONResponse(content=[r for r in review_query]) @router.get("/review_ids", response_model=list[ReviewSegmentResponse]) @@ -134,7 +165,15 @@ def review_ids(ids: str): @router.get("/review/summary", response_model=ReviewSummaryResponse) -def review_summary(params: ReviewSummaryQueryParams = Depends()): +async def review_summary( + params: ReviewSummaryQueryParams = Depends(), + current_user: dict = Depends(get_current_user), +): + if isinstance(current_user, JSONResponse): + return current_user + + user_id = current_user["username"] + hour_modifier, minute_modifier, seconds_offset = get_tz_modifiers(params.timezone) day_ago = (datetime.datetime.now() - datetime.timedelta(hours=24)).timestamp() month_ago = (datetime.datetime.now() - datetime.timedelta(days=30)).timestamp() @@ -160,10 +199,7 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): (ReviewSegment.data["objects"].cast("text") % f'*"{label}"*') | (ReviewSegment.data["audio"].cast("text") % f'*"{label}"*') ) - - label_clause = reduce(operator.or_, label_clauses) - clauses.append((label_clause)) - + clauses.append(reduce(operator.or_, label_clauses)) if zones != "all": # use matching so segments with multiple zones # still match on a search where any zone matches @@ -172,21 +208,20 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): for zone in filtered_zones: zone_clauses.append( - (ReviewSegment.data["zones"].cast("text") % f'*"{zone}"*') + ReviewSegment.data["zones"].cast("text") % f'*"{zone}"*' ) + clauses.append(reduce(operator.or_, zone_clauses)) - zone_clause = reduce(operator.or_, zone_clauses) - clauses.append((zone_clause)) - - last_24 = ( + last_24_query = ( ReviewSegment.select( fn.SUM( Case( None, [ ( - (ReviewSegment.severity == SeverityEnum.alert), - ReviewSegment.has_been_reviewed, + (ReviewSegment.severity == SeverityEnum.alert) + & (UserReviewStatus.has_been_reviewed == True), + 1, ) ], 0, @@ -197,8 +232,9 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == SeverityEnum.detection), - ReviewSegment.has_been_reviewed, + (ReviewSegment.severity == SeverityEnum.detection) + & (UserReviewStatus.has_been_reviewed == True), + 1, ) ], 0, @@ -229,6 +265,13 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): ) ).alias("total_detection"), ) + .left_outer_join( + UserReviewStatus, + on=( + (ReviewSegment.id == UserReviewStatus.review_segment) + & (UserReviewStatus.user_id == user_id) + ), + ) .where(reduce(operator.and_, clauses)) .dicts() .get() @@ -248,14 +291,12 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): for label in filtered_labels: label_clauses.append( - (ReviewSegment.data["objects"].cast("text") % f'*"{label}"*') + ReviewSegment.data["objects"].cast("text") % f'*"{label}"*' ) - - label_clause = reduce(operator.or_, label_clauses) - clauses.append((label_clause)) + clauses.append(reduce(operator.or_, label_clauses)) day_in_seconds = 60 * 60 * 24 - last_month = ( + last_month_query = ( ReviewSegment.select( fn.strftime( "%Y-%m-%d", @@ -271,8 +312,9 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == SeverityEnum.alert), - ReviewSegment.has_been_reviewed, + (ReviewSegment.severity == SeverityEnum.alert) + & (UserReviewStatus.has_been_reviewed == True), + 1, ) ], 0, @@ -283,8 +325,9 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): None, [ ( - (ReviewSegment.severity == SeverityEnum.detection), - ReviewSegment.has_been_reviewed, + (ReviewSegment.severity == SeverityEnum.detection) + & (UserReviewStatus.has_been_reviewed == True), + 1, ) ], 0, @@ -315,28 +358,59 @@ def review_summary(params: ReviewSummaryQueryParams = Depends()): ) ).alias("total_detection"), ) + .left_outer_join( + UserReviewStatus, + on=( + (ReviewSegment.id == UserReviewStatus.review_segment) + & (UserReviewStatus.user_id == user_id) + ), + ) .where(reduce(operator.and_, clauses)) .group_by( - (ReviewSegment.start_time + seconds_offset).cast("int") / day_in_seconds, + (ReviewSegment.start_time + seconds_offset).cast("int") / day_in_seconds ) .order_by(ReviewSegment.start_time.desc()) ) data = { - "last24Hours": last_24, + "last24Hours": last_24_query, } - for e in last_month.dicts().iterator(): + for e in last_month_query.dicts().iterator(): data[e["day"]] = e return JSONResponse(content=data) @router.post("/reviews/viewed", response_model=GenericResponse) -def set_multiple_reviewed(body: ReviewModifyMultipleBody): - ReviewSegment.update(has_been_reviewed=True).where( - ReviewSegment.id << body.ids - ).execute() +async def set_multiple_reviewed( + body: ReviewModifyMultipleBody, + current_user: dict = Depends(get_current_user), +): + if isinstance(current_user, JSONResponse): + return current_user + + user_id = current_user["username"] + + for review_id in body.ids: + try: + review_status = UserReviewStatus.get( + UserReviewStatus.user_id == user_id, + UserReviewStatus.review_segment == review_id, + ) + # If it exists and isn’t reviewed, update it + if not review_status.has_been_reviewed: + review_status.has_been_reviewed = True + review_status.save() + except DoesNotExist: + try: + UserReviewStatus.create( + user_id=user_id, + review_segment=ReviewSegment.get(id=review_id), + has_been_reviewed=True, + ) + except (DoesNotExist, IntegrityError): + pass return JSONResponse( content=({"success": True, "message": "Reviewed multiple items"}), @@ -389,6 +463,9 @@ def delete_reviews(body: ReviewModifyMultipleBody): # delete recordings and review segments Recordings.delete().where(Recordings.id << recording_ids).execute() ReviewSegment.delete().where(ReviewSegment.id << list_of_ids).execute() + UserReviewStatus.delete().where( + UserReviewStatus.review_segment << list_of_ids + ).execute() return JSONResponse( content=({"success": True, "message": "Deleted review items."}), status_code=200 @@ -502,7 +579,15 @@ def get_review(review_id: str): @router.delete("/review/{review_id}/viewed", response_model=GenericResponse) -def set_not_reviewed(review_id: str): +async def set_not_reviewed( + review_id: str, + current_user: dict = Depends(get_current_user), +): + if isinstance(current_user, JSONResponse): + return current_user + + user_id = current_user["username"] + try: review: ReviewSegment = ReviewSegment.get(ReviewSegment.id == review_id) except DoesNotExist: @@ -513,8 +598,15 @@ def set_not_reviewed(review_id: str): status_code=404, ) - review.has_been_reviewed = False - review.save() + try: + user_review = UserReviewStatus.get( + UserReviewStatus.user_id == user_id, + UserReviewStatus.review_segment == review, + ) + # we could update here instead of delete if we need + user_review.delete_instance() + except DoesNotExist: + pass # Already effectively "not reviewed" return JSONResponse( content=({"success": True, "message": f"Set Review {review_id} as not viewed"}), diff --git a/frigate/config/classification.py b/frigate/config/classification.py index 07d986d7d..30cd12b7c 100644 --- a/frigate/config/classification.py +++ b/frigate/config/classification.py @@ -55,7 +55,13 @@ class FaceRecognitionConfig(FrigateBaseModel): gt=0.0, le=1.0, ) - threshold: float = Field( + detection_threshold: float = Field( + default=0.7, + title="Minimum face detection score required to be considered a face.", + gt=0.0, + le=1.0, + ) + recognition_threshold: float = Field( default=0.9, title="Minimum face distance score required to be considered a match.", gt=0.0, diff --git a/frigate/data_processing/common/license_plate/mixin.py b/frigate/data_processing/common/license_plate/mixin.py index 751a674f5..c07163819 100644 --- a/frigate/data_processing/common/license_plate/mixin.py +++ b/frigate/data_processing/common/license_plate/mixin.py @@ -937,12 +937,6 @@ class LicensePlateProcessingMixin: if not license_plate: return - if license_plate.get("score") < self.lpr_config.detection_threshold: - logger.debug( - f"Plate detection score is less than the threshold ({license_plate['score']:0.2f} < {self.lpr_config.detection_threshold})" - ) - return - license_plate_box = license_plate.get("box") # check that license plate is valid diff --git a/frigate/data_processing/real_time/bird.py b/frigate/data_processing/real_time/bird.py index d942edf6f..ba6d4f08c 100644 --- a/frigate/data_processing/real_time/bird.py +++ b/frigate/data_processing/real_time/bird.py @@ -115,10 +115,10 @@ class BirdRealTimeProcessor(RealTimeProcessorApi): x:x2, ] - cv2.imwrite("/media/frigate/test_class.png", input) + if input.shape != (224, 224): + input = cv2.resize(input, (224, 224)) input = np.expand_dims(input, axis=0) - self.interpreter.set_tensor(self.tensor_input_details[0]["index"], input) self.interpreter.invoke() res: np.ndarray = self.interpreter.get_tensor( diff --git a/frigate/data_processing/real_time/face.py b/frigate/data_processing/real_time/face.py index c88aae027..7d97f8586 100644 --- a/frigate/data_processing/real_time/face.py +++ b/frigate/data_processing/real_time/face.py @@ -88,7 +88,7 @@ class FaceRealTimeProcessor(RealTimeProcessorApi): os.path.join(MODEL_CACHE_DIR, "facedet/facedet.onnx"), config="", input_size=(320, 320), - score_threshold=0.8, + score_threshold=self.face_config.detection_threshold, nms_threshold=0.3, ) self.landmark_detector = cv2.face.createFacemarkLBF() @@ -367,9 +367,9 @@ class FaceRealTimeProcessor(RealTimeProcessorApi): os.makedirs(folder, exist_ok=True) cv2.imwrite(file, face_frame) - if score < self.config.face_recognition.threshold: + if score < self.config.face_recognition.recognition_threshold: logger.debug( - f"Recognized face distance {score} is less than threshold {self.config.face_recognition.threshold}" + f"Recognized face distance {score} is less than threshold {self.config.face_recognition.recognition_threshold}" ) self.__update_metrics(datetime.datetime.now().timestamp() - start) return diff --git a/frigate/models.py b/frigate/models.py index 11b25b938..5aa0dc5b2 100644 --- a/frigate/models.py +++ b/frigate/models.py @@ -3,6 +3,7 @@ from peewee import ( CharField, DateTimeField, FloatField, + ForeignKeyField, IntegerField, Model, TextField, @@ -92,12 +93,20 @@ class ReviewSegment(Model): # type: ignore[misc] camera = CharField(index=True, max_length=20) start_time = DateTimeField() end_time = DateTimeField() - has_been_reviewed = BooleanField(default=False) severity = CharField(max_length=30) # alert, detection thumb_path = CharField(unique=True) data = JSONField() # additional data about detection like list of labels, zone, areas of significant motion +class UserReviewStatus(Model): # type: ignore[misc] + user_id = CharField(max_length=30) + review_segment = ForeignKeyField(ReviewSegment, backref="user_reviews") + has_been_reviewed = BooleanField(default=False) + + class Meta: + indexes = ((("user_id", "review_segment"), True),) + + class Previews(Model): # type: ignore[misc] id = CharField(null=False, primary_key=True, max_length=30) camera = CharField(index=True, max_length=20) diff --git a/frigate/ptz/autotrack.py b/frigate/ptz/autotrack.py index c1184f5b5..81e54c6d7 100644 --- a/frigate/ptz/autotrack.py +++ b/frigate/ptz/autotrack.py @@ -584,19 +584,31 @@ class PtzAutoTracker: # Extract areas and calculate weighted average # grab the largest dimension of the bounding box and create a square from that + # Filter out the initial frame and use a recent time window + current_time = obj.obj_data["frame_time"] + time_window = 1.5 # seconds + history = [ + entry + for entry in self.tracked_object_history[camera] + if not entry.get("is_initial_frame", False) + and current_time - entry["frame_time"] <= time_window + ] + if not history: # Fallback to latest if no recent entries + history = [self.tracked_object_history[camera][-1]] + areas = [ { - "frame_time": obj["frame_time"], - "box": obj["box"], + "frame_time": entry["frame_time"], + "box": entry["box"], "area": max( - obj["box"][2] - obj["box"][0], obj["box"][3] - obj["box"][1] + entry["box"][2] - entry["box"][0], entry["box"][3] - entry["box"][1] ) ** 2, } - for obj in self.tracked_object_history[camera] + for entry in history ] - filtered_areas = remove_outliers(areas) if len(areas) >= 2 else areas + filtered_areas = remove_outliers(areas) if len(areas) > 3 else areas # Filter entries that are not touching the frame edge filtered_areas_not_touching_edge = [ diff --git a/frigate/ptz/onvif.py b/frigate/ptz/onvif.py index 1a813c799..dea7f5b77 100644 --- a/frigate/ptz/onvif.py +++ b/frigate/ptz/onvif.py @@ -2,6 +2,7 @@ import asyncio import logging +import time from enum import Enum from importlib.util import find_spec from pathlib import Path @@ -39,6 +40,10 @@ class OnvifController: self, config: FrigateConfig, ptz_metrics: dict[str, PTZMetrics] ) -> None: self.cams: dict[str, ONVIFCamera] = {} + self.failed_cams: dict[str, dict] = {} + self.max_retries = 5 + self.reset_timeout = 900 # 15 minutes + self.config = config self.ptz_metrics = ptz_metrics @@ -47,26 +52,37 @@ class OnvifController: continue if cam.onvif.host: - try: - self.cams[cam_name] = { - "onvif": ONVIFCamera( - cam.onvif.host, - cam.onvif.port, - cam.onvif.user, - cam.onvif.password, - wsdl_dir=str( - Path(find_spec("onvif").origin).parent / "wsdl" - ), - adjust_time=cam.onvif.ignore_time_mismatch, - encrypt=not cam.onvif.tls_insecure, - ), - "init": False, - "active": False, - "features": [], - "presets": {}, - } - except ONVIFError as e: - logger.error(f"Onvif connection to {cam.name} failed: {e}") + result = self._create_onvif_camera(cam_name, cam) + if result: + self.cams[cam_name] = result + + def _create_onvif_camera(self, cam_name: str, cam) -> dict | None: + """Create an ONVIF camera instance and handle failures.""" + try: + return { + "onvif": ONVIFCamera( + cam.onvif.host, + cam.onvif.port, + cam.onvif.user, + cam.onvif.password, + wsdl_dir=str(Path(find_spec("onvif").origin).parent / "wsdl"), + adjust_time=cam.onvif.ignore_time_mismatch, + encrypt=not cam.onvif.tls_insecure, + ), + "init": False, + "active": False, + "features": [], + "presets": {}, + } + except ONVIFError as e: + logger.error(f"Failed to create ONVIF camera instance for {cam_name}: {e}") + # track initial failures + self.failed_cams[cam_name] = { + "retry_attempts": 0, + "last_error": str(e), + "last_attempt": time.time(), + } + return None async def _init_onvif(self, camera_name: str) -> bool: onvif: ONVIFCamera = self.cams[camera_name]["onvif"] @@ -548,7 +564,7 @@ class OnvifController: self, camera_name: str, command: OnvifCommandEnum, param: str = "" ) -> None: if camera_name not in self.cams.keys(): - logger.error(f"Onvif is not setup for {camera_name}") + logger.error(f"ONVIF is not configured for {camera_name}") return if not self.cams[camera_name]["init"]: @@ -576,23 +592,94 @@ class OnvifController: except ONVIFError as e: logger.error(f"Unable to handle onvif command: {e}") - def get_camera_info(self, camera_name: str) -> dict[str, any]: - if camera_name not in self.cams.keys(): - logger.debug(f"Onvif is not setup for {camera_name}") + async def get_camera_info(self, camera_name: str) -> dict[str, any]: + """ + Get ptz capabilities and presets, attempting to reconnect if ONVIF is configured + but not initialized. + + Returns camera details including features and presets if available. + """ + if not self.config.cameras[camera_name].enabled: + logger.debug( + f"Camera {camera_name} disabled, won't try to initialize ONVIF" + ) return {} - if not self.cams[camera_name]["init"]: - asyncio.run(self._init_onvif(camera_name)) + if camera_name not in self.cams and ( + camera_name not in self.config.cameras + or not self.config.cameras[camera_name].onvif.host + ): + logger.debug(f"ONVIF is not configured for {camera_name}") + return {} - return { - "name": camera_name, - "features": self.cams[camera_name]["features"], - "presets": list(self.cams[camera_name]["presets"].keys()), - } + if camera_name in self.cams and self.cams[camera_name]["init"]: + return { + "name": camera_name, + "features": self.cams[camera_name]["features"], + "presets": list(self.cams[camera_name]["presets"].keys()), + } + + if camera_name not in self.cams and camera_name in self.config.cameras: + cam = self.config.cameras[camera_name] + result = self._create_onvif_camera(camera_name, cam) + if result: + self.cams[camera_name] = result + else: + return {} + + # Reset retry count after timeout + attempts = self.failed_cams.get(camera_name, {}).get("retry_attempts", 0) + last_attempt = self.failed_cams.get(camera_name, {}).get("last_attempt", 0) + + if last_attempt and (time.time() - last_attempt) > self.reset_timeout: + logger.debug(f"Resetting retry count for {camera_name} after timeout") + attempts = 0 + self.failed_cams[camera_name]["retry_attempts"] = 0 + + # Attempt initialization/reconnection + if attempts < self.max_retries: + logger.info( + f"Attempting ONVIF initialization for {camera_name} (retry {attempts + 1}/{self.max_retries})" + ) + try: + if await self._init_onvif(camera_name): + if camera_name in self.failed_cams: + del self.failed_cams[camera_name] + return { + "name": camera_name, + "features": self.cams[camera_name]["features"], + "presets": list(self.cams[camera_name]["presets"].keys()), + } + else: + logger.warning(f"ONVIF initialization failed for {camera_name}") + except Exception as e: + logger.error( + f"Error during ONVIF initialization for {camera_name}: {e}" + ) + if camera_name not in self.failed_cams: + self.failed_cams[camera_name] = {"retry_attempts": 0} + self.failed_cams[camera_name].update( + { + "retry_attempts": attempts + 1, + "last_error": str(e), + "last_attempt": time.time(), + } + ) + + if attempts >= self.max_retries: + remaining_time = max( + 0, int((self.reset_timeout - (time.time() - last_attempt)) / 60) + ) + logger.error( + f"Too many ONVIF initialization attempts for {camera_name}, retry in {remaining_time} minute{'s' if remaining_time != 1 else ''}" + ) + + logger.debug(f"Could not initialize ONVIF for {camera_name}") + return {} def get_service_capabilities(self, camera_name: str) -> None: if camera_name not in self.cams.keys(): - logger.error(f"Onvif is not setup for {camera_name}") + logger.error(f"ONVIF is not configured for {camera_name}") return {} if not self.cams[camera_name]["init"]: @@ -622,7 +709,7 @@ class OnvifController: def get_camera_status(self, camera_name: str) -> None: if camera_name not in self.cams.keys(): - logger.error(f"Onvif is not setup for {camera_name}") + logger.error(f"ONVIF is not configured for {camera_name}") return {} if not self.cams[camera_name]["init"]: diff --git a/frigate/record/cleanup.py b/frigate/record/cleanup.py index e526b020d..c86c81859 100644 --- a/frigate/record/cleanup.py +++ b/frigate/record/cleanup.py @@ -12,7 +12,7 @@ from playhouse.sqlite_ext import SqliteExtDatabase from frigate.config import CameraConfig, FrigateConfig, RetainModeEnum from frigate.const import CACHE_DIR, CLIPS_DIR, MAX_WAL_SIZE, RECORD_DIR -from frigate.models import Previews, Recordings, ReviewSegment +from frigate.models import Previews, Recordings, ReviewSegment, UserReviewStatus from frigate.record.util import remove_empty_directories, sync_recordings from frigate.util.builtin import clear_and_unlink, get_tomorrow_at_time @@ -90,6 +90,10 @@ class RecordingCleanup(threading.Thread): ReviewSegment.delete().where( ReviewSegment.id << deleted_reviews_list[i : i + max_deletes] ).execute() + UserReviewStatus.delete().where( + UserReviewStatus.review_segment + << deleted_reviews_list[i : i + max_deletes] + ).execute() def expire_existing_camera_recordings( self, expire_date: float, config: CameraConfig, reviews: ReviewSegment diff --git a/frigate/test/http_api/base_http_test.py b/frigate/test/http_api/base_http_test.py index 35cda7b79..3c4a7ccdc 100644 --- a/frigate/test/http_api/base_http_test.py +++ b/frigate/test/http_api/base_http_test.py @@ -157,16 +157,14 @@ class BaseTestHttp(unittest.TestCase): start_time: float = datetime.datetime.now().timestamp(), end_time: float = datetime.datetime.now().timestamp() + 20, severity: SeverityEnum = SeverityEnum.alert, - has_been_reviewed: bool = False, data: Json = {}, - ) -> Event: + ) -> ReviewSegment: """Inserts a review segment model with a given id.""" return ReviewSegment.insert( id=id, camera="front_door", start_time=start_time, end_time=end_time, - has_been_reviewed=has_been_reviewed, severity=severity, thumb_path=False, data=data, diff --git a/frigate/test/http_api/test_http_review.py b/frigate/test/http_api/test_http_review.py index ee7d96bc5..19c589a67 100644 --- a/frigate/test/http_api/test_http_review.py +++ b/frigate/test/http_api/test_http_review.py @@ -1,16 +1,29 @@ from datetime import datetime, timedelta from fastapi.testclient import TestClient +from peewee import DoesNotExist -from frigate.models import Event, Recordings, ReviewSegment +from frigate.api.auth import get_current_user +from frigate.models import Event, Recordings, ReviewSegment, UserReviewStatus from frigate.review.types import SeverityEnum from frigate.test.http_api.base_http_test import BaseTestHttp class TestHttpReview(BaseTestHttp): def setUp(self): - super().setUp([Event, Recordings, ReviewSegment]) + super().setUp([Event, Recordings, ReviewSegment, UserReviewStatus]) self.app = super().create_app() + self.user_id = "admin" + + # Mock get_current_user for all tests + async def mock_get_current_user(): + return {"username": self.user_id, "role": "admin"} + + self.app.dependency_overrides[get_current_user] = mock_get_current_user + + def tearDown(self): + self.app.dependency_overrides.clear() + super().tearDown() def _get_reviews(self, ids: list[str]): return list( @@ -24,6 +37,13 @@ class TestHttpReview(BaseTestHttp): Recordings.select(Recordings.id).where(Recordings.id.in_(ids)).execute() ) + def _insert_user_review_status(self, review_id: str, reviewed: bool = True): + UserReviewStatus.create( + user_id=self.user_id, + review_segment=ReviewSegment.get(ReviewSegment.id == review_id), + has_been_reviewed=reviewed, + ) + #################################################################################################################### ################################### GET /review Endpoint ######################################################## #################################################################################################################### @@ -43,11 +63,14 @@ class TestHttpReview(BaseTestHttp): now = datetime.now().timestamp() with TestClient(self.app) as client: - super().insert_mock_review_segment("123456.random", now - 2, now - 1) + id = "123456.random" + super().insert_mock_review_segment(id, now - 2, now - 1) response = client.get("/review") assert response.status_code == 200 response_json = response.json() assert len(response_json) == 1 + assert response_json[0]["id"] == id + assert response_json[0]["has_been_reviewed"] == False def test_get_review_with_time_filter_no_matches(self): now = datetime.now().timestamp() @@ -391,37 +414,27 @@ class TestHttpReview(BaseTestHttp): with TestClient(self.app) as client: five_days_ago_ts = five_days_ago.timestamp() for i in range(10): + id = f"123456_{i}.random_alert_not_reviewed" super().insert_mock_review_segment( - f"123456_{i}.random_alert_not_reviewed", - five_days_ago_ts, - five_days_ago_ts, - SeverityEnum.alert, - False, + id, five_days_ago_ts, five_days_ago_ts, SeverityEnum.alert ) for i in range(10): + id = f"123456_{i}.random_alert_reviewed" super().insert_mock_review_segment( - f"123456_{i}.random_alert_reviewed", - five_days_ago_ts, - five_days_ago_ts, - SeverityEnum.alert, - True, + id, five_days_ago_ts, five_days_ago_ts, SeverityEnum.alert ) + self._insert_user_review_status(id, reviewed=True) for i in range(10): + id = f"123456_{i}.random_detection_not_reviewed" super().insert_mock_review_segment( - f"123456_{i}.random_detection_not_reviewed", - five_days_ago_ts, - five_days_ago_ts, - SeverityEnum.detection, - False, + id, five_days_ago_ts, five_days_ago_ts, SeverityEnum.detection ) for i in range(5): + id = f"123456_{i}.random_detection_reviewed" super().insert_mock_review_segment( - f"123456_{i}.random_detection_reviewed", - five_days_ago_ts, - five_days_ago_ts, - SeverityEnum.detection, - True, + id, five_days_ago_ts, five_days_ago_ts, SeverityEnum.detection ) + self._insert_user_review_status(id, reviewed=True) response = client.get("/review/summary") assert response.status_code == 200 response_json = response.json() @@ -447,6 +460,7 @@ class TestHttpReview(BaseTestHttp): #################################################################################################################### ################################### POST reviews/viewed Endpoint ################################################ #################################################################################################################### + def test_post_reviews_viewed_no_body(self): with TestClient(self.app) as client: super().insert_mock_review_segment("123456.random") @@ -473,12 +487,11 @@ class TestHttpReview(BaseTestHttp): assert response["success"] == True assert response["message"] == "Reviewed multiple items" # Verify that in DB the review segment was not changed - review_segment_in_db = ( - ReviewSegment.select(ReviewSegment.has_been_reviewed) - .where(ReviewSegment.id == id) - .get() - ) - assert review_segment_in_db.has_been_reviewed == False + with self.assertRaises(DoesNotExist): + UserReviewStatus.get( + UserReviewStatus.user_id == self.user_id, + UserReviewStatus.review_segment == "1", + ) def test_post_reviews_viewed(self): with TestClient(self.app) as client: @@ -487,16 +500,15 @@ class TestHttpReview(BaseTestHttp): body = {"ids": [id]} response = client.post("/reviews/viewed", json=body) assert response.status_code == 200 - response = response.json() - assert response["success"] == True - assert response["message"] == "Reviewed multiple items" - # Verify that in DB the review segment was changed - review_segment_in_db = ( - ReviewSegment.select(ReviewSegment.has_been_reviewed) - .where(ReviewSegment.id == id) - .get() + response_json = response.json() + assert response_json["success"] == True + assert response_json["message"] == "Reviewed multiple items" + # Verify UserReviewStatus was created + user_review = UserReviewStatus.get( + UserReviewStatus.user_id == self.user_id, + UserReviewStatus.review_segment == id, ) - assert review_segment_in_db.has_been_reviewed == True + assert user_review.has_been_reviewed == True #################################################################################################################### ################################### POST reviews/delete Endpoint ################################################ @@ -672,8 +684,7 @@ class TestHttpReview(BaseTestHttp): "camera": "front_door", "start_time": now + 1, "end_time": now + 2, - "has_been_reviewed": False, - "severity": SeverityEnum.alert, + "severity": "alert", "thumb_path": "False", "data": {"detections": {"event_id": event_id}}, }, @@ -708,8 +719,7 @@ class TestHttpReview(BaseTestHttp): "camera": "front_door", "start_time": now + 1, "end_time": now + 2, - "has_been_reviewed": False, - "severity": SeverityEnum.alert, + "severity": "alert", "thumb_path": "False", "data": {}, }, @@ -719,6 +729,7 @@ class TestHttpReview(BaseTestHttp): #################################################################################################################### ################################### DELETE /review/{review_id}/viewed Endpoint ################################## #################################################################################################################### + def test_delete_review_viewed_review_not_found(self): with TestClient(self.app) as client: review_id = "123456.random" @@ -735,11 +746,10 @@ class TestHttpReview(BaseTestHttp): with TestClient(self.app) as client: review_id = "123456.review.random" - super().insert_mock_review_segment( - review_id, now + 1, now + 2, has_been_reviewed=True - ) - review_before = ReviewSegment.get(ReviewSegment.id == review_id) - assert review_before.has_been_reviewed == True + super().insert_mock_review_segment(review_id, now + 1, now + 2) + self._insert_user_review_status(review_id, reviewed=True) + # Verify it’s reviewed before + response = client.get(f"/review/{review_id}") response = client.delete(f"/review/{review_id}/viewed") assert response.status_code == 200 @@ -749,5 +759,9 @@ class TestHttpReview(BaseTestHttp): response_json, ) - review_after = ReviewSegment.get(ReviewSegment.id == review_id) - assert review_after.has_been_reviewed == False + # Verify it’s unreviewed after + with self.assertRaises(DoesNotExist): + UserReviewStatus.get( + UserReviewStatus.user_id == self.user_id, + UserReviewStatus.review_segment == review_id, + ) diff --git a/migrations/030_create_user_review_status.py b/migrations/030_create_user_review_status.py new file mode 100644 index 000000000..d24738438 --- /dev/null +++ b/migrations/030_create_user_review_status.py @@ -0,0 +1,85 @@ +"""Peewee migrations -- 030_create_user_review_status.py. + +This migration creates the UserReviewStatus table to track per-user review states, +migrates existing has_been_reviewed data from ReviewSegment to all users in the user table, +and drops the has_been_reviewed column. Rollback drops UserReviewStatus and restores the column. + +Some examples (model - class or model_name):: + + > Model = migrator.orm['model_name'] # Return model in current state by name + > migrator.sql(sql) # Run custom SQL + > migrator.python(func, *args, **kwargs) # Run python code + > migrator.create_model(Model) # Create a model (could be used as decorator) + > migrator.remove_model(model, cascade=True) # Remove a model + > migrator.add_fields(model, **fields) # Add fields to a model + > migrator.change_fields(model, **fields) # Change fields + > migrator.remove_fields(model, *field_names, cascade=True) + > migrator.rename_field(model, old_field_name, new_field_name) + > migrator.rename_table(model, new_table_name) + > migrator.add_index(model, *col_names, unique=False) + > migrator.drop_index(model, *col_names) + > migrator.add_not_null(model, *field_names) + > migrator.drop_not_null(model, *field_names) + > migrator.add_default(model, field_name, default) + +""" + +import peewee as pw + +from frigate.models import User, UserReviewStatus + +SQL = pw.SQL + + +def migrate(migrator, database, fake=False, **kwargs): + User._meta.database = database + UserReviewStatus._meta.database = database + + migrator.sql( + """ + CREATE TABLE IF NOT EXISTS "userreviewstatus" ( + "id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, + "user_id" VARCHAR(30) NOT NULL, + "review_segment_id" VARCHAR(30) NOT NULL, + "has_been_reviewed" INTEGER NOT NULL DEFAULT 0, + FOREIGN KEY ("review_segment_id") REFERENCES "reviewsegment" ("id") ON DELETE CASCADE + ) + """ + ) + + # Add unique index on (user_id, review_segment_id) + migrator.sql( + 'CREATE UNIQUE INDEX IF NOT EXISTS "userreviewstatus_user_segment" ON "userreviewstatus" ("user_id", "review_segment_id")' + ) + + # Migrate existing has_been_reviewed data to UserReviewStatus for all users + def migrate_data(): + all_users = list(User.select()) + if not all_users: + return + + cursor = database.execute_sql( + 'SELECT "id" FROM "reviewsegment" WHERE "has_been_reviewed" = 1' + ) + reviewed_segment_ids = [row[0] for row in cursor.fetchall()] + + for segment_id in reviewed_segment_ids: + for user in all_users: + UserReviewStatus.create( + user_id=user.username, + review_segment=segment_id, + has_been_reviewed=True, + ) + + if not fake: # Only run data migration if not faking + migrator.python(migrate_data) + + migrator.sql('ALTER TABLE "reviewsegment" DROP COLUMN "has_been_reviewed"') + + +def rollback(migrator, database, fake=False, **kwargs): + migrator.sql('DROP TABLE IF EXISTS "userreviewstatus"') + # Restore has_been_reviewed column to reviewsegment (no data restoration) + migrator.sql( + 'ALTER TABLE "reviewsegment" ADD COLUMN "has_been_reviewed" INTEGER NOT NULL DEFAULT 0' + ) diff --git a/web/src/components/camera/AutoUpdatingCameraImage.tsx b/web/src/components/camera/AutoUpdatingCameraImage.tsx index d97a9214a..95d90d9bd 100644 --- a/web/src/components/camera/AutoUpdatingCameraImage.tsx +++ b/web/src/components/camera/AutoUpdatingCameraImage.tsx @@ -78,7 +78,10 @@ export default function AutoUpdatingCameraImage({ let baseParam = ""; if (periodicCache && !isCached) { - baseParam = "store=1"; + const date = new Date(key); + date.setMinutes(date.getMinutes() - (date.getMinutes() % 10), 0, 0); + + baseParam = `store=1&cache=${date.getTime() / 1000}`; } else { baseParam = `cache=${key}`; } diff --git a/web/src/components/menu/GeneralSettings.tsx b/web/src/components/menu/GeneralSettings.tsx index b4d3479e7..8e131b513 100644 --- a/web/src/components/menu/GeneralSettings.tsx +++ b/web/src/components/menu/GeneralSettings.tsx @@ -275,6 +275,19 @@ export default function GeneralSettings({ className }: GeneralSettingsProps) { )} + {isAdmin && isMobile && ( + <> + + + + Configuration editor + + + + )} {t("menu.appearance")} diff --git a/web/src/components/overlay/dialog/SearchFilterDialog.tsx b/web/src/components/overlay/dialog/SearchFilterDialog.tsx index 509169602..3ffb7a552 100644 --- a/web/src/components/overlay/dialog/SearchFilterDialog.tsx +++ b/web/src/components/overlay/dialog/SearchFilterDialog.tsx @@ -256,11 +256,13 @@ function TimeRangeFilterContent({ const [endOpen, setEndOpen] = useState(false); const [afterHour, beforeHour] = useMemo(() => { - if (!timeRange || !timeRange.includes(",")) { - return [DEFAULT_TIME_RANGE_AFTER, DEFAULT_TIME_RANGE_BEFORE]; + if (Array.isArray(timeRange) && timeRange.length === 2) { + return timeRange; } - - return timeRange.split(","); + if (typeof timeRange === "string" && timeRange.includes(",")) { + return timeRange.split(","); + } + return [DEFAULT_TIME_RANGE_AFTER, DEFAULT_TIME_RANGE_BEFORE]; }, [timeRange]); const [selectedAfterHour, setSelectedAfterHour] = useState(afterHour); diff --git a/web/src/pages/FaceLibrary.tsx b/web/src/pages/FaceLibrary.tsx index 91cfa54a9..5756967d1 100644 --- a/web/src/pages/FaceLibrary.tsx +++ b/web/src/pages/FaceLibrary.tsx @@ -347,7 +347,7 @@ function TrainingGrid({ key={image} image={image} faceNames={faceNames} - threshold={config.face_recognition.threshold} + threshold={config.face_recognition.recognition_threshold} selected={selectedFaces.includes(image)} onClick={() => onClickFace(image)} onRefresh={onRefresh} diff --git a/web/src/pages/Settings.tsx b/web/src/pages/Settings.tsx index 452abf41f..0ee657e00 100644 --- a/web/src/pages/Settings.tsx +++ b/web/src/pages/Settings.tsx @@ -35,7 +35,7 @@ import MotionTunerView from "@/views/settings/MotionTunerView"; import MasksAndZonesView from "@/views/settings/MasksAndZonesView"; import AuthenticationView from "@/views/settings/AuthenticationView"; import NotificationView from "@/views/settings/NotificationsSettingsView"; -import SearchSettingsView from "@/views/settings/SearchSettingsView"; +import ClassificationSettingsView from "@/views/settings/ClassificationSettingsView"; import UiSettingsView from "@/views/settings/UiSettingsView"; import { useSearchEffect } from "@/hooks/use-overlay-state"; import { useSearchParams } from "react-router-dom"; @@ -47,7 +47,7 @@ import { useTranslation } from "react-i18next"; const allSettingsViews = [ "uiSettings", - "exploreSettings", + "classificationSettings", "cameraSettings", "masksAndZones", "motionTuner", @@ -247,8 +247,8 @@ export default function Settings() {
{page == "uiSettings" && } - {page == "exploreSettings" && ( - + {page == "classificationSettings" && ( + )} {page == "debug" && ( diff --git a/web/src/types/frigateConfig.ts b/web/src/types/frigateConfig.ts index 9b3d60606..2910118f4 100644 --- a/web/src/types/frigateConfig.ts +++ b/web/src/types/frigateConfig.ts @@ -333,7 +333,8 @@ export interface FrigateConfig { face_recognition: { enabled: boolean; - threshold: number; + detection_threshold: number; + recognition_threshold: number; }; ffmpeg: { diff --git a/web/src/views/settings/ClassificationSettingsView.tsx b/web/src/views/settings/ClassificationSettingsView.tsx new file mode 100644 index 000000000..f6ce3c37d --- /dev/null +++ b/web/src/views/settings/ClassificationSettingsView.tsx @@ -0,0 +1,449 @@ +import Heading from "@/components/ui/heading"; +import { FrigateConfig, SearchModelSize } from "@/types/frigateConfig"; +import useSWR from "swr"; +import axios from "axios"; +import ActivityIndicator from "@/components/indicators/activity-indicator"; +import { useCallback, useContext, useEffect, useState } from "react"; +import { Label } from "@/components/ui/label"; +import { Button } from "@/components/ui/button"; +import { Switch } from "@/components/ui/switch"; +import { Toaster } from "@/components/ui/sonner"; +import { toast } from "sonner"; +import { Separator } from "@/components/ui/separator"; +import { Link } from "react-router-dom"; +import { LuExternalLink } from "react-icons/lu"; +import { StatusBarMessagesContext } from "@/context/statusbar-provider"; +import { + Select, + SelectContent, + SelectGroup, + SelectItem, + SelectTrigger, +} from "@/components/ui/select"; + +type ClassificationSettings = { + search: { + enabled?: boolean; + reindex?: boolean; + model_size?: SearchModelSize; + }; + face: { + enabled?: boolean; + }; + lpr: { + enabled?: boolean; + }; +}; + +type ClassificationSettingsViewProps = { + setUnsavedChanges: React.Dispatch>; +}; +export default function ClassificationSettingsView({ + setUnsavedChanges, +}: ClassificationSettingsViewProps) { + const { data: config, mutate: updateConfig } = + useSWR("config"); + const [changedValue, setChangedValue] = useState(false); + const [isLoading, setIsLoading] = useState(false); + + const { addMessage, removeMessage } = useContext(StatusBarMessagesContext)!; + + const [classificationSettings, setClassificationSettings] = + useState({ + search: { + enabled: undefined, + reindex: undefined, + model_size: undefined, + }, + face: { + enabled: undefined, + }, + lpr: { + enabled: undefined, + }, + }); + + const [origSearchSettings, setOrigSearchSettings] = + useState({ + search: { + enabled: undefined, + reindex: undefined, + model_size: undefined, + }, + face: { + enabled: undefined, + }, + lpr: { + enabled: undefined, + }, + }); + + useEffect(() => { + if (config) { + if (classificationSettings?.search.enabled == undefined) { + setClassificationSettings({ + search: { + enabled: config.semantic_search.enabled, + reindex: config.semantic_search.reindex, + model_size: config.semantic_search.model_size, + }, + face: { + enabled: config.face_recognition.enabled, + }, + lpr: { + enabled: config.lpr.enabled, + }, + }); + } + + setOrigSearchSettings({ + search: { + enabled: config.semantic_search.enabled, + reindex: config.semantic_search.reindex, + model_size: config.semantic_search.model_size, + }, + face: { + enabled: config.face_recognition.enabled, + }, + lpr: { + enabled: config.lpr.enabled, + }, + }); + } + // we know that these deps are correct + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [config]); + + const handleClassificationConfigChange = ( + newConfig: Partial, + ) => { + setClassificationSettings((prevConfig) => ({ + search: { + ...prevConfig.search, + ...newConfig.search, + }, + face: { ...prevConfig.face, ...newConfig.face }, + lpr: { ...prevConfig.lpr, ...newConfig.lpr }, + })); + setUnsavedChanges(true); + setChangedValue(true); + }; + + const saveToConfig = useCallback(async () => { + setIsLoading(true); + + axios + .put( + `config/set?semantic_search.enabled=${classificationSettings.search.enabled ? "True" : "False"}&semantic_search.reindex=${classificationSettings.search.reindex ? "True" : "False"}&semantic_search.model_size=${classificationSettings.search.model_size}`, + { + requires_restart: 0, + }, + ) + .then((res) => { + if (res.status === 200) { + toast.success("Classification settings have been saved.", { + position: "top-center", + }); + setChangedValue(false); + updateConfig(); + } else { + toast.error(`Failed to save config changes: ${res.statusText}`, { + position: "top-center", + }); + } + }) + .catch((error) => { + const errorMessage = + error.response?.data?.message || + error.response?.data?.detail || + "Unknown error"; + toast.error(`Failed to save config changes: ${errorMessage}`, { + position: "top-center", + }); + }) + .finally(() => { + setIsLoading(false); + }); + }, [updateConfig, classificationSettings.search]); + + const onCancel = useCallback(() => { + setClassificationSettings(origSearchSettings); + setChangedValue(false); + removeMessage("search_settings", "search_settings"); + }, [origSearchSettings, removeMessage]); + + useEffect(() => { + if (changedValue) { + addMessage( + "search_settings", + `Unsaved Classification settings changes`, + undefined, + "search_settings", + ); + } else { + removeMessage("search_settings", "search_settings"); + } + // we know that these deps are correct + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [changedValue]); + + useEffect(() => { + document.title = "Classification Settings - Frigate"; + }, []); + + if (!config) { + return ; + } + + return ( +
+ +
+ + Classification Settings + + + + Semantic Search + +
+
+

+ Semantic Search in Frigate allows you to find tracked objects + within your review items using either the image itself, a + user-defined text description, or an automatically generated one. +

+ +
+ + Read the Documentation + + +
+
+
+ +
+
+ { + handleClassificationConfigChange({ + search: { enabled: isChecked }, + }); + }} + /> +
+ +
+
+
+
+ { + handleClassificationConfigChange({ + search: { reindex: isChecked }, + }); + }} + /> +
+ +
+
+
+ Re-indexing will reprocess all thumbnails and descriptions (if + enabled) and apply the embeddings on each startup.{" "} + Don't forget to disable the option after restarting! +
+
+
+
+
Model Size
+
+

+ The size of the model used for Semantic Search embeddings. +

+
    +
  • + Using small employs a quantized version of the + model that uses less RAM and runs faster on CPU with a very + negligible difference in embedding quality. +
  • +
  • + Using large employs the full Jina model and will + automatically run on the GPU if applicable. +
  • +
+
+
+ +
+
+ +
+ + + + Face Recognition + +
+
+

+ Face recognition allows people to be assigned names and when + their face is recognized Frigate will assign the person's name + as a sub label. This information is included in the UI, filters, + as well as in notifications. +

+ +
+ + Read the Documentation + + +
+
+
+ +
+
+ { + handleClassificationConfigChange({ + face: { enabled: isChecked }, + }); + }} + /> +
+ +
+
+
+ + + + + License Plate Recognition + +
+
+

+ Frigate can recognize license plates on vehicles and + automatically add the detected characters to the + recognized_license_plate field or a known name as a sub_label to + objects that are of type car. A common use case may be to read + the license plates of cars pulling into a driveway or cars + passing by on a street. +

+ +
+ + Read the Documentation + + +
+
+
+ +
+
+ { + handleClassificationConfigChange({ + lpr: { enabled: isChecked }, + }); + }} + /> +
+ +
+
+
+ + + +
+ + +
+
+
+
+ ); +} diff --git a/web/src/views/settings/SearchSettingsView.tsx b/web/src/views/settings/SearchSettingsView.tsx deleted file mode 100644 index 89ca1dcc3..000000000 --- a/web/src/views/settings/SearchSettingsView.tsx +++ /dev/null @@ -1,325 +0,0 @@ -import Heading from "@/components/ui/heading"; -import { FrigateConfig, SearchModelSize } from "@/types/frigateConfig"; -import useSWR from "swr"; -import axios from "axios"; -import ActivityIndicator from "@/components/indicators/activity-indicator"; -import { useCallback, useContext, useEffect, useState } from "react"; -import { Label } from "@/components/ui/label"; -import { Button } from "@/components/ui/button"; -import { Switch } from "@/components/ui/switch"; -import { Toaster } from "@/components/ui/sonner"; -import { toast } from "sonner"; -import { Separator } from "@/components/ui/separator"; -import { Link } from "react-router-dom"; -import { LuExternalLink } from "react-icons/lu"; -import { StatusBarMessagesContext } from "@/context/statusbar-provider"; -import { - Select, - SelectContent, - SelectGroup, - SelectItem, - SelectTrigger, -} from "@/components/ui/select"; -import { Trans, useTranslation } from "react-i18next"; - -type ExploreSettingsViewProps = { - setUnsavedChanges: React.Dispatch>; -}; - -type ExploreSettings = { - enabled?: boolean; - reindex?: boolean; - model_size?: SearchModelSize; -}; - -export default function ExploreSettingsView({ - setUnsavedChanges, -}: ExploreSettingsViewProps) { - const { t } = useTranslation("views/settings"); - const { data: config, mutate: updateConfig } = - useSWR("config"); - const [changedValue, setChangedValue] = useState(false); - const [isLoading, setIsLoading] = useState(false); - - const { addMessage, removeMessage } = useContext(StatusBarMessagesContext)!; - - const [exploreSettings, setExploreSettings] = useState({ - enabled: undefined, - reindex: undefined, - model_size: undefined, - }); - - const [origExploreSettings, setOrigExploreSettings] = - useState({ - enabled: undefined, - reindex: undefined, - model_size: undefined, - }); - - useEffect(() => { - if (config) { - if (exploreSettings?.enabled == undefined) { - setExploreSettings({ - enabled: config.semantic_search.enabled, - reindex: config.semantic_search.reindex, - model_size: config.semantic_search.model_size, - }); - } - - setOrigExploreSettings({ - enabled: config.semantic_search.enabled, - reindex: config.semantic_search.reindex, - model_size: config.semantic_search.model_size, - }); - } - // we know that these deps are correct - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [config]); - - const handleSearchConfigChange = (newConfig: Partial) => { - setExploreSettings((prevConfig) => ({ ...prevConfig, ...newConfig })); - setUnsavedChanges(true); - setChangedValue(true); - }; - - const saveToConfig = useCallback(async () => { - setIsLoading(true); - - axios - .put( - `config/set?semantic_search.enabled=${exploreSettings.enabled ? "True" : "False"}&semantic_search.reindex=${exploreSettings.reindex ? "True" : "False"}&semantic_search.model_size=${exploreSettings.model_size}`, - { - requires_restart: 0, - }, - ) - .then((res) => { - if (res.status === 200) { - toast.success(t("explore.toast.success"), { - position: "top-center", - }); - setChangedValue(false); - updateConfig(); - } else { - toast.error( - t("toast.save.error", { - errorMessage: res.statusText, - ns: "common", - }), - { - position: "top-center", - }, - ); - } - }) - .catch((error) => { - const errorMessage = - error.response?.data?.message || - error.response?.data?.detail || - "Unknown error"; - toast.error( - t("toast.save.error", { - errorMessage, - ns: "common", - }), - { - position: "top-center", - }, - ); - }) - .finally(() => { - setIsLoading(false); - }); - }, [ - updateConfig, - exploreSettings.enabled, - exploreSettings.reindex, - exploreSettings.model_size, - t, - ]); - - const onCancel = useCallback(() => { - setExploreSettings(origExploreSettings); - setChangedValue(false); - removeMessage("search_settings", "search_settings"); - }, [origExploreSettings, removeMessage]); - - useEffect(() => { - if (changedValue) { - addMessage( - "search_settings", - `Unsaved Explore settings changes`, - undefined, - "search_settings", - ); - } else { - removeMessage("search_settings", "search_settings"); - } - // we know that these deps are correct - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [changedValue]); - - useEffect(() => { - document.title = "Explore Settings - Frigate"; - }, []); - - if (!config) { - return ; - } - - return ( -
- -
- - {t("explore.title")} - - - - {t("explore.semanticSearch.title")} - -
-
-

- explore.semanticSearch.desc -

- -
- - {t("explore.semanticSearch.readTheDocumentation")} - - -
-
-
- -
-
- { - handleSearchConfigChange({ enabled: isChecked }); - }} - /> -
- -
-
-
-
- { - handleSearchConfigChange({ reindex: isChecked }); - }} - /> -
- -
-
-
- - explore.semanticSearch.reindexOnStartup.desc - -
-
-
-
-
- {t("explore.semanticSearch.modelSize.label")} -
-
-

- - explore.semanticSearch.modelSize.desc - -

-
    -
  • - - explore.semanticSearch.modelSize.small.desc - -
  • -
  • - - explore.semanticSearch.modelSize.large.desc - -
  • -
-
-
- -
-
- - -
- - -
-
-
- ); -}