Files
frigate/frigate/models.py
T
Josh HawkinsandGitHub 4eb60af5a7 Implement opt-in analytics (#24440)
* Add analytics opt-in setting, prompt notice kind, and notice watermarks

* Add the analytics report schema

* Add the analytics state file

* Add the install and hardware analytics collectors

* Add the detection analytics collector

* Add the cameras analytics collector

* Add the features analytics collector

* Add the health analytics collector

* Build analytics reports from the section collectors

* Add the analytics report transport

* Send the daily analytics report from the main process

* Add the analytics preview API

* Generate the analytics schema and check it in CI

* Record the image variant in each Dockerfile

* Show the analytics preview in telemetry settings

* Document anonymous analytics

* change wording

* fixes

* update network requirements docs

* adapt analytics to free-form model scenes, split notice counts, and peewee 4

* fixes

* change log levels

* docstrings

* docstrings

* honor file-only analytics opt-outs and skip pre-consent notices

An opt-out saved through the YAML editor's "Save only" or a hand edit never reached the live config, so reports kept going out under the old install ID until a restart. The reporter now reads telemetry.analytics from the config file when a report is due, validated with TelemetryConfig so yes, on, and 1 still count as consent, and an opt-out there deletes the state file and skips the send. The first report under a new install ID also carried every notice since the counters started, including ones from before the opt-in, so a fresh state file now resets the notice watermarks before the report's snapshot.

* take the notice baseline at opt-in instead of the first send

The watermark reset ran at the first send attempt, so notices raised between the opt-in and that send, up to 45 minutes after a startup, were absorbed into the reset and never reported. The reporter now writes the new install ID and resets the watermarks as soon as it sees consent without a state file.

* report notice counts cumulatively from a database baseline

Moving watermarks after each accepted send kept the install ID and the notice baseline out of step: pre-consent notices went out, notices raised before the first send were dropped, and a send that finished after an off/on toggle overwrote the new identity's baseline. health.notices now carries cumulative counts since the install ID was created. Creating an ID copies the lifetime counts into the existing reported_* columns before the state file names it, and nothing is written after a send, so the backend diffs consecutive reports per install ID.

* write held notice repeats before the analytics baseline reset

Repeats of batched kinds like failed_login wait in memory until the stats emitter flushes them, so a baseline reset taken while some were held left them out, and the next flush counted those pre-opt-in repeats as post-consent. The reset now writes held repeats to the notice history under the same lock before copying the counts.

* seed detector inference speed from the first inference

The average started at 0 so idle detectors read 0, but that made every detector's speed climb from 0 for its first couple dozen inferences after a start. Detectors now use the same InferenceSpeed helper as enrichments, so the first inference seeds the average and idle detectors still read 0.
2026-11-06 07:43:38 -07:00

216 lines
7.2 KiB
Python

from peewee import (
BlobField,
BooleanField,
CharField,
CompositeKey,
DateTimeField,
FloatField,
ForeignKeyField,
IntegerField,
Model,
TextField,
)
from playhouse.sqlite_ext import JSONField
class Event(Model):
id = CharField(null=False, primary_key=True, max_length=30)
label = CharField(index=True, max_length=20)
sub_label = CharField(max_length=100, null=True)
camera = CharField(index=True, max_length=20)
start_time = DateTimeField()
end_time = DateTimeField()
top_score = (
FloatField()
) # TODO remove when columns can be dropped without rebuilding table
score = (
FloatField()
) # TODO remove when columns can be dropped without rebuilding table
false_positive = BooleanField()
zones = JSONField()
thumbnail = TextField()
has_clip = BooleanField(default=True)
has_snapshot = BooleanField(default=True)
region = (
JSONField()
) # TODO remove when columns can be dropped without rebuilding table
box = (
JSONField()
) # TODO remove when columns can be dropped without rebuilding table
area = (
IntegerField()
) # TODO remove when columns can be dropped without rebuilding table
retain_indefinitely = BooleanField(default=False)
ratio = FloatField(
default=1.0
) # TODO remove when columns can be dropped without rebuilding table
plus_id = CharField(max_length=30)
model_hash = CharField(max_length=32)
detector_type = CharField(max_length=32)
model_type = CharField(max_length=32)
data = JSONField() # ex: tracked object box, region, etc.
class Timeline(Model):
timestamp = DateTimeField()
camera = CharField(index=True, max_length=20)
source = CharField(index=True, max_length=20) # ex: tracked object, audio, external
source_id = CharField(index=True, max_length=30)
class_type = CharField(max_length=50) # ex: entered_zone, audio_heard
data = JSONField() # ex: tracked object id, region, box, etc.
class Regions(Model):
camera = CharField(null=False, primary_key=True, max_length=20)
grid = JSONField() # json blob of grid
last_update = DateTimeField()
class Recordings(Model):
id = CharField(null=False, primary_key=True, max_length=30)
camera = CharField(index=True, max_length=20)
path = CharField(unique=True)
start_time = DateTimeField()
end_time = DateTimeField()
duration = FloatField()
motion = IntegerField(null=True)
objects = IntegerField(null=True)
dBFS = IntegerField(null=True)
segment_size = FloatField(default=0) # this should be stored as MB
regions = IntegerField(null=True)
motion_heatmap = JSONField(null=True) # 16x16 grid, 256 values (0-255)
keyframes = JSONField(null=True) # ms offsets; NULL = unprobed (legacy rows)
stream_type = CharField(default="main", max_length=8)
has_audio = BooleanField(null=True) # NULL = unknown (legacy rows)
audio_rate = IntegerField(null=True) # Hz; NULL = unknown (legacy rows)
audio_codec = CharField(null=True, max_length=20) # NULL = unknown (legacy rows)
video_codec = CharField(null=True, max_length=20) # NULL = unknown (legacy rows)
audio_duration = FloatField(null=True) # seconds of audio samples; NULL = unknown
class ExportCase(Model):
id = CharField(null=False, primary_key=True, max_length=30)
name = CharField(index=True, max_length=100)
description = TextField(null=True)
created_at = DateTimeField()
updated_at = DateTimeField()
class Export(Model):
id = CharField(null=False, primary_key=True, max_length=30)
camera = CharField(index=True, max_length=20)
name = CharField(index=True, max_length=100)
date = DateTimeField()
video_path = CharField(unique=True)
thumb_path = CharField(unique=True)
in_progress = BooleanField()
export_case = ForeignKeyField(
ExportCase,
null=True,
backref="exports",
column_name="export_case_id",
)
# peewee adds this accessor for the export_case column at runtime
export_case_id: str | None
class ReviewSegment(Model):
id = CharField(null=False, primary_key=True, max_length=30)
camera = CharField(index=True, max_length=20)
start_time = DateTimeField()
end_time = DateTimeField()
severity = CharField(max_length=30) # alert, detection
thumb_path = CharField(unique=True)
data = JSONField() # additional data about detection like list of labels, zone, areas of significant motion
class UserReviewStatus(Model):
user_id = CharField(max_length=30)
review_segment = ForeignKeyField(ReviewSegment, backref="user_reviews")
has_been_reviewed = BooleanField(default=False)
class Meta:
indexes = ((("user_id", "review_segment"), True),)
class Previews(Model):
id = CharField(null=False, primary_key=True, max_length=30)
camera = CharField(index=True, max_length=20)
path = CharField(unique=True)
start_time = DateTimeField()
end_time = DateTimeField()
duration = FloatField()
# Used for temporary table in record/cleanup.py
class RecordingsToDelete(Model):
id = CharField(null=False, primary_key=False, max_length=30)
class Meta:
temporary = True
class User(Model):
username = CharField(null=False, primary_key=True, max_length=30)
role = CharField(
max_length=20,
default="admin",
)
password_hash = CharField(null=False, max_length=120)
password_changed_at = DateTimeField(null=True)
notification_tokens = JSONField()
@classmethod
def get_allowed_cameras(
cls, role: str, roles_dict: dict[str, list[str]], all_camera_names: set[str]
) -> list[str]:
if role not in roles_dict:
return [] # Invalid role grants no access
allowed = roles_dict[role]
if not allowed: # Empty list means all cameras
return list(all_camera_names)
return [cam for cam in allowed if cam in all_camera_names]
class Trigger(Model):
camera = CharField(max_length=20)
name = CharField()
type = CharField(max_length=10)
data = TextField()
threshold = FloatField()
model = CharField(max_length=30)
embedding = BlobField()
triggering_event_id = CharField(max_length=30)
last_triggered = DateTimeField()
class Meta:
primary_key = CompositeKey("camera", "name")
class Notice(Model):
id = CharField(null=False, primary_key=True, max_length=150)
kind = CharField(index=True, max_length=50)
scope = CharField(max_length=100, null=True)
params = JSONField()
first_seen = DateTimeField()
last_seen = DateTimeField()
count = IntegerField(default=1)
# hidden until the next occurrence
acknowledged_at = DateTimeField(null=True)
# hidden for good
muted_at = DateTimeField(null=True)
class NoticeStats(Model):
kind = CharField(null=False, primary_key=True, max_length=50)
occurrences = IntegerField(default=0)
acknowledgements = IntegerField(default=0)
mutes = IntegerField(default=0)
first_seen = DateTimeField()
last_seen = DateTimeField()
# lifetime counts when the current analytics install ID was created
reported_occurrences = IntegerField(default=0)
reported_acknowledgements = IntegerField(default=0)
reported_mutes = IntegerField(default=0)