* Fix content type for latest image API endpoint
Extension is an enum and .value needed to be appended. Additionally, fastapi's Response() automatically sets the content type when media_type is specified, so a Content-Type in the headers was redundant.
* Remove another unneeded Content-Type
* Generate review item summaries with requests
* Adjust logic to only send important items
* Don't mention ladder
* Adjust prompt to be more specific
* Add more relaxed nature for normal activity
* Cleanup summary
* Update ollama client
* Add more directions to analyze the frames in order
* Remove environment from prompt
* Add enum for type of classification for objects
* Update recognized license plate topic to be used as attribute updater
* Update attribute for attribute type object classification
* Cleanup
* semantic trigger test
* database and model
* config
* embeddings maintainer and trigger post-processor
* api to create, edit, delete triggers
* frontend and i18n keys
* use thumbnail and description for trigger types
* image picker tweaks
* initial sync
* thumbnail file management
* clean up logs and use saved thumbnail on frontend
* publish mqtt messages
* webpush changes to enable trigger notifications
* add enabled switch
* add triggers from explore
* renaming and deletion fixes
* fix typing
* UI updates and add last triggering event time and link
* log exception instead of return in endpoint
* highlight entry in UI when triggered
* save and delete thumbnails directly
* remove alert action for now and add descriptions
* tweaks
* clean up
* fix types
* docs
* docs tweaks
* docs
* reuse enum
* Add base class for global config updates
* Add or remove camera states
* Move camera process management to separate thread
* Move camera management fully to separate class
* Cleanup
* Stop camera processes when stop command is sent
* Start processes dynamically when needed
* Adjust
* Leave extra room in tracked object queue for two cameras
* Dynamically set extra config pieces
* Add some TODOs
* Fix type check
* Simplify config updates
* Improve typing
* Correctly handle indexed entries
* Cleanup
* Create out SHM
* Use ZMQ for signaling object detectoin is completed
* Get camera correctly created
* Cleanup for updating the cameras config
* Cleanup
* Don't enable audio if no cameras have audio transcription
* Use exact string so similar camera names don't interfere
* Add ability to update config via json body to config/set endpoint
Additionally, update the config in a single rather than multiple calls for each updated key
* fix autotracking calibration to support new config updater function
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Implement model training via ZMQ and add model states to represent training
* Get model updates working
* Improve toasts and model state
* Clean up logging
* Add back in
* Setup basic training structure
* Build out route
* Handle model configs
* Add image fetch APIs
* Implement model training screen with dataset selection
* Implement viewing of training images
* Adjust directories
* Implement viewing of images
* Add support for deleting images
* Implement full deletion
* Implement classification model training
* Improve naming
* More renaming
* Improve layout
* Reduce logging
* Cleanup
* install new packages for transcription support
* add config options
* audio maintainer modifications to support transcription
* pass main config to audio process
* embeddings support
* api and transcription post processor
* embeddings maintainer support for post processor
* live audio transcription with sherpa and faster-whisper
* update dispatcher with live transcription topic
* frontend websocket
* frontend live transcription
* frontend changes for speech events
* i18n changes
* docs
* mqtt docs
* fix linter
* use float16 and small model on gpu for real-time
* fix return value and use requestor to embed description instead of passing embeddings
* run real-time transcription in its own thread
* tweaks
* publish live transcriptions on their own topic instead of tracked_object_update
* config validator and docs
* clarify docs
* Indicate no recordings on the history timeline with gray hash marks
This commit includes a new backend API endpoint and the frontend changes needed to support this functionality
* don't show slashes for now
* Include config publisher in api
* Call update topic for passed topics
* Update zones dynamically
* Update zones internally
* Support zone and mask reset
* Handle updating objects config
* Don't put status for needing to restart Frigate
* Cleanup http tests
* Fix tests
* Fetch all license plates outside of filter component
If the swr call took a long time, the entire select component may not display. This change moves the fetch to the parent component (like sub labels).
* add loading indicator
* improve query
* Remove torch install
* notification fixes
the pubkey was not being returned if notifications was not enabled at the global level
* Put back
* single condition check for fetching and disabling button
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
When an IPv6 address that doesn't map to an IPv4 address was checked
against an IPv4 trusted proxy, we'd hit an exception because
ip.ipv4_mapped was None. Fix this by verifying ipv4_mapped is not None
Co-authored-by: me <me@me>
* Fix showing review items that span over multiple days
* Simplify
* Fix tests
* Fix unchanged value
* Allow admin as default role and viewer as passed header for proxy auth
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* fix i18n keys
* hide disable from context menu for viewers
* Fix auto live check for default dashboard and camera groups
Disabling the Automatic Live View switch in Settings should prevent streaming from occurring. Overriding any settings in a camera group will override the global setting. The check here incorrectly always returned false instead of undefined.
* clarify hardware accelerated enrichments
* clarify
* add note about detect stream to face rec docs
* add note about low end Dahuas for autotracking
* Catch invalid face box / image
* Video tab tweaks
With the changes in https://github.com/blakeblackshear/frigate/pull/18220, the video tab in the Tracked Object Details pane now correctly trims the in-browser HLS video. Because of keyframes and record/detect stream differences, we can manually subtract a couple of seconds from the event start_time to ensure the first few frames aren't cut off from the video
* Clarify
* Don't use Migraphx by default
* Provide better support for running embeddings on GPU
* correctly join cameras
* Adjust blur confidence reduction
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Only check if an object is stationary to avoid mqtt snapshot
* docs heading tweak
* Add more API descriptions
* Add missing lib for new rocm onnxruntime whl
* Update inference times to reflect better rocm performance
* Cleanup resetting tracked object activity
* remove print
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Add option to not trim clip
* Improve API
* Update snapshot for new best objects
* Fix missing strings
* Convert to separate key
* Always include bounding box on snapshots
* improve autotracking relative zooming time calculation
* update proxy docs to note the need for comma separated header roles
* Add count translation
* tracked object lifecycle i18n fix
* update speed estimation docs
* clarity
* Re-initialize onvif information when toggling camera on live view
* Move time ago to card info and add face area
* Clarify face recognition docs
* Increase minimum face recognition area
* use clipFrom to in vod module endpoint to start at the correct time
* Cleanup media api
* Don't change duration
* Use search detail dialog for face library
* Move to segment based
* Cleanup
* Add back duration modification
* clean up docs
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Fix the `Any` typing hint treewide
There has been confusion between the Any type[1] and the any function[2]
in typing hints.
[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any
* Fix typing for various frame_shape members
Frame shapes are most likely defined by height and width, so a single int
cannot express that.
* Wrap gpu stats functions in Optional[]
These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.
* Fix return type in get_latest_segment_datetime
Returns a datetime object, not an integer.
* Make the return type of FrameManager.write optional
This is necessary since the SharedMemoryFrameManager.write function can
return None.
* Fix total_seconds() return type in get_tz_modifiers
The function returns a float, not an int.
https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds
* Account for floating point results in to_relative_box
Because the function uses division the return types may either be int or
float.
* Resolve ruff deprecation warning
The config has been split into formatter and linter, and the global
options are deprecated.
* face library i18n fixes
* face library i18n fixes
* add ability to use ctrl/cmd S to save in the config editor
* Use datetime as ID
* Update metrics inference speed to start with 0 ms
* fix android formatted thumbnail
* ensure role is comma separated and stripped correctly
* improve face library deletion
- add a confirmation dialog
- add ability to select all / delete faces in collections
* Implement lazy loading for video previews
* Force GPU for large embedding model
* GPU is required
* settings i18n fixes
* Don't delete train tab
* webpush debugging logs
* Fix incorrectly copying zones
* copy path data
* Ensure that cache dir exists for Frigate+
* face docs update
* Add description to upload image step to clarify the image
* Clean up
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix onvif reinitialization
* api docs: clarify usage of clip.mp4 endpoint
* Always show train tab
* Add description to API
* catch lpr model inference exceptions
* always apply motion mask when using yolov9 plate detection
* lpr faq
* fix incorrect focus when reopening search detail dialog on video tab
* only use keyboard listener in face library when train tab is active
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* use async/await instead of asyncio.run()
* fix autotracking
* create cameras in same event loop that will use them
* more debug
* try using existing event loop instead of creating a new one
* merge dev
* fixes
* run get_camera_info onvifcontroller calls in dedicated loop
* move coroutine call with loop to api
* use asyncio for autotracking move queues
* clean up
* fix calibration
* improve exception logging
* frigate+ pane i18n fix
* catch more exceptions
* explore search result tooltip i18n fix
* i18n fix
* remove comments about deprecated strftime_fmt
* Catch producers exists but is None
* Formatting
* fix live camera view i18n
* Add default role config for proxy users
This allows users to specify a default role for users when using a proxy for auth. This can be useful for users who can't/don't want to define a header mapping for the remote-role header.
* update reference config and auth docs
* clarify face rec camera level config
* clarify auth docs
* Fix onnx not working with openvino
* Update openvino to fix failed npu plugin check
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* fix check for snapshot score
for manual events, snapScore is 0, so "0" gets displayed instead of a condition being evaluated.
* fix ongoing events from being returned for all review queries
The existing condition will include any record with a null end_time regardless of when it started, as long as the start_time is greater than the after param. With this fix, both the start time needs to be within the specified range, and for events that have already ended, their end time must be before the before param
* Base alert toggles on width not device class
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Don't limit review summary query
The /review/summary endpoint was being restricted to the last 30 days, so unreviewed alerts and detections weren't showing in the calendar before that point. This change removes the WHERE clause so that the endpoint returns all results.
* fix
* remove unneeded test
we are now returning all results so we don't need to test for > month_ago
* fix
* Correctly ensure updates are more periodic when lpr or face detection is needed
* Cleanup
* Update api schema
* Don't update for stationary objects
* Simplify check
* Remove
* Section faces by event id
* Make score keeping more robust
* layout improvements
* Cleanup dialog
* Fix clicking behavior
* Add view in explore option
* math.round
* Don't require events
* Cleanup
* Remove selection
* Don't require
* Change dialog size with snapshot
* Use filename as key
* fix key
* Rework layout for mobile
* Handle mobile landscape
* Fix train issue
* Match logic
* Move deletion logic
* Fix reprocessing
* Support creating a new face
* Translations
* Do sorting in frontend
* Adjust unknown
* Cleanup
* Set max limit to faces to recognize
* Fix sorting
* Fix
* Don't assume landmark file is downloaded
* Rewrite build model task to be asynchronous so it doesn't block the pipeline
* Handle case where face recognition does not respond
* Cleanup
* Make daemon thread
* Add ability to update Frigate+ model to latest
* UI tweaks
* further UI tweaks
* UI tweaks: add width and height, fix select
* Add placeholder while API call in progress
* Fix Frigate+ enabled check
* Fix config change lost when reloading page
* Add persistent message requiring restart
* Drop down supported detectors and dimensions
* Add width and height to display
* Update FrigatePlusSettingsView.tsx
* Temp fix for Codespaces not loading
* Add i18n, format
* remove unneeded brackets
* missing colon
* Revert "Temp fix for Codespaces not loading"
This reverts commit 75b19674ce.
* Move database and config from homeassistant /config to addon /config
* Re-implement config migration for the add-on
* Align some terms
* Improve function name
* Use local variables
* Add model.path migration
* Fix homeassistant config path
* Ensure migration scripts run before go2rtc and frigate
* Migrate all files I know
* Add ffmpeg.path migration
* Update docker/main/rootfs/etc/s6-overlay/s6-rc.d/prepare/run
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Improve some variable names and organization
* Update docs to reflect addon config dir
* Update live.md with /addon_configs
* Move addon config section to configuration doc
* Align several terminologies and improve text
* Fix webrtc example config title
* Capitalize Add-on in more places
* Improve specific add-on config dir docs
* Align bash and python scripts to prefer config.yml over config.yaml
* Support config.json in migration shell scripts
* Change docs to reflect config.yml is preferred over config.yaml
* If previous config was yaml, migrate to yaml
* Fix typo in edgetpu.md
* Fix formatting of Python files
* Remove HailoRT Beta add-on variant from docs
* Add migration for labelmap and certs
* Fix variable name
* Fix new_config_file var unset
* Fix addon config directories table
* Improve db migration to avoid migrating files like .db.bak
* Fix echo location
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Ensure doesn't fail due to missing dir
* Remove redundant settings from tabs
* Adjust selection method for mobile
* Fix button descendent error
* Ensure train is option on mobile
* Cleanup face images
* Cleanup
* Add api to run face recognition on image
* Rework save attempts option
* Cleanup mobile object pane buttons
* Adjust api signature
* Remove param
* Cleanup
* Set maximum number of face images to be kept
* Fix vertical camera scaling
* adjust wording
* Add attributes to search data
* Add button to train face from event
* Handle event id saving in API
* backend
* backend fixes
* api for search queries
* frontend
* docs
* add filterable scroll list to more filters pane for identifiers
* always publish identifier
* db migration
* db model
* assign admin role on password reset
* add role to jwt and api responses
* don't restrict api access for admins yet
* use json response
* frontend auth context
* update auth form for profile endpoint
* add access denied page
* add protected routes
* auth hook
* dialogs
* user settings view
* restrict viewer access to settings
* restrict camera functions for viewer role
* add password dialog to account menu
* spacing tweak
* migrator default to admin
* escape quotes in migrator
* ui tweaks
* tweaks
* colors
* colors
* fix merge conflict
* fix icons
* add api layer enforcement
* ui tweaks
* fix error message
* debug
* clean up
* remove print
* guard apis for admin only
* fix tests
* fix review tests
* use correct error responses from api in toasts
* add role to account menu
* recordings data pub/sub
* function to process recording stream frames
* model runner
* lpr model runner
* refactor to mixin class and use model runner
* separate out realtime and post processors
* move model and mixin folders
* basic postprocessor
* clean up
* docs
* postprocessing logic
* clean up
* return none if recordings are disabled
* run postprocessor handle_requests too
* tweak expansion
* add put endpoint
* postprocessor tweaks with endpoint
* Remove thumbnail from dict
* Create thumbnail diectory
* Cleanup handling of tracked object images
* Make thumbnail optional
* Handle cases where thumbnail is used
* Expand options for thumbnail api
* Fix up the does not exist condition
* Remove absolute usages of thumbnails
* Write thumbnails for external events
* Reduce webp quality
* Use webp everywhere in frontend
* Formatting
* Always consider all events when re-indexing
* Add thumbnail deletion and cleanup path management
* Cleanup imports
* Rename def
* Don't save thumbnail for every object
* Correct event count
* Use correct function
* Include thumbnail in query
* Remove unused
* Fix requiring exception
* add path point tracking to backend
* types
* draw paths on lifecycle pane
* make points clickable
* don't display a path if we don't have any saved path points
* only object lifecycle points should have a click handler
* change to debug log
* better debug log message
* utility functions
* backend config
* backend object speed tracking
* draw speed on debug view
* basic frontend zone editor
* remove line sorting
* fix types
* highlight line on canvas when entering value in zone edit pane
* rename vars and add validation
* ensure speed estimation is disabled when user adds more than 4 points
* pixel velocity in debug
* unit_system in config
* ability to define unit system in config
* save max speed to db
* frontend
* docs
* clarify docs
* utility functions
* backend config
* backend object speed tracking
* draw speed on debug view
* basic frontend zone editor
* remove line sorting
* fix types
* highlight line on canvas when entering value in zone edit pane
* rename vars and add validation
* ensure speed estimation is disabled when user adds more than 4 points
* pixel velocity in debug
* unit_system in config
* ability to define unit system in config
* save max speed to db
* frontend
* docs
* clarify docs
* fix duplicates from merge
* include max_estimated_speed in api responses
* add units to zone edit pane
* catch undefined
* add average speed
* clarify docs
* only track average speed when object is active
* rename vars
* ensure points and distances are ordered clockwise
* only store the last 10 speeds like score history
* remove max estimated speed
* update docs
* update docs
* fix point ordering
* improve readability
* docs inertia recommendation
* fix point ordering
* check object frame time
* add velocity angle to frontend
* docs clarity
* add frontend speed filter
* fix mqtt docs
* fix mqtt docs
* don't try to remove distances if they weren't already defined
* don't display estimates on debug view/snapshots if object is not in a speed tracking zone
* docs
* implement speed_threshold for zone presence
* docs for threshold
* better ground plane image
* improve image zone size
* add inertia to speed threshold example
* Actually send result to face registration
* Define postprocessing api and move face processing to fit
* Standardize request handling
* Standardize handling of processors
* Rename processing metrics
* Cleanup
* Standardize object end
* Update to newer formatting
* One more
* One more
* use ruamel to parse and preserve line numbers for config validation
* maintain exception for non validation errors
* fix types
* include input in log messages
* Validate faces using cosine distance and SVC
* Formatting
* Use opencv instead of face embedding
* Update docs for training data
* Adjust to score system
* Set bounds
* remove face embeddings
* Update writing images
* Add face library page
* Add ability to select file
* Install opencv deps
* Cleanup
* Use different deps
* Move deps
* Cleanup
* Only show face library for desktop
* Implement deleting
* Add ability to upload image
* Add support for uploading images
* Add basic config and face recognition table
* Reconfigure updates processing to handle face
* Crop frame to face box
* Implement face embedding calculation
* Get matching face embeddings
* Add support face recognition based on existing faces
* Use arcface face embeddings instead of generic embeddings model
* Add apis for managing faces
* Implement face uploading API
* Build out more APIs
* Add min area config
* Handle larger images
* Add more debug logs
* fix calculation
* Reduce timeout
* Small tweaks
* Use webp images
* Use facenet model
* Started unit tests for the review controller
* Revert "Started unit tests for the review controller"
This reverts commit 7746eb146f.
* Started unit tests for GET /review/activity/motion Endpoint
* Started unit tests for GET /review/event/{event_id} Endpoint
* Continued unit tests for GET /review/event/{event_id} Endpoint
* Continued unit tests for GET /review/{event_id} Endpoint
* Continued unit tests for GET /review/{review_id} Endpoint
* Added unit tests for GET /review/{review_id}/viewed Endpoint
* Added unit tests for GET /stats Endpoint
* Added unit tests for GET /events Endpoint
* Updated unit tests for GET /events Endpoint
* Deleted unit tests for /events from test_http (updated tests are now in test_http_event.py)
* Removed duplicated test for GET /review/activity/motion Endpoint
* Organize api files
* Add more API definitions for events
* Add export select by ID
* Typing fixes
* Update openapi spec
* Change type
* Fix test
* Fix message
* Fix tests
* use id instead of index for object details and scrolling
* long press package and hook
* fix long press in review
* search action group
* multi select in explore
* add bulk deletion to backend api
* clean up
* mimic behavior of review
* don't open dialog on left click when mutli selecting
* context menu on container ref
* revert long press code
* clean up
* Fix audio events in explore section
Make sure that audio events are listed in the explore section
* Update audio.py
* Hide other submit options
Only allow submits for objects only
* Use custom body for the export recordings endpoint
* Fixed usage of ExportRecordingsBody
* Updated docs to reflect changes to export endpoint
* Fix friendly name and source
* Updated openAPI spec
* Updated documentation for the review endpoint
* Updated documentation for the review/summary endpoint
* Updated documentation for the review/summary endpoint
* Documentation for the review activity audio and motion endpoints
* Added responses for more review.py endpoints
* Added responses for more review.py endpoints
* Fixed review.py responses and proper path parameter names
* Added body model for /reviews/viewed and /reviews/delete
* Updated OpenAPI specification for the review controller endpoints
* Run ruff format frigate
* Drop significant_motion
* Updated frigate-api.yaml
* Deleted total_motion
* Combine 2 models into generic
* Add reindex progress to mobile bottom bar status alert
* move menu to new component
* actions component in search footer thumbnail
* context menu for explore summary thumbnail images
* readd top_score to search query for old events
* Fix access
* Reorganize tracked object for imports
* Separate out rockchip build
* Formatting
* Use original ffmpeg build
* Fix build
* Update default search type value
* backend score filtering and sorting
* score filter frontend
* use input for score filtering
* use correct score on search thumbnail
* add popover to explain top_score
* revert sublabel score calc
* update filters logic
* fix rounding on score
* wait until default view is loaded
* don't turn button to selected style for similarity searches
* clarify language
* fix alert dialog buttons to use correct destructive variant
* use root level top_score for very old events
* better arrangement of thumbnail footer items on smaller screens
* Use cosine distance metric for vec tables
* Only apply normalization to multi modal searches
* Catch possible edge case in stddev calc
* Use sigmoid function for normalization for multi modal searches only
* Ensure we get model state on initial page load
* Only save stats for multi modal searches and only use cosine similarity for image -> image search
* :Add support for nvidia driver info
* Don't show temperature if detector isn't called coral
* Add encoder and decoder info for Nvidia GPUs
* Fix device info
* Implement GPU info for nvidia GPU
* Update web/src/views/system/GeneralMetrics.tsx
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Update web/src/views/system/GeneralMetrics.tsx
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* swap sqlite_vec for chroma in requirements
* load sqlite_vec in embeddings manager
* remove chroma and revamp Embeddings class for sqlite_vec
* manual minilm onnx inference
* remove chroma in clip model
* migrate api from chroma to sqlite_vec
* migrate event cleanup from chroma to sqlite_vec
* migrate embedding maintainer from chroma to sqlite_vec
* genai description for sqlite_vec
* load sqlite_vec in main thread db
* extend the SqliteQueueDatabase class and use peewee db.execute_sql
* search with Event type for similarity
* fix similarity search
* install and add comment about transformers
* fix normalization
* add id filter
* clean up
* clean up
* fully remove chroma and add transformers env var
* readd uvicorn for fastapi
* readd tokenizer parallelism env var
* remove chroma from docs
* remove chroma from UI
* try removing custom pysqlite3 build
* hard code limit
* optimize queries
* revert explore query
* fix query
* keep building pysqlite3
* single pass fetch and process
* remove unnecessary re-embed
* update deps
* move SqliteVecQueueDatabase to db directory
* make search thumbnail take up full size of results box
* improve typing
* improve model downloading and add status screen
* daemon downloading thread
* catch case when semantic search is disabled
* fix typing
* build sqlite_vec from source
* resolve conflict
* file permissions
* try build deps
* remove sources
* sources
* fix thread start
* include git in build
* reorder embeddings after detectors are started
* build with sqlite amalgamation
* non-platform specific
* use wget instead of curl
* remove unzip -d
* remove sqlite_vec from requirements and load the compiled version
* fix build
* avoid race in db connection
* add scale_factor and bias to description zscore normalization
* Allow embedding of snapshot for description via config option
* docs
* frontend button
* Backend
* crop snapshot to region
* only show dropdown when event has snapshot
* fix cursor on dropdown
* crop on initial generation as well
* use enum for type
* fix type
* Portal tooltips
* Add ability to time_range filter chroma searches
* centering and padding consistency
* add event id back to chroma metadata
* query sqlite first and pass those ids to chroma for embeddings search
* ensure we pass timezone to the api call
* remove object lifecycle from search details for non-object events
* simplify hour calculation
* fix query without filters
* bump chroma version
* chroma 0.5.7
* fix selecting camera group in cameras filter button
