Container security hardening (phase 3, breaking) (#24081)

* Run the frigate service as the frigate user

* Run go2rtc as its own restricted user

* Run nginx as the frigate user with writable state in /tmp/nginx

* Disable bandwidth stats gracefully when not running as root

* Hand TensorRT model cache ownership to the runtime user

* Document non-root operation and per-hardware device access

* Create /media/frigate after the ownership sweep

* Assert non-root services, JWT migration, and escape hatch in CI

* only write the sweep sentinel when a media volume is mounted

* tolerate homekit config chown failures in the go2rtc run script

* chown the s6 log pipe so non-root nginx can reopen /dev/stdout

* set HOME to /config for non-root services

* run smoke nginx -t and the write probe as the runtime user

* re-own the nginx shm cache on service restart

* discard stdout for the unprivileged smoke nginx -t

* unwrap hard-wrapped prose in the installation docs

* report progress during the ownership sweep

* document EXTRA_GROUPS as the only device access path for dropped services

* expand the non-root device access docs with diagnosis steps and udev rules

* document network storage ownership and the remaining detector hardware

* skip lost+found during the ownership sweep

* hand /tmp/cache to the runtime user before services start

* make bundled models readable by the runtime user

* reload nginx by signaling the master instead of parsing its config as root

* harden root writes into unprivileged-owned paths

Restrict the sweep sentinel to a mount at or below /media/frigate so a
parent /media mount cannot bless a later-shadowed volume. Rebuild
/tmp/nginx root-owned each start so root's cp and tempio writes cannot
follow a symlink an unprivileged nginx planted in the previous run.

* collapse the duplicated sentinel comment

* add a service-runs-as-root helper for granular root services

* validate FRIGATE_ROOT_SERVICES and fail fast on unknown names

* let services listed in FRIGATE_ROOT_SERVICES skip the privilege drop

* record the root-services mode in the sentinel and sweep small trees each boot

* cache the runtime ids in the ownership helper

* chown recordings, previews, and exports to the runtime user at create

* chown the database files after init

* recommend FRIGATE_ROOT_SERVICES in the bandwidth stats warning

* assert granular root services in CI

* document FRIGATE_ROOT_SERVICES

* own every directory level created for a recording segment

* clear the cached runtime ids when ownership tests finish

* skip missing media paths in the per-boot ownership sweep

* clarify granular root services docs

* clean up

* install acl for device access grants

* grant runtime users access to mapped device nodes at boot

* assert device access grants in CI

* document automatic device access grants

* stop telling users device access needs host side setup

* clarify the non-root docs

* link the migration script to the repo

* group the manual device setup under one section

* harden against symlink attacks

/config is owned by the unprivileged runtime user after the ownership sweep, so root operations on files there could be redirected by a planted symlink.

- go2rtc HomeKit setup: replace the root yq/jq normalization and chown with an O_NOFOLLOW helper (prepare_homekit.py), so a symlink at go2rtc_homekit.yml can't redirect a root write or chown onto another file
- go2rtc binary override: ignore /config/go2rtc whenever the service runs as root, so a planted binary can't exec as root under FRIGATE_ROOT_SERVICES
- sweep sentinel: read and write it through safe-sentinel, which trusts only a root-owned regular file and never follows a symlink, so it can't be forged to skip the migration or symlinked to clobber a root file
- ownership sweep: chown with -execdir so a parent directory swapped for a symlink mid-walk can't redirect the chown out of the volume
- validate inputs: restrict DEVICE_ACL_PATHS to /dev, require nonzero numeric EXTRA_GROUPS, and reject PUID/PGID that collide with the go2rtc ids
- docs: correct the TLS key ownership note to match what actually happens

* tweak docs

* stop the ownership sweep chasing entries other mechanisms own

* keep custom binaries out of root services only under granular root
This commit is contained in:
Josh Hawkins
2026-09-12 07:30:04 -06:00
committed by Nicolas Mowen
parent 1693415375
commit 3be59c9c18
38 changed files with 1427 additions and 83 deletions
@@ -6,6 +6,24 @@ set -o errexit -o nounset -o pipefail
# Logs should be sent to stdout so that s6 can collect them
# Not `nginx -s reload`: that has root parse /tmp/nginx/conf, which the
# unprivileged nginx user can rewrite, and nginx chowns path directives on load.
function reload_nginx() {
local pid
if ! pid=$(cat /tmp/nginx/nginx.pid 2>/dev/null); then
echo "[ERROR] No nginx pid file found, not reloading"
return 0
fi
if [[ ! "$pid" =~ ^[0-9]+$ ]] || [[ "$(cat "/proc/${pid}/comm" 2>/dev/null)" != "nginx" ]]; then
echo "[ERROR] nginx pid file does not name a running nginx process, not reloading"
return 0
fi
kill -HUP "$pid"
}
echo "[INFO] Starting certsync..."
lefile="/etc/letsencrypt/live/frigate/fullchain.pem"
@@ -49,7 +67,7 @@ do
then
echo "[INFO] Reloading nginx to refresh TLS certificate"
echo "$lefile: $leprint"
/usr/local/nginx/sbin/nginx -s reload
reload_nginx
fi
sleep 60
@@ -4,6 +4,19 @@
set -o errexit -o nounset -o pipefail
runs_as_root=0
if [[ "$(id -u)" -eq 0 ]]; then
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]] || /usr/local/bin/service-runs-as-root frigate; then
runs_as_root=1
fi
fi
# /root survives s6-setuidgid and breaks cache writes after the drop; set
# before opt_in_out so the opt-out marker lands where the service will look
if [[ "$runs_as_root" -eq 0 ]]; then
export HOME=/config
fi
# opt out of openvino telemetry
if [ -e /usr/local/bin/opt_in_out ]; then
/usr/local/bin/opt_in_out --opt_out > /dev/null 2>&1
@@ -30,4 +43,8 @@ cd /opt/frigate || echo "[ERROR] Failed to change working directory to /opt/frig
# Replace the bash process with the Frigate process, redirecting stderr to stdout
exec 2>&1
exec python3 -u -m frigate
if [[ "$(id -u)" -ne 0 || "$runs_as_root" -eq 1 ]]; then
exec python3 -u -m frigate
else
exec s6-setuidgid frigate python3 -u -m frigate
fi
@@ -4,6 +4,20 @@
set -o errexit -o nounset -o pipefail
runs_as_root=0
if [[ "$(id -u)" -eq 0 ]]; then
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]] || /usr/local/bin/service-runs-as-root go2rtc; then
runs_as_root=1
fi
fi
# Root via FRIGATE_ROOT_SERVICES only; the escape hatch sweeps nothing and
# leaves no unprivileged service, so /config/go2rtc stays as safe as pre-drop.
granular_root=0
if [[ "$runs_as_root" -eq 1 && "${FRIGATE_RUN_AS_ROOT:-false}" != "true" ]]; then
granular_root=1
fi
# Logs should be sent to stdout so that s6 can collect them
function get_ip_and_port_from_supervisor() {
@@ -50,42 +64,6 @@ function set_libva_version() {
export LIBAVFORMAT_VERSION_MAJOR
}
function setup_homekit_config() {
local config_path="$1"
if [[ ! -f "${config_path}" ]]; then
echo "[INFO] Creating empty config file for HomeKit..."
: > "${config_path}"
fi
# Convert YAML to JSON for jq processing
local temp_json="/tmp/cache/homekit_config.json"
yq eval -o=json "${config_path}" > "${temp_json}" 2>/dev/null || {
echo "[WARNING] Failed to convert HomeKit config to JSON, skipping cleanup"
return 0
}
# Use jq to extract the homekit section, if it exists
local homekit_json
homekit_json=$(jq '
if has("homekit") then {homekit: .homekit} else null end
' "${temp_json}" 2>/dev/null) || homekit_json="null"
# If no homekit section, write an empty config file
if [[ "${homekit_json}" == "null" ]]; then
: > "${config_path}"
else
# Convert homekit JSON back to YAML and write to the config file
echo "${homekit_json}" | yq eval -P - > "${config_path}" 2>/dev/null || {
echo "[WARNING] Failed to convert cleaned config to YAML, creating minimal config"
: > "${config_path}"
}
fi
# Clean up temp files
rm -f "${temp_json}"
}
set_libva_version
if [[ -f "/dev/shm/go2rtc.yaml" ]]; then
@@ -106,13 +84,23 @@ else
echo "[WARNING] Unable to remove existing go2rtc config. Changes made to your frigate config file may not be recognized. Please remove the /dev/shm/go2rtc.yaml from your docker host manually."
fi
# HomeKit configuration persistence setup
# HomeKit persistence. The helper is symlink-safe; hand off to go2rtc only when dropping.
readonly homekit_config_path="/config/go2rtc_homekit.yml"
setup_homekit_config "${homekit_config_path}"
if [[ "$(id -u)" -eq 0 && "$runs_as_root" -eq 0 ]]; then
python3 /usr/local/go2rtc/prepare_homekit.py "${homekit_config_path}" --chown
chown go2rtc:go2rtc /dev/shm/go2rtc.yaml 2>/dev/null || true
else
python3 /usr/local/go2rtc/prepare_homekit.py "${homekit_config_path}"
fi
readonly config_path="/config"
if [[ -x "${config_path}/go2rtc" ]]; then
# the sweep hands /config to uid 1000, so a root service must not exec from it
if [[ "$granular_root" -eq 1 && -x "${config_path}/go2rtc" ]]; then
echo "[WARN] Ignoring '${config_path}/go2rtc' because FRIGATE_ROOT_SERVICES runs this service as root and /config is owned by the runtime user; using the embedded binary"
echo "[WARN] Use FRIGATE_RUN_AS_ROOT=true instead if you need both a custom go2rtc build and root"
readonly binary_path="/usr/local/go2rtc/bin/go2rtc"
elif [[ -x "${config_path}/go2rtc" ]]; then
readonly binary_path="${config_path}/go2rtc"
echo "[WARN] Using go2rtc binary from '${binary_path}' instead of the embedded one"
else
@@ -125,4 +113,8 @@ echo "[INFO] Starting go2rtc..."
# Use HomeKit config as the primary config so writebacks go there
# The main config from Frigate will be loaded as a secondary config
exec 2>&1
exec "${binary_path}" -config="${homekit_config_path}" -config=/dev/shm/go2rtc.yaml
if [[ "$(id -u)" -ne 0 || "$runs_as_root" -eq 1 ]]; then
exec "${binary_path}" -config="${homekit_config_path}" -config=/dev/shm/go2rtc.yaml
else
exec s6-setuidgid go2rtc "${binary_path}" -config="${homekit_config_path}" -config=/dev/shm/go2rtc.yaml
fi
+103
View File
@@ -0,0 +1,103 @@
#!/command/with-contenv bash
# shellcheck shell=bash
# Grant the runtime users access to mapped-in device nodes with POSIX ACLs,
# so --device works without host-side group or udev setup.
# No-op when: started with --user (euid != 0), FRIGATE_RUN_AS_ROOT=true,
# or FRIGATE_DEVICE_ACLS=false.
set -o errexit -o nounset -o pipefail
if [[ "$(id -u)" -ne 0 ]]; then
exit 0
fi
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]]; then
exit 0
fi
if [[ "${FRIGATE_DEVICE_ACLS:-true}" == "false" ]]; then
echo "[INFO] FRIGATE_DEVICE_ACLS=false: skipping device access grants"
exit 0
fi
shopt -s nullglob
device_globs=(
"/dev/dri/*"
"/dev/apex_*"
"/dev/hailo*"
"/dev/video*"
"/dev/kfd"
"/dev/rknpu*"
"/dev/mpp_service"
"/dev/rga"
"/dev/dma_heap/*"
"/dev/nvhost*"
"/dev/nvmap"
"/dev/nvidia*"
"/dev/memx*"
)
IFS=',' read -ra extra_globs <<< "${DEVICE_ACL_PATHS:-}"
for extra in "${extra_globs[@]}"; do
extra="${extra//[[:space:]]/}"
if [[ -z "$extra" ]]; then
continue
fi
if [[ "$extra" != /dev/* || "$extra" == *..* ]]; then
echo "[ERROR] DEVICE_ACL_PATHS entries must be under /dev, got '${extra}'" >&2
exit 1
fi
device_globs+=("$extra")
done
granted=0
failed=0
grant() {
local node="$1"
# nullglob only drops patterns that hold a metacharacter, so a literal
# table entry for absent hardware arrives here verbatim. Warn only about
# nodes that exist and could not be granted.
if [[ ! -e "$node" ]]; then
return 0
fi
local spec="u:frigate:rw,u:go2rtc:rw"
# directories need traverse or nothing under them is reachable
if [[ -d "$node" ]]; then
spec="u:frigate:rwx,u:go2rtc:rwx"
fi
if setfacl -m "$spec" "$node" 2>/dev/null; then
granted=$((granted + 1))
else
failed=$((failed + 1))
echo "[WARN] could not grant device access on ${node}; see EXTRA_GROUPS in the non-root docs for the fallback"
fi
}
for glob in "${device_globs[@]}"; do
# shellcheck disable=SC2231
for node in $glob; do
grant "$node"
done
done
# USB devices re-enumerate (the Coral uploads firmware and reattaches as a new
# node), so the directories also get a default ACL new nodes inherit. The
# inherited grant is clamped by the creating mode's group bits, which is rw on
# udev hosts (0664) and nothing on raw devtmpfs (0600); hardware-verified.
if [[ -d /dev/bus/usb ]]; then
while IFS= read -r -d '' node; do
grant "$node"
done < <(find /dev/bus/usb -mindepth 1 -print0)
while IFS= read -r -d '' dir; do
setfacl -d -m "u:frigate:rw,u:go2rtc:rw" "$dir" 2>/dev/null || \
echo "[WARN] could not set a default ACL on ${dir}; a re-enumerating USB device may lose access"
done < <(find /dev/bus/usb -type d -print0)
fi
if [[ "$failed" -gt 0 ]]; then
echo "[INFO] device access: granted ${granted} node(s), ${failed} failed"
elif [[ "$granted" -gt 0 ]]; then
echo "[INFO] device access: granted ${granted} node(s) to the runtime users"
fi
@@ -0,0 +1 @@
oneshot
@@ -0,0 +1 @@
/etc/s6-overlay/s6-rc.d/init-devices/run
@@ -2,7 +2,7 @@
# shellcheck shell=bash
# Remap the frigate user to PUID/PGID and register EXTRA_GROUPS.
# No-op when: started with --user (euid != 0), FRIGATE_RUN_AS_ROOT=true,
# or PUID/PGID already match.
# or PUID/PGID already match. FRIGATE_ROOT_SERVICES is validated here too.
set -o errexit -o nounset -o pipefail
@@ -12,10 +12,31 @@ if [[ "$(id -u)" -ne 0 ]]; then
fi
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]]; then
if [[ -n "${FRIGATE_ROOT_SERVICES:-}" ]]; then
echo "[INFO] FRIGATE_RUN_AS_ROOT=true: ignoring FRIGATE_ROOT_SERVICES"
fi
echo "[INFO] FRIGATE_RUN_AS_ROOT=true: skipping user remapping"
exit 0
fi
# a typo must fail the boot, not silently drop a service to non-root
if [[ -n "${FRIGATE_ROOT_SERVICES:-}" ]]; then
IFS=',' read -ra root_services <<< "${FRIGATE_ROOT_SERVICES}"
for entry in "${root_services[@]}"; do
entry="${entry//[[:space:]]/}"
if [[ -z "$entry" ]]; then
continue
fi
case "$entry" in
frigate|go2rtc|nginx) ;;
*)
echo "[ERROR] FRIGATE_ROOT_SERVICES contains unknown service '${entry}'; valid names are frigate, go2rtc, nginx" >&2
exit 1
;;
esac
done
fi
puid="${PUID:-1000}"
pgid="${PGID:-1000}"
@@ -32,6 +53,15 @@ if [[ "$puid" -eq 0 || "$pgid" -eq 0 ]]; then
exit 1
fi
# Colliding with the go2rtc ids would merge the two users and collapse the
# separation between the main process and the network-facing restreamer.
go2rtc_uid="$(id -u go2rtc)"
go2rtc_gid="$(id -g go2rtc)"
if [[ "$puid" -eq "$go2rtc_uid" || "$pgid" -eq "$go2rtc_gid" ]]; then
echo "[ERROR] PUID/PGID must not equal the go2rtc service ids (${go2rtc_uid}:${go2rtc_gid})." >&2
exit 1
fi
current_uid="$(id -u frigate)"
current_gid="$(id -g frigate)"
@@ -50,6 +80,10 @@ fi
# EXTRA_GROUPS: numeric host GIDs granting device access (e.g. host render/video)
if [[ -n "${EXTRA_GROUPS:-}" ]]; then
for gid in ${EXTRA_GROUPS//,/ }; do
if ! [[ "$gid" =~ ^[0-9]+$ ]] || [[ "$gid" -eq 0 ]]; then
echo "[ERROR] EXTRA_GROUPS must be nonzero numeric GIDs, got '${gid}'" >&2
exit 1
fi
if ! getent group "$gid" >/dev/null; then
groupadd -o -g "$gid" "frigate-extra-${gid}"
fi
@@ -2,4 +2,4 @@
set -e
# Wait for PID file to exist.
while ! test -f /run/nginx.pid; do sleep 1; done
while ! test -f /tmp/nginx/nginx.pid; do sleep 1; done
@@ -4,6 +4,13 @@
set -o errexit -o nounset -o pipefail
runs_as_root=0
if [[ "$(id -u)" -eq 0 ]]; then
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]] || /usr/local/bin/service-runs-as-root nginx; then
runs_as_root=1
fi
fi
# Logs should be sent to stdout so that s6 can collect them
echo "[INFO] Starting NGINX..."
@@ -59,10 +66,18 @@ function set_worker_processes() {
cpus=4
fi
# we need to catch any errors because sed will fail if user has bind mounted a custom nginx file
sed -i "s/worker_processes auto;/worker_processes ${cpus};/" /usr/local/nginx/conf/nginx.conf || true
sed -i "s/worker_processes auto;/worker_processes ${cpus};/" /tmp/nginx/conf/nginx.conf
}
# Rebuilt root-owned every start: a symlink planted by the previously
# unprivileged nginx would redirect the root cp/tempio writes below onto any
# root file. rm does not traverse symlinks; the bare mkdir fails closed if raced.
rm -rf /tmp/nginx
mkdir /tmp/nginx
mkdir -p /tmp/nginx/conf /tmp/nginx/client_body /tmp/nginx/proxy \
/tmp/nginx/fastcgi /tmp/nginx/uwsgi /tmp/nginx/scgi
cp -r /usr/local/nginx/conf/. /tmp/nginx/conf/
set_worker_processes
# ensure the directory for ACME challenges exists
@@ -87,15 +102,37 @@ nginx_settings=$(python3 /usr/local/nginx/get_nginx_settings.py)
# build templates for optional FRIGATE_BASE_PATH environment variable
echo "$nginx_settings" | \
tempio -template /usr/local/nginx/templates/base_path.gotmpl \
-out /usr/local/nginx/conf/base_path.conf
-out /tmp/nginx/conf/base_path.conf
# build templates for additional network settings
echo "$nginx_settings" | \
tempio -template /usr/local/nginx/templates/listen.gotmpl \
-out /usr/local/nginx/conf/listen.conf
-out /tmp/nginx/conf/listen.conf
if [[ "$(id -u)" -eq 0 && "$runs_as_root" -eq 0 ]]; then
chown -R frigate:frigate /tmp/nginx
# heal the cache: a root `nginx -t` chowns every cycle path to the `user` directive user
if [ -d /dev/shm/nginx_cache ]; then
chown -R frigate:frigate /dev/shm/nginx_cache
fi
# nginx reopens /dev/stdout by path for its logs, and s6 made the pipe
# root-owned 0600; without this the non-root master exits EACCES
chown frigate /dev/stdout
# self-signed certs are root-generated; tolerant because mounted certs may be :ro
if [ -f "$letsencrypt_path/privkey.pem" ]; then
chown frigate:frigate "$letsencrypt_path/privkey.pem" "$letsencrypt_path/fullchain.pem" 2>/dev/null || true
fi
fi
# Replace the bash process with the NGINX process, redirecting stderr to stdout
exec 2>&1
exec \
s6-notifyoncheck -t 30000 -n 1 \
nginx
# -e stderr: the compiled-in error log path is not writable by the runtime user
if [[ "$(id -u)" -ne 0 || "$runs_as_root" -eq 1 ]]; then
exec \
s6-notifyoncheck -t 30000 -n 1 \
nginx -e stderr -c /tmp/nginx/conf/nginx.conf
else
exec \
s6-notifyoncheck -t 30000 -n 1 \
s6-setuidgid frigate nginx -e stderr -c /tmp/nginx/conf/nginx.conf
fi
@@ -153,7 +153,50 @@ if [[ "$(id -u)" -eq 0 ]]; then
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" == "true" ]]; then
rm -f /config/.permissions_version
else
/usr/local/bin/fix-ownership --sentinel /config/.permissions_version \
# Only when a mount backs /media/frigate itself: under a parent /media
# mount, a dedicated volume added later would be shadowed and skipped
sentinel_args=(--sentinel /config/.permissions_version)
root_services_mode=""
if [[ -n "${FRIGATE_ROOT_SERVICES:-}" ]]; then
# || true: an all-empty list (",") fails grep -v and errexit would kill the boot
root_services_mode=$(tr ',' '\n' <<< "${FRIGATE_ROOT_SERVICES//[[:space:]]/}" | grep -v '^$' | sort -u | paste -sd, - || true)
if [[ -n "$root_services_mode" ]]; then
sentinel_args+=(--mode "$root_services_mode")
fi
fi
if ! awk '$2 == "/media/frigate" || $2 ~ /^\/media\/frigate\//' /proc/mounts | grep -q .; then
sentinel_args=()
fi
/usr/local/bin/fix-ownership "${sentinel_args[@]}" \
"${PUID:-1000}" "${PGID:-1000}" /config /media/frigate
# Root services write clips stragglers and caches mid-run; realign the
# small trees every boot. Recordings are chowned at create instead.
if [[ -n "$root_services_mode" ]]; then
# only sweep what exists; clips and exports appear after the first run
boot_sweep_paths=(/config)
for extra in /media/frigate/clips /media/frigate/exports; do
if [[ -d "$extra" ]]; then
boot_sweep_paths+=("$extra")
fi
done
/usr/local/bin/fix-ownership \
"${PUID:-1000}" "${PGID:-1000}" "${boot_sweep_paths[@]}"
fi
fi
fi
# Must stay after the sweep, which reads an absent /media/frigate as an
# unmounted volume rather than a swept one
if [[ "$(id -u)" -eq 0 && ! -d /media/frigate ]]; then
mkdir -p /media/frigate
if [[ "${FRIGATE_RUN_AS_ROOT:-false}" != "true" ]]; then
chown "${PUID:-1000}:${PGID:-1000}" /media/frigate
fi
fi
# usually a tmpfs mount: root-owned on arrival and outside the swept volumes
if [[ "$(id -u)" -eq 0 && "${FRIGATE_RUN_AS_ROOT:-false}" != "true" ]]; then
mkdir -p /tmp/cache
chown "${PUID:-1000}:${PGID:-1000}" /tmp/cache
fi