Reduce idle CPU from thread pools (#24652)
CI / AMD64 Build (push) Canceled after 0s
CI / ARM Build (push) Canceled after 0s
CI / Jetson Jetpack 6 (push) Canceled after 0s
CI / AMD64 Smoke Test (push) Canceled after 0s
CI / AMD64 Extra Build (push) Canceled after 0s
CI / ARM Extra Build (push) Canceled after 0s
CI / Synaptics Build (push) Canceled after 0s
CI / Assemble and push default build (push) Canceled after 0s

* reduce idle cpu from onnxruntime and opencv thread pools

ONNX Runtime's intra-op threads kept spinning for about 1 s after every run, so a CPU enrichment called once a second or more kept most cores busy. On a 16 thread machine, arcface at one call per second used 9.4 s of CPU per call with ~360 ms latency. Enrichment sessions on the CPU execution provider now set `session.intra_op.allow_spinning` to 0, which drops that to 1.9 s with no latency change. Detector sessions and GPU providers are unchanged, since disabling spinning doubled yolov9-t latency on CPU (12 to 23 ms).

The output and embeddings processes now run OpenCV single threaded, like the camera tracking process. A 4 camera birdseye redraw at 1080p drops from 13.7 to 4.0 ms of CPU, and the full frame conversion in the enrichment processors from 23 to 2.7 ms, each adding about 1 ms of wall time.

* test that only cpu enrichment sessions disable ort spinning

* patch out the neural engine in the spinning test
This commit is contained in:
Josh Hawkins
2026-10-13 07:46:55 -06:00
committed by GitHub
parent e61ad52b67
commit b79e043976
5 changed files with 107 additions and 18 deletions
+6
View File
@@ -10,6 +10,7 @@ from json.decoder import JSONDecodeError
from multiprocessing.synchronize import Event as MpEvent
from typing import Any
import cv2
import regex
from pathvalidate import ValidationError, sanitize_filename
@@ -51,6 +52,11 @@ class EmbeddingProcess(FrigateProcess):
from .maintainer import EmbeddingMaintainer
self.pre_run_setup(self.config.logger)
# frame conversions and crops here are one-off calls, and OpenCV's
# per-core thread pool burns ~8x the CPU of running them inline
cv2.setNumThreads(1)
maintainer = EmbeddingMaintainer(
self.config,
self.metrics,