build(tensorrt): JetPack 7 / L4T R39 image with GPU-accelerated ONNX detector

Adds a `-tensorrt-jp7` Frigate image for JetPack 7.2 / L4T R39.2 Jetson hosts,
built on nvcr.io/nvidia/tensorrt:26.02-py3-igpu (TensorRT 10.11, CUDA 13, py3.12),
keeping the existing JP6 path unchanged. ONNX Runtime GPU is built from source
(no public aarch64 onnxruntime-gpu wheel), TensorRT-Python branch is selected by
the base image, and TensorRT runtime library checks are major-version aware.

VALIDATED on a real AGX Orin (L4T R39.2 / nv_tegra_release R39 rev 2.0):
- image frigate:test-tensorrt-jp7 builds (rc=0, 16.3GB,
  sha256:c9e4d382f1603ee130ee4a7315b4f71f9461405e3785707251505e2d6d088df3)
- ONNX Runtime 1.25.1 exposes CUDAExecutionProvider, and a real Add-model
  inference RAN on the iGPU CUDA EP (functional, not just listed)
- frigate.util.model.get_ort_providers(False,"AUTO") = [CUDA, CPU] (CUDA first,
  CPU last, TensorRT EP excluded) -> the ONNX detector GPU-accelerates on JP7
- /etc/TENSORRT_VER = 10.11.0

Native `type: tensorrt` (.trt gen) + the ORT TensorRT EP stay DRAFT-GATED: the
L4T R39 host ships NO libnvdla_compiler.so (absent from host AND base image), so
`import tensorrt` and libonnxruntime_providers_tensorrt.so fail to load. ONNX
detector GPU acceleration is the supported JP7 path; native trt is deferred.

Build fixes the new noble/CUDA-13 base surfaced (beyond the plan):
- build_nginx.sh: enable deb-src for the deb822 ubuntu.sources (Ubuntu 24.04)
- tensorrt_libyolo.sh: strip -lnvToolsExt (removed in CUDA 13) + -lnvparsers
  (dropped in TensorRT 10) when those libs are absent
- docker/main: noble/py3.12 build adjustments (Dockerfile, build_sqlite_vec.sh)

Reproducible: `make -C docker/tensorrt ... local-trt-jp7` (JETPACK7_ARGS) on any
arm64 builder; built on-device only because the GPU smoke test needs the iGPU.
This commit is contained in:
Jorge Ramirez
2026-06-27 17:14:01 -07:00
parent 1ec511b66c
commit 243172fbf4
19 changed files with 486 additions and 73 deletions
@@ -10,11 +10,12 @@ set -o errexit -o nounset -o pipefail
MODEL_CACHE_DIR=${MODEL_CACHE_DIR:-"/config/model_cache/tensorrt"}
TRT_VER=${TRT_VER:-$(cat /etc/TENSORRT_VER)}
TRT_MAJOR=${TRT_VER%%.*}
OUTPUT_FOLDER="${MODEL_CACHE_DIR}/${TRT_VER}"
YOLO_MODELS=${YOLO_MODELS:-""}
# Create output folder
mkdir -p ${OUTPUT_FOLDER}
mkdir -p "${OUTPUT_FOLDER}"
FIRST_MODEL=true
MODEL_DOWNLOAD=""
@@ -28,9 +29,9 @@ fi
for model in ${YOLO_MODELS//,/ }
do
# Remove old link in case path/version changed
rm -f ${MODEL_CACHE_DIR}/${model}.trt
rm -f "${MODEL_CACHE_DIR}/${model}.trt"
if [[ ! -f ${OUTPUT_FOLDER}/${model}.trt ]]; then
if [[ ! -f "${OUTPUT_FOLDER}/${model}.trt" ]]; then
if [[ ${FIRST_MODEL} = true ]]; then
MODEL_DOWNLOAD="${model%-dla}";
MODEL_CONVERT="${model}"
@@ -40,7 +41,7 @@ do
MODEL_CONVERT+=",${model}";
fi
else
ln -s ${OUTPUT_FOLDER}/${model}.trt ${MODEL_CACHE_DIR}/${model}.trt
ln -s "${OUTPUT_FOLDER}/${model}.trt" "${MODEL_CACHE_DIR}/${model}.trt"
fi
done
@@ -50,8 +51,8 @@ if [[ -z ${MODEL_CONVERT} ]]; then
fi
# Setup ENV to select GPU for conversion
if [ ! -z ${TRT_MODEL_PREP_DEVICE+x} ]; then
if [ ! -z ${CUDA_VISIBLE_DEVICES+x} ]; then
if [ -n "${TRT_MODEL_PREP_DEVICE+x}" ]; then
if [ -n "${CUDA_VISIBLE_DEVICES+x}" ]; then
PREVIOUS_CVD="$CUDA_VISIBLE_DEVICES"
unset CUDA_VISIBLE_DEVICES
fi
@@ -67,12 +68,30 @@ if [[ "$(arch)" == "aarch64" ]]; then
if [[ ! -e /usr/lib/aarch64-linux-gnu/tegra && ! -e /usr/lib/aarch64-linux-gnu/tegra-egl ]]; then
echo "ERROR: Container must be launched with nvidia runtime"
exit 1
elif [[ ! -e /usr/lib/aarch64-linux-gnu/libnvinfer.so.8 ||
! -e /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8 ||
! -e /usr/lib/aarch64-linux-gnu/libnvparsers.so.8 ||
! -e /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.8 ]]; then
echo "ERROR: Please run the following on the HOST:"
echo " sudo apt install libnvinfer8 libnvinfer-plugin8 libnvparsers8 libnvonnxparsers8 nvidia-container"
fi
TRT_LIB_DIR=/usr/lib/aarch64-linux-gnu
REQUIRED_TRT_LIBS=(
"${TRT_LIB_DIR}/libnvinfer.so.${TRT_MAJOR}"
"${TRT_LIB_DIR}/libnvinfer_plugin.so.${TRT_MAJOR}"
"${TRT_LIB_DIR}/libnvonnxparser.so.${TRT_MAJOR}"
)
if (( TRT_MAJOR < 10 )) || [[ -e "${TRT_LIB_DIR}/libnvparsers.so.${TRT_MAJOR}" ]]; then
REQUIRED_TRT_LIBS+=("${TRT_LIB_DIR}/libnvparsers.so.${TRT_MAJOR}")
fi
MISSING_TRT_LIBS=()
for TRT_LIB in "${REQUIRED_TRT_LIBS[@]}"; do
if [[ ! -e "${TRT_LIB}" ]]; then
MISSING_TRT_LIBS+=("${TRT_LIB}")
fi
done
if (( ${#MISSING_TRT_LIBS[@]} > 0 )); then
echo "ERROR: Missing TensorRT runtime libraries:"
printf ' %s\n' "${MISSING_TRT_LIBS[@]}"
echo "ERROR: Install the matching TensorRT ${TRT_MAJOR} runtime packages and nvidia-container on the HOST."
exit 1
fi
fi
@@ -83,33 +102,33 @@ echo "Generating the following TRT Models: ${MODEL_CONVERT}"
cd /usr/local/src/tensorrt_demos/yolo
echo "Downloading yolo weights"
./download_yolo.sh $MODEL_DOWNLOAD 2> /dev/null
./download_yolo.sh "${MODEL_DOWNLOAD}" 2> /dev/null
for model in ${MODEL_CONVERT//,/ }
do
python3 yolo_to_onnx.py -m ${model%-dla} > /dev/null
python3 yolo_to_onnx.py -m "${model%-dla}" > /dev/null
echo -e "\nGenerating ${model}.trt. This may take a few minutes.\n"; start=$(date +%s)
if [[ $model == *-dla ]]; then
cmd="python3 onnx_to_tensorrt.py -m ${model%-dla} --dla_core 0"
cmd=(python3 onnx_to_tensorrt.py -m "${model%-dla}" --dla_core 0)
else
cmd="python3 onnx_to_tensorrt.py -m ${model}"
cmd=(python3 onnx_to_tensorrt.py -m "${model}")
fi
$cmd > /tmp/onnx_to_tensorrt.log || { cat /tmp/onnx_to_tensorrt.log && continue; }
"${cmd[@]}" > /tmp/onnx_to_tensorrt.log || { cat /tmp/onnx_to_tensorrt.log && continue; }
mv ${model%-dla}.trt ${OUTPUT_FOLDER}/${model}.trt;
ln -s ${OUTPUT_FOLDER}/${model}.trt ${MODEL_CACHE_DIR}/${model}.trt
mv "${model%-dla}.trt" "${OUTPUT_FOLDER}/${model}.trt";
ln -s "${OUTPUT_FOLDER}/${model}.trt" "${MODEL_CACHE_DIR}/${model}.trt"
echo "Generated ${model}.trt in $(($(date +%s)-start)) seconds"
done
# Restore ENV after conversion
if [ ! -z ${TRT_MODEL_PREP_DEVICE+x} ]; then
if [ -n "${TRT_MODEL_PREP_DEVICE+x}" ]; then
unset CUDA_VISIBLE_DEVICES
if [ ! -z ${PREVIOUS_CVD+x} ]; then
if [ -n "${PREVIOUS_CVD+x}" ]; then
export CUDA_VISIBLE_DEVICES="$PREVIOUS_CVD"
fi
fi
# Print which models exist in output folder
echo "Available tensorrt models:"
cd ${OUTPUT_FOLDER} && ls *.trt;
cd "${OUTPUT_FOLDER}" && ls -- *.trt;