mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-11-01 19:30:33 +03:00
build(tensorrt): JetPack 7 / L4T R39 image with GPU-accelerated ONNX detector
Adds a `-tensorrt-jp7` Frigate image for JetPack 7.2 / L4T R39.2 Jetson hosts, built on nvcr.io/nvidia/tensorrt:26.02-py3-igpu (TensorRT 10.11, CUDA 13, py3.12), keeping the existing JP6 path unchanged. ONNX Runtime GPU is built from source (no public aarch64 onnxruntime-gpu wheel), TensorRT-Python branch is selected by the base image, and TensorRT runtime library checks are major-version aware. VALIDATED on a real AGX Orin (L4T R39.2 / nv_tegra_release R39 rev 2.0): - image frigate:test-tensorrt-jp7 builds (rc=0, 16.3GB, sha256:c9e4d382f1603ee130ee4a7315b4f71f9461405e3785707251505e2d6d088df3) - ONNX Runtime 1.25.1 exposes CUDAExecutionProvider, and a real Add-model inference RAN on the iGPU CUDA EP (functional, not just listed) - frigate.util.model.get_ort_providers(False,"AUTO") = [CUDA, CPU] (CUDA first, CPU last, TensorRT EP excluded) -> the ONNX detector GPU-accelerates on JP7 - /etc/TENSORRT_VER = 10.11.0 Native `type: tensorrt` (.trt gen) + the ORT TensorRT EP stay DRAFT-GATED: the L4T R39 host ships NO libnvdla_compiler.so (absent from host AND base image), so `import tensorrt` and libonnxruntime_providers_tensorrt.so fail to load. ONNX detector GPU acceleration is the supported JP7 path; native trt is deferred. Build fixes the new noble/CUDA-13 base surfaced (beyond the plan): - build_nginx.sh: enable deb-src for the deb822 ubuntu.sources (Ubuntu 24.04) - tensorrt_libyolo.sh: strip -lnvToolsExt (removed in CUDA 13) + -lnvparsers (dropped in TensorRT 10) when those libs are absent - docker/main: noble/py3.12 build adjustments (Dockerfile, build_sqlite_vec.sh) Reproducible: `make -C docker/tensorrt ... local-trt-jp7` (JETPACK7_ARGS) on any arm64 builder; built on-device only because the GPU smoke test needs the iGPU.
This commit is contained in:
+72
@@ -0,0 +1,72 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -euxo pipefail
|
||||
|
||||
ONNXRUNTIME_VERSION=${ONNXRUNTIME_VERSION:-1.25.1}
|
||||
ONNXRUNTIME_BRANCH=${ONNXRUNTIME_BRANCH:-rel-1.25.1}
|
||||
ORT_PARALLEL=${ORT_PARALLEL:-4}
|
||||
ORT_NVCC_THREADS=${ORT_NVCC_THREADS:-1}
|
||||
|
||||
mkdir -p /trt-wheels /workspace
|
||||
|
||||
pip3 install --upgrade pip setuptools wheel packaging numpy
|
||||
|
||||
command -v nvcc
|
||||
test -f /usr/local/cuda/include/cuda.h
|
||||
|
||||
require_header() {
|
||||
local pattern="$1"
|
||||
local header
|
||||
|
||||
for header in /usr/include/${pattern} /usr/include/aarch64-linux-gnu/${pattern}; do
|
||||
if [[ -e "${header}" ]]; then
|
||||
echo "${header}"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Missing required header matching ${pattern}" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
require_ldconfig_entry() {
|
||||
local lib_name="$1"
|
||||
|
||||
if ! ldconfig -p | grep -q "${lib_name}\\.so"; then
|
||||
echo "Missing ldconfig entry for ${lib_name}" >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
require_header "cudnn.h"
|
||||
require_header "cudnn_version*.h"
|
||||
require_header "NvInfer.h"
|
||||
require_header "NvOnnxParser.h"
|
||||
|
||||
require_ldconfig_entry "libnvinfer"
|
||||
require_ldconfig_entry "libnvinfer_plugin"
|
||||
require_ldconfig_entry "libnvonnxparser"
|
||||
|
||||
cd /workspace
|
||||
rm -rf onnxruntime
|
||||
git clone --recursive --branch "${ONNXRUNTIME_BRANCH}" https://github.com/microsoft/onnxruntime.git onnxruntime
|
||||
|
||||
cd /workspace/onnxruntime
|
||||
./build.sh \
|
||||
--config Release \
|
||||
--update \
|
||||
--build \
|
||||
--build_wheel \
|
||||
--parallel "${ORT_PARALLEL}" \
|
||||
--nvcc_threads "${ORT_NVCC_THREADS}" \
|
||||
--allow_running_as_root \
|
||||
--compile_no_warning_as_error \
|
||||
--skip_tests \
|
||||
--use_cuda \
|
||||
--cuda_home /usr/local/cuda \
|
||||
--use_tensorrt \
|
||||
--cudnn_home /usr/lib/aarch64-linux-gnu \
|
||||
--tensorrt_home /usr/lib/aarch64-linux-gnu \
|
||||
--cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF
|
||||
|
||||
cp build/Linux/Release/dist/onnxruntime_gpu-"${ONNXRUNTIME_VERSION}"-*.whl /trt-wheels/
|
||||
Reference in New Issue
Block a user