OpenCV 5 release – New DNN engine with enhanced ONNX and LLM/VLM support, Intel, Arm, and RISC-V hardware optimizations

OpenCV 5 open-source computer vision library has recently been released with a brand-new DNN (Deep Neural Network) engine that provides better ONNX coverage and enables LLM/VLM support.

The fifth version of the popular CV library also adds support for Intel, Arm, Qualcomm, and RISC-V hardware acceleration, improved 3D vision, and various new core features such as new data types, real N-dimensional and scalar support, and performance improvements.

OpenCV 5’s DNN Engine

OpenCV 4.x supports about 22% of ONNX operators, and the new DNN engine in OpenCV 5 brings coverage to over 80%.  That means models with dynamic shapes that used to fail on OpenCV 4.x, should now work, as the 5.x engine was rebuilt around a typed operation graph with proper shape inference, constant folding, and operator fusion.

The table below shows the main difference between OpenCV 4.x and OpenCV 5

Aspect	Classic engine (4.x)	New engine (5.x)
Model representation	One struct per layer, walked in order	A typed graph the engine can analyze
Shapes	Static only	Symbolic, dynamic
Subgraphs	Not supported	If and Loop supported
Fusion	Limited	QDQ, BatchNorm, Attention, MatMul, Softmax, and more
Memory	Reused per layer	A unified buffer pool that reuses memory aggressively

Since it’s quite a big change, to make sure nothing breaks, you can select four engine options:

• ENGINE_CLASSIC (1) – Force the old 4.x-style engine. Supports non-CPU backends and targets such as CUDA and OpenVINO.
• ENGINE_NEW (2) – Force the new graph engine, with fusion and dynamic shapes. Important: the new engine is CPU-only for now, although work on GPU acceleration for inference and non-CPU HAL for pre-/post-processing has started for the new DNN
• ENGINE_AUTO (3) – The default. Try the new engine first, and fall back to the classic engine if the model fails to load.
• ENGINE_ORT (4) – Use the bundled ONNX Runtime wrapper. ONNX models only, and the build must be configured with WITH_ONNXRUNTIME=ON.

While the CV in OpenCV stands for Computer Vision, OpenCV 5 can also run large language models (LLMs) and vision-language models (VLMs) directly inside the DNN module, with no separate runtime thanks to a native tokenizer and KV-cache for autoregressive decoding. Qwen 2.5, Gemma 3, PaliGemma, and the GPT-2 / GPT-4 family models work through the same Net API used for a YOLO model.

In practical terms, that means various Detection, segmentation, vision-language, and generative AI models are now supported in OpenCV 5.

Performance also looks fairly good, as shown in the table below comparing OpenCV 5’s DNN to the ONNX runtime on an Intel Core i9-14900KS machine running Ubuntu 24.04 LTS.

Model	OpenCV 5 DNN (ms)	ONNX Runtime (ms)	Difference
XFeat	6.56	8.61	31.25% faster
YOLOv8n	10.9	12.15	11.5% faster
YOLOX-S	23.46	25.16	7.24% faster
DINOv2 small	23.78	29.58	24.4% faster
RF-DETR	102.01	106.49	4.4% faster
OWLv2	1,090	1,489	36.6% faster
BiRefNet	7,178	9,503.14	32.4% faster

You’ll find more benchmarks on various platforms in the wiki on GitHub.

New Core features

Besides deep learning changes, the OpenCV core also got improvements:

• New data types. OpenCV 5 adds first-class FP16 (cv::hfloat, CV_16F) and BF16 (cv::bfloat, CV_16BF) types, plus bool, 64-bit integers, and more.
• Real N-dimensional and scalar support. cv::Mat can now represent 0D (scalar) and 1D arrays; OpenCV 5 also adds broadcasting and first-class N-D operations like transposeND and flipND
• Better performance. Up to 2x improvements on mathematical workloads, and the same code now runs across CPUs and accelerators without modification.

Some cleanups were made on the language side:

• The legacy C API is officially deprecated
• C++17 is now the minimum recommended standard, with C++20 modules planned for later 5.x releases.
• Python – NumPy 2.x support, deeper integration, and named (keyword) arguments for C++ algorithms, so you can write cv.someAlgorithm(threshold=0.5) instead of memorizing positional order.

Hardware acceleration

OpenCV 5 benefits from a redesigned Hardware Acceleration Layer (HAL) that relies on Universal Intrinsics 2.0, a single vector codebase that maps to SSE, AVX2/512, NEON, SVE, RVV, and more. It supports the following accelerators:

• Intel IPP (IPPICV) – Restructured from the original x86/x64 acceleration path. A free subset (ICV) ships by default, dispatching to SSE/AVX-optimized kernels for filtering, color conversion, and geometric transforms.
• Arm KleidiCV – HAL for AArch64 that accelerates core image processing and DNN kernels using NEON, SVE, and SME, validated on AWS Graviton 4 and Cortex-A chips. Enabled automatically on supported operations; up to 3-4x speedups measured on Arm operations like resizing and warping.
• Qualcomm FastCV – Acceleration on Snapdragon targets through the Hexagon DSP and NPU.
• RISC-V Vector (RVV) – Scalable vector support mostly enabled by the OpenCV China organization.

It’s the same high-level code for all four, and OpenCV automatically uses the best path on the hardware it runs on.

Better 3D Vision

OpenCV 5’s 3D vision supports multi-camera calibration, point cloud and mesh I/O, dense RGB-D fusion. It implements three modules:

• 3d: basic 3D geometry and vision, including I/O, geometry primitives, algorithms like ICP, and parts of SLAM.
• calib: camera calibration, including single-camera calibration and a refactored multi-camera pipeline.
• stereo: depth from stereo.

The developers claim it’s a meaningful upgrade for people working on structure-from-motion, robotics, or any kind of reconstruction.

You’ll find fore more details about OpenCV 5 in the announcement and the revamped documentation website, while the source code is hosted on GitHub.   The announcement mentions that the “pip version of OpenCV 5 will be released on 8th June”, but I can only see “4.13.0.92” so far, so you’d still need to build it from source for now.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress. We also use affiliate links in articles to earn commissions if you make a purchase after clicking on those links.