Vivante NPU (found in Amlogic A311D) gets its open-source driver upstreamed to Mesa

The Teflon project is an open-source TensorFlow Lite delegate that currently supports VeriSilicon Vivante NPU and aims to support several other AI accelerators over time. The project has recently been upstreamed to Mesa meaning AI inference with open-source software will work out of the box in supported hardware once Mesa 24.1 is released in May 2024.

Teflon relies on the Etnaviv open-source driver better known for working with Vivante GPUs found, for instance, in NXP i.MX processors, but Tomeu Vizoso also worked on implementing Vivante NPU support since May 2022 thanks to a sponsorship from Libre Computer and documented the development progress on this blog.

Vivante NPU open source

The VeriSilicon Vivante “VIPNano-QI.7120” 5 TOPS NPU is found in the Amlogic A311D processor, and Tomeu tested Teflon with the Etnaviv NPU drive on both the Libre Computer AML-A311D-CC Alta and Khadas VIM3 single board computers with decent performance although still slower than with the closed-source binary provided by VeriSilicon.

  • MobileNet V1 (UINT8) – ~15 ms inference speed
  • MobileNet V2 (UINT8) – ~15.5 ms inference speed (about 6 times faster than the CPU and twice as slow as the proprietary driver)
  • SSDLite MobileDet (UINT8) – ~53 ms inference speed (about 4 times faster than the CPU and 3 times slower than what the proprietary driver)

Getting upstreamed to Mesa provides several benefits. First, the userspace driver will be automatically packaged by distros using Mesa 3D graphics library such as Debian, Ubuntu, Fedora, and Yocto, so the Vivante NPU will be supported out of the box with the open-source driver. Teflon will benefit from the Mesa project’s infrastructure (Gitlab, continuous integration, etc…) and leverage the Mesa codebase with features such as NIR intermediate representation and the Gallium internal API. You’ll find more details in the blog post linked in the second paragraph of this post.

All that means is that we now have a relatively inexpensive ($65 on Amazon) Arm SBC running mainline Linux with a 5 TOPS NPU supported by open-source drivers.

Share this:

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK Pi 4C Plus
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
3 Comments
oldest
newest
Da Xue
Da Xue
2 months ago

Tomeu has done some amazing work but also credit goes out to the others he mentioned on his blog. Neil Armstrong and BayLibre have been instrumental in making this even possible since a whole platform bringup in multiple open source projects is a major feat. Now that the initial support is in mesa, I expect the performance to exceed the proprietary implementations in a year’s time. Good times ahead. And thanks to those who purchase the our SBCs that make this work and other open source work possible.

fenom
fenom
2 months ago

!remind 1year

Khadas VIM4 SBC