Allwinner VPU gets open-source Linux driver for its H.264 hardware video encoder

Long-time readers of CNX Software may remember Bootlin’s crowdfunding campaign launched in 2018 to bring open-source Allwinner VPU drivers to take care of hardware video encoding and decoding in mainline Linux. They managed to raise enough funds (over 30,000 Euros) from small donors and several companies (Olimex, Pine64, Libre Computer, FriendlyELEC, and Orange Pi) to work on open-source VPU drivers for mainline Linux, but only for MPEG2, H264, and H265 decoding for in a range of Allwinner SoCs such as the A20 and H5, but not quite enough to cover the cost of H264 video encoding.

Five years later, Bootlin took it upon themselves to complete the work without backing or support from Allwinner and have now released an open-source Linux-kernel based V4L2 driver to support the H.264 video encoder found in Allwinner V3, V3s, and S3 camera SoCs.

Allwinner V3s open source Linux video encoding driver

The new driver builds upon earlier work by the company to support the MIPI CSI-2 camera interface and the ISP  (Image Signal Processor) in Allwinner V3/V3s/S3 processors both of which were added to mainline Linux in 2021.

Bootlin’s Linux kernel support for H.264 video encoding with the Allwinner V3/V3s/S3 platforms consists of patchsets on top of the mainline Linux Cedrus driver  (which already supports decoding) which can be found in the h264-encoding branch on Bootlin’s Linux repo on GitHub, and a dedicated userspace test tool called v4l2-cedrus-enc-test.

cedrus driver video0 decoding video1 encoding

The Cedrus driver now uses /dev/video0 for video decoding and /dev/video1 for video encoding in Allwinner V3, V3s, and S3 camera processors. Bootlin explains it’s not quite ready for inclusion in mainline Linux and full support for H.264 encoding due to the following points:

  • A new userspace API (uAPI) is needed for exposing stateless encoders in mainline Linux. See the discussion on the linux-media mailing list for details
  • Rework of the driver needs to be submitted and merged upstream;
  • Rate-control is currently not implemented and only direct QP controls are available
  • The new driver only supports the Allwinner V3/V3s/S3 platforms, but other generations use a different, yet rather similar, H.264 encoder units that could also be supported with extra work
  • Pre-processing features such as scaling and pixel format conversion are not yet supported
  • FFmpeg, GStreamer, and/or other user space libraries would need to be updated to support the stateless encoder uAPI

Bootlin’s announcement has more technical details and the company also welcomes funding from companies that want a more complete open-source Allwinner video encoder driver.

Share this:

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK 5 ITX RK3588 mini-ITX motherboard
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
23 Comments
oldest
newest
Chris
Chris
6 months ago

Five years later… It’s true that’s is a long time now we know we cannot trust this company. And it’s a waste because they have has a good start, but a true wickness in software support.

David Willmore
David Willmore
6 months ago

What are you talking about? They completed the work they started 5 years ago way back then. This is new work for a completely different chip.

sander
sander
6 months ago

“we cannot trust this company.” … which company?

I do not trust Allwinner.

Jerry
Jerry
6 months ago

Wouldn’t it be easier to just use the original RPi instead of these pirate boards?

Nita Vesa
Nita Vesa
6 months ago

What, exactly, makes these “pirate boards”? Where’s the piracy? They don’t use Raspberry Foundation’s proprietary software, they don’t use any illegally gotten chips…

tkaiser
tkaiser
6 months ago

> What, exactly, makes these “pirate boards”?

It’s called ‘Jerry’.

The person is either a really stupid Raspberry Pi Ltd. employee (who hasn’t been briefed about the lack of any HW accelerated encoding on BCM2712) or a funny troll who praises everything RPi since years here in the comments section 🙂

TrollDetector
TrollDetector
6 months ago

User Jerry identified as troll with a chance of 99.68%.

back2future
back2future
6 months ago

[ Allw. known for not supporting a very active freeware community, organized within the name linux-sunxi, developing on that company’s hardware Sunxi about Allw. “With an increased importance of cheap ARM development boards, and the size and activity of the sunxi community, Allwinner started to take notice of our community. They were beginning to show signs of opening up to linux-sunxi, and direct communication was becoming possible, giving us all hope of better future support of Allwinner SoCs. Sadly, these hopes were crushed when Allwinner joined linaro, ticked their “open source” marketing box, and cut off communication altogether.” (joined Linaro… Read more »

back2future
back2future
6 months ago

[ 2014, cnx-software Interview with Allwinner Regarding their Linaro Membership and about software development ]

Ahmed ARIF
6 months ago

What makes that chip stand out? Broadcom already offers superior encoding/decoding support and more reliable drivers. If my memory serves me right, Allwinner was hesitant to release all its driver sources publicly, mainly due to its competition with Rockchip. This seems like a waste of time, especially now that the era of RISC-V is beginning to gain momentum. It’s quite disappointing😕

Paul Kocialkowski
6 months ago

Allwinner SoCs are interesting in general because of the long-lasting community effort around them and the fact that they can run with more or less entirely free software. They are quite capable chips with a very low price tag too. The fact that Allwinner is not involved in mainline is a bit sad but it doesn’t disqualify the chip.

But yeah thanks for calling all this hard work a waste of time, it’s much appreciated 😉

6 months ago

@paul This is definitely not a “waste of time” ! 9 years ago, with my limited abilities, I tried https://github.com/stulluk/FFmpeg-Cedrus Since then, dozens of people tried to use that , and later on bootlin did a proper work and gave us HW accelerated decoding. I didn’t had enough money (AFAIR 100K USD) to hire bootin to do remaining work to have HW accelerated decoding & encoding. I am proud to meet you during ELC2019 and I loved your presentation there. Please continue your amazing work, we appreciate. Edit: And please don’t get affected by random toxicity all over the internet.… Read more »

Paul Kocialkowski
Paul Kocialkowski
6 months ago

Thanks for the kind words! We now have free software support for decode on Allwinner (cedrus), Rockchip (verisilicon/hantro and rkvdec) and a few others that also use a Hantro unit. Others that are supported (stateful decoders and encoders) rely on non-free firmwares even though they have free software mainline Linux drivers. For GPUs the situation is pretty good with Mali (lima/panfrost), Vivante (Etnaviv) and a few less interesting ones (VC4/V3D/Freedreno) that only support platforms where there is no free boot software support. Imagination is now coming up with a proper free driver, but it involves a proprietary firmware running in… Read more »

6 months ago

Right but I am a little confused. On Allwinner A20 / A64, we have the “free” software for decoding, but none of the players are supported if I am not mistaken: https://linux-sunxi.org/Sunxi-Cedrus
As an example, if I want to develop a QT application which need to play some video in windowed mode, this is currently impossible, or am I misunderstanding ?

Paul Kocialkowski
Paul Kocialkowski
5 months ago

There is now GStreamer support for stateless decoding. It hasn’t yet been integrated with FFmpeg so we recommend using GStreamer at this point.

Unturned3
Unturned3
6 months ago

Some people here knows very little beyond the Raspberry Pi. This is awesome work!

David Willmore
David Willmore
6 months ago

Don’t let the bastards wear you down, Paul.

Paul Kocialkowski
Paul Kocialkowski
6 months ago

Thanks! No worries I’m very immune to questionable opinions from random dudes on the Internet. It’s just fun to reply sometimes 🙂

David Willmore
David Willmore
6 months ago

If you guys ever do any more crowdfunding campaigns one something even near my interests, consider me in–again. You did a great job with the last one. I backed the video decoder/encoder even though I run all my SBCs headless. But it really seemed like a useful thing to have available. This work looks to be the same. I’ve never used one of the V3/S3 chips, but I know several people who’ve worked on them, so I can see the value of this. Know that there’s a lot of us out here who appreciate your work even when it doesn’t… Read more »

Paul Kocialkowski
Paul Kocialkowski
5 months ago

Thanks for the kind words!

Grumpy Old Coot
Grumpy Old Coot
6 months ago

We must remember that Allwinner’s engineers are “lazy” when it comes to changing the internal workings of a SOC’s VPU – they don’t like fixing hardware that works. It is incredibly likely that the API for the H.264 hardware in the V3/s and S3 SOCs will be translatable to future Allwinner H.264 implementations.

Paul Kocialkowski
Paul Kocialkowski
6 months ago

Fingers crossed! They have changed it a bit from the A10/A20-era, but not all that much either. And some SoCs also have a H.265 encoder nowadays!

Khadas VIM4 SBC