Long-time readers of CNX Software may remember Bootlin’s crowdfunding campaign launched in 2018 to bring open-source Allwinner VPU drivers to take care of hardware video encoding and decoding in mainline Linux. They managed to raise enough funds (over 30,000 Euros) from small donors and several companies (Olimex, Pine64, Libre Computer, FriendlyELEC, and Orange Pi) to work on open-source VPU drivers for mainline Linux, but only for MPEG2, H264, and H265 decoding for in a range of Allwinner SoCs such as the A20 and H5, but not quite enough to cover the cost of H264 video encoding.
Five years later, Bootlin took it upon themselves to complete the work without backing or support from Allwinner and have now released an open-source Linux-kernel based V4L2 driver to support the H.264 video encoder found in Allwinner V3, V3s, and S3 camera SoCs.
The new driver builds upon earlier work by the company to support the MIPI CSI-2 camera interface and the ISP (Image Signal Processor) in Allwinner V3/V3s/S3 processors both of which were added to mainline Linux in 2021.
Bootlin’s Linux kernel support for H.264 video encoding with the Allwinner V3/V3s/S3 platforms consists of patchsets on top of the mainline Linux Cedrus driver (which already supports decoding) which can be found in the h264-encoding branch on Bootlin’s Linux repo on GitHub, and a dedicated userspace test tool called v4l2-cedrus-enc-test.
The Cedrus driver now uses /dev/video0 for video decoding and /dev/video1 for video encoding in Allwinner V3, V3s, and S3 camera processors. Bootlin explains it’s not quite ready for inclusion in mainline Linux and full support for H.264 encoding due to the following points:
- A new userspace API (uAPI) is needed for exposing stateless encoders in mainline Linux. See the discussion on the linux-media mailing list for details
- Rework of the driver needs to be submitted and merged upstream;
- Rate-control is currently not implemented and only direct QP controls are available
- The new driver only supports the Allwinner V3/V3s/S3 platforms, but other generations use a different, yet rather similar, H.264 encoder units that could also be supported with extra work
- Pre-processing features such as scaling and pixel format conversion are not yet supported
- FFmpeg, GStreamer, and/or other user space libraries would need to be updated to support the stateless encoder uAPI
Bootlin’s announcement has more technical details and the company also welcomes funding from companies that want a more complete open-source Allwinner video encoder driver.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
Five years later… It’s true that’s is a long time now we know we cannot trust this company. And it’s a waste because they have has a good start, but a true wickness in software support.
What are you talking about? They completed the work they started 5 years ago way back then. This is new work for a completely different chip.
“we cannot trust this company.” … which company?
I do not trust Allwinner.
Wouldn’t it be easier to just use the original RPi instead of these pirate boards?
What, exactly, makes these “pirate boards”? Where’s the piracy? They don’t use Raspberry Foundation’s proprietary software, they don’t use any illegally gotten chips…
> What, exactly, makes these “pirate boards”?
It’s called ‘Jerry’.
The person is either a really stupid Raspberry Pi Ltd. employee (who hasn’t been briefed about the lack of any HW accelerated encoding on BCM2712) or a funny troll who praises everything RPi since years here in the comments section 🙂
User Jerry identified as troll with a chance of 99.68%.
[ Allw. known for not supporting a very active freeware community, organized within the name linux-sunxi, developing on that company’s hardware
Sunxi about Allw.
“With an increased importance of cheap ARM development boards, and the size and activity of the sunxi community, Allwinner started to take notice of our community. They were beginning to show signs of opening up to linux-sunxi, and direct communication was becoming possible, giving us all hope of better future support of Allwinner SoCs. Sadly, these hopes were crushed when Allwinner joined linaro, ticked their “open source” marketing box, and cut off communication altogether.”
(joined Linaro ~10yrs ago 2014 and Linux Foundation 2015
LF, ‘About’: “The Linux Foundation is the world’s leading home for collaboration on open source software, hardware, standards, and data. [etc.]”)
(Allw. seems having difficulties with completely understanding hard open-source requirements and GPL licenses)
about Cedrus from Sunxi community:
“The hardware block which accelerates decoding and encoding of video codecs by hardware, called Video Engine, Cedar Engine or also known as VPU, was successfully reversed-engineered to the point of allowing the use of hardware decoding for the most popular video codecs. Find more at the Cedrus project wiki page.”
Linaro, ~2014:
“Linaro is the place where engineers from the world’s leading technology companies define the future of Linux on Arm. The company is a not-for-profit engineering organization with over 150 engineers working on consolidating and optimizing open source software for the Arm architecture, including developer tools, the Linux kernel, Arm power management, and other software infrastructure. Linaro is distribution neutral [etc.]”
Allwinner, ~2014:
“We will take an active role in the organization and work with Linaro and the open source community to drive new Arm technologies.”
Allwinner, ‘About’:
“Allwinner Technology, founded in 2007, is an outstanding designer dedicated to intelligent application SoC, high performance analog component and wireless connectivity IC. It is headquartered in Zhuhai China, with other R&D centers and offices in Shenzhen, Xi’an,Shanghai,Chengdu,Hengqin,Guangzhou,Beijing and HongKong,. Listed on the GEM of the Shenzhen Stock Exchange in 2015, with the stock code 300458.
Motivated by customer-oriented strategy, Allwinner aligns remarkable R&D teams with long-term core-technology investment in UHD video processing, high-performance multi-core CPU/GPU integration with AI and advanced manufacturing process in terms of high integration , ultra-low power consumption and full-stack integration platform, providing competitive turnkey solutions with considerate services. The products powered by Allwinner spread across from industry control, smart home, smart hardware, tablet, automotive electronics, robot, virtual reality, OTT box, wireless communication to analog products.”
If You are the Jerry, with some comments known for being far off from mainstream or collective knowledge, Your burden is with putting straight prejudice and biases, because of Your username here on ‘www'(?) ]
[ 2014, cnx-software Interview with Allwinner Regarding their Linaro Membership and about software development ]
What makes that chip stand out? Broadcom already offers superior encoding/decoding support and more reliable drivers. If my memory serves me right, Allwinner was hesitant to release all its driver sources publicly, mainly due to its competition with Rockchip. This seems like a waste of time, especially now that the era of RISC-V is beginning to gain momentum. It’s quite disappointing😕
The chip itself was released in 2015 (Allwinner V3) but the main news is that it has open-source Linux drivers for video encoding and decoding with the latter already working with mainline, and the former should soon. I’m not sure there are even other chips with open-source video encoding drivers. I did not know Broadcom made camera SoCs, but now I can see it.
Allwinner SoCs are interesting in general because of the long-lasting community effort around them and the fact that they can run with more or less entirely free software. They are quite capable chips with a very low price tag too. The fact that Allwinner is not involved in mainline is a bit sad but it doesn’t disqualify the chip.
But yeah thanks for calling all this hard work a waste of time, it’s much appreciated 😉
@paul This is definitely not a “waste of time” !
9 years ago, with my limited abilities, I tried https://github.com/stulluk/FFmpeg-Cedrus
Since then, dozens of people tried to use that , and later on bootlin did a proper work and gave us HW accelerated decoding.
I didn’t had enough money (AFAIR 100K USD) to hire bootin to do remaining work to have HW accelerated decoding & encoding.
I am proud to meet you during ELC2019 and I loved your presentation there.
Please continue your amazing work, we appreciate.
Edit:
And please don’t get affected by random toxicity all over the internet. OpenSource Community deserve non-binary / mainline HW encoding & decoding / 2D/3D GPU acceleration on ARM/RISC-V. If I am not mistaken, as of 2023, we still do not have this for any vendor on the mainline. Please correct me if I am wrong.
Thanks for the kind words!
We now have free software support for decode on Allwinner (cedrus), Rockchip (verisilicon/hantro and rkvdec) and a few others that also use a Hantro unit. Others that are supported (stateful decoders and encoders) rely on non-free firmwares even though they have free software mainline Linux drivers.
For GPUs the situation is pretty good with Mali (lima/panfrost), Vivante (Etnaviv) and a few less interesting ones (VC4/V3D/Freedreno) that only support platforms where there is no free boot software support. Imagination is now coming up with a proper free driver, but it involves a proprietary firmware running in the GPU. So all in all it’s not too bad and it has improved significantly over the past 10 years.
Right but I am a little confused. On Allwinner A20 / A64, we have the “free” software for decoding, but none of the players are supported if I am not mistaken: https://linux-sunxi.org/Sunxi-Cedrus
As an example, if I want to develop a QT application which need to play some video in windowed mode, this is currently impossible, or am I misunderstanding ?
There is now GStreamer support for stateless decoding. It hasn’t yet been integrated with FFmpeg so we recommend using GStreamer at this point.
Some people here knows very little beyond the Raspberry Pi. This is awesome work!
Don’t let the bastards wear you down, Paul.
Thanks! No worries I’m very immune to questionable opinions from random dudes on the Internet. It’s just fun to reply sometimes 🙂
If you guys ever do any more crowdfunding campaigns one something even near my interests, consider me in–again. You did a great job with the last one. I backed the video decoder/encoder even though I run all my SBCs headless. But it really seemed like a useful thing to have available. This work looks to be the same. I’ve never used one of the V3/S3 chips, but I know several people who’ve worked on them, so I can see the value of this. Know that there’s a lot of us out here who appreciate your work even when it doesn’t directly benefit us.
Thanks for the kind words!
We must remember that Allwinner’s engineers are “lazy” when it comes to changing the internal workings of a SOC’s VPU – they don’t like fixing hardware that works. It is incredibly likely that the API for the H.264 hardware in the V3/s and S3 SOCs will be translatable to future Allwinner H.264 implementations.
Fingers crossed! They have changed it a bit from the A10/A20-era, but not all that much either. And some SoCs also have a H.265 encoder nowadays!