The other day I tested Raspberry Pi 4 with an heatsink since previous multi-threaded benchmarks clearly made the board throttle when running those without any cooling solution.
The guys at the Raspberry Pi Foundation somehow noticed my post, and I received an email from Eben Upton explaining a new Raspberry Pi 4 VLI firmware had “some thermal optimizations that are not installed by default on early production units.” I did not understand VLI at first, but eventually understood this referred to the firmware for VIA VL805 PCIe USB 3.0 controller on the board.
The Raspberry Pi Foundation provided me with a test version of the firmware, which they’ll release in the next few days, or weeks after testing is completed.
Now if you’re going to test a platform that will throttle due to overheating, it’s very important you do so at constant room temperature. I work in a office where the air conditioner is set to 28°C, so that’s about the temperature I have here.
Before going on with the test I’ve installed rpi-monitor to have nice CPU temperature charts later on:
|
1 2 3 4 5 |
sudo apt-get install dirmngr sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 2C0D3C0F sudo wget http://goo.gl/vewCLL -O /etc/apt/sources.list.d/rpimonitor.list sudo apt-get update sudo apt-get install rpimonitor |
Let’s run sbc-bench without heatsink with old VLI firmware (version 00013701):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
sudo /bin/bash ./sbc-bench.sh -c sbc-bench v0.6.7 Installing needed tools. This may take some time... Done. Checking cpufreq OPP... Done. Executing tinymembench. This will take a long time...^[[15~ Done. Executing OpenSSL benchmark. This will take 3 minutes... Done. Executing 7-zip benchmark. This will take a long time..../sbc-bench.sh: line 600: 9284 Killed "${SevenZip}" b >> ${TempLog} ./sbc-bench.sh: line 600: 9454 Killed "${SevenZip}" b >> ${TempLog} ./sbc-bench.sh: line 600: 9615 Killed "${SevenZip}" b >> ${TempLog} Done. Checking cpufreq OPP... Done. ATTENTION: Throttling has occured. Check the log for details. Memory performance: memcpy: 2695.2 MB/s memset: 3703.2 MB/s (0.5%) 7-zip total scores (3 consecutive runs): OpenSSL results: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 62423.76k 76661.25k 83079.59k 84544.17k 84863.66k 85082.11k aes-128-cbc 62536.07k 76638.74k 83104.00k 84629.16k 84956.50k 85185.88k aes-192-cbc 56289.64k 67320.53k 72100.18k 73361.75k 73569.62k 73667.93k aes-192-cbc 56288.07k 67469.87k 71959.30k 73264.47k 73566.89k 73536.85k aes-256-cbc 51019.20k 60045.33k 63557.38k 64655.02k 64948.91k 64957.10k aes-256-cbc 50838.41k 60060.05k 63685.29k 64467.29k 64935.25k 64951.64k Full results uploaded to http://ix.io/1NaQ. Please check the log for anomalies (e.g. swapping or throttling happenend) and otherwise share this URL. |
7-zip never completed, as it was killed three times due to running out of memory. Maybe it happened because with a 1GB RPi 4, we’re right at the limit. Enabling ZRAM may help.
But we do have our temperature data for the full benchmark. We started at 67°C idle, and the spike to over 80°C (11:26 to 11:30) is exactly during 7-zip multi-threaded benchmark:
Now let’s install the firmware in a terminal in the Raspberry Pi 4:
|
1 2 3 4 5 6 |
unzip vl805_update_0137a8.zip chmod a+x vl805 sudo ./vl805 -w vl805_fw_0137a8.bin VL805 FW version: 00013701 VL805 EEPROM successfully reprogrammed - please reboot to activate. sudo reboot |
For reference the tool can also be used to backup the firmware, and write to any location in the EEPROM:
|
1 2 3 4 5 6 7 8 9 10 |
./vl805 -h Usage: vl805 [<options>] where <options> are taken from: -h Show this help message -r <file> Read EEPROM -w <file> Write EEPROM -v <file> Verify EEPROM -o <offset> Specify an offset to read from/write to (for testing). N.B. Erases to the end of the EEPROM! -d Verbose debugging |
If you mess up you’ll lose USB connectivity, but user could have to ssh or serial into the device and re-run the tool to flash an older firmware to recover. It’s unclear whether early adapter will have to update the firmware manually, or it will be done automatically as part of the update process. That’s one of the reason I can’t share the files now.
Nevertheless it does seem to have some effect on idle temperature. Previously I got just under 65°C, but now I get just above 61°C once it stabilizes, so the new firmware does lower the temperature by 3 to 4°C thanks to lower power consumption. Sadly, I can’t measure the latter as my power meter is dead.
Now let’s run sbc-bench again without heatsink and the new VLI firmware (version 0137a8):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
sudo /bin/bash ./sbc-bench.sh -c WARNING: this tool is meant to run only on Debian Stretch or Ubuntu Bionic. When running on other distros results are partially meaningless or can't be collected. Press [ctrl]-[c] to stop or [enter] to continue. sbc-bench v0.6.7 Installing needed tools. This may take some time... Done. Checking cpufreq OPP... Done. Executing tinymembench. This will take a long time... Done. Executing OpenSSL benchmark. This will take 3 minutes... Done. Executing 7-zip benchmark. This will take a long time..../sbc-bench.sh: line 600: 5469 Killed taskset -c 0 "${SevenZip}" b -mmt 1 >> ${TempLog} Done. Checking cpufreq OPP... Done. ATTENTION: Throttling has occured. Check the log for details. Memory performance: memcpy: 2662.9 MB/s memset: 3719.3 MB/s (0.5%) 7-zip total scores (3 consecutive runs): 5473,5483,4938 OpenSSL results: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 62552.61k 76725.93k 83106.05k 84651.01k 85273.26k 85185.88k aes-128-cbc 62791.37k 76941.34k 82987.52k 84688.90k 85054.81k 85229.57k aes-192-cbc 56272.02k 67331.56k 72086.95k 73305.43k 73591.47k 73596.93k aes-192-cbc 56187.43k 67463.00k 71973.12k 73198.25k 73706.15k 73596.93k aes-256-cbc 51015.38k 60031.36k 63559.94k 64579.58k 64918.87k 64957.10k aes-256-cbc 46234.68k 58296.15k 63157.42k 64423.94k 64875.18k 64809.64k Full results uploaded to http://ix.io/1NaS. Please check the log for anomalies (e.g. swapping or throttling happenend) and otherwise share this URL. |
This time all three runs for 7-zip could complete for some reasons, and while throttling still did occur, it did to a lesser extent, and the temperature was clearly lower during to single thread benchmarks (~70°C vs 75°C with old firmware).
For reference, 7-zip benchmark score with heatsink averaged 5,397 points, without heatsink + old VLI firmware 4,423 points, but the “no heatsink with new VLI firmware” results are much better at 5,298 points. You’ll also note the first two runs were as good as the results with heatsink, but the last one dropped to just under 5,000 points, so for full load under and extended period of time a heatsink is still recommended for full performance. It’s still impressive what a new firmware can achieve.
You may wonder what the Raspberry Pi Foundation has changed. Thomas Kaiser may have found the reason in advance, as now ASPM (Active-State Power Management) seems to be enabled:
|
1 2 3 4 5 6 7 8 9 10 |
sudo apt install pciutils sudo lspci -vvv | grep ASPM LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <4us ClockPM+ Surprise- LLActRep- BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp- LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+ |
This was not the case with the old VLI firmware. The full lspci output can be found here.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
The latest VLI firmware is now publicly available @ https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=243500&p=1490467#p1490467
Quick test using iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 and iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1. Before: VL805 FW version: 00013701 random random kB reclen write rewrite read reread read write 102400 4 20812 25482 27329 27361 18825 25112 102400 16 70033 79674 105836 105819 72370 79878 102400 512 255415 263070 281536 291803 280565 258067 102400 1024 277021 280541 303274 315428 305137 280623 102400 16384 291279 270583 341552 344350 336558 299291 1024000 16384 320589 320593 368595 369111 12345678… Read more »
LOL, another user reports a drop in sequential read speed from 350 MB/s down to less than 2MB/s: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=243500&sid=9fc4fdb61bf2d775e4d45df107fff9aa&start=100#p1490588
Is it just me but if I ignore the Raspberry foundation guff, the the Pi4 is a Broadcom set top box failure and the worst fit for a Pi yet. I keep reading and seeing youtube reviews gushing with how this thing is amazing and it would seem the oppisite is true. Upton said 2020 for a reason, is this the most rushed non design Raspberry yet? The form factor is totally broken, the GPU is still bad ole VC4, the PSU is 5.1v! on usb-C. I have to be really critical because with the resources Raspberry have they could… Read more »
The GPU is VC6, which is totally different, for one. Form factor is meant to stay the same, as is backwards compatibility with older software. The I/O bottleneck is fixed. The myriad of other improvements are self explanatory. These are at least some of the things you fail to understand.
Interesting follow-up: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=244421#p1490852
https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/9fa62b1a31c96715aef34f25000e882ed4ac4876%5E!/#F0
usb-storage: Fix scsi-sd failure “Invalid field in cdb” for USB adapter JMicron
The patch extends the family of SATA-to-USB JMicron adapters that need
FUA to be disabled and applies the same policy for uas driver.
See details in
https://gist.github.com/lucabelluccini/9a11c48dcf1d627bbcbd8213f6de3eae Raspberry 4B – Boot from SD and rootfs on USB 6.3. If you’re using this USB drive adapter (use lsusb): Bus 002 Device 002: ID 152d:0578 JMicron Technology Corp. / JMicron USA Technology Corp. JMS567 SATA 6Gb/s bridge The disk is slow and reports errors in dmesg like the following: [ 543.071702] sd 0:0:0:0: [sda] tag#16 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD IN [ 543.071719] sd 0:0:0:0: [sda] tag#16 CDB: opcode=0x28 28 00 00 42 80 88 00 00 08 00 [ 543.111634] scsi host0: uas_eh_device_reset_handler start [ 543.262296] usb 2-1: reset SuperSpeed Gen 1 USB device number 2… Read more »
> The disk is slow and reports errors in dmesg like the following
Then don’t make the mistake and blame UAS or the UAS driver if strings containing ‘uas’ appear in
dmesgoutput. Most probably it’s just the usual ‘USB peripherals underpowered’ shit show or some firmware mismatches.Transcend TS120GMTS420 SSD in JMS578 ‘enclosure’ with JMS578 firmware v173.1.0.2 and VL805 firmware 013701 (the old one) no problems whatsoever: https://pastebin.com/raw/a1riqqu1
The net is full of stupid recommendations to disable UAS, please don’t add another one…
dont tell me, tell him, as you see Thomas already did.
also read Hans de Goedes patch below this https://bugzilla.redhat.com/show_bug.cgi?id=1315013#c11
people also seem to forget it wasnt that long ago they where debating how to optimise linux scsi as it was limiting thoughput… https://lwn.net/Articles/548368/ LSFMM: Reducing SCSI latency …By Jonathan Corbet April 25, 2013 LSFMM Summit 2013 As was noted often during the 2013 LSFMM Summit, the speed of storage devices is increasing rapidly, with the result that the Linux storage stack is having a hard time driving those devices at their full speed. For much of that hardware, one of the more significant parts of the stack is the SCSI layer. A session led by Bart van Assche examined… Read more »
Technically speaking, VLI/VIA Labs, is a separate business entity from VIA, I don’t know why they did this, but a lot business things like this makes little to no sense. Usually upgrading USB host controller firmwares is painless, as long as you don’t lost power, not much to mess up imho. Back in the days, NEC used to release firmwares on an almost bi-monthly schedule, but I guess that was when USB 3.0 was far from as mature as it is today. This is still an odd thing, as USB host controllers are generally not all that power hungry on… Read more »
Curios whether this USB storage torture test will still work with the new firmware: https://forum.openmediavault.org/index.php/Thread/27710-Raspberry-pi-4-announced-better-than-3/?postID=207330#post207330
BTW: A simple way to improve performance on Raspbian especially with those 1GB variants running low on memory and then starting to swap on SD card is:
Modifying
zram-swap-config.confis optional. I changedCOMP_ALG=lzoandSWAPPINESS=100.You can install the zram-tools package in debian buster to enable zram swap.
Thanks, didn’t know that.
Hmm… just installed it. But I don’t like the defaults and will keep
zram-swap-configon non Armbian SBC (on Armbian zram is enabled by default).Are there suggestions to speed up zram device changes if swapoff is slow (refreshing swap storage by changing swap priority from device zram0 to zram1 needs swapoff)?
The multiple zram devices is a legacy thing way back to 3.14 think armbian still support 3.14.
But nope swap priority is the same and the system will handle swap with what is available without need to swapoff.
It is interesting how other users like ETA PRIME on YouTube talk about having over clocked both CPU and GPU but don’t mention throttling issues, odd ? Confusing at least.
It depends how they tested it. If they didn’t check multi-threaded workloads then it may be fine.
Room temperature also matters. If the board is in a 20C room it may not suffer from throttling at all (TBC).
Is your air con really set to 28 degrees C? That seems incredibly warm.
Yes, it’s true. I like that temperature. It’s around 32-35C outside, so it feels cool enough. Humidity matters too, so if weather outside is not that hot, and humid, I may lower it to 26 degrees C.
Lower than that I start wearing a sweater and woolly hat :).
Because they are amateurs. They watch other YouTubers and make the same videos. Never found anything new on ETA PRIME channel.
Just waiting for idle at 2.5W and i will buy this nice board or his sister!
Hope firmware update will put idle under 2.5W!
With an USB Network boot this board is a killer… Who want to wait 5 years or more to have h264 encode, get usable GPU acceleration for browser or QT framework?
Real opensource soc like H3 or RK3399 miss something important, maybe crow-founding Dev delivering lib binary to be paid and after release the code when it’s finish?
This is why the people will buy the RPis. The support and community is just fantastic. H265 4k desktop and bit banging i3c using pure python provided by the same board. You could even use the same board as a NAS (containers).
Didn’t they start an allwinner kickstarter but there’s still no h264 support out of the box? HDMI audio pass through and cec are still missing ?
Perhaps read this?
https://www.kickstarter.com/projects/bootlin/allwinner-vpu-support-in-the-official-linux-kernel/posts/2377685
That’s what Jerry was referring to. They raised enough money for MPEG-2, H.265 and H.264 decoding, but not H.264 encoding. https://www.cnx-software.com/2018/07/23/open-source-vpu-driver-allwinner-mpeg2-h-264-video-decoding/
> allwinner … there’s still no h264 support out of the box? BS. There was h.264 support from the very beginning, h.265 and Allwinner works since 2015. But in the past only with BLOBs which is pretty much the same situation as on the RPi. The Kickstarter is about to get HW accelerated video encoding on some Allwinner SoCs without any proprietary BLOBs usable with mainline kernel. Something that’s impossible on the RPi since the whole video thing happens completely inside the closed source proprietary domain called ThreadX. The only area of change with RPi 4 is now 3D/GPU acceleration… Read more »
From a potential user’s point of view, the most important question is, whether the board supports fully GPU accelerated video when using Kodi, mplayer, Firefox, or Chromium. I don’t think LibreELEC provided images for Allwinner boards even if they provided some of these blobs. Furthermore, Kodi users expect full HDMI CEC support. Odroid C1 users learnt the hard way that the advertised CEC is not true CEC. That’s why they released C1+. Just like RPi3 gigabit ethernet is only 300 Mbps, the HDMI CEC on Odroid C1 is no CEC. http://linux-sunxi.org/Linux_mainlining_effort#Status_Matrix also mentions no HDMI audio passthrough support. DD/DTS audio… Read more »
Just like RPI 4 4K YouTube video playing screws up, even 1080p full screen hurts the eyes.
Many tester showing the truth on YouTube.
> consumers who prefer ripped or stolen pirate movies
Silly me always forgetting about the RPi’s main use case…
> I don’t think LibreELEC provided images for Allwinner boards even if they provided some of these blobs. There are LibreELEC nightly images for Allwinner A64, H3 and H6 that use mainline V4L2 stateless decoder + additional patches to provide HW accelerated MPEG-2, H264 and HEVC free from blobs. (libmali blobs is used for GUI acceleration). MPEG-2 decoder is already in mainline linux and initial H264 decoder should land in v5.3. > Something that’s impossible on the RPi since the whole video thing happens completely inside the closed source proprietary domain called ThreadX. There is a V4L2 stateful interface (bcm2835-codec)… Read more »
> MMAL interface
MMAL means ‘video done on the VideoCore by ThreadX’? And interaction happens through
/dev/vchiq?> And interaction happens through /dev/vchiq?
The bcm2835-codec driver accesses the multimedia service running on VideoCore using the vchiq-mmal driver, see https://github.com/raspberrypi/linux/blob/rpi-4.19.y/drivers/staging/vc04_services/bcm2835-codec/bcm2835-v4l2-codec.c for bcm2835-codec details.
User-space only needs to interact with a standard V4L2 m2m stateful decoder instead of using the mmal/omx library. Like other V4L2 m2m stateful decoders proprietary firmware is part of the video decoding/encoding pipeline.
> Like other V4L2 m2m stateful decoders proprietary firmware is part of the video decoding/encoding pipeline
Thanks a lot for all the explanations. Highly appreciated.
But what I still don’t understand: BLOBs are needed everywhere? Even on Allwinner hardware?
> But what I still don’t understand: BLOBs are needed everywhere? Even on Allwinner hardware? There is generally two types of decoders, stateful and stateless, and V4L2 have two API flavors to access them, regular V4L2 m2m stateful and the new V4L2 m2m stateless (request api) added in v4.20+. For stateful decoders the user-space have to provide the full bitstream and will receive decoded frames in display order. Bitstream parsing can be done in firmware code with possible assistance from HW block or the firmware could just work as a mailbox to expose HW block to ARM core or anything… Read more »
are you refering to https://lwn.net/Articles/790006/ ? oops ocn, Allwinner is not Amlogic… still interesting. Add Amlogic video decoder driver From: Maxime Jourdan To: Mauro Carvalho Chehab , Hans Verkuil Subject: [PATCH v7 0/4] Add Amlogic video decoder driver Date: Fri, 31 May 2019 11:31:22 +0200 …[V7] The Driver was moved to staging until it can pass future specification & compliance tools. [V6] Good news, the firmware situation is resolved. We have received a redistributable license from Amlogic and the firmwares have been merged in linux-firmware[5]. …It was tested primarily with ffmpeg’s v4l2-m2m implementation. For instance: $ ffmpeg -c:v mpeg2_v4l2m2m -i… Read more »
The Raspberry Pi foundation is working on further thermal/power optimizations with the SDRAM controller.
Would be funny if this would result in lower memory performance scores. Same with the new VL805 firmware and USB3 performance. The benchmark numbers generated last week will be those that remain. Even if further attempts to get the RPi in a sane consumption range will slow the board down from then on. Something similar has happened last year with the RPi 3B+ — with ThreadX release 4800f08a139d6ca1c5ecbee345ea6682e2160881 from Jun 7 2018 RPi Trading guys trashed performance on all RPi 3B+ out there by downclocking the CPU cores to 1200 MHz when 70°C have been exceeded. Details in the Explanations… Read more »
it would be nice if Eben got someone on the docs team to update their pages as no bcm2711 library or bcm2711 reference for that matter is found, in a search for “qspi” bcm2711 library only https://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md#hardware mentions Hardware The BCM2835 on the Raspberry Pi has 3 SPI Controllers. Only the SPI0 controller is available on the header. Chapter 10 in the BCM2835 ARM Peripherals datasheet describes this controller. Master modes Signal name abbreviations SCLK – Serial CLocK CE – Chip Enable (often called Chip Select) MOSI – Master Out Slave In MISO – Master In Slave Out MOMI –… Read more »
Add an issue on the docs github repo, and someone will get round to it at some point.
oc the obvious problem there is to even tell them theres a problem,you need to actually register a github account… not going to happen for the casual user, why do you think we use cnx as our prefered point of contact.. no fuss, simply post your issue and talk about it,simples
The development happens on Github, so it makes sense to post issues in the tracker over there. Not all developers will read CNX Software comment section, and it’s not the most convenient way to handle bugs.
I’m also not sure it’s good for casual users to report bugs in Github, because as casual users they may not know what they are doing, so it’s going to add noise. Most skilled people on CNX, likely already have an account on Github.
thats all true jean, but oc using the cnx no fuss way ,even Eben can come here pick a handle, talk,learn and get down to the things that bug people and so make changes at his end if he wants, and no one would be any the wiser.
something to consider on their part at least, to get away from the yes men etc
That’s all fine and dandy but the Foundation has to somehow shave another 20-30C at idle. I wish them luck in that.
I’ve already been there and bought the t-shirt: http://sarpi.fatdog.eu/index.php?p=rpi-cooling#rpi4b_thermotest
I don’t think the RPi Foundation can fully resolve these RPi4 thermal issues through firmware updates. Therefore, I prefer to use my own cooling solutions.
You wish them luck. They will need more than just luck, I believe.
>Now let’s run sbc-bench again without heatsink and the new VLI firmware (version 0137a8):
Is it true that the second test was tested without the heat sink or the new firmware?
If so, why did the temperature drop in the second test?
The second test was without heatsink (like the first), but with the new VLI firmware instead of the one that shipped with the board.
I find my Pi4 gets pretty hot simply trying to stream h264 video at 1080p60 off a TVHeadend server, and I have to set VLC to render at half size otherwise the Pi can’t keep up and drops a lot of frames.
what about disabling CPU cores?
for x in /sys/devices/system/cpu/cpu[1-3]*/online; do
echo 0 >”$x”
done
can decrease temperature?