Khadas VIM3 Linux Benchmarks and Comparison to Raspberry Pi 4

I received Khadas VIM3 Amlogic A311D SBC in early July and started testing it with Android running some benchmarks and playing games last month.

I was impressed by graphics performance and overall benchmark results in Android, especially the results I got with a heatsink matched Khadas own results with heatsink + fan. So I installed the latest Ubuntu 18.04 available at the time (July 19) in order to repeat benchmarks in Linux and see how it goes.

System info in Ubuntu 18.04:


I decided to install armbianmonitor to draw some nice temperature charts as I did with  Raspberry Pi 4:


But I had some error during installation:


And while I can load the webpage with top menu appearing, it won’t show any data, as its name implies it may only work in Armbian.

SBC Bench on Khadas VIM3

Let’s download SBC bench:


Note that I had to comment out CheckLoad in the script because Amlogic processor’s average will converge to 1.0 instead of 0.0 at idle. This happens on ODROID-N2 as well. Now we can run the benchmark on VIM3 with the provided heatsink:


Some ugly errors, but the test completed, and the full results are available here.  Wow, it’s not pretty with lots of throttling:


If we compared the results to Raspberry Pi 4 with and without heatsink, it shows something is really wrong for the multi-threaded 7-zip benchmark:

Raspberry Pi 4 vs Khadas VIM3 Linux Benchmarks
Higher is better. Units: 7-zip: MIPS; memset/memcpy: MB/s

memset and memcpy are quite good since single-threaded. AES is about 10 times faster on VIM3.

OpenSSL VIM3 vs Raspberry Pi 4
khash per second

If you wonder why, here’s a clue:


Khadas VIM3 runs Ubuntu 18.04 64-bit, against Debian 10 32-bit on RPi 4, which means it can use Armv8 crypto extensions while the Raspberry Pi Foundation board cannot.  Note this will only affect a few applications.

I was surprised the temperature would never go above 73°C, so I contacted Khadas and they told me the board will throttle if the temperature reaches 75°C. For some reasons the system runs at full speed in Android with just the heatsink, but Linux appears to need further cooling. As reference, Broadcom BCM2711 processor in Raspberry Pi can reach up to 85°C, so I don’t know if Amlogic is being extra cautious here.

Khadas VIM3 Fan Installation

Nevertheless, the company sent me their 3705 cooling fan specifically designed for their VIM and Edge boards. It costs $14.95 on their store.

Khadas 3705 Cooling Fan
Click to Enlarge

The fan has four wire as it’s controlled via PWM, meaning it can be turned off, and/or the speed can be adjusted depending on the temperature or user settings.

It comes with four screws which we’ll use on the heatsink we already got with VIM3 SBC the first time.

Khadas VIM3 fan heatsink

Tighten the heatsink on the SBC, and connect the cable to the 4-pin fan header.

Khadas VIM3 fan heatsink installation

We can now place the top cover and tighten everything. As a side-note, I had not used the board for a while, so I upgrade the system as follows:


This upgraded Ubuntu 18.04.2 to Ubuntu 18.04.3 and the system still boot, but with some funny colors.

Khadas VIM3 Ubuntu Strange Colors

It turns out it was because u-boot package had changed name, and was not automatically upgraded. So I reinstalled it manually:


I was using an older version of the OS, and it’s unlikely to happen to anybody else. But if it does, you know where to look.

Nevertheless, we can run the benchmark again:


That’s more like it although sbc-bench detected throttling on the big cores, but not the little ones this time:


It we really look at the details, throttling did not really occur in any meaning ful manner, as frequency dropped to 1908 MHz for seven tenth of a second only, and temperature did not go over 72°C.

After boot, the fan is basically off, and as the workload increases the speed and noise go up. The fan is really noisy during cpuminer. To avoid throttling completely, I was told to edit /boot/env.txt and change “auto”  to “high”, and reboot. This makes the fan rotate at high speed at all times.

Khadas VIM3 Fan vs Raspberry Pi 4
Higher is better. Units: 7-zip: MIPS; memset/memcpy: MB/s

Let’s see what our new results benchmarks look against Raspberry Pi 4 both at the stock 1.5 GHz frequency and overclocked to 2.0 GHz.This time everything is as expected with Khadas VIM3 taking the lead for 7-zip with the help of its fan. Memset and memcpy scores although slightly improved.

OpenSSL VIM3 Fan vs Raspberry Pi 4
khash per second

OpenSSL / AES benchmark as well, meaning Khadas VIM3 is almost 20 times faster than Raspberry Pi 4 at stock frequency thanks to 64-bit userspace.

Speedometer 2.0

SBC bench is fine to test headless use case, but for people who rely on the board for desktop use cases like web browsing, I’ve also run Speedometer 2.0 in Khadas VIM3 equipped with heatsink and fan using the pre-loaded Chromium.Khadas VIM3 Speedometer 2.0

Speedometer 2.0 Details

We can see the results are pretty constant, and VIM3 achieves 25.6 runs per minutes. Let’s see what a Raspberry Pi 4 overclocked to 2.0 GHz can do.

Raspberry Pi 4 Speedometer 2.0Khadas VIM3 is indeed faster but not overly so. Note that Speedometer 2.0 runs over https, so SSL/TLS is used, but it’s only a small part of the workload here.

I also wanted to test GPU performance, but Ubuntu 18.04.3 in Khadas VIM3 does not seem to support it yet, or there has been an issue during the upgrade:


Once it works, I’ll write a separate post about GPU performance.

Note that Khadas VIM3 (2GB RAM) sells for $99 while the equivalent Raspberry Pi 4 (2GB) goes for $45. VIM3 does have other advantages however in terms of media playback, and proper HDMI 2.0 support (4K YUV420 supported).

Support CNX Software - Donate via PayPal or become a Patron on Patreon

24
Leave a Reply

avatar
7 Comment threads
17 Thread replies
1 Followers
 
Most reacted comment
Hottest comment thread
14 Comment authors
Jean-Luc Aufranc (CNXSoft)Kenneth EkmanRay KnightPopolontheguyuk Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Member

Random link to install 64b ubuntu server on RaspPI 4.
https://jamesachambers.com/raspberry-pi-ubuntu-server-18-04-2-installation-guide/

Gouwa
Guest

Hi Jean,
Wanna confirm with you that if the Pi 4 you used for testing is 2GB model? Just wondering why the 7-zip results of VIM3 is lower then Pi4 🙂

BTW, the cooling fan you installed with wrong direction, the air outlet of the 3705 cooling fan needs to face to the fins of the Heatsink 🙂

NicoD
Guest

7-zip multicore benchmark doesn’t work on multi-core systems with different sized clusters. You can see that on the percentage it uses.
So with the Raspberry Pi 4 it uses 400%. While with the vim3 it’s less than 500% of 600%.
It is only able to run at the smallest sized cores their speed. You could do
sudo taskset -c 0-1 7z b (to run the 2 small cores seperate)
sudo taskset -c 2-5 7z b (run the big cores seperate)
And then add the numbers together. Only then you’ll have used the full potential.

CPUMiner is better, but you need to use the same distro. Bionic vs Stretch gives a big difference, even on the same board.
Greetings, NicoD

tkaiser
Guest
tkaiser

> 7-zip multicore benchmark doesn’t work on multi-core systems with different sized clusters.

Nope. You just refuse to realize that if many CPU cores fight for memory then CPU utilization and performance will suffer. Since I already tried to explain this multiple times to you, only one last time…

What you call ‘multi-core systems with different sized clusters’ are in reality CPUs with many cores. The more cores in such a use case as the 7-zip benchmark is designed for fight for memory the lower the overall CPU utilization.

> You can see that on the percentage it uses

That’s an octa-core NanoPi Fire3: http://ix.io/1jiU

Count of CPU cores is 8, memory interface (latency) rather slow. And that’s all that matters (75% CPU utilization on average):

With the A311D above it’s 6 cores but a faster memory interface and we’re at close to 85% CPU utilization on average:

> So with the Raspberry Pi 4 it uses 400%

Not on mine. RPi 4 with just 4 CPU cores but a rather slow memory interface and we’re at above 85% CPU utilization on average:

Everything as expected. This benchmark represents ‘server workloads in general’ and just like the real use case it is dependent on the memory interface and performance does not scale indefinitely with more CPU cores since memory access becomes a bottleneck.

> You could do … And then add the numbers together. Only then you’ll have used the full potential.

No, you simply generated numbers without meaning.

tkaiser
Guest
tkaiser

> Just wondering why the 7-zip results of VIM3 is lower then Pi4

Since massive throttling happened as can be seen clearly in the output? The thermal design of all your boards unfortunately always requires an expensive fansink to allow for full performance 🙁

BTW: any particular reason to start throttling already at 75°C? By exchanging the &pcontrol trip point with &phot throttling should start at 85°C: https://github.com/khadas/linux/blob/67a9d9d585d4070e3a2f051945e71ba25e5828f3/arch/arm/boot/dts/amlogic/mesong12b.dtsi#L1545-L1576

willy
Guest
willy

The difference between &pcontrol and &pcritical seems huge to me. I suspect that the current thresholds (including &phot) are extremely conservative, and it’s sad because due to thermal resistance of the air, the board itself, the enclosure and the desk, it’s way harder to reach high temperatures than low ones, and the difference between these can directly change from “never throttle with a small heatsink” to “requires a noisy fan”. Of course I’m not advocating for running at &pcritical by default in a final product, but there might be a sweet point between the two where the fan doesn’t start to spin too fast.

Just checked now. The A311D’s datasheet mentions 105°C as the recommended Tj and 125°C as the maximum absolute ratings, which tends to confirm that &pcritical is relevant. If I had one I’d probably change to something like 90°=> fan, 100°C=> throttling, 110°C => critical, thus gaining 25°C of extra budget without affecting performance :-/

Ray Knight
Guest
Ray Knight

What’s the issue with supply for the 3705 fan? I’ve ordered 3 of them now and have yet to receive a single one! I first ordered one when the VIM2 v1.4 was released and ordered 2 more with my VIM3, but haven’t seen one yet!

Michał
Guest
Michał

Can you please include the units in graphs with results? I have no idea what 8000 means and is it better or not than 9000 unless you provide units.

ddiss
Guest
ddiss

IIUC, the A311D has AES 256 CBC HW acceleration but the rpi4 doesn’t, so the openssl delta shouldn’t just be due to 64-bit userspace.

halherta
Guest
halherta

So basically, not only does the VIM3 need a heatsink but also a fan. ( Rpi4 stays below 55C with just a fan). the VIM3 exhibits much faster performance in a few use cases, but overall it is not that much faster. The VIM3 costs almost twice as much for the 4GB version and the software support is not as great.

But the VIM3 is somehow the winner….

theguyuk
Guest
theguyuk

The Vim does not need extrem over clocking like the rpi4.

Kenneth Ekman
Guest
Kenneth Ekman

I’m looking for AI / NPU benchmarks, but can’t find it here http://ai-benchmark.com/ranking_processors.html for example.