Khadas VIM3 Linux Benchmarks and Comparison to Raspberry Pi 4

I received Khadas VIM3 Amlogic A311D SBC in early July and started testing it with Android running some benchmarks and playing games last month.

I was impressed by graphics performance and overall benchmark results in Android, especially the results I got with a heatsink matched Khadas own results with heatsink + fan. So I installed the latest Ubuntu 18.04 available at the time (July 19) in order to repeat benchmarks in Linux and see how it goes.

System info in Ubuntu 18.04:


I decided to install armbianmonitor to draw some nice temperature charts as I did with Raspberry Pi 4:


But I had some error during installation:


And while I can load the webpage with top menu appearing, it won’t show any data, as its name implies it may only work in Armbian.

SBC Bench on Khadas VIM3

Let’s download SBC bench:


Note that I had to comment out CheckLoad in the script because Amlogic processor’s average will converge to 1.0 instead of 0.0 at idle. This happens on ODROID-N2 as well. Now we can run the benchmark on VIM3 with the provided heatsink:


Some ugly errors, but the test completed, and the full results are available here.  Wow, it’s not pretty with lots of throttling:


If we compare the results to Raspberry Pi 4 with and without heatsink, it shows something is really wrong for the multi-threaded 7-zip benchmark:

Higher is better. Units: 7-zip: MIPS; memset/memcpy: MB/s

memset and memcpy are quite good since single-threaded. AES is about 10 times faster on VIM3.

khash per second

If you wonder why, here’s a clue:


Khadas VIM3 runs Ubuntu 18.04 64-bit, against Debian 10 32-bit on RPi 4, which means it can use Armv8 crypto extensions while the Raspberry Pi Foundation board cannot.  Note this will only affect a few applications.

I was surprised the temperature would never go above 73°C, so I contacted Khadas and they told me the board will throttle if the temperature reaches 75°C. For some reasons the system runs at full speed in Android with just the heatsink, but Linux appears to need further cooling. As reference, Broadcom BCM2711 processor in Raspberry Pi can reach up to 85°C, so I don’t know if Amlogic is being extra cautious here.

Khadas VIM3 Fan Installation

Nevertheless, the company sent me their 3705 cooling fan specifically designed for their VIM and Edge boards. It costs $14.95 on their store.

Khadas 3705 Cooling Fan
Click to Enlarge

The fan has four wire as it’s controlled via PWM, meaning it can be turned off, and/or the speed can be adjusted depending on the temperature or user settings.

It comes with four screws which we’ll use on the heatsink we already got with VIM3 SBC the first time.

Khadas VIM3 fan heatsink

Tighten the heatsink on the SBC, and connect the cable to the 4-pin fan header.

Khadas VIM3 fan heatsink installation

We can now place the top cover and tighten everything. As a side-note, I had not used the board for a while, so I upgrade the system as follows:


This upgraded Ubuntu 18.04.2 to Ubuntu 18.04.3 and the system still boot, but with some funny colors.

Khadas VIM3 Ubuntu Strange Colors

It turns out it was because u-boot package had changed name, and was not automatically upgraded. So I reinstalled it manually:


I was using an older version of the OS, and it’s unlikely to happen to anybody else. But if it does, you know where to look.

Nevertheless, we can run the benchmark again:


That’s more like it although sbc-bench detected throttling on the big cores, but not the little ones this time:


It we really look at the details, throttling did not really occur in any meaning ful manner, as frequency dropped to 1908 MHz for seven tenth of a second only, and temperature did not go over 72°C.

After boot, the fan is basically off, and as the workload increases the speed and noise go up. The fan is really noisy during cpuminer. To avoid throttling completely, I was told to edit /boot/env.txt and change “auto”  to “high”, and reboot. This makes the fan rotate at high speed at all times.

Higher is better. Units: 7-zip: MIPS; memset/memcpy: MB/s

Let’s see what our new results benchmarks look against Raspberry Pi 4 both at the stock 1.5 GHz frequency and overclocked to 2.0 GHz.This time everything is as expected with Khadas VIM3 taking the lead for 7-zip with the help of its fan. Memset and memcpy scores although slightly improved.

khash per second

OpenSSL / AES benchmark as well, meaning Khadas VIM3 is almost 20 times faster than Raspberry Pi 4 at stock frequency thanks to 64-bit userspace.

Speedometer 2.0

SBC bench is fine to test headless use case, but for people who rely on the board for desktop use cases like web browsing, I’ve also run Speedometer 2.0 in Khadas VIM3 equipped with heatsink and fan using the pre-loaded Chromium.

Speedometer 2.0 Details

We can see the results are pretty constant, and VIM3 achieves 25.6 runs per minutes. Let’s see what a Raspberry Pi 4 overclocked to 2.0 GHz can do.

Khadas VIM3 is indeed faster but not overly so. Note that Speedometer 2.0 runs over https, so SSL/TLS is used, but it’s only a small part of the workload here.

I also wanted to test GPU performance, but Ubuntu 18.04.3 in Khadas VIM3 does not seem to support it yet, or there has been an issue during the upgrade:


Once it works, I’ll write a separate post about GPU performance.

Note that Khadas VIM3 (2GB RAM) sells for $99 while the equivalent Raspberry Pi 4 (2GB) goes for $45. VIM3 does have other advantages however in terms of media playback, and proper HDMI 2.0 support (4K YUV420 supported).

Support CNX Software - Donate via PayPal, become a Patron on Patreon, or buy review samples
Subscribe
Notify of
guest
26 Comments
oldest
newest most voted
Jon Smirl
1 year ago

Random link to install 64b ubuntu server on RaspPI 4.
https://jamesachambers.com/raspberry-pi-ubuntu-server-18-04-2-installation-guide/

willy
willy
1 year ago

I’ve installed the first version of this image on mine a few weeks ago. Nothing amazing to report to be honest. The RAM bandwidth seems low, the build time drops from 23s to 21s when overclocking from 1.5 to 2.0 GHz. I’d need to retest in 32-bits, as it’s likely that it will be faster with such a slow RAM (shorter pointers and structures). The CPU doesn’t have crypto extensions so in fact you don’t really benefit from armv8 improvements. It might be better for NEON possibly, I’ve not checked.

m][sko
1 year ago

From my experience that image(64bit) just work, at least for your benchmarks it should be good enough
it has only one limit that I aware of and that is 1GB ram limit

lets hope that people from ubuntu will provide official image for Pi 4
or you can try this
ubuntu 19.10 🙂
http://raspex.exton.se/

Gouwa
1 year ago

Hi Jean,
Wanna confirm with you that if the Pi 4 you used for testing is 2GB model? Just wondering why the 7-zip results of VIM3 is lower then Pi4 🙂

BTW, the cooling fan you installed with wrong direction, the air outlet of the 3705 cooling fan needs to face to the fins of the Heatsink 🙂

NicoD
1 year ago

7-zip multicore benchmark doesn’t work on multi-core systems with different sized clusters. You can see that on the percentage it uses. So with the Raspberry Pi 4 it uses 400%. While with the vim3 it’s less than 500% of 600%. It is only able to run at the smallest sized cores their speed. You could do sudo taskset -c 0-1 7z b (to run the 2 small cores seperate) sudo taskset -c 2-5 7z b (run the big cores seperate) And then add the numbers together. Only then you’ll have used the full potential. CPUMiner is better, but you need… Read more »

tkaiser
tkaiser
1 year ago

> 7-zip multicore benchmark doesn’t work on multi-core systems with different sized clusters. Nope. You just refuse to realize that if many CPU cores fight for memory then CPU utilization and performance will suffer. Since I already tried to explain this multiple times to you, only one last time… What you call ‘multi-core systems with different sized clusters’ are in reality CPUs with many cores. The more cores in such a use case as the 7-zip benchmark is designed for fight for memory the lower the overall CPU utilization. > You can see that on the percentage it uses That’s… Read more »

tkaiser
tkaiser
1 year ago

> Memory does not matter in benchmarks unless swapping starts to occur

You meant ‘amount of memory’, right? Since of course memory (performance) matters for a lot of benchmarks and associated use cases (e.g. 7-zip being dependent on low memory latency) 🙂

tkaiser
tkaiser
1 year ago

> Just wondering why the 7-zip results of VIM3 is lower then Pi4

Since massive throttling happened as can be seen clearly in the output? The thermal design of all your boards unfortunately always requires an expensive fansink to allow for full performance 🙁

BTW: any particular reason to start throttling already at 75°C? By exchanging the &pcontrol trip point with &phot throttling should start at 85°C: https://github.com/khadas/linux/blob/67a9d9d585d4070e3a2f051945e71ba25e5828f3/arch/arm/boot/dts/amlogic/mesong12b.dtsi#L1545-L1576

willy
willy
1 year ago

The difference between &pcontrol and &pcritical seems huge to me. I suspect that the current thresholds (including &phot) are extremely conservative, and it’s sad because due to thermal resistance of the air, the board itself, the enclosure and the desk, it’s way harder to reach high temperatures than low ones, and the difference between these can directly change from “never throttle with a small heatsink” to “requires a noisy fan”. Of course I’m not advocating for running at &pcritical by default in a final product, but there might be a sweet point between the two where the fan doesn’t start… Read more »

Ray Knight
Ray Knight
1 year ago

What’s the issue with supply for the 3705 fan? I’ve ordered 3 of them now and have yet to receive a single one! I first ordered one when the VIM2 v1.4 was released and ordered 2 more with my VIM3, but haven’t seen one yet!

Michał
Michał
1 year ago

Can you please include the units in graphs with results? I have no idea what 8000 means and is it better or not than 9000 unless you provide units.

ddiss
ddiss
1 year ago

IIUC, the A311D has AES 256 CBC HW acceleration but the rpi4 doesn’t, so the openssl delta shouldn’t just be due to 64-bit userspace.

Popolon
Popolon
1 year ago

I don’t know how the Panfrost driver works on Mali-G52, but it could be interesting for this test on Khadas VIM3. I tested the one from Mesa-19.2.0RC1 yesterday, 19.2 is the first version that basically work on RK3288 (most of development by Alyssa Rosenzweig was done on RK3399. It works pretty nice for a first start. I am able to run Weston/wayland smoothly, and even some full GL applications on Wayland/X11. On X11 : *glmark works perfectly (yes ES2 30% slower than ARM blob but without problems and OpenGL version too, every bench worked fine even glass bunny. * wz2100 (with… Read more »

m][sko
1 year ago

Panfrost don’t support Mali-G52 (Bifrost architecture) 🙁 for now
only Midgard
But it will be nice to test it on Odroid XU4(Mali-T628) if it is supported 🙂

I am using this PPA for ubuntu
https://launchpad.net/~oibaf/+archive/ubuntu/graphics-drivers
it is mesa git master 🙂
but no 32bit packages but it is really easy and fast to build from source

Popolon
Popolon
1 year ago

There is at least some WIP on bitfrost driver, I don’t know the state of the driver :

https://gitlab.freedesktop.org/mesa/mesa/tree/master/src/panfrost/bifrost

halherta
halherta
1 year ago

So basically, not only does the VIM3 need a heatsink but also a fan. ( Rpi4 stays below 55C with just a fan). the VIM3 exhibits much faster performance in a few use cases, but overall it is not that much faster. The VIM3 costs almost twice as much for the 4GB version and the software support is not as great.

But the VIM3 is somehow the winner….

theguyuk
theguyuk
1 year ago

The Vim does not need extrem over clocking like the rpi4.

Kenneth Ekman
Kenneth Ekman
11 months ago

I’m looking for AI / NPU benchmarks, but can’t find it here http://ai-benchmark.com/ranking_processors.html for example.

clort81
clort81
5 months ago

Why is pi4 memcpy and memset shown in the graph as slower at 2.0 ghz than 1.5 ghz?

Advertisements