FriendlyElec NanoPi NEO4 is currently the cheapest and smallest SBC powered by Rockchip RK3399 hexa-core processor which packs two Cortex A72 “fast” 64-bit cores, and four Cortex-A53 “efficiency” cores, so it should be an obvious candidate if you plan on building an Arm build farm costs to its low cost, small form factor, and relatively good performance.
As part of his work on HAProxy load balancer, Willy Tarreau often has to run time-consuming builds
for Arm targets, and to speed up the builds he’s put together several Arm based build farms powered by low cost development boards / SBCs. Up to now he had a build farm powered by five MIQI boards featuring Rockchip RK3288 processor with four Cortex-A17 “fast” 32-bit processor, and controlled with a ClearFog Pro networking board. He’s now decided to build another similar build farm but with NanoPi NEO4 boards instead.
Willy goes through the hardware setup, and software into much details in a blog post, so I’ll try to give a summary highlighting the key points in this article.
As you can see from the photo above, the setup comes with five NanoPi NEO4 boards connected to ClearFog Pro board over a Gigabit Ethernet connection. But there’s something funny about the heatsink, it does not look like the heatsink provided by FriendlyElec at all… Willy wanted to save some horizontal space, so instead he made his own heatsinks out of an L-shaped aluminum block that 5.2cm wide and comes with a 2mm thick aluminum corner.
He drilled four mounting holes into the aluminum block, and added a ceramic pad for cooling the processor at first, but eventually switched to copper pads coupled with screws and spring to kept the processor, pad and aluminum block in good contact without too much force applied.
Once all five boards are prepared in a similar fashion, we’re still not finished with the five boards themselves connected to a much larger heatsink taken from an old Pentium2 system using a large band of thermal tape since going with screws was not practical.
That part was fun, and I could even have published a separate post just about the DIY heatsink. The hardware build needs to be completed USB-C cables and a 5-port USB power supply such as this one.
The default FriendlyElec Ubuntu images were installed to eMMC flash module, and Willy disabled “systemd related crap”, as well as graphics mode since it’s needed for the use case. An important tip during installation is not to duplicate the flash from one board to install on the others, as it will also replace the MAC address, and you obviously don’t want 5 boards with the same MAC address in your local network 🙂
Since that’s a build farm for C programs, distcc was installed as well as various compilers from kernel.org.
In order to get the most performance of the board he overclocked them with the Cortex-A72 cores reaching up to 2.2 GHz, and Cortex-A53 cores up to 1.8 GHz, except for one were it had to be limited to 1.7 GHz, so finally all boards for their LITTLE cores limited to 1.7 GHz for practical reasons. The overclocking requires some kernel patches for linux 4.4.138 and a script to change the frequencies:
# cat set-speed-neo4-1.sh
echo 2 > /sys/kernel/debug/clk/sclk_ddrc/clk_enable_count
echo 928000000 > /sys/kernel/debug/clk/sclk_ddrc/clk_rate
echo 1 > /sys/devices/system/cpu/cpufreq/boost
echo 1704000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 2208000 > /sys/devices/system/cpu/cpufreq/policy4/scaling_max_freq
echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
echo performance > /sys/devices/system/cpu/cpufreq/policy4/scaling_governor
echo performance > /sys/devices/platform/dmc/devfreq/dmc/governor
Some improvements are also discussed such as replacing the overpowered ClearFog Pro board by a NanoPi NEO2 and an 8-port Gigabit Ethernet switch, as well as adding a reset button to the board to facilitate debugging.
Willy concludes as follows:
This constitutes a nice upgrade to the previous farm and I feel more confident hacking a bit with it thanks to the removable eMMC that I can easily re-flash from my PC. The boards are easy to hack on since all sources and docs are available, which is a real joy.