NanoPi R5S preview – Part 2: Ubuntu 20.04 (FriendlyCore)

I started the NanoPi R5S review with an unboxing, a teardown, a quick try of the pre-installed OpenWrt-based FriendlyWrt, and some iperf3 benchmarks on the 2.5GbE interfaces that were rather disappointing. I test further I switched to the Ubuntu 20.04-based FriendlyCore image since I’m more familiar with Debian-based operating systems, and some tools will not run on OpenWrt. Note the performance is still not quite optimal, and that’s why I call this a preview since numbers should improve in the next few months as more people tweak the software.

OpenWrt optimizations?

But feature jumping to Ubuntu, I gave an updated version of FriendlyWrt a give as FriendElec told me they had added some optimizations:

We have made some optimizations on the new image, such as NIC interrupt settings, and offload support…

So I downloaded “rk3568-eflasher-friendlywrt-20220526.img.gz” found on Google Drive, and flashed it to a microSD card with USBImager, and booted it to the router.

It will automatically flash the image to the eMMC flash, If you connect a monitor you can follow the result. Once it’s done, remove the microSD card, and power cycle the router.

eflasher friendlywrt

You can check the status by connecting an HDMI monitor (as shown above), or checking out the LEDs on the device. It’s very fast and the installation to the eMMC flash takes only a few seconds.

The main changes were made to the 40-net-smp-affinity file. In the pre-installed  FriendlyWrt, it looks like that:


While the new 40-net-smp-affinity file is indeed different:


Willy Tarreau explains the changes made for the eth1 interface:

It involves RPS …. i.e. they receive this IRQ on core 2 and redistribute the incoming traffic to cores 0,1,3. That’s the right way to use RPS. However you have to manually assign iperf and watch the first core that saturates. If it’s saturating core 2 with ksoftirqd first, make sure that iperf runs on any of the other 3. If core2 is slightly idle, try to put iperf on it. If putting it on it makes ksoftirqd pop up, then they’re hindering each other, and you’d rather change the RPS setting to free another core and use it for iperf.

I did not try this method before testing and switching to Ubuntu, and my results were even worse with the new FriendlyWrt image:


So this would have to be revisited.

M.2 NVMe SSD installation in NanoPi R5S

I’ve just purchased an APACER AS2280 (AP256GAS2280P4-1) PCIe Gen 3.0 x4 SSD that can achieve up to 1,800 MB/s sequential read and up to 1,100 MB/s sequential write speeds on the right hardware.

Apacer M.2 2280 PCIe SSD

The installation is straightforward as I just had to loosen four screws to remove the bottom cover, install the SSD, and fasten it with the provided screw.

NanoPi R5S M.2 NVMe SSD

Installing Ubuntu 20.04 FriendlyCore on NanoPi R5S

I first attempted to install FriendlyCore using the eflasher image.

eflasher friendlycore

It looked good, so I restarted the router, but then I noticed the WAN interface link would not show on the TP-Link switch, and only the power LED was turned on (It happens the latter is normal for the FriendlyCore/Ubuntu image). I tried again, going into eflasher UI settings by clicking on Finish, but still no luck.

Eflasher FriendlyCore install NanoPi RS5

So instead I downloaded the “SD” image to boot directly from the microSD card and run the OS from there. It works fine. If you intend to use NanoPi R5S for multiple purposes and expected a desktop environment in the Ubuntu 20.04 image, you’ll be disappointed, as the HDMI output is currently only used to access the terminal.

Ubuntu 20.04 FriendlyElec Login

FriendlyCore system information

You’ll find the boot log on CNX Software Pastebin. I logged in with SSH using pi/pi credentials (username/password), and upgraded the system to the latest packages with:


Let’s run some commands to get system information:


It looks all good, except the NVMe drive has not been mounted automatically. Let’s find more details with inxi:


Only eth0 WAN port is up, and eth1/eth2 2.5GbE ports are down, and not configured at all. FriendlyElec appears to mostly focus on the FriendlyWrt image, and they told me optimizations were not implemented on FriendlyCore yet, so most people should probably use FriendlyWrt instead since it will be easier to configure the network and router settings. I can see the Apacer AS2280P4 SSD is detected, and it turns out it’s not formatted out of the box, so I just formatted it with mkfs.ext4.

Benchmarking NanoPi R5S

Let’s run SBC Bench on the router to benchmark the CPU and potentially find some issues:


I launched this almost immediately after boot, so the dmesg output should be there in full (see boot load earlier in this preview/review), but the script is missing some information from it. The full output from sbc-bench.sh script can be found on pastebin, and we can notably see the “1992” MHz advertised frequency is tested to be 1845 MHz in reality, so some optimization may be possible here.

7zip is still faster than on NanoPi R2S router (3871), or about a 23% boost in performance, while AES-256-CBC 16KB is about 22% faster (704,872.45 vs 861,334.19kH/s)

NVMe benchmarking

I tested the NVMe SSD three times with iozone 3 three times, one with 100MB file:


then a 500MB file:


and finally a 1GB file:


The results are more or less consistent across all three tests without massive variations, and in last we’ve got about 380MB/s for read and write, well below the SSD advertised write/read speeds, and results for ODROID-M1, but that’s because of the PCIe 2.0 x1 interface used in this design, instead of the PCIe Gen 3.0 x2 interface used in the Hardkernel board.

Here’s lspci output for reference:

2.5GbE interfaces configuration and benchmarking

NanoPi R5S router review

Since only the eth0 Gigabit Ethernet “WAN” interface is configured out of the box, we have to configure the two 2.5GbE ports manually. I used the same testbed as in the first part of the review with FriendlyWrt, namely a Ubuntu 20.04 laptop with a Realtek RTL8156BG USB 3.0 to 2.5GbE dongle connected to eth1, and UP Xtreme i11 mini PC connected to eth2. Instead of using a bridge interface like in FriendlyWrt, I’ll configure two different subnets: 192.168.2.0 for eth1, and 192.168.3.0 for eth2.

Let’s create two new files in /etc/network/interfaces.d/:

  • eth1

  • eth2


Now install a DHCP server


edit /etc/dhcp/dhcpd.conf file with our two subnets:


… before restarting the dhcp server:


At this point the laptop and mini PC should get their IP address from NanoPi R5S on the respective subnets. We can start benchmarking the interfaces:

iperf3 download (Rx from R5S point-of-view) using eth1 connected to the laptop:


That’s a bit slower than what I got (1.85 Gbps) in OpenWrt, and there are retransmissions. I also monitored the system with sbc-bench.sh during the transfer:


The system performs at its maximum advertised frequency during the test, and I don’t any obvious bottleneck shere.

We can also check some information and stats with ethtool:


We did get some rx_mac_missed.

Let’s do that in reverse (Tx):


It looks quite better than in OpenWrt (1.12 Gbps).


IRQ percentages are much lower but I suppose that’s normal for Tx.

Let’s switch to eth2 connected to UP Xtreme i11:


Oh great! It’s the first time I get a decent 2.35 Gbps transfer, so there’s hope!


Unless I’m mistaken the 25% of IRQ should mean a core is fully utilized handling those.

Let’s try Tx:


1.59 Gbps. Not quite perfect, but still better than in OpenWrt.


Again the CPU operates are full speed, and far from 100% utilization so the bottlenecks must be somewhere else.  We can again check eth2 info and stats with ethtool.



We’ve got more rx_mac_missed here. So there will be some tweaks to improve the performance, but based on my experience with RTL8156B adjusting settings is really tricky, and experienced people don’t seem to agree on what to adjust, I’m talking about Realtek engineers working on RTL8156/8125 drivers vs regular readers here that are experts in networking.

Configuring NAT between the two 2.5GbE interfaces

Since the 2.5GbE interfaces don’t work optimally with iperf3, I did not bother testing the router performance in FriendlyWrt, but several people still asked. So I’ll show how I configured NAT in Ubuntu 20.04, and still test NAT performance bearing in mind it will certainly have improved in a few weeks or months.

We’ll need to enable IP forwarding and NAT. I used instructions adapted from a post on networkreverse.

Edit /etc/sysctl.conf to enable IP forwarding (uncomment the following line):


Apply the changes:


Now let’s enable NAT:


We can now ping UP Xtreme i11 on 192.168.3.0 subnet from my laptop on 192.168.2.0 subnet:


If you want to make the changes permanent:


Let’s try iperf3 between UP Xtreme i11 and my laptop with the data routed through the NanoPi R5S router.


768 Mbps in one direction and 937 Mbps in the other.


Monitoring with sbc-bench.sh show the processor runs at 1992 MHz (or 1845 MHz real), and again the 25% IRQ should mean one core is fully utilized to handle IRQs.

The mpstat command shows this should handled by core #0


And this can be confirmed by using top and htop.

top htop irq

NanoPi R5S power consumption

I’ve just got a wall power meter to check power consumption:

  • Idle – 5.1 Watts
  • Iperf3 to eth1 – 6.3 to 6.7 Watts
  • NAT test between laptop and mini PC – 6.2W

The numbers above are with Ubuntu 20.04 and the NVMe SSD installed.

I’ve also tested on NanoPi R5S with OpenWrt and no SSD:

  • Idle – 4.6 Watts
  • Iperf3  – 6.0 to 6.2 Watts

Note that since I’m using a wall power meter any efficiency loss in the power adapter (Khadas VIM4 USB-C power adapter) will be included and the values may be higher than with a USB-C power meter. It might also be possible to optimize power consumption with settings.

Final words

I’ll stop here for today. Optimizations should include changing the firmware to get Rockchip cores to run at 1992 MHz, and adjusting various settings related to PCIe and Ethernet settings, most of which I’m not familiar with (yet).

I’d like to thank FriendlyElec for sending review samples of NanoPi R5S mini router. The router with metal enclosure can be purchased on FriendlyElec website for $75, or you can get the board only for $59. The router can also be found on Aliexpress from several sellers, some of which also sell a 4GB RAM version, which is odd since FriendlyElec only sells the model with 2GB RAM at this time.

 

Share this:

Support CNX Software! Donate via PayPal or cryptocurrencies, become a Patron on Patreon, or buy review samples

ROCK Pi 4C Plus
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
30 Comments
oldest
newest
evadim
evadim
4 months ago

> which is odd since FriendlyElec only sells the model with 2GB RAM at this time.

In description at one of those sellers from Ali I see note:

> Nanopi R5S 4GB Ram are presale, and it is expected to be shipped at June 15 

Willy
Willy
4 months ago

At first I was concerned about the bad ramlat perf report that could have indicated a deeper issue, but that’s just due to a missing gcc optimization. @tkaiser, I’ve just added a makefile to ramlat address this, let’s just run “make” now. I’ve run similar network tests on my StationPC-M2 ([email protected]) with its native NIC, and monitored the performance and CPU usage. IRQs are only delivered to core 0. It correctly reaches line rate at Gbps in Tx (minus overhead since iperf3 only reports payload, i.e. 941 Mbps), and 95% in Rx. The CPU usage reaches 18% of a core… Read more »

labracherie
labracherie
4 months ago

Hello,

I’m looking for a relatively simple and affordable way to set up a control, especially parental control, for a home network. As I want to control all connections, I would like to install the control router between the fiber box and the ISP box. Can an R5S do the job in your opinion?

Looking forward to your feedback.

Best regards,

Renaud

tkaiser
tkaiser
4 months ago

> the “1992” MHz advertised frequently is tested to be 1845 MHz in reality Quick digging through sbc-bench results collection shows that so far only Radxa and Firefly managed to get RK3568 to clock at advertised speeds: grep 'SoC guess' *.txt | grep RK3568 | cut -f1 -d':' | while read ; do echo -e "$(tail -n1 "${REPLY}" | awk -F"|" '{print $2}')/ $(awk -F": " '/^SoC guess/ {print $2}' "${REPLY}" | sed 's/Rockchip //') ($(grep '^Linux' "${REPLY}" | head -n1 | awk -F" " '{print $2}' | cut -f1 -d'-')): $(grep -i 'measured' "${REPLY}" | head -n1)"; done 1… Read more »

9b2bcc232498eee3a3e1b81c730ac322bf74fea9d71ce47568
9b2bcc232498eee3a3e1b81c730ac322bf74fea9d71ce47568
4 months ago

it might be useful for practicability having some sorting function on github (html, javascript? ) tables ( https://github.com/ThomasKaiser/sbc-bench/blob/master/Results.md#some-numbers )? (thx)

tkaiser
tkaiser
3 months ago

Raw Markdown table data is available to be fed into whatever converter to end up with something sortable.

Sbc-bench’s intention is producing insights and not numbers/graphs for brainless consumption. There are so many different use cases and the only number where sorting would make sense is the 7-zip score which vaguely represents ‘server tasks in general’. But if someone intends to run a server both questioning the numbers and switching to a better benchmarking mode is a great idea. 🙂

9b2bcc232498eee3a3e1b81c730ac322bf74fea9d71ce47568
9b2bcc232498eee3a3e1b81c730ac322bf74fea9d71ce47568
3 months ago

just suggesting it might be useful (several github sites, this), if tables increase in size (low, high, mean/mode/median [(n) values first]). Some examples show, this would be 15-30 lines javascript ( compared to browser extensions for several os/provider or external programs )
‘a/p benchmarking’ needs knowledge about hardware internals for being useful ( this depending on companies and policies, Rockchip seems getting China’s Rpi Ltd./ARM for people’s (advanced) education, at the moments )

hja
hja
4 months ago

In real life performance only works for devices that you work with interactively. For a router or NAS it seems to me the power efficiency is also important.

Did you run the test with the below setting:
$ cat /sys/module/pcie_aspm/parameters/policy
default [performance] powersave powersupersave

(I think there are more settings needed to trim the power consumption of an NVMe, but this is a big one.)

tkaiser
tkaiser
4 months ago

> /sys/module/pcie_aspm/parameters/policy This affects everything PCIe (also the two 2.5GbE NICs on the faster PCIe lanes). For initial benchmarking using anything other than ‘performance’ would be weird while for ‘real world operation’ settings need to be determined that provide a good compromise between consumption and sustained/peak performance. For SSD power consumption the SSD in question is the most important factor and there’s more than ASPM, check for example the ‘Supported Power States’ using ‘nvme’ or ‘smartctl -a’ :    Transcend TS2TMTE220S:    St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat  … Read more »

hja
hja
4 months ago

You are right in making the distinction between initial tests and subsequent tests. With mature software a powersaving mode should have increased performance where the (current) performance mode is a practical limit. I have found no way to check if ASPM works right. The way I see it my device advertises 0,0040W when idle (P3 -> 0.0450W). What I measure when using NVMe instead of an SD-card is my SBC still uses around 0.5W extra when idle, even using the powersupersafe. I hope 0.3-0.4W extra savings are still possible. Currently, I don’t see the NVME as a positive addition since… Read more »

tkaiser
tkaiser
3 months ago

I found it hard to measure consumption with more complex scenarios just with a primitive meter only outputting actual amps/volts. Now happy with NetIO equipment I had to write a monitoring plugin for to not only measure/graph ‘current’ data but also ‘consumption since last check’ since being way more precise with low loads after being averaged over a time period. Major downside: you need a lot of patience since after adjusting settings it requires a long(er) time span for the new averaged consumption value to settle. But you get precise consumption data and not just fluctuating values you almost always… Read more »

Melrick
Melrick
3 months ago

I was thinking the optimizations, first is to enable the rss queue flag to y on the r8125 driver source code, and also disable the ASPM, the last thing is to bind the irqs to the cpus

And also it will be very nice to add the xt_FLOWOFFLOAD.

I ordered this sbc on ali and cost 72usb, it’s a very nice sbc

tkaiser
tkaiser
3 months ago
phinn
phinn
3 months ago

Looks like there is a lot of work to do on performance for the R5S right now. I know it’s new but these results are not impressive. I’ll give it some time and if FriendlyWrt improves when OpenWrt 22.03 is officially released that might be the time to get one.

Nola
Nola
3 months ago

Thanks. That kinda kills the device for me though, way to much for a device that will idle most of the day.

The numbers are *bare* power or with nvme?

Ping
Ping
3 months ago

Does this board has an RTC clock? Do we need an external RTC clock?

Ping
Ping
3 months ago

Thanks Jean! I’ve got quite a few uses for the NanoPi R5S if it comes with the RTC capability. But you’re right that the documentation on the RTC and RTC battery input is lacking.

silentcreek
silentcreek
3 months ago

I’m a bit surprised to see that the driver for the NIC is r8125 version 9.008, because that means that they use the out-of-tree driver by Realtek while Linux 5.10 should have native support for the 2.5 GBE NICs in mainline with the r8169 kernel module. I’m wondering if it would make any difference if you used the mainline driver for those NICs. I have used the r8125 driver on my desktop for quite some time and while I had no performance issues, it flooded my kernel logs with error messages that I could only suppress with some kernel command… Read more »

TheSwede86
TheSwede86
3 months ago

Sorry for asking what might be stupid questions; 1) Can you access the USB 3.0-ports from Docker? So if I want to setup Home Assistant running on a Docker container and use 1* Z-Wave USB-stick and 1* Zigbee USB-stick can I expose both of them to the Home Assistant Docker container? I guess right? Mount a USB drive in a Docker container (sevenbridges.com) 2) I was thinking of either using a “USB2LCD” to monitor the IP-address and temperature or / and a small touch screen to be able to view the actual output. However I guess since the Wiki mentions… Read more »

wmw
wmw
3 months ago

Can it boot from M.2 SSD?

Nicholas Pires
2 months ago

How do you use an m.2 SSD with openwrt? I’ve tried both SATA and NVMe and neither show in “Mount Points”.

Simon S.
Simon S.
2 months ago

Sorry FriendlyElec – not touching it. Its just kind of a waste since they actually never fix any serious firmware related issues or provide to mainline. Give this 2-4 months or even a year and it will be left to rot on bugged drivers and old OS snapshots at the point of release. They also never care to make their hardware stable at release so that both added together is an absolute no-go to ever touch their stuff again for anthing other then throwing your money away for some playing around. Imagine the Rasperry-Pi foundation doing that. Its really a… Read more »

tkaiser
tkaiser
2 months ago

> they actually never fix any serious firmware related issues

Which firmware are you talking about? This device is not meant to run Android but some sort of Linux or *BSD. If you want OpenWRT it should already run with mainline kernel/u-boot

Advertisement