Archive

Posts Tagged ‘benchmark’

Getting Started with MediaTek X20 Android Development Board

July 17th, 2017 No comments

Thanks to CNX for helping me get a hand on the 96Boards compliant Mediatek X20 board that was generously donated by Seeed Studio. In this article, I will walk through the steps to get the board up and running and also compile Android from the source code. The current Android is version 6.

Unboxing the Beast

Figure-1 : DHL Packet

Figure-2 : MediaTek X20 Box

Figure-3 : Standoff, board and instructions

Figure-4 : Front Facing

Figure-5 : Powerful tiny MediateTek chip

Figure-6 : Side Shot

Figure-7 : Backside Shot

Figure-8 : Multiple Antenna

First Boot Up

The board boots up from the eMMC, and the first time you boot up you will get Android screen as shown in Figure-9. This is the default Android image from the factory, which surprisingly looks like it was setup for a phone screen mode, which is not sufficient for a HDMI monitor. It would be better to install the images that are made available at Linaro website or build your own. See the other section to flash the board with different images.

Figure-9 : Out-of-the-box Android

Figure-10 : Partition mount information

Switching to Fastboot Mode

Flashing image files are done using fastboot tool in bootloader mode. There are 2 ways to switch to bootloader mode. To prepare the board to be flashed it will need pin 3 (USB Host Set) located at the back of the board as shown in Figure-11 to be set to OFF

Figure-11 : Switch OFF pin 3

Method 1

The first method requires that you boot your board into Android. Power the board and let it boot to Android. Once it boots to Android you can switch to bootloader mode by typing

Once it switch to bootloader mode you can use the fastboot to flash the image

Method 2

The 2nd method require the xflash tool which can be downloaded from the following link http://builds.96boards.org/releases/helio-x20/mediatek/aosp/16.10/mediatek-x20-aosp-16.10.tar.xz. Unzip the file and you will see something like Figure-12.

Figure-12 : Tools and Image files

Extract xflash.tar.gz and you will see something like Figure-13.

Figure-13 : Inside xflash.tar.gz

Unplug the power supply, and plug your computer USB cable to the micro USB cable of the board and run the xflash tool as follow

The location of MT6797_Android_scatter.txt can be found inside the <your_unzip_mediatek>/Images/Normal Image/ as shown in Figure-14

Figure-14: Scatter File

Power up your board after running the xflash tool. You will see print out on the screen as shown below.

Once you see the text ‘END’ the board has been switched to bootloader mode, and is ready to be flashed.

Flashing Android Image

Before flashing the new Android image make sure your board is indeed in bootloader mode by running the following command

You know that you are in bootloader mode, once you get a reply like the following

You can either flash using the image files provided by Linaro or build your own custom image. You can download a ready made image file from http://builds.96boards.org/releases/helio-x20/mediatek/aosp/16.10/mediatek-x20-aosp-16.10.tar.xz (the image file are inside the <directory>/Images/Normal Image).

The extracted mediatek-x20-aosp-16.10.tar.xz wil look like Figure-15.

Figure-15: All image files

Copy all the different files inside /Normal Image and /Special Image to a separate folder and flash the files using the fastboot command as follows:

Building From Source

Android 6.0 is supported on the X20 board. Use the following command to checkout the AOSP source code

You will need to download the binary drivers from Linaro website. The driver binary can be downloaded from https://builds.96boards.org/releases/helio-x20/mediatek/aosp/latest/. Download the file called sla.tar.gz and unzip it. You will see something like Figure-16.

Figure-16 : Content of sla.tar.gz

Copy the contents of device/, prebuilts/ and vendor/ into the AOSP directory. After completing the copy steps follow the steps below to start compiling

  1. source build/envsetup.sh
  2. lunch
  3. You will be shown the selection like Figure-17

    Figure-17 : Lunch menu

  4. Select 8 (or even 9)
  5. make -j10

Once the build process is complete, you will see list of files as shown in Figure-18.

Figure-18 : Local image files

The image files are now ready to be flashed to the board. Use the same flashboot commands as above to flash the new compiled image.

Mediatek X20 Board Info and Antutu Benchmark

I’ll complete this guide by showing the info provided by CPU-Z and Antutu benchmark for the board for people wanting such details.

Click to Enlarge

Click to Enlarge

Click to Enlarge

If you’re interested in the board, you can purchase it for $199 plus shipping on Seeed Studio.

References:

  1. http://builds.96boards.org/releases/helio-x20/mediatek/aosp/16.10/
  2. http://www.96boards.org/documentation/ConsumerEdition/MediaTekX20/Downloads/ThirdParty/AOSP/LinuxFastboot.md/

ROCK64 Board Review – Part 1: eMMC Flash Module, Android 7.1 Firmware, Benchmarks, and Kodi

July 16th, 2017 17 comments

ROCK64 is the second ARM Linux development board by Pine64 that is based on Rockchip RK3328 processor, instead of Allwinner 64 for Pine A64 board, and while both processors comes with four Cortex A53 cores, the Rockchip processor offers a faster GPU, 4K @ 60 Hz video support, as well as USB 3.0 support. I’ve got a sample with an eMMC flash module, and I’m going to test it first with Android 7.1, since those were the only firmware images currently available on the Wiki, but Linux will be available before the board ships publicly at the end of July.

Click to Enlarge

Rock64 Linux Ports Status

There are now at least three Linux ports in progress:

  1. ayufan’s Debian Jessie and Ubuntu Xenial images which are now in pre-release, but should officially be released next week.
  2. Arch Linux ARM with no pre-built Rock64 images yet
  3. Fedora 25 minimal Image
  4. The Yocto Project support is also being worked on with the first release planed for August

I’ve got some comments on IRC, that ayufan images supports GPIO(s), Mali hardware graphics acceleration – but performance needs to be tuned -, as well as hardware video decoding but 4K videos are not playing that well right now. I’ll make sure to try one of ayufan image, before the boards ship on July 31st.

eMMC Flash Module Compatibility with ODROID Boards

ROCK64 board can boot from micro SD card or an eMMC flash module, with the latter likely to provide better performance than most SD cards, and you won’t have to worry about fakes. The eMMC flash module is not a new idea, as Hardkernel has offered those with their ODROID boards for year. And I read that Pine64 eMMC flash module would be compatible with ODROID boards, which would be good news if you already own any of Hardkernel ones. I tested compatibility the easy way, as I connected Pine64’s eMMC flash to Hardkernel eMMC to micro SD adapter…

and plugged it in to a card reader in my computer, and I could access “system” and “Volumn” partitions on the eMMC flash module part of the pre-installed Android image.So ODROID users can rejoice, as you may not need to buy extra eMMC flash modules to play around with ROCK64 board.

Installing Android 7.1 on ROCK64 Board

ROCK64 board firmware update to the eMMC flash module works just like firmware update on Rockchip TV boxes, i.e using Android_Tool program in Windows, or upgrade_tool (now renamed rkdeveloptool) in Linux. You’ll also need a USB male to male connected between your computer and the board. If you plan to run Android from a micro SD card instead, you can use the same image, but flashing it via SD_Firmware_Tool from Windows.  Eventually Pine64 installer program, based on Etcher, will probably be updated to support ROCK64 board, and handle both download and flashing to the micro SD card.

Currently there are two Android images provided via the Wiki:

  • Stock Android 7.1 [20170712] Engineering version released by the company
  • Community Build Android TV 7.1 [ver 0.1.8] by ayufan

I’ll use “Stock Android 7.1” for this review, since I’m not a big fan of Android TV, and both download are slow, with the Pine64 server easily winning the download race against github. 🙂

Since I’m a Ubuntu user, I went with the Linux method. Let’s install the update_tool, and uncompress the firmware image first. After download run:

After connecting a jumper – or something else to short both pins – as shown in the picture above with a red circle, I connected the power supply, and a USB cable between the top USB 2.0 port and my computer in order to enter MaskROM mode, and finally removed the jumper before running:

This should take a couple of minutes with the following output:

Firmware update is done.

Android 7.1 “Engineering Version”  Launcher, Settings and Benchmarks

So we can connect the board with everything we need.

The first time. I’m asked to choose between “ROCK64 Launcher” and “ROCK64 Launcher”, and you’ve got to love the choice offer here.
I just selected the first one (com.cloudmedia.rock64launcher) and it did not work quite as expected…

I tried to open the app again, but it did not work, so I just restarted the board, and could finally access the launcher [Update: The second launcher works better apparently]. Note that the framebuffer resolution is only 1280×720 instead of the more common 1920×1080 on this type of hardware.

Click for Original Size

One small problem though… While the USB keyboard would work, my air mouse’s RF dongle connected to the top USB dongle would not be recognized. I had to move it to the lower part to make it work. Nothing would work on that upper USB port, so I guess it’s stuck in device mode, and could not find an option in Android to change that. So I connect a USB Hub with the USB keyboard and RF dongle, and it would not work either. After unplugging and replugging the hub, I was finally able to use at the same time.

Click to Enlarge

The settings will show on the right side of the screen, I just composed the screenshot above to show both resolution options up to 4096x2160p60(YCrCr420), and maybe 4096x2160p60(YCrCb444), which may have seen filtered out since LG 42UB820T television used for review is not supporting that mode.


The storage section shows 3.08GB used out of 14.45GB from the 16GB eMMC flash module, and the system could mount both NTFS and EXT4 partitions from the USB 3.0 hard drive, but not exFAT nor BTRFS.That image is based on Android 7.1.2 plus Linux 3.10.104, and a fairly recent April 5,2017 security patch.


Above is the list of pre-installed app in that image. I went to Google Play Store to install a few apps, and many were grayed out and impossible to install. I still managed to install Antutu and DRM Info. I ran the former, keeping in mind that without heatsink, the score may not be that good. 34,811 points in Antutu 6.x without heatsink is not bad, especially considering the room temperature was close to 30 ºC. I actually got a lower score with Rockchip RK3328 TV boxes like A95X R2 with only 33,117 points, also using the same 1280×720 resolution. DRM Info returns no support for Widevine DRM at all, not really surprising.

In order to install more apps, I decided to download APKPure apk via the installed Lightning web browser, and somehow the board became rather unresponsive, as is there was lot of I/Os in the background. After killing the browser twice when getting the “app not responding” window, I finally manage to download and install APKPure.

Damn, it will launch in portrait, a frequent bug on TV boxes running Android…. 4 or 5 years ago. So I decided that maybe I should give ayufan Android image a try after all, since the “Engineering Version” image has soon many bugs right now. As I finished testing I was also informed that “the stock build listed as engineering, we still tuning on 4K playback”, and that “eventually this tuning will roll into ayufan TV build around end of this month when ROCK64 production board roll out”. So eventually people will be directed to use Android TV 7.1 OS, and you should still keep in mind that there’s still 2 weeks before the board ships so many bugs will be ironed out.

Android TV 7.1 “ayufan” Firmware

As I tested the “Engineering” image, Android TV 7.1 firmware could complete to download (it tool around 6 hours), and I uncompress it, and flashed it using the same method as above:

I had no troubles to access to the launcher this time, and both USB 2.0 ports are working.

Click to Enlarge

The resolution is still 1920×1080, so that’s also an improvement.

Android TV firmware does not play nice with my air mouse however, as in many sections, mouse clicks are not supported, and the Home key has no effect. Doing searches in Google Play is a nightmare, as it will randomly switch to “voice mode” when I type with a keyboard or air mouse.

I could not find Chrome, nor Firefox in Google Play, so instead I installed some other “TV Browser”, and the download would not start, so instead I download APKPure in my computer, and installed it from ES File Explorer. The same portrait orientation occurred again, so I gave up, and side-loaded CPU-Z, A1SD Bench, and iperf from ES File Explorer.


CPU-Z shows a quad core Cortex A53 processor clocked at up to 1.51 GHz with a Mali-450MP GPU…

… and that I got the 1GB RAM version of the board (there are also 2GB and 4GB version), and the “screen resolution” is 1920×1080 as we’ve seen above.

This image is also based on Android 7.1.2 with Linux 3.10.104 kernel.

I used A1SD bench for storage benchmarks in Android which reports sequential speeds only.

Click to Enlarge

The “Cache read” is the internal memory so the 90.80 MB/s read speed is invalid. The 35.39MB/s write speed should be correct, and is not too bad with a flash. We’ll find more about the eMMC flash module once I run iozone3 benchmark in Linux. USB storage is working OK, with the EXT-4 partition (Red SD card) achieving 94.13 and 90.93 MB/s W/W speeds, and the NTFS partition (Yellow SD card) getting 92.82 MB/s and 62.66 MB/s not that from what I get on my computer with that mechanical hard drive. Other people have tested performance up to 300 MB/s on SSD drive connected to the USB 3.0 port of ROCK64 board. The important part is that performance maxes out Gigabit Ethernet bandwidth, which with EXT-4 file system it nearly does.

Speaking about Ethernet, I tested Gigabit performance by using iperf for 60 seconds for a full duplex transfer:

Not too bad, but it looks like it can not (currently) transfer close to max speed in both direction at the same time. If you are using it as TV box, it will not matter a bit however.

I also tested a few 4K videos from the USB 3.0 drive in Kodi 17.3 installed from Google Play since I was told it should work OK…:

  • HD.Club-4K-Chimei-inn-60mbps.mp4 (H.264, 30 fps) – Audio OK, but video is a slidehow despite hardware video decoding with amc-h264 codec
  • sintel-2010-4k.mkv (H.264, 24 fps, 4096×1744) – Shows previous video still with many green lines. I had very difficult controls from the air mouse, so I powered cycled the board.
  • BT.2020.20140602.ts (Rec.2020 compliant video; 36 Mbps; 59.97 Hz) – Audio OK, but video is a slideshow despite using hardware decoding with amc-h265
  • Fifa_WorldCup2014_Uruguay-Colombia_4K-x265.mp4 (4K, H.265, 60 fps) – Audio OK, but video is a slideshow with the picture refresh every ~6 seconds
  • Samsung_UHD_Dubai_10-bit_HEVC_51.4Mbps.ts (10-bit HEVC / MPEG-4 AAC) – Several audio cuts, and slideshow
  • 暗流涌动-4K.mp4 (10-bit H.264; 120 Mbps) – Video will freeze after a few seconds
  • tara-no9-vp9.webm (4K VP9 YouTube video @ 60 fps, Vorbis audio) – Maybe 1 fps with amc-VP9 (HW)

All videos above are using hardware video decoding according to Kodi overlay info. I tried Big Buck Bunny 1080p60 sample, but the video was not watchable either. It would play, but not smoothly. CPU temperature should not be the problem, as I measured around 65 °C with a IR thermometer. I skipped HDMI audio pass-through, since I was told it would not work, and I can’t play videos properly in Kodi with the current firmware. Better video playback and HDMI audio pass-through are all being worked on right now.

So Android support does not look too good right now, and while I’m hopeful the most critical issues will be fixed by the time, people get the board next months, there are so many problems that, it’s likely many other bugs will still have to be fixed, so you may have to be patient if you have ordered  a board in order to run Android.

ROCK64 can be pre-ordered for $24.95 with 1GB RAM, $34.95 with 2GB RAM, and $44.95 with 4GB RAM on Pine64 website. You may want to add a power supply, and optionally the eMMC flash module to your order. Remember that you’ll also need a USB male to male if you want to flash the firmware to the eMMC flash. It’s not needed if you run Android from a micro SD card. You can find some tips to order a proper micro SD card in the article entitled  “Micro SD Cards for Development Boards – Classes, Tools, Benchmarks, Reliability, and Tips & Tricks“.

Review of MeLE PCG03 Apo Fanless 4K Mini PC – Part 2: Windows 10, Benchmarks, and Kodi

July 12th, 2017 20 comments

MeLE PCG03 Apo is an update to MeLE PCG03 mini PC, and one of the rare Apollo Lake mini PCs to be both fanless, and support HDMI 2.0 output. I’ve already checked out the hardware design in “MeLE PCG03 Apo Fanless Apollo Lake mini PC Review – Part 1: Unboxing and Teardown“, so in the second part of the review I tested Windows 10, focusing on HDMI 2.0 features, audio pass-through in Kodi, and performance and stability tests to see how well it compared to similar actively cooled mini PCs such as Voyo V1 VMac mini.

MeLE PCG03 Apo Setup and System Information

I connected a USB 3.0 drive to one of the USB 3.0 ports, USB keyboard and mouse, and RF dongle for a wireless gamepad to the other USB 2.0/3.0 ports, as well as USB type C to micro USB adapter itself connected to a micro USB OTG adapter in order to add a USB flash drive. Finally I added Ethernet, VGA and HDMI cables, and of course the power supply to complete the setup.

Click to Enlarge

A typical boot to the login window takes around 15 seconds, but the very first boot, I went through the usual Windows 10 setup wizard to select the language, create a user, etc…, as well as retrieve the latest Windows update. So that’s better than some other Windows 10 mini PCs which are already configured with a default user, and may raise suspicions.

I still wen to Control Panel->System and Security->System to check Windows 10 is indeed activated, and the mini PC is running Windows 10 Home 64-bit on an Intel Celeron N3450 with 4GB RAM as expected.

Click to Enlarge

The next step was to check HDMI 2.0 support that should allow 4K resolutions @ 60 Hz, and I could select and use 3840×2160 or 4096×2160 up to “60p Hz” without any issues.

Click to Enlarge

Since we have a VGA port too, I tested dual display setup using extended desktop mode with 3840×2160 resolution @ 60 Hz on my 4K TV, and 1600×1050 on my Full HD TV, as it was the maximum resolution I was offered.

No problem here again, and I could use to independent display one connected via HDMI 2.0, and the other via VGA.

Click to Enlarge

I’ve also included a screenshot of the dual display setup for those interested.

Click for Original Size

I took the screenshot below after starting to download a few programs (but no installation), and 28.9GB storage is really tight for Windows 10, I could not install all programs used for the review, without uninstalling one or more. For example, if I install PCMark 8, complete the benchmark, I had to remove it before install PCMark 10, even when moving all download the USB partitions.

So may want to run Disk Cleanup from time to time, and uninstall some pre-installed games. I may also consider disabling hiberfil.sys file, learn how to do folder redirection and filesystem junction with mklink, which I used for Package Cache directory. You could do this to external USB hard drive, but performance may suffer while loading programs or during databases accesses, so you could consider adding a 80mm M.2 SSD inside the mini PC. I could not get one to test, but the company told me they tested three models available on Aliexpress:

Another thing I discovered is that when you “uninstall” Windows Store apps, there are not deleted, but for some reasons kept in C:\Programs Files\WindowsApps\Deleted directory, so I’d have to take ownership of the directory and delete it if you want to free up some more space.

The 32GB eMMC flash capacity is the most negative point I found about this mini PC, I wished the company could offer a 64GB version, or better a pre-installed 128 M.2 SSD [Update: I forgot this would be a problem with the discounted Windows 10 license]. This will not be a problem if you only plan to use the box as an HTPC, but for desktop use, you really need more external storage.

The mini PC recognized the NTFS and exFAT partition in my USB 3.0 drive, but the USB flash drive which I connected the USB type C port was not found. I tried to connect the keyboard there instead, and then to my computer via a USB type C to USB type A cable, but again no luck in both cases. It looks like the USB C port is not usable for anything. Maybe my sample has some issues.

I took a screenshot of the Device Manager for people who want a few more technical details about peripherals.

Click to Enlarge

… as well as HWiNFO64 which basically reports the same info as on Voyo V1 Vmac Mini since it’s based on the same Celeron N3450 processor.

Click to Enlarge

MeLE PCG03 Apo Benchmarks

Let’s start with PCMARK 8  HOME ACCELERATED 3.0 benchmark at 1080p60 resolution and framerate.

Click to Enlarge

The score here is surprisingly higher than on the fan cooled Voyo mini PC (1,566 points), and not too far to the score I got (1,846)with the Pentium N4200 version of Voyo V1 VMac Mini.

Since PCG03 Apo is a candidate to use as 4K desktop for simple tasks, I run the same benchmark using 3840×2160 @ 60p Hz video output, and the score dropped a little to 1,431 points.

Click to Enlarge

Please note that only two passes out of three could complete, as the benchmark failed somewhere during the third pass, but the average should not change, it’s just we can 2 test samples, instead of 3. I tried the benchmarks 3 times in total, and the two other times it failed during the first pass ending with no score. You can find the details results here.

FutureMark has recently released PCMark 10, so for future reference I also ran that version of the benchmark using 1080p60 output.

Click to Enlarge

Check this link for full results. All other benchmarks below were done using 1920×1080 @ 60 Hz video output / resolution.

Passmark PerformanceTest 9.0 confirmed the good performance of the device with 995.70 points, which compares to 998.4 points for Voyo V1 (N3450), and 1087 points for Voyo V1 (N4200).

Note the performance of Disk Mark is quite weaker here, and the Voyo models who got close to 3,000 points, but the latter had the C: drive in a 128GB SSD, instead of a 32GB flash, which explains the massive performance difference here.

I ran three 3DMark tests showing performance that’s almost as good as Voyo mini PC based on Pentium N4200 processor.

Click to Enlarge

Details for all three results can be found below:

CrystalDiskMark reports up to 258 MB/s sequential read speed, and 51 MB/s write speed, with random I/O up to 28 MB/s for the C: drive (32GB eMMC flash). That’s actually roughly the same as the 32GB eMMC flash in Voyo mini PC, but a big difference compared to the 500MB/s+, you got from the 128GB FORESEE SSD installed in the same devices.

Random I/O performance will be better in the SSD too, so you may considering re-installing Windows 10 in an M.2 SSD if you decide to purchase one [Update: Not a good idea, the Genuine Windows key in the device will be ignore, and Windows 10 will not be activated]. You’ll find BIOS, drivers, and instructions to re-install Windows 10 in MeLE’s forums.

USB 3.0 performance is fine with my USB driver achieving around 100MB/s read and write sequential speed.
The random I/Os number shows why you don’t want to install Windows or apps in such drive.

I used iperf 2.x to measure network performance with using dual duplex transfer over Gigabit Ethernet:

All good, so I connected the mini PC to my AC router…

… and performed WiFi upload and download tests with iperf:

  • Upload
  • Download

Those are decent results with my setup, i.e. the AC router is located about 4 meter from the DUT and with a wall in between. You can see a comparison with some 802.11ac Android TV boxes I’ve recently reviewed.

802.11ac WiFi Download and Upload Speed in Mbps

To give a better idea of the performance I compare it against other low power mini PCs based on Braswell (MINIX NGC-1, Vorke V1), Cherry Trail (Voyo V3, MINIX NEO Z83-4), Apollo Lake (Voyo V1 VMac Mini), and Skylake (Compute Stick) for various benchmarks.

Click to Enlarge

Note: Ice Storm scores divided by 10, Fire Strike scores multiplies by 4 for scale.

The Skylake compute stick really stands out despite having similar TDP, but it’s also much more expensive. Other mini PCs are closely matched, but the good news is the MeLE PCG03 Apo mini PC fairs well in all benchmarks, except for storage speeds, but it can be made to match other systems if you use a M.2 SSD to run Windows instead of the 32GB eMMC flash.

Kodi 4K Video Playback and HDMI Audio Pass-through

I’ve installed the latest Kodi 17.3, and run it using 1920×[email protected] output.

Click to Enlarge

You don’t need to set the Windows resolution to 3840×2160 to watch videos, since Kodi will automatically do that if you go to Settings->Player Settings->Videos, and set Adjust display refresh rate to On start / stop, as it will also automatically adjust to the best resolution for the video.

Once I’d done that I tested my usual 4K video samples via SAMBA over Gigabit Ethernet unless otherwise stated:

  • HD.Club-4K-Chimei-inn-60mbps.mp4 – Mostly OK, but the video seems to skip frames a few times
  • sintel-2010-4k.mkv – OK
  • Beauty_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) – OK
  • Bosphorus_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) – OK
  • Jockey_3840x2160_120fps_420_8bit_HEVC_TS.ts (H.265) – OK
  • MHD_2013_2160p_ShowReel_R_9000f_24fps_RMN_QP23_10b.mkv (10-bit HEVC, 24 fps) – OK
  • phfx_4KHD_VP9TestFootage.webm (VP9) – 4 to 6 fps (Software decode)
  • BT.2020.20140602.ts (Rec.2020 compliant video) – SAMBA: Audio cuts and buffering issue; HDD: OK
  • big_buck_bunny_4k_H264_30fps.mp4 – OK
  • big_buck_bunny_4k_H264_60fps.mp4 – Not super smooth, but no audio delay like on ARM TV boxes. Almost watchable
  • Fifa_WorldCup2014_Uruguay-Colombia_4K-x265.mp4 (4K, H.265, 60 fps) – OK
  • Samsung_UHD_Dubai_10-bit_HEVC_51.4Mbps.ts (10-bit HEVC / MPEG-4 AAC) – OK
  • Astra-11479_V_22000-Canal+ UHD Demo 42.6 Mbps bitrate.ts (10-bit H.265 from DVB-S2 stream) – OK
  • 暗流涌动-4K.mp4 (10-bit H.264; 120 Mbps) – Maybe 10 fps (Software decode) and buffering issues
  • Ducks Take Off [2160p a 243 Mbps].mkv (4K H.264 @ 29.97 fps; 243 Mbps; no audio) – HDD: OK
  • tara-no9-vp9.webm (4K VP9 YouTube video @ 60 fps, Vorbis audio) – 4 to 6 fps (Software decode) + buffering issues
  • The.Curvature.of.Earth.4K.60FPS-YT-UceRgEyfSsc.VP9.3840×2160.OPUS.160K.webm (4K VP9 @ 60 fps + opus audio) – 4 to 6 fps (Software decode) + buffering issues

Automatic frame rate switching is working well, but playing videos with bitrate over 50Mbps over SAMBA seems to be an issue with this mini PC. VP9 and 10-bit H.264 codecs are not supported by Apollo Lake processor, so Kodi revert to software decoding, but the processor is not powerful enough to handle those codecs at 4K. Hi10p up to 1080p is fine. One small issue worth noting is that almost all videos had a short audio cut after 16 to 19 seconds, but the problem would not reoccur at other points in the videos.

Apollo Lake mini PC are supposed to support audio pass-through, but HDMI 2.0 is implemented via an eDP to HDMI bridge which in the past has created issue with this features. So I enabled audio pass-through in Kodi by going to Settings->System Settings->Audio, enabling Allow passthrough, and selecting DIRECTSOUND: TX-NR636…. as the Passthrough output device before running the tests with various audio codecs.

Video HDMI Pass-through
AC3 / Dolby Digital 5.1 OK
E-AC-3 / Dolby Digital+ 5.1 PCM 2.0
Dolby Digital+ 7.1 PCM 2.0
TrueHD 5.1 PCM 2.0
TrueHD 7.1 PCM 2.0
Dolby Atmos 7.1 PCM 2.0
DTS HD Master PCM 2.0
DTS HD High Resolution PCM 2.0
DTS:X PCM 2.0

So only AC3 is supported. I changed to WASAPI output device instead, and the results are not much better.

Video HDMI Pass-through
AC3 / Dolby Digital 5.1 OK
E-AC-3 / Dolby Digital+ 5.1 OK
Dolby Digital+ 7.1 PCM 2.0
TrueHD 5.1 PCM 2.0
TrueHD 7.1 PCM 2.0
Dolby Atmos 7.1 PCM 2.0
DTS HD Master PCM 2.0
DTS HD High Resolution PCM 2.0
DTS:X PCM 2.0

Disappointing, as HDMI 1.4 Apollo Lake mini PCs can normally handle DTS 5.1 as well even with DTS HD files.

User Experience, Stress Test, and Power Consumption

I did a user experience test like with other Windows 10 PCs with multi-tasking by launching an using ThunderBird, Firefox, Libre Office, and Gimp at the same, multi-tab browsing in Firefox playing some Flash games, and watching 4K YouTube videos. I also played Asphalt 8, and as shown in the section above used Kodi to watch videos. The experience felt very similar to other Apollo Lake mini PC with maybe apps not launching as fast due to the eMMC flash. I also run HWiNFO64 in sensor only mode during my tests and benchmarks, and CPU throttling was never reported by the program, so MeLE PCG03 Apo is a solid device with good thermal design.

I have not done any video this time, but if you’re new to Apollo Lake system, you may want to watch Voyo V1 Vmac Mini video below which should give you an idea of the performance.

I also ran AIDA64 Extreme stability test during 2 hours with HWiNFO64 also running side by side, and the CPU temperature never went above 79 °C with the average CPU clock speed being 1.6 GHz right between the base frequency (1.1 GHz) and turbo frequency (2.2 GHz).

Click to Enlarge

Power consumption is about 6.4W in idle mode with the USB 3.0 drive connected, 1.0W in sleep mode, and 0.0W in power off mode.

Conclusion

MeLE PCG03 Apo is a solid device that stays cool enough under load despite thanks to a good fanless thermal design, and HDMI 2.0 works as expected with 4K @ 60 Hz supported. The VGA port also allows for dual independent display setups. Gigabit Ethernet, and 802.11ac WiFi are performing very well. The main downsides I can see are the small eMMC flash, USB type C port that would not work for anything (sample issue?), and HDMI audio pass-through is limited to Dolby Digital 5.1. The first issue can easily be solved by installing a larger (and faster) 80mm M.2 SSD inside the device.

If you are interested in this mini PC, you can purchase MeLE PCG03 Apo for $159.20 including shipping on Aliexpress. If the price is higher when you check it out, it may pay to wait until the week-end to get a better price.

[Update: MeLE’s answers to some of the issues raised in this review:

Read / Write speed
1. It is clear that MeLE PCG03 Apo (N3450) is far behind VOYO VMac Mini (N3450 and N4200) because MeLE follows Microsoft’s policy strictly to install the genuine Windows 10 Home as C: Disk on 32GB eMMC while VOYO does that on the 64GB or 128GB SSD.
2. According to the policy, the unit price for genuine Windows 10 Home on Apollo Lake mini PC is USD 45 at least if the storage capacity (as C: Disk) is equal or over 64GB. That is why user may see a 64GB SSD as C: Disk on VOYO VMac Mini while there is still a 32GB eMMC as D: Disk on the PCB board. It is a trick which has just been discovered and warned by Microsoft in China.
Audio Setting
1. I will check with technical team on how to make DD & DTS 7.1 working in Kodi as well, I will keep you updated.
2. It will get Audio 7.1 DD & DTS with LAV codecs using MPC-HC as player for example.
USB Type-C
1. It is actually a standard USB 3.0 interface converted into USB Type-C shape.
2. It only supports normal (not fast) power charging, and data transfer directly to USB Drive or HDD in external enclosure with Type-C interface.
3. For this point, we will update our product description on our official store on Aliexpress to make it more specific for every buyer to avoid any misunderstanding.
]

Vorke Z3 Rockchip RK3399 TV Box Review – Part 2: Android 6.0 Firmware

July 6th, 2017 11 comments

Vorke Z3 is another mini PC / TV box powered by Rockchip RK3399 hexa core processor with two Cortex A72 cores, and two Cortex A53 cores making it theoretically one of the fastest TV boxes on the market, excluding NVIDIA Shield Android TV which is well ahead of the competition, albeit with poor worldwide availability. I’ve have already shown Vorke Z3 hardware inside out, so in the second part of the review, I’ll focus on testing the firmware including video playback, and the system performance, and see how it compares to the similar Yundoo Y8, which I reviewed last month.

First Boot, Setup, and First Impressions

One the selling point of Vorke Z3 is its SATA connector, so I connected a 1TB 3.5″ SATA drive first, as well as Seagate USB 3.0 drive (1TB) to the USB 3.0 port, I also added a USB hub to connect a USB keyboard, as well as two USB RF dongles for Tronsmart Mars G01 gamepad. and MINIX NEO A2 Lite air mouse. I did not use the USC type C port at that point, but we’ll see later that it can be used for firmware update, remote storage access, and video output.  I completed the hardware setup by plugging in Ethernet, HDMI, and power cables.

Click to Enlarge

Press the mechanical power switch on the rear panel in order to start the box, with a typical boot taking around 30 seconds. That’s acceptable, but I was expecting a faster boot with the high end eMMC flash used together with the powerful processor. The first time, you may be asked to choose between Launcher3 (Stock Android Home screen similar to what you get on your phone) better if you are close to the screen using the box as a mini PC,

or the familiar MediaBox Launcher better suited to TV use, if you are seated a few meters from the screen.

Click for Original Size

Below are the pre-installed app (minus RKMC which I installed manually) in the stock firmware.

YouTube, Hulu Plus, and Netflix are installed, but you’ll notice Kodi is missing, and there’s a good reason for this as we’ll find out below…

Click to Enlarge

The Settings are pretty much standard, but when you go to Display Output, you’ll see HDMI and HDMI1 output, because the mini PC can handle two display if you connect a compatible USB type C to HDMI dock to the device, and you can either mirror the displays, or extend the desktop. I don’t have such dock, and GeekBuying did not seem interested in sending me one, so I did not test that part, but it’s something to keep in mind.

I could still go to the HDMI section, and configure video output up to 4096×2160 @ 60 Hz. What I found out is that the system will not keep the setting, and it will just jump aroud 4K @ 30 Hz, 1080p60 or 720p60 between reboots. The Sound & Notification section does not include “Sound Devices Manager”, so you can’t enable audio pass-through, so if you want to do that you’d have to use the blue “Settings” app instead. The settings do not have any options for CEC or HDR, with the latter not supported by the hardware.

The screenshot above was taken at the end of the review, and I still had plenty of space out of the 26.74GB partition. USB3_NTFS is the NTFS partition of my four partitions USB drive, meaning exFAT, EXT-4 and BTRFS file systems are not supported. “USB Drive” is actually the SATA drive, and is a misnomer as the hardware implementation relies on a PCIe to SATA bridge. In case you planed to use the device as a mini PC connected to your printer, you may want to know Printing settings are missing.

The About section shows the device name is indeed VORKE Z3, and it runs Android 6.0.1 on top of Linux 4.4.166 with the Android security patch dated August 2016. The build machine’s hostname is sunchip-CS24-TY, so it’s quite possible Sunchip is being the software and hardware design. Two versions of the firmware are available with either root or no root, so you could install the one you prefer using AndroidTool (Windows) or upgrade_tool (Linux).

The IR remote control is pretty basic, and I assume most people will used the own input device be it a an air mouse or the smartphone app. It did the job but only up to 4 meters, farther than this, and some key presses will be missed. I had no troubles installing apps via Google Play and Amazon Underground stores.

It’s not possible to cleanly power off the device with the remote control, only the mechanical switch on the back can do this, so instead you can only go in or out of standby.  I measured the power consumption with or without SATA and/or USB hard drive(s) attached in power off, standby, and idle modes:

  • Power off – 0.0 Watt
  • Standby – 5.1 Watts
  • Idle – 5.1 Watts
  • Power off + USB HDD – 0.0 Watt
  • Standby + USB HDD – 8.1 Watts
  • Idle + USB HDD – 9.1 Watts
  • Power off + USB HDD – 0.0 Watt
  • Standby + USB HDD – 10.3 Watts
  • Idle + USB HDD – 11.2 Watts

After playing a 2-hour H.264 video in Kodi, I measured maximum temperatures of 37 and 40°C on the top and bottom of the case with an IR thermometer, but the temperature felt a little higher than that when touching the surface with my hand, maybe it’s made of a material that interferes with measurements. After playing Riptide GP2 for around 15 minutes, the temperatures went up a little to 39 and 42°C, and gameplay was OK overtime, but only similar to what I experience on Yundoo Y8 or Amlogic S912 TV Boxes, and not as smooth as on Xiaomi Mi Box 3 Enhanced. CPU-Z always reports 26 °C, so it’s not usable as an alternative temperature measure. I also noticed the box got quite hot (44 °C with IR thermometer) when I turned off the display, and let the UI in the launcher doing nothing. I did not notice any sharp drop in performance during use, but thermal throttling is happening as we’ll see in the Benchmark section.

I like the SATA port and USB type C port supporting data and video in Vorke Z3, and I found the firmware to be rather stable and very responsive. However, there are many small issues like no clean power off, HDMI setting is not remembered properly, so settings like Printing, CEC, Audio device, and automatic frame rate switching are missing from the main Setting app. The IR remote control does not feel very good either, and range was rather short.

Kodi & RKMC Video Playback, DRM Info, and YouTube

Kodi is not installed so I went to the Google Play store to install Kodi 17.3, and naively expected most videos to play fine. Those are my results for 4K video samples:

  • HD.Club-4K-Chimei-inn-60mbps.mp4 – OK, but not perfectly smooth
  • sintel-2010-4k.mkv – Stays in UI
  • Beauty_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) –  OK
  • Bosphorus_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) – OK
  • Jockey_3840x2160_120fps_420_8bit_HEVC_TS.ts (H.265) – Stays in UI
  • MHD_2013_2160p_ShowReel_R_9000f_24fps_RMN_QP23_10b.mkv (10-bit HEVC, 24 fps) – OK
  • phfx_4KHD_VP9TestFootage.webm (VP9) – ~5 fps (software decode)
  • BT.2020.20140602.ts (Rec.2020 compliant video) – Stays in UI
  • big_buck_bunny_4k_H264_30fps.mp4 – OK
  • big_buck_bunny_4k_H264_60fps.mp4 – Slideshow + audio delay  (4K H.264 @ 60 fps is not supported by the VPU in Rockchip RK3399 SoC)
  • Fifa_WorldCup2014_Uruguay-Colombia_4K-x265.mp4 (4K, H.265, 60 fps) – OK
  • Samsung_UHD_Dubai_10-bit_HEVC_51.4Mbps.ts (10-bit HEVC / MPEG-4 AAC) – Stays in UI
  • Astra-11479_V_22000-Canal+ UHD Demo 42.6 Mbps bitrate.ts (10-bit H.265 from DVB-S2 stream) – Stays in UI
  • 暗流涌动-4K.mp4 (10-bit H.264; 120 Mbps) – First try: Kodi hangs; Second try: ~2fps + artifacts (software decode)
  • Ducks Take Off [2160p a 243 Mbps].mkv (4K H.264 @ 29.97 fps; 243 Mbps; no audio) – HDD: OK
  • tara-no9-vp9.webm (4K VP9 YouTube video @ 60 fps, Vorbis audio) – ~5 fps, massive artifacts (software decode)
  • The.Curvature.of.Earth.4K.60FPS-YT-UceRgEyfSsc.VP9.3840×2160.OPUS.160K.webm (4K VP9 @ 60 fps + opus audio) – ~5 fps, massive artifacts (software decode)

Wow… I don’t think I can remember any platform that failed that badly. But I usually use the pre-installed Kodi app, which in this case was missing, so maybe that’s why. I’ll give up on Kodi 17.3 for now, but I remember somebody mentioned RKMC 16.1 would work better on Rockchip device. I did some research, and installed RKMC in the box. I made some mistake doing so, reinstalled the firmware and lost all my screenshots in the process. So maybe sure you backup any files before messing with the system partition or config files.

But what about the results with RKMC and my 4K videos samples?:

  • HD.Club-4K-Chimei-inn-60mbps.mp4 – OK, but not perfectly smooth
  • sintel-2010-4k.mkv – OK, but not perfectly smooth
  • Beauty_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) –  Crash
  • Bosphorus_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) – Crash
  • Jockey_3840x2160_120fps_420_8bit_HEVC_TS.ts (H.265) – Stays in UI + hang
  • MHD_2013_2160p_ShowReel_R_9000f_24fps_RMN_QP23_10b.mkv (10-bit HEVC, 24 fps) – Crash
  • phfx_4KHD_VP9TestFootage.webm (VP9) – ~5 fps (software decode)
  • BT.2020.20140602.ts (Rec.2020 compliant video) – Stays in UI + hang
  • big_buck_bunny_4k_H264_30fps.mp4 – OK
  • big_buck_bunny_4k_H264_60fps.mp4 – Almost smooth + audio delay  (4K H.264 @ 60 fps is not supported by the VPU in Rockchip RK3399 SoC)
  • Fifa_WorldCup2014_Uruguay-Colombia_4K-x265.mp4 (4K, H.265, 60 fps) – Crash
  • Samsung_UHD_Dubai_10-bit_HEVC_51.4Mbps.ts (10-bit HEVC / MPEG-4 AAC) – Stays in UI + audio
  • Astra-11479_V_22000-Canal+ UHD Demo 42.6 Mbps bitrate.ts (10-bit H.265 from DVB-S2 stream) – Stays in UI + audio
  • 暗流涌动-4K.mp4 (10-bit H.264; 120 Mbps) – OK (hardware decode)
  • Ducks Take Off [2160p a 243 Mbps].mkv (4K H.264 @ 29.97 fps; 243 Mbps; no audio) – HDD: OK
  • tara-no9-vp9.webm (4K VP9 YouTube video @ 60 fps, Vorbis audio) – ~5 fps, massive artifacts (software decode)
  • The.Curvature.of.Earth.4K.60FPS-YT-UceRgEyfSsc.VP9.3840×2160.OPUS.160K.webm (4K VP9 @ 60 fps + opus audio) – ~5 fps, massive artifacts (software decode)

So H.265 video won’t play, VP9 will, but are unwatchable with software decode, and only H.264 videos are playing relatively well.

I got more insights in my RKMC post with Superceleron commenting:

Well don’t expect miracles, besides that is a old kodi fork dont have python 2.7 and it have subtitles security flaw.
On my tests, on A7 version of rk sdk use kodi 18 nightly it will play ok almost all codecs. (i just made a custom rom for A95X-R2 so i know it plays it ok)
On A6 forget it, i had to make some mix custom roms to make it work with FTMC.. it now plays almost all codecs (it crashes with RKCodec on H264 10bits, but mediacodec play ok but max 720p)
Rockchip never learns….

and

Yes need to wait for it, or simple try one of my roms for 3399 like for the Yundoo Y8 or H96 Max.
It will work lot better than stock, i fixed the play of VP9 in Hw in FTMC but still crash in H264 10bits, and cant play Youtube 4k due to codec issue!

So at this stage, I’d recommend not to buy any Rockchip RK3399 TV box if you want to play videos in Kodi, and wait for Android 7.1 firmware which is expected sometimes in the next few weeks or months. If you already have one, and/or are ready to waste some time, you could try TVMC that works somewhat in Yundoo Y8, or use one of the aforementioned ROMs on Freaktab.

I’ll skip video testing in this review, and if time permits perform tests again once Android 7.1 firmware and Kodi 18.0 are released.

Click to Enlarge

I still checked DRM info, and there’s no DRM support at all, except for something called CENC ClearKey.

YouTube worked well for me up to 1080p. Video is rendered to the framebuffer (I can take screenshot of it), which should explain the resolution limitation, and mean that it’s likely using software decode.

Network Performance (Wi-Fi and Ethernet)

As I connected to my 802.11ac router, I noticed the Link Speed was 526 Mbps when I checked, so better than the usual 433 Mbps you get on most other devices.

I first tested WiFi 802.11ac throughput by transferring a 278MB file beetween SAMBA and the internal flash and vice versa using ES File Explorer. There’s again a serious problem with SAMBA implementation, as download speed was fairly good @ 3.00 MB/s, but upload speed drop to 1.06 MB/s only, leading to a well below average average speed.

Throughput in MB/s

But iperf tests below show 802.11ac performance is actually quite good in both directions, so the real problem is with SAMBA implementation/configuration in the firmware.

WiFi 802.11ac upload:

WiFi 802.11ac download:

I repeated the SAMBA test with a larger 885 MB file over Gigabit Ethernet, and I confirmed the same issue as it took 49 seconds to upload the file from the server, and 1 minute and 52 seconds to upload the file.

I also run iperf again for Gigabit Ethernet using full duplex option:

I was expecting higher numbers, but those values will be good enough for most people.

Storage Performance

I used A1SD Bench to evaluate sequential performance of internal storage, USB 3.0 and SATA interfaces.

Click to Enlarge

In theory, the Samsung eMMC flash used is very good, and results from the benchmark confirm this with 157.63 MB/s read speed, and 124.80 MB/s write speed. That’s the best performance I’ve ever gotten from an Android device.

Read & Write Speeds in MB/s – Click to Enlarge

Sure enough, I never had troubles with “app is not responding” or app beings slow to load.

USB 3.0 and SATA performance is also pretty solid, especially sequential read speed. Write speed was actually 100 MB/s the first time I tried with SATA, but after I had to reinstall the firmware, I never managed to get back to that result with the speed limited to around 72 MB/s.

Read & Write Speeds in MB/s – Click to Enlarge

Nevertheless, results are fairly good, and SATA should provide a little more performance than USB 3.0.

Vorke Z3 Benchmark & System Info

CPU-Z reports a dual cluster “RK3066” processor with two Cortex A72 cores @ 1.99 GHz, and four Cortex A53 cores @ 1.51 GHz, and an ARM Mali-T860 GPU.

Click to Enlarge

VORKE Z3 (rk3399_box) uses a 1920×1080 framebuffer resolution, comes with 3878MB total RAM (the rest being used by hardware buffers), and 26.74 GB interface storage.

Antutu 6.x score varies a lot between 69k and 78k due to thermal throttling. But if I run the benchmark right after boot, I get the result below, roughly the same as Yundoo Y8 one (76,819 points).

Vellamo 2.x results would also varies due to thermal throttling, but also because for some reasons SunSpider test would fail to run from time to time, as shown by the yellow mark on the first Chrome Browser test.
For some reasons, Chrome Browser result is much lower (4,512) compared to the 5,275 points I got with Yundoo Y8, but Multicore (2,587 vs 2,492) and Metal (2,311 vs 2,332) results are roughly the same.

3DMark’s Ice Storm Extreme results was also slightly lower with 9,726 points compared to 9,906 points for Yundoo Y8.

Click to Enlarge

At one point I only got 6,7xx points, but it was not because of overheating, and was instead due to the system randomly changing resolution and refresh rate, with the video output set to 3840×2160 @ 30 Hz at the time, limiting the framerate to 30 fps max.

Conclusion

I did not have the best experience with Vorke Z3 due to my struggle with Kodi and RKMC, and various smaller issues like HDMI output resolution & framerate randomly changing between reboots, lack of proper power off mode, some overheating, etc… But there are also some positives like excellent internal storage performance, good USB 3.0 and SATA performance, a USB type C port supporting data, and video output, and very good WiFi performance, so I’m hoping the upcoming Android 7.1 Nougat firmware will greatly improve the device usefulness.

PROS

  • Good overall performance and stable firmware
  • Fastest internal storage I’ve seen in any TV box
  • Fast USB 3.0 and SATA interfaces for external storage
  • Very good 802.11ac WiFi performance
  • USB type C port with support for data and video output (via an external dock).
  • OTA firmware update appears to be supported

CONS

  • The device is unusable with Kodi 17.3, or RKMC with most videos failing to play properly
  • Overheating leading to CPU / GPU throttling (The performance degradation is noticeable in benchmarks, but I have not really experienced it during normal use after playing a 2-hour video, or playing games for 15 minutes)
  • HDMI video output setting is not properly remembered, and it may be 720p, 4K30, 1080p60 at next boot.
  • No clean power off mode (mechanical switch only)
  • Only NTFS and FAT32 files systems are supported, no EXT-4, no exFAT
  • Lack of DRM support
  • Poor upload speed to SAMBA server

GeekBuying sent the device for review, and in case you are interested you could buy Vorke Z3 for $149.99 shipped with VORKEZ3F coupon on their website. You’ll also find the device from various sellers on Aliexpress.

A95X R2 TV Box Review – Part 2: Android 7.1, Video & Audio Tests, and Benchmarks

June 26th, 2017 6 comments

In the first part of A95X R2 Rockchip RK3328 TV box review, I listed the specifications of the device, took a few photos, and reported about the chip used in the PCBA. I’ve now had time to play with the box, so I can report about my experience with Android 7.1, video & audio capabilities, and the performance of the device in the second part of the review.

First Boot, Setup, and First Impressions

I connected a USB 3.0 hard drive to the USB 3.0 port of the box, a USB keyboard to one of the USB port to take screenshots, and a USB hub with the RF dongles for my air mouse and gamepad in the remaining USB 2.0 port. I completed the hardware setup with Ethernet and HDMI cables, as well as the power supply.

Click to Enlarge

I then pressed the power button on the unit to start it up, the front panel display showed a “Boot” string, and the box booted to recovery mode automatically.

I remember I had a “update.zip” file  for U5 PVR Deluxe in my USB hard drive, so maybe that was the issue. So I deleted it, and it can boot normally now most of the time, as sometimes it will still go into recovery mode, maybe a power issue since the 5V/2A adapter is just on the limit… So the only way to reliable boot the device is to remove the USB hard drive during boot. A typical boot is very fast, as it only takes about 19 seconds, faster than all devices I’ve reviewed so far.

Click for Original Size

It’s the first time I see this launcher, but the features are pretty standard with status icons, weather, date & time on the top, some shortcuts to the Play Store, web Browser, File Explorer, App list…., and a bottom row with customizable shortcuts. The HELP icon redirects to www.tvboxceo.com with a Q&A and Solutions sections explaining how to solve some common problems (e.g. how to install adult add-ons….), and download apps like TVMC (Kodi fork), Add-ons, Plex, Netflix, etc… Note that the resolution is only 1280×720, instead of 1920×1080 on most devices.

Click to Enlarge

Clicking on Settings will bring a right panels with various options, instead of starting it full screen. The settings are pretty standard with Network for WiFi and Ethernet, Sound including Audio device to select Default Output, Spdif passthrough, or HDMI bitstream, Display to select resolution from 720x480p-60 to 4096x2160p-60(YCbCr420), and other typical settings you’d normally find in Android.  What’s missing however are settings for HDMI CEC, HDR, and automatic frame rate switching that you’d normally find in (Amlogic) TV boxes.

Click to Enlarge

The box could detect EXT-4 and NTFS partitions in my hard drive, albeit with a wrong 32GB total size instead of around 250 MB, meaning BTRFS and exFAT are not supported. The Internal storage is reported to be 8.0 GB, but the actual available space is 4.7 GB before installing apps. The About Section shows A95x_R2 device runs Android 7.1.1 on top of Linux 3.10.104 with the latest Android security patch level dated February 5, 2017. The device is rooted, and OTA firmware appears to be working, but I could not confirm since I did not get any firmware update.

The box comes with Google Play for TVs, but search is not working as the search field will disappear about 2 seconds after you enable it, so I could only install some apps from the Top Free section like ES File Explorer File Manager, and Beach Buggy Blitz.

Instead I reverted to APKPure to install most apps, including Amazon Underground, which I then used to install the free version of Riptide GP2.

Click to Enlarge

The screenshot above is after I installed APKPure, Antutu and CPU-Z, but it shows few apps are pre-installed, and Kodi is even missing from the list. However, when I tried to install Kodi from APKPure, it showed it was already installed, and I could only open it with started TVMC 16.1, but more on that latter.

The Setup Wizard app will allow you to run some basic configuration like language, overscan adjustment, and networking. It was not triggered during the first boot for me.

The status bar can be hidden or shown are you prefer, and includes volume, Android buttons, and a screenshot button.

I tested the IR remote control, which worked reliably up to 7 meters, and further than that I started to experience key misses. IR learning function is also working. As usual, I spent most of the time using MINIX NEO A2 Lite air mouse instead since it’s so much more convenient in Android.

Power handling works a little differently than most TV boxes. The only way to turn on the device after you connect the power is to press the power button on the unit, but if your turn off the box with the remote control, it’s possible to turn it back on with either the remote control or the power button on the device. A short press on the remote control’s power button will put the box in standby mode, while a long press will show a menu to either power off or restart the device.

I’ve measured power consumption with and without USB 3.0 hard drive , and differentiated between “hard” power off (connected power supply, but not started), and soft power off (power off from remote control):

  • Hard power off – 0.0 Watt
  • Soft power off – 1.0 Watt
  • Standby – 1.0 Watt
  • Idle – ~4.0 Watts
  • Hard power off + USB HDD – 0.0 Watt
  • Soft power off + USB HDD – ~4.0 Watts
  • Standby + USB HDD – ~4.0 Watts
  • Idle + USB HDD – ~5.3 Watts

Soft power off mode is pretty much useless, as it consumes just as much as standby mode, and you need to go through the full boot sequence. If you want to properly turn off the device, you’ll need to press the button on the unit for about five seconds, release it, and you’ll see the “power off” sequence like you would on your phone. Power consumption will be zero watt in that case.

A95X R2 box is not super cool during use, but I have not noticed obvious CPU throttling during use. After playing a 2-hour video in Kodi / TVMC, the temperature measured with an IR thermometer on the top and bottom of the device was 49 and 54 °C respectively, and after playing Riptide GP2 for over 15 minutes, it went up to 50 and 57 °C. I also checked the temperature reported by CPU-Z after both test: 82.3 and 89.2 °C, so it looks to be on the limit. Gamin performance in Beach Buggy Racing and Riptide GP2 was very good, even with maximum settings, but it was certainly helped with the 1280×720 resolution.

My first impressions with the TV box were rather mixed as beside the first boot into recovery, a 720p user interface, and an unusable Google Play, I also had many “App isn’t responding window” due to the slow storage. The box basically comes to a halt when an app is being installed/updated in the background, so that’s certainly a major downside for the box, especially when it does so in the background while you are using another program.

Video & Audio Playback in TVMC, DRM Info, and YouTube

TVMC 16.1, a fork of Kodi 16.1, is installed the box. However, by default, no icon is shown. I first found the app via APKPure, as when I search for Kodi, I could not install it, instead I was offered to Open it, and TVMC was launched.

Click for Original Size

If you want easy access to TVMC icon, install it via the HELP section of the main launched. I enabled automatic frame rate switching in Kodi/TVMC, before trying a few videos (Linaro Media Samples) played from a SAMBA share over Ethernet:

  • H.264 codec / MP4 container (Big Buck Bunny) – 1080p – OK
  • MPEG2 codec / MPG container – 1080p – OK
  • MPEG4 codec, AVI container – 1080p – OK
  • VC1 codec (WMV) – 1080p – OK
  • Real Media (RMVB), 720p / 5Mbps – OK
  • WebM / VP8 – 1080p – OK
  • H.265 codec / MPEG TS container – 1080p – OK

All good, and to my surprise, automatic frame rate switching also worked just fine, so I played a few more videos, still from SAMBA, unless other stated (HDD = USB hard drive):

  • ED_HD.avi (MPEG-4/MSMPEG4v2 – 10 Mbps) – OK
  • big_buck_bunny_1080p_surround.avi (1080p H.264 – 12 Mbps) – OK
  • h264_1080p_hp_4.1_40mbps_birds.mkv (40 Mbps) – OK
  • hddvd_demo_17.5Mbps_1080p_VC1.mkv (17.5Mbps) – OK
  • Jellyfish-120-Mbps.mkv (120 Mbps video without audio) – HDD: OK

Still very good, so let’s switch to some 4K video samples:

  • HD.Club-4K-Chimei-inn-60mbps.mp4 (H.264, 30 fps) – OK
  • sintel-2010-4k.mkv (H.264, 24 fps, 4096×1744) –  SAMBA: Frequent buffering; HDD: OK
  • Beauty_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) –  OK
  • Bosphorus_3840x2160_120fps_420_8bit_HEVC_MP4.mp4 (H.265) – OK
  • Jockey_3840x2160_120fps_420_8bit_HEVC_TS.ts (H.265) – OK
  • MHD_2013_2160p_ShowReel_R_9000f_24fps_RMN_QP23_10b.mkv (10-bit HEVC) – OK
  • phfx_4KHD_VP9TestFootage.webm (VP9) – 1 to 2 fps (software decode)
  • BT.2020.20140602.ts (Rec.2020 compliant video; 36 Mbps; 59.97 Hz) – OK
  • big_buck_bunny_4k_H264_30fps.mp4 – OK
  • big_buck_bunny_4k_H264_60fps.mp4 – Not 100% smooth, and audio delay (H.264 @ 4K60fps not supported by VPU)
  • Fifa_WorldCup2014_Uruguay-Colombia_4K-x265.mp4 (4K, H.265, 60 fps) – OK
  • Samsung_UHD_Dubai_10-bit_HEVC_51.4Mbps.ts (10-bit HEVC / MPEG-4 AAC) – SAMBA: Some audio cuts due to buffering; HDD: OK
  • Astra-11479_V_22000-Canal+ UHD Demo 42.6 Mbps bitrate.ts (10-bit H.265 from DVB-S2 stream) –  OK
  • 暗流涌动-4K.mp4 (10-bit H.264; 120 Mbps) – HDD: perfect.
  • Ducks Take Off [2160p a 243 Mbps].mkv (4K H.264 @ 29.97 fps; 243 Mbps; no audio) – HDD: Slideshow
  • tara-no9-vp9.webm (4K VP9 YouTube video @ 60 fps, Vorbis audio) – Massive artifacts (software decode)
  • The.Curvature.of.Earth.4K.60FPS-YT-UceRgEyfSsc.VP9.3840×2160.OPUS.160K.webm (4K VP9 @ 60 fps + opus audio) – Lots of artifacts (software decode)

We can start to see some “cracks” here, with some 50 to 60 Mbps videos failing to play smoothly over Ethernet + SAMBA, and VP9 hardware decoding not implemented in TVMC despite being supported by the processor (in theory). So I tried again the VP9 videos in FileExplorer, but I was not able to login to the SAMBA share, so I played them from the hard drive instead. It worked, but with large black bars on the bottom, left and right on the screen, and rendered on the framebuffer, meaning 720p instead of 4K.

The option to adjust Zoom/Aspect Ratio is also not available while playing videos in Kodi either.

Click to Enlarge

TVMC has option for audio pass-through for AC3, E-AC3, DTS, TrueHD, and DTS HD, so I tested those over HDMI with Onkyo TX-NR636 AV receiver, as well as stereo output (PCM 2.0) downsampling.

Audio Codec in Video PCM 2.0 Output HDMI Pass-through
AC3 / Dolby Digital 5.1 Audio OK,
video: 1:1 aspect ratio
Audio OK,
video: 1:1 aspect ratio
E-AC-3 / Dolby Digital+ 5.1 OK OK
Dolby Digital+ 7.1 OK OK
TrueHD 5.1 OK OK
TrueHD 7.1 OK OK
Dolby Atmos 7.1 OK Dolby True HD 7.1
DTS HD Master OK OK
DTS HD High Resolution OK OK
DTS:X OK DTS HD MSTR 7.1ch B

My AV receiver does not support Dolby Atmos, nor DTS:X, so it correctly falls back to Dolby True HD and DTS HD Master. I’m pleasantly surprise here again. Note that the first time, I had some background noise for all pass-through options, but once I enabled HDMI BitStream in Android settings, and restarted TVMC, everything worked fine.

Playing Blu-Ray ISOs would often results in TVMC crashing, and when working AMAT.iso would play very very slowly, while Sintel.iso would play just fine. MPEG2 1080i videos & 720p/1080p Hi10p videos played perfectly well. That makes A95X R2 the very first TV box I own capable of handling all three Hi10p (10-bit H.264) videos samples I own at 720p, 1080p, and 4K resolution with video, audio, and subtitles working.

The box could also decode SBS and over/under 3D videos, but I could not confirm whether 3D is supported since my TV is not 3D capable. Various MKV, AVI, XViD/DViX, MP4, VOB/IFO, and FLV videos could play, and the TV box passed the reliability test with a 2-hour 1080p H.264 movie played over a SAMBA share. So while the Android performance is quite poor due to the slow storage, the video & audio capabilities are not too bad. The only problem is that at the end of the review, TVMC started crashing each time I played a video, and I could not fix it even after clearing cache and data….

Click to Enlarge

DRM Info app shows Google Widevine Level 3 is supported, meaning premium apps like Netflix won’t be able to support HD or UHD video playback.

YouTube works, but can only play videos up to 720p (1280×720) resolution likely before the framebuffer is set to that resolution. The video are also rendered to the framebuffer, instead of the hardware video buffer, as I could take screenshots with the video, something that is not supposed to be possible when playing video on the hardware video buffer.

Networking & Storage Performance

A95X R2 is limited to 802.11n @ 2.4 GHz, so that’s what I tested by copying a 278MB file between SAMBA and the internal flash, and vice versa, using ES File Explorer. I could transfer that file at 3.2MB/s on average, a very good results on this type of connection.

Throughput in MB/s – Click to Enlarge

I also used iperf -t 60 -c ip_server to test raw speeds.

  • WiFi upload:

  • WiFi download:

and results are again pretty good. I also quickly tested Fast Ethernet using full duplex transfer, and the bandwidth was maxed out in both directions:

I won’t test Bluetooth with this device, simply because it does not support it.

Switching to A1SD bench app for storage performance, I can confirm the cheap Samsung eMMC flash used in the device has poor write speed (6.46 MB/s), and read speed (71.14MB/s) should be ignored due to cached read. That flash is the reason of some of the very poor performance with the TV box at times, especially when write operation (e.g. installing/updating app) occur in the background.

USB 3.0 performance is however impressive with 100.25 MB/s and 80.21 MB/s read and write speed on the NTFS partition, and 94.52 MB/s and 90.73 MB/s on the EXT-4 partition.

Read & Write Speeds in MB/s – Click to Enlarge

The performance is excellent, and is close to the performance I got on my main computer with that drive. That’s very promising for other RK3328 hardware platforms, as sadly A95X R2 is bottlenecked by the Fast Ethernet connection.

A95X R2 TV Box / Rockchip RK3328 Benchmarks and System Info

CPU-Z reports a quad core Cortex A53 processor @ 408 MHz to 1.51 GHz with an ARM Mali-450MP GPU. The model is A95x_R2 (A95x_R2_8189), probably meaning there may be other models with a different WiFi module, and as mentioned previously the “screen resolution” is limited to 1280×720. 990 MB of total RAM is available to the system, and 5.27 GB of internal storage.

Click to Enlarge

A95X R2 scores 33,117 points in Antutu 6.x , or about the same as TV boxes with Amlogic S905X, which should be expected, as both processors have similar features, except RK3328 has one less GPU cores. This is mitigated by the 1280×720 UI instead of 1920×1080. If the developer had chosen the latter, the score should have been a little lower.

Vellamo 3.x also reports similar scores with 1,943 point for Chrome Browser test, 1,464 points for Multicore (one test failed), and 937 for the Metal score, against 1,855 points for Browser test (note: not Chrome), 1,491 points for multicore, and 910 for Metal for an Amlogic S905X TV box

3DMark ICE Storm Extreme v1.2 reveals the weakness of the CPU with just 2,252 points, against 4,183 points on Amlogic S905X processor.

Click to Enlarge

Conclusion

My review of A95X R2 did not start so well with boot problems when connecting a USB hard drive (using a beefier power supply may help), and frequent sluggishness and app is not responding pop-up windows appearing. However, video and audio playback are quite well support with automatic frame rate switching and HD audio pass-through working well. USB 3.0 performance is excellent, and I was also pleased with WiFi performance.

PROS

  • Latest Android 7.1.1 Nougat firmware
  • Good 4K video & audio support in TVMC (Kodi’s fork) with automatic frame rate switching and HD audio pass-through working for DTS HD and Dolby TrueHD
  • Very good WiFi performance and stability (for a device limited to 802.11n)
  • Excellent USB 3.0 storage performance similar to what I get on my main computer
  • NTFS, EXT-4, and FAT32 file systems supported
  • Pretty design with compact box and front panel LCD display
  • Fast boot (< 20 seconds)
  • OTA firmware update likely supported (but not tested, since no new firmware)

CONS

  • Slow eMMC flash leading to sluggishness, and poor performance at times
  • Connecting a USB 3.0 hard drive may lead to booting into recovery (random issue)
  • Google Play is not usable, due to non-accessible search function
  • TVMC/Kodi issues – no zoom option during playback, VP9 hardware decoding not working, random crash when starting to play Blu-ray ISO’s, failed to play any videos at the end of review
  • YouTube limited to 720p (due to 1280×720 UI)
  • No visible options for HDR, HDMI CEC, Deep Color, etc…
  • No Bluetooth support
  • DRM limited to Widevine Level 3

It’s hard to recommend A95X R2 TV box due to the serious cons, but I find Rockchip RK3328 good be a good base on hardware with a faster eMMC flash, and Gigabit Ethernet, even potentially suitable for a NAS + TV Box combo due to the excellent USB 3.0 storage performance.

I’d like to thank GearBest for providing a sample for review. If you are still interested, you could purchase A95X R2 for $32.99 including shipping using GBA95XR2 coupon. The TV box can also be found on Banggood, GeekBuying, and Aliexpress for just under $40.

NanoPi NEO NAS Kit Review – Assembly, OpenMediaVault Installation & Setup, and Benchmarks

June 18th, 2017 55 comments

NAS Dock v1.2 for Nano Pi NEO / NEO 2 is, as the name implies, a complete mini NAS kit for 2.5″ drive for NanoPi NEO or NEO 2 board. The NEO 2 board is strongly recommended, since it’s not much more expensive, but should deliver much better results due to its Gigabit Ethernet interface. I’ve received two of those kits together with several other boards & accessories from FriendlyELEC, and today I’ll show how to assemble the kit, configure OpenMediaVault, and run some benchmarks.

NAS Kit V1.2 Assembly with NanoPi NEO 2 Board

The only extra tool you’ll need is a screwdriver, and potentially a soldering iron as we’ll see further below.
The metal box is stuff wih accessories so the first thing is to open one or two sides to take out the content. We have the mainboard, NanoPi NEO back plate, NanoPi NEO 2 back plater, a heatsink and thermal set, and a set of 5 screws to tighten the hard drive which mean there’s one extra screw. FriendlyELEC always adds extra screws, and I find it’s a nice touch, as it can be a real pain if you happen to lose one.

Click to Enlarge

Let’s have a closer look at the “1-bay NAS Dock v1.2 for NanoPi NEO/NEO2” board. We have a UAS capable USB 3.0 to SATA brige chip between the two header for NanoPi NEO board (note that the USB connection will be limited to USB 2.0 since the board only supports that), an LED, a USB 2.0 host port for a printer, WiFi dongle, or webcam, the power switch, the power jack, a 3-pin serial header, an I2C connector for Grove modules, and of course the SATA connector.

Click to Enlarge

There’s not much on the other side of the board, except a CR2032 battery slot for the RTC.

Before going further, you’ll need to go to the Wiki, and get the latest OpenMediaVault firmware, in my case nanopi-neo2_debian-nas-jessie_4.11.2_20170531.img.zip, which I then flashed with Ether program to a micro SD card..

Once this is done, install the heatsink and thermal to your NanoPi NEO 2 board, and insert the micro SD card into the board.

Notice that I also soldered the headers. While it would be obvious to people would have looked at the pinout diagram, I’ve read some people have justed connect the board using the (pre-soldered) 4-pin header, as they may have believed it was a USB header, but it’s just the serial console instead, and obviously the hard drive was not detected. If you don’t feel like soldering the headers to the board yourself, make sure you tick the option “with pin headers soldered” when ordering. It just costs $1 extra.

Now we can insert our board into the “1-bay NAS Dock” board, instead the hard drive, and optionally an I2C module. I connected an I2C OLED display i the picture below for illustrate, as using the display would require cutting out the case. Some people may want to connect an I2C temperature sensor instead.

Click to Enlarge

I used four screws to tighen the hard drive on the other side of the board, and install a CR2032 battery for the real-time clock.


Finally, you’ll need a 12V power supply with at least 1A, but I could not find any (safe) spare ones so I used Maxoak K2 power bank instead, since it can output 12V @ 2.5 A max.


OpenMediaVault Setup on NanoPi NEO 2 Board

So I connected everything, and applied power, but the board would not boot with the Ethernet Link LED blinking in a regular fashion, meaning something was very wrong. So I took out the board, and connected a serial debug board, connect to the console via minicom using 115200 8N1, and that’s what I got:

The boot was just stuck there. I re-inserted the micro SD in my PC, and I could see both boot and rootfs partitions, so everything looked good.
Then I powered the NanoPi NEO 2 board with a 5V/2A power supply only, and the boot succeeded:

Then I went back to the 12V power input on NAS Kit with the power bank and the boot succeeded. Very strange. It turns out the board would not boot most of the time, but the symptoms are not reproducible 100% of the time. This kind of random behavior is usually a timing or distorted signal issue. So I thought the micro SD card might not play well with the board, and the power bank signal might not be so clean. So I first flashed another micro SD card, but same results. I used another 12V/5A power supply, and it did not really help either. Finally, I used another NanoPi NEO 2 board and it appears to be stable.

You can find the board using FriendlyELEC.local if bonjour services are running in your computer:

Alternatively, you could check out the IP address in other ways. In my case, I just type friendlyelec.local in Firefox to access the web interface. The default username and password are admin and openmediavault.

Click to Enlarge

After login, you can access the dashboard showing system information, and which services are running. You may want to disable the services you don’t need.

Click to Enlarge

You can go to Storage->Physical Disks to check if your hard drive has been detected. No problem for me here with a 931.51 GiB drive detected.

Click to Enlarge

You may then want to setup a fix IP address. There are various ways to do this but I went to Network->Interfaces and set eth0 to a fixed IP address. You’ll be asked to apply the changes once it’s done.

Click to Enlarge

I also changed the hostname to CNX-NEO2-NAS in the General tab.

After that I decided to address some security issues. First by changing the administrator password in General Settings->Web Administrator Password.

I then went to Access Rights Management->User to find out there were two pre-configured users: pi and fa. I deleted fa user, changed pi’s user password, and added it to ssh group. It’s actually even probably better to just delete both user, and create your own.

The root user is not shown, but you’ll want to login as root through ssh first and change the password, as the default password is fa. Once it’s done, you’ll have better security, and your system should not be easily accessible via basic “hacks”. For more security, you’ll still want to install an RSA certificate. A self-signed one should do if you plan to use it only in the local network, but you may also consider a free Let’s Encrypt certificate instead.

We can now take care of the hard drive. I went to Storage->File Systems, and clicked on +Create file system which will let you choose between BTRFS, EXT3, EXT4, XFS, and JFS. I’ve gone with EXT4 first.

Click to Enlarge

After a few minutes you drive should be formatted, so we can configure network shares. I want to use SAMBA and SFTP to transfer files for the purpose of this review, so I went to Access Rights Management->Shared Folders to add a new share called HDD for the root of of hard drive. You may want to add multiple share if you plan to split videos, documents, music and so on.

Click to Enlarge

I clicked Save, and selected ACL to add permissions to pi and admin users. You can add whatever users you plan to use to access the share.

Click to Enlarge

That share3d folder can now be assigned to the services you plan to use. SFTP is enabled by default when SSH is running, so I create a SAMA/CIFS share by going to Services->SMB/CIFS->Shares to add the share.

Click to Enlarge

Browsing the Network with Nautilus would show both cnx-neo2-NAS – SMB.CIFS and cnx-neo2-nas – SSH (SFTP) shares.

Configuration is now complete. I have not find a clean way to power off the system, so I normally open a terminal session via ssh and run the shutdown now command. A software button to turn of the NAS would have been a nice features on the kit.

I also often encountered the error “Software Failure. Press left mouse button to continue. Session not authenticated.” before the session timeout is set to 5 minutes. If you prefer a longer timeout, you can change it in General Settings->Web Administration.

In case you want to use the RTC, you may first want to set the timezone:

Check the date is correct, and write it to the hardware clock:

before reading it back.

You can test it by rebooting the board without the Ethernet cable:

Perfect! You’d just have to make sure the “set” command is run automatically at boot time if the time in the RTC is set. It would be good if FriendlyELEC updated their image to do that automatically at boot time.

NAS Dock V1.2 + NanoPi NEO 2 Benchmarks

Since I can now copy files and folders over SAMBA and SFTP, we can start running some benchmarks to evaluate performance. I’ll use EXT-4, BTRFS, and XFS file systems on the hard drive, and run iozone to specicially test storage performance, following by copying large and small files over SAMBA or SFTP to test real-life NAS performance. For large file copy, I’ll use a folder with 7 large files totaling 6.5 GB, and for small files, I’ve done a fresh checkout of the Linux kernel in my computer:

and removed symlinks since they may cause issues during copy, as well as .git directory with a huge 1.8GB file:

The end result is a directory with 64,013 files totaling 748.6 MB.

Iozone results

EXT-4:

BTRFS:

XFS:

I’ve taken results with 16384kB reclen for read, write, random read and random write values to draw a chart, since most people are likely going to store large files in their NAS. The smaller reclen could be interesting if you plan to handle smaller files.

All three file systems have a very good read speed of around 40 MB/s, but BTRFS write appear to be the fastest among the three, with EXT-4 being the weakest at around 25 MB/s. But for some reasons, those results are useless in practice, as we’ll see below. Finding out the exact reason would possibly require studying and profiling iozone and the kernel source code which would be outside of the scope of this review.

File copy over SAMBA and SFTP

Results for large files in minutes and seconds.

File Copy  Large Files SMB SFTP
Write Read Write Read
EXT4 02:49.00 02:40.00 03:54.00 04:15.00
BTRFS 03:20.00 02:40.00 03:48.00 04:32.00
XFS 02:45.00 02:38.00 03:36.00 04:23.00

Chart converted to MB/s.

Read and Write Speeds in MB/s

First, we can see very good read performance from the NAS (NAS to my PC)  with 41 to 42 MB/s close to the theorethical limit of a USB 2.0 connection. Write speed is a a little different as the files were transferred more slowly with BTRS, and around 40MB/s with EXT-4 and XFS.  Since SFTP is encrypted the transfer speed is roughly the same for all three file systems. Overall the file system you choose does not really impact performance with large files.

Results for small files in minutes and seconds.

File Copy  Small Files SMB SFTP
Write Read Write Read
EXT4 15:26.00 18:34.00 09:02.00 12:48.00
BTRFS 18:48.00 18:02.00 10:30.00 11:30.00
XFS 17:33.00 18:22.00 09:18.00 12:35.00

Chart converted to MB/s.

Transferring a large number of small files over SAMBA is really slow, and barely faster over SFTP. Again,there aren’t any significant differences between file systems here.  If you are going to transfer a large number of small file over the network, you may want to either compress the files before transfer, or compress the files on the fly using the command line:

It took just 1 minute and 49 seconds to transfer all 64,013 files, or over five times faster than SFTP write to XFS, at around an effective 6.86 MB/s. So knowing your tools may matter as much as having the right hardware.

I was going to run a last part after enabling optimizations provided by tkaiser, but it turns out FriendELEC has already done that in their firmware image.

If you want to reproduce the setup above, you’ll need to purchase NAS Kit v1.2 for $12.99, and a NanoPi NEO 2 with soldered headers for $15.99. If you don’t have a 2.5″ hard drive, you’ll need to add this, as well as a 12V power supply which you could purchase locally, or on FriendlyELEC website for under $10. All in all that’s cheaper than a similar kit with a Raspberry Pi 3 board, and you’ll get close to four times the SAMBA performance for large files since RPi 3 will be limited to 10 to 12 MB/s due to the Fast Ethernet connection.

Micro SD Cards for Development Boards – Classes, Tools, Benchmarks, Reliability, and Tips & Tricks

June 13th, 2017 38 comments

When people plan to use a development board for their project, they mainly focus on the requirements of the development board itself, as well as software support. But selecting the right accessories may be just as crucial for good performance and stability. For example, selecting a proper power supply is important, as the board may freeze or randmly rebooted if it is not feed at the right voltage. Part of this is selecting a micro USB cable, as you’ll want a cable with minimal resistance which can be achieved through shorter cables and/or a low AWG value. Another important item that can impact stability and performance of the systems are micro SD cards when used to run the operating system in development boards.

Understanding SD Card Performance Metrics & Classes

Until a few years ago, (Micro) SD cards were primary used to store data such as photos, videos and music. In those use cases, you have large files that benefit from high sequential read and write speeds. That’s why the SD association created different classes to specify a minimum write speed with speed class, UHS speed class, and more recently Video Speed Class.

That’s useful as you won’t need the same write speeds to copy music files, or record videos in a 4K camera, and the cards with a lower class are normally cheaper. Note that most (all?) low cost development board do not support UHS-II or UHS-III interface, so it may work, but you won’t be able to reach the maximum performance of the card. Micro SD card controllers are also often connected via SDIO which limits the sequential performance to around 23 MB/s.

However, those classes are only marginally useful for micro SD cards used in development boards, since operating systems generate a lot of small read and write operations, for example for databases and web caches, rather than handling large files.  That’s why storage devices may also include read and write IOPS (I/O per seconds) which are more useful for this type of use case. The table below about Samsung eMMC and UFS flash chip shows some of those numbers.

Click to Enlarge

The only problem is that until very recently, there was no way you could know minimal IOPS R/W numbers for SD cards since they were simply not made available by the manufacturers, so instead people were recommending Class 10 micro SD cards which may or may not best suited to their use in development boards. However, since the introduction of Adoptable Storage in Android, where micro SD cards could be used for apps instead of just data, IOPS values became more important, and the SD association has now introduced Class A1 & Class A2 application performance classes for SD cards specifying minimum IOPS R/W values together with a 10MB/sec minimum sustained sequential write speed.

That’s great! Problem solved then? Not quite, as manufacturers have only started to use application performance class logos in their higher-end SD cards with high capacity like 256GB, which usually cost several times the price of your typical development board. So we need to rely on the community to test IOPS R/W values, or random read and write speeds.

Software Tools for SD Cards

There are several program that can test performance. hdparm and dd are popular, but the former only test sequential speed for a very short time, and the latter only test sequential speed, and many people may not include the time it takes to flash the cache to the actual card (e.g. with sync), leading to potentially misleading results, and  mostly irrelevant to our use case.

So instead, there are specific tools to tests both sequential speeds and random I/O performance. Bonnie++ is one, but recently iozone has become the reference for disk I/O testing, and that’s the one used in the benchmarks below.

Many tutorials recommend to flash firmware using Win32DiskImager in Windows, or dd in Linux. However, while I have not experience the issue myself, I’ve been told the latter may not always detect errors while flashing, and a new tool is now recommended: Etcher. It works in Windows, Linux, or Mac OS using a GUI or the command line, and will verify the SD card after flashing making sure nothing has gone wrong during the process.

SD Cards Benchmarks

With that in mind we need micro SD cards with good random R/W performance, which we can test with iozone3, and for many applications 8 to 32 GB capacities are enough. We were planning to test ourselves, but Andreas Spiess has already published a videos – embedded further below- leveraging pidramble work, with benchmarks using several micro SD cards on a Raspberry Pi 3 board. You’ll also find more results and discussion on Armbian forums. Linaro also made a survey of micro SD cards a while ago.

Random Read & Write Speeds in MB/s – Cards Sorted by Random Write Speed – Click to Enlarge

Here are the main takeaways from the tests:

  • Larger cards are usually faster but not always (See Samsung EVO+ 64GB results).
  • Benchmarks vary a lot between runs for a given card with 10 to 20% variations
  • Most low cost micro SD cards / clones have really bad random write performance.
  • Samsung and Sandisk cards get the best random I/O performance
  • It’s not always necessary to buy the most expensive cards for usable performance.

The conclusion is that for the best performance select cards like Samsung EVO+ 32GB or Samsung Pro+ 32GB selling for respectively around $18 and about $25 on Amazon US. A cheaper card that should work decently is Sandisk Extreme Pro 8GB sold for $8.

The Problem of Fake Micro SD Cards

It looks like we have now found a decent list of micro SD cards to use with our development boards. There are still problem has some are copy, even on sites like Amazon , as manufacturer of fake card may insert them in the supply chain. One way to double check if you have an original is to check the CID (Card Identification) number. In Linux, you can do so as follows:

The first two digit represents the manufacturer ID. There’s no public list, but “1b” is for Samsung. You can decode the full CID on here.

If you’ve bought a Samsung SD card, and the Manufacturer ID is difference from 0x1b, then you almost certainly have a fake Samsung SD card. If the number is 0x1b, you can’t be 100% sure you have a genuine SD card however, as the clones may have also cloned that part.

Sadly there’s no easy other way to check if a card is real or fake. Checking the performance is similar to other reports online may help confirm you have a proper card. That 2010 article by Bunnie has more details about fake SD cards.

Longevity and Reliability of micro SD Cards

I’ve been using micro SD cards for about 5 years both in development boards, and my phone, and I have to say I’ve had to throw away many of them due to I/O errors after a while. I’ve also found my card readers also only last a few months. So far, I mostly bought cheaper ones, so it could be an explanation. Sadly, manufacturers do not provide any MTBF (Mean Time Between Failure) for SD cards so there’s no data like on hard drives and SSDs.

However, there are ways to limit wear and tear on micro SD cards by limited writes, either by making the rootfs read-only which is fine for embedded projects, less if you plan on using your board as a computer, or using log2ram, extending the write buffer commit interval, etc… More discussion about SD card lifespan can be found on Pete Scargill’s blog.

Some people will also encounter corrupted flash that won’t boot anymore after a power failure or a crash because the file system has been corrupted. It is not directly related to the SD card itself, and it’s more likely to occur if you have a poor power supply. Steps to reduce the probably of corruption involves the same steps as previously mentioned to limit wear and tear with a read-only file system and so on.

Another solution to avoid most the issues mentioned above is to use boards with eMMC flash, but of course going that route would mean that you’d have to avoid  Raspberry Pi boards, baring the RPi Compute Module. Neverhteless, a combination of carefully selected power supply, USB cable, and micro SD card should still give you a good experience, and avoid many potential headaches, with your board.

Thanks to Karl and Thomas for tips and insights.

ARM Chromebooks Run Android Apps Better, Exhibit Longer Battery Life than Intel Chromebooks (Study)

May 24th, 2017 4 comments

Google has been working on supporting Android apps and the Play Store on Chromebooks, which are normally sold with either ARM or Intel processors. So the ability to run Android apps well is one of the things to consider before purchasing a Chromebook. Shrout Research has published a paper entitled “Chromebook Platform Choice Important for Android App Performance” comparing an Acer Chromebook R13 with a Mediatek MT8173C ARM Cortex A72/A53 processor to Acer Chromebook R11 with an Intel Celeron N3060.

Click to Enlarge

The Intel Chromebook has a smaller resolution so this could be an advantage, so less resources are needed to update the display. However, the ARM processor is significantly more powerful than the Intel one according to GeekBench results, and Chromebook R13 is sold for $399 on Amazon US, while Chromebook R11 goes for $299 (and lower during promotions). So it’s not a perfect comparison, but it should give an idea especially when it comes to app stability.

Click to Enlarge

The ARM Chromebook can run Android apps “well” (only minor issues) and “very well”, while the Intel Chromebook also did a good job for entertainment and productivity apps, but performed unreliably, and in some cases very poorly with some social media apps, and games. Since most mobile phones are based on ARM, developers spent more time optimization code for the platform. Some performance issues may also be partially due to different CPU and GPU performance, but the study did not address this at all, except when saying R11 screen resolution was lower.

Click to Enlarge

The research company also ran “education simulation” in both Chromebooks, not using Android apps, but instead various website and apps in the web browser such as Edmodo, Google Docs, Solar Explorer, etc… and found out the Intel chromebooks was depleting the battery faster with the ARM Chromebook uses 11.5% less power.

One last point shown in the disclaimer at the end of the paper:

This paper was commissioned by ARM Holdings. All testing, evaluation and analysis was performed in-house by Shrout Research and its contractors. Shrout Research provides consulting and research services for many companies in the technology field, other of which are mentioned in this work

Via Brent Sullivan on G+