Posted on by Jean-Luc Aufranc (CNXSoft) - No Comments on UP TWL AI Dev Kit review – Benchmarks, features testing, and AI workloads on Ubuntu 24.04

UP TWL AI Dev Kit review – Benchmarks, features testing, and AI workloads on Ubuntu 24.04

Earlier this month, I started the review of the Intel-based UP AI development kits with an unboxing of the UP TWL, UP Squared Pro TWL, and UP Xtreme ARL single board computers. I’ve now had time to test the first model, the credit card-sized, Intel Processor N150-based UP TWL SBC with 64GB eMMC flash preloaded with Ubuntu 24.04.

As usual, I’ll run a few benchmarks and test the board’s key hardware features, but I’ll then focus on the AI part since that’s what the kit is for. Note that the UP TWL AI Dev Kit is an entry-level solution, and all AI workloads will be running on the CPU or the integrated GPU, since there’s no dedicated AI accelerator or an M.2 slot to add one on this model. In the next parts of the review, the UP Squared Pro TWL adds an Hailo-8L AI accelerator, and the UP Xtreme ARL delivers up to 83 TOPS through a 14-core Intel Core Ultra 5 225H “Arrow Lake” processor.

UP TWL AI Dev Kit Review

UP TWL SBC system information

The UP TWL quad-core SBC comes preloaded with Ubuntu 24.04.3 LTS installed on a 64GB (62.6GB) eMMC flash, and the system also features 8GB of RAM.

UP TWL Ubuntu 24.04 About

We can get more information with the inxi utility:


All main features seem to be detected properly, including Gigabit Ethernet and the USB camera I connected to the board.

Benchmarks

Since the performance of the Intel Processor N150 and other Intel Alder Lake-N/Twin Lake processors at large is well known, I’ve just run sbc-bench.sh in this review:


The UP TWL runs super cool thanks to its built-in fansink, and the heatsink is cool at all times. It’s, however, rather noisy like most other hardware platforms from AAEON.  They design hardware for the industrial market, so low noise may not be as important as in consumer devices.

In terms of performance, the UP TWL SBC achieved 6810 MIPS in the 7-zip benchmark on average, which compares to 9730 MIPS on the Zimaboard 2 with the same Intel N150 SoC and rather poor cooling.

The PL1/PL2 power limits can explain the difference:


PL1 set to 6W and PL2 to 25W, which compares to 12W and 20W for the Zimaboard 2 and 6W and 12W for the Intel N100-based MINIX Z100 0dB fanless mini PC. The results are widely different. I ran the test again, this time without background activity detected, and the results were the same. I tried to change the PL1 to 12W in the BIOS…

UP TWL BIOS Power Limits

… and started again:


11,690 MIPS in 7-zip is more like it. Note that all other tests below were done with PL1 set to 6W (default), rather than 12W. It’s not the first time AAEON set their board to a conservation PL1 value, potentially for improved stability in high temperature environments, and they let customers select an optimal value in the BIOS as needed..

UP TWL SBC features testing

I’ve also checked the key hardware features of the UP TWL SBC as follows:

  • HDMI – Video OK, Audio OK
  • Storage – eMMC flash OK: 317 MB/s sequential reads, 230 MB/s sequential writes.
  • Gigabit Ethernet – OK (iperf3 DL: 942 Mbps, UL: 942 Mbps, full-duplex: 938/935 Mbps)
  • USB ports tested with an ORICO NVMe SSD enclosure (EXT-4 partition), USB mouse, RF dongle for a wireless keyboard, and USB camera
    • USB 3.0 combo jack
      • Top – 10 Gbps; tested up to 999 MB/s with iozone3
      • Bottom – 10 Gbps; tested up to 1,007 MB/s with iozone3
    • USB 3.0 on Ethernet combo jack – 10 Gbps –  tested up to 1,009 MB/s with iozone3
  • RTC – OK
  • GPIOS – OK – Also see the 40-pin GPIO header layout for all UP boards.

Everything works as expected from those tests.

AI testing on the UP TWL Intel N150 SBC

Since it’s an AI development kit, I ran several AI workloads on the system using Network Optix Nx Meta and the AAEON UP AI toolkit

Network Optix Nx Meta

I started with the Nx AI Certification Test. Let’s install it first:


The last command will download models (about 3.5GB of data) and may take a while. We can now run all the tests:


Everything happens within the terminal, and there’s no visualization. There are two main parts: benchmarks and stability tests.

Here’s the end of the AI benchmarks log:


Some of the tests run at an acceptable speed, while others struggle at under 1 FPS. We’ll use this performance data to compare it against the results for the Intel N150+Hailo-8L and Intel Core Ultra 5 225H in the next parts of the review. You can check the full benchmarks log if interested.

The stability test was successful:


Again, I saved the full log for that part.

AAEON UP AI toolkit demos

In the second part of the AI demos, I’ll use the UP AI toolkit examples available on GitHub.

Those are the steps to install and launch the AAEON UP AI toolkit:


I didn’t go exactly smoothly for me, as the first time, the prepare command ended as follows:


A download error occurred, but if you only read the last few lines, it looks like the installation was successful, while it was not (OpenVino was not installed). So, I had to run the command again, and the second time it progressed further, but I encountered another timeout (twice) when the script attempted to download TensorFlow. It might be useful to change the pip mirror, but I could not find any in Thailand. Nevertheless, the fourth time, the installation was (almost) successful:


It turns out all those AI demos take a lot of space, and a 64GB eMMC flash is a bit tight:


So I deleted the nxai_test, some cached files from pip, and finally, the installation was successful:


The up-ai directory is 33GB after installation is complete:


This does not include the several GB needed for pip packages. As I used the AI demo, I got into more storage capacity issues, and eventually connected a USB SSD to temporarily move some of the files to it to complete this review. 64GB of flash is clearly not enough for an AI development kit.

After a reboot, we can try the application by running the command:


I was expecting a menu here, but instead it launched Firefox and opened localhost:8080, giving us access to the UP Edge AI Sizing Tool.

UP Edge AI Sizing Tool dashboard

It is a zero-code configuration dashboard that allows users to easily set up AI applications by selecting inputs, accelerators, performance modes, and AI models right from a web browser. To get started, click the Add demo button on the left panel.  We can then select Computer Vision, Natural speed, or Audio demo. I went with a Computer Vision demo for Object Detection using the UP USB camera. I added one Yolo8s demo using the Intel N150 CPU, and another identical demo relying on the “Intel Graphics (GPU)”.

UP Edge AI Sizing Tool Add demo

We can click on the demo in the left panel to start it, and the camera output will show up with boxes to highlight detected objects. The dashboard also reports the frame rate, about 1.25 FPS on the CPU.

UP TWL DLStreamer object detection CPU framerate

We can also check the CPU, GPU, and memory usage as the AI workload is running. We’re close to 100% CPU usage here, and in the screenshot below, the frame rate dropped further to 1.04 FPS.

Up Edge AI Sizing Tool CPU object detection CPU GPU usage

Switching to GPU-accelerated object detection improves the inference speed to about 6-7 FPS.

UP TWL DLStreamer object detection GPU framerate

CPU usage is still close to 100%, and the GPU is now being used a bit more. Memory usage is about the same at 39% (of ~8GB)

Up Edge AI Sizing Tool GPU object detection CPU GPU usage

After looking at the script code, I realized I needed to add a parameter (any will do) to get a menu:


The first one will use videos stored in the eMMC flash, but since we have the camera connected, I went with option 2 to run object detection with the USB camera, so it should similar to above, just not in a web browser:

Intel N150 Camera Object detection installationI was told OpenVino Object Detect was already installed, but since I had never run that test before, I just asked the script to delete and reinstall it. Once done, we are asked to select the hardware (Intel Device was the only option), and where we want to run it on the CPU or GPU, and I went with the latter.

Intel GPU Object Detection

A window will open with the camera output, detection boxes, and red text with inference time (38.8ms) and frame rate (25.9 FPS). That’s much faster than in the web browser…

GPU Object detection Intel N150

We can also press the “a” key to see CPU and memory usage.

UP TWL AI Object Detection CPU memory usage

I went back to the menu to install the Chatbot. I was again told it was already installed, so I went forward without requesting a new install, and it failed due to a missing demo environment…

UP TWL AI Dev Kit Chatbot installation failedSo I reinstalled OpenVino Chatbot, selected Intel Device, and tiny-llama-1b-chat, the only options I was offered.

UP TWL Chatbot installation

We now have a Chatbot up and running in the web browser. The speed is not too bad since it’s only a 1 billion parameter model, but you can expect it to know too much… It can be useful for small, custom language models.

OpenVino tiny llama 1b chat chatbot

Power consumption

Since it’s often requested, I also measured the power consumption of the development kit using a wall power meter:

  • Power off – 1.6 -1.7 Watts
  • Idle – 5.2 – 5.5 Watts (fan active at all times)
  • Stress test (stress -c 4)
    • First few seconds – 18.2 – 18.6 Watts
    • Longer runs – 16.7 – 16.8 Watts
  • Object detection – Camera + GPU – 17.6 – 18.1 Watts

Remarks: PL1 was set to 12W. An HDMI monitor (Eazeye Radiant), an Ethernet cable, a USB mouse, and a wireless USB dongle were connected to the board. I also added the UP USB camera for the object detection test.

Conclusion

The UP TWL AI Dev Kit is an entry-level artificial intelligence development kit relying only on the CPU and GPU of the Intel Processor N150 Twin Lake SoC. It’s clearly not an AI powerhouse, but it can be suitable for some AI workloads. The system ships with Meta Nx support, as well as the AAEON UP AI toolkit, to easily experiment with AI workloads. The main downside I found was that the 64GB eMMC flash can be filled pretty quickly, and the UP TWL SBC does not offer storage expansion options, except via USB 3.2 (10 Gbps) ports.

Otherwise, everything works as expected, including Gigabit Ethernet, a relatively fast eMMC flash, HDMI video and audio output, all three 10 Gbps USB ports, the RTC, GPIOs, etc…  As usual, the company is rather conservative with power limits, and you may extract more performance by changing the power limits in the BIOS, as we showed in this review.

Next up is the UP Squared Pro TWL AI Dev Kit, also based on an Intel N150 SoC coupled with 8GB RAM and a 64GB eMMC flash, but shipping with an Hailo-8L M.2 AI accelerator module for higher AI performance. I’ll also make sure to install an M.2 NVMe SSD to avoid the storage issues I had with the UP TWL AI Dev Kit.

I’d like to thank AAEON for sending the UP TWL AI Dev Kit for review. It can be purchased for $279 on the UP shop. The kit includes the board, a 12V/5A power supply, and a USB camera.

Share this:

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress. We also use affiliate links in articles to earn commissions if you make a purchase after clicking on those links.

Radxa Orion O6 Armv9 mini-ITX motherboard
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
0 Comments
oldest
newest
Boardcon MINI1126B-P AI vision system-on-module wit Rockchip RV1126B-P SoC