Posted on by Jean-Luc Aufranc (CNXSoft) - 8 Comments on UP Squared Pro TWL AI Dev Kit Review – Intel N150 + Hailo-8L accelerator tested on Ubuntu 24.04

UP Squared Pro TWL AI Dev Kit Review – Intel N150 + Hailo-8L accelerator tested on Ubuntu 24.04

I’ve been asked to review three Intel-based UP AI development kits running Ubuntu 24.04 Pro. Last time, I tested the UP TWL SBC with Nx Meta and UP AI Toolkit, and most AI workloads would pass, but since these were running on the Intel N150 CPU or GPU, the performance was not optimal for most. I’ll now switch to the UP Squared Pro TWL “mid-range” AI devkit review with another Intel Processor N150 SBC fitted with a 13 TOPS Hailo-8L M.2 AI accelerator.

Both the UP TWL and UP Squared Pro TWL come with a 64GB eMMC flash, and I found out it was rather tight since AI software and models can take a lot of space. The UP Squared Pro TWL has a few M.2 sockets, so I’ll install an NVMe SSD to expand storage before installing the UP AI toolkit. As usual, I’ll run a few benchmarks and test the board’s key hardware features before focusing on the AI part.

UP Squared Pro TWL SBC system information

Like the UP TWL, the UP Squared Pro TWL Intel N150 SBC is preloaded with Ubuntu 24.04.3 LTS installed on a 64GB (62.6GB) eMMC flash, and ships with 8GB of RAM.

devkit-UPN-TWL01 Ubuntu 24.04

Let’s get more details with the inxi utility:


Most main features seem to be detected properly, including the two 2.5GbE RJ45 ports (only one connected) and the UP USB camera connected to the board. The Hailo-8L module is not listed here, but we can locate it with lspci.

Benchmarks

Before I run any benchmark on the UP Squared Pro, I’ll check the power limits:


PL1 is set to 6W and PL2 to 25W by default. As we’ve seen in the UP TWL review, this limits the performance in benchmarks like sbc-bench.sh. Since we are not talking about a consumer device here, it may make sense since the UP TWL and UP Squared Pro TWL SBCs are designed to operate in the -20°C to 70°C temperature range with their respective active coolers, and reliability may be more important than performance. It’s up to the customer to decide whether increasing power limits is appropriate for their application.

For comparison purposes, I went to the BIOS to set PL1 to 12W since this value is used in other Intel Alder Lake-N/Twin Lake processors.

Intel N150 BIOS PL1 12W

Time to run sbc-bench.sh:


There’s no thermal throttling here with the CPU temperature measured at 66.0°C after the benchmark. Performance is roughly in line with other Intel N100/N150 systems based on the average 7-zip score: 11,440 MIPS. For reference, the UP TWL achieved 11,690 MIPS in the same test. If you need to extract a bit more performance, switching to PL1=15W is also possible, and for reference, the GEEKOM Mini Air12 Lite mini PC reached 12750 MIPS with this power limit.

I switched back to PL1 = 6W for the rest of the test since it is the default value.

UP Squared Pro TWL SBC features testing

I’ve also checked most hardware features of the UP Squared Pro TWL SBC:

  • HDMI – Video OK, Audio OK
  • DisplayPort – Video OK, Audio OK (improvement over the UP 7100 SBC for which DisplayPort audio was not working for me)
  • eDP – Skipped, no hardware to test
  • Storage
    • eMMC flash – OK: 314 MB/s sequential reads, 230 MB/s sequential writes.
    • M.2 2280 PCIe socket tested with 2TB WD Black SN850X NVMe SSD – OK for PCIe Gen3 x2: 1,703 MB/s sequential reads, 1,700 MB/s sequential writes.
  • 2.5 Gbps Gigabit Ethernet
    • LAN1 (top) – OK (iperf3 DL: 2.35 Gbps, UL: 2.35 Gbps, full-duplex: 2.35/2.35 Gbps)
    • LAN2 (bottom) – OK (iperf3 DL: 2.35 Gbps, UL: 2.35 Gbps, full-duplex: 2.35/2.35 Gbps)
  • USB ports tested with an ORICO NVMe SSD enclosure (EXT-4 partition), RF dongle for a wireless keyboard/mouse combo, and USB camera
    • Top – 10 Gbps; tested up to 1,0007 MB/s with iozone3
    • Bottom – 10 Gbps; tested up to 1,008 MB/s with iozone3
  • RTC – OK
  • GPIOS – OK – Also see the 40-pin GPIO header layout for all UP boards.

I also tested a dual display setup using a portable HDMI full HD monitor (CrowView) and a 4K Google TV monitor (KTC A32Q8) with a DisplayPort input, and no problems here. It should even support up to three displays using the eDP connector.

AAEON UP Squared Pro TWL HDMI DisplayPort

I had no audio at first because the HDMI display lacks speakers, but I could switch to HDMI / DisplayPort 2 to get audio through the KTC monitor’s speakers.

Ubuntu 24.04 HDMI DisplayPort Audio Output

Besides the two audio output devices, the microphone from the UP USB camera also showed up as a n audio input device.

UP Squared Pro TWL Ubuntu Audio Output Input options

So all features work to expectation, and no major issue here.

AI testing on the UP Squared Pro TWL Intel N150 SBC with Hailo-8 AI accelerator

I will now run several AI workloads on the system using Network Optix Nx Meta and the AAEON UP AI toolkit, as I did on the UP TWL, but leveraging the 13 TOPS Hailo-8L AI accelerator included in the kit.

Network Optix Nx Meta

For this test, I’ll use the UP Squared Pro TWL’s built-in eMMC flash and the Hailo-8L equipped with a heatsink for cooling.

Intel N150 SBC Hailo 8L module with Heatsink

Let’s install with Nx AI Certification Test:


It’s basically the same process as for the UP TWL AI Dev Kit, except for the install_acceleration_library.py step, where I selected ONNX-HailoRT:


Once the installation is complete, we can run the test with the following command:


Everything runs within the terminal as expected, but I got a lot of failed benchmarks:


Eventually, only 6 benchmarks would complete successfully against 46 when using the Nx CPU option:


Several stability tests could not run:


… but the three that could, passed just fine:


See the full log on CNX Software’s pastebin.

I asked AAEON if it was normal, and here’s their answer:

Certain AI accelerators have compatibility with only with some models so it is expected.
If CNX Software want to run the tests for Hailo, they have to select hailo and perform the tests.
Then they can do the same for openvino and select the specific compute unit (e.g. GPU) to run the tests on the Intel SoC), Nx CPU is generic and I would not do it.

I selected Nx CPU for the UP TWL AI Dev Kit, so I decided to try OpenVino accelerator libraries and models since the UP Squared Pro TWL is based on the same Intel N150 CPU:


Results:


See full output on pastebin.

We now have some data so we can compare Intel N150 with Nx CPU, OpenVino, and Hailo-8L AI accelerator.

UP TWL
UP Squared Pro TWL
Nx CPU
OpenVino
Hailo-8L
80-classes-object-detector[640x640]
3.91 FPS
3.73 FPS
38.04 FPS
80-classes-object-detector[320x320]
15.24 FPS
14.73 FPS
90.31 FPS
postprocessor-python-example
14.98 FPS
14.73 FPS
88.40 FPS
Postprocessor-python-image-example
15.73 FPS
15.17 FPS
90.55 FPS
postprocessor-c-image-example
14.77 FPS
14.75 FPS
83.18 FPS
postprocessor-c-example
14.75 FPS
14.76 FPS
89.15 FPS
Model-Yolov9-e-converted-[640x640]
0.32 FPS
0.59 FPS
Failed
Model-Yolov4-[320x320]
15.25 FPS
14.68 FPS
Failed
Model-Mobilenet-V3
48.45 FPS
56.08 FPS
Failed

The first remark is that there’s little difference between Nx CPU and OpenVino in most cases. While Hailo-8L support is only implemented in a few tests in Nx Meta, it does offer significant benefits in the tests where it is, with 6 to 10+ times higher FPS.

Since I’d like to have more storage for AI testing, I also installed a 2TB NVMe SSD. But the way the board has been designed, it also means I had to remove the Hailo-8L heatsink. So let’s run the Nx AI certification test one more time.

UP Squared Pro TWL with Hailo-8L and M.2 NVMe SSD

Note that I had issues executing commands on the NVMe SSD because, for whatever reason, it was mounted with the noexec tag. I had to remount it as follows:


Alternatively, the following command does the trick:


I ran the NX AI Benchmarks again with the same six successful tests:


You can check the full log if you want. The postprocessor tests are not impacted, but the 80-classes-object-detector results vary quite a lot: 110.10 FPS with 320×320 images (instead of 90.31 FPS) and 29.80 FPS with 640×640 images (instead of 38.04 FPS).

AAEON UP AI toolkit demos

In the second part of the AI demos, I planned to use the UP AI toolkit examples available on GitHub.

Those are the steps to install and launch the AAEON UP AI toolkit on the NVMe SSD:


Like in the UP TWL AI Dev Kit review, I had to run the prepare.sh script several times due to HTTP errors, maybe because I’m based in Thailand.


It looks like there was a hardcoded path in the script, so I ran it manually.


At this point, I rebooted the system as recommended and started the script in a terminal on the Ubuntu desktop:


The UP Edge Sizing Tool dashboard was opened in Firefox, as in our review of the UP TWL AI Dev Kit.

UP Edge AI Sizing Tool Hailo 8L NPU not detected

But there’s just a little problem… The NPU is “not available”. So I tried other options by launching the menu:


But I encountered issues due to a mismatch between the setuptools version and Python 3.12. I tried to install new Hailo-8/8L packages from the Hailo website, but it didn’t help. Eventually, I contacted AAEON, and they provided an updated script within 3 days. It still failed with the error:


At this point, I was told that the UP AI toolkit only works with Hailo-8 and Hailo-10H AI accelerators, but not the Hailo-8L, which happens to be the default option (M.2 2230-Hailo 8L) when somebody purchases the kit. The Hailo-8L will be supported in an upcoming version of the toolkit, but it may take a while, especially since holidays are coming soon. So I’ll skip it in this review…

Power Consumption

I also measured the power consumption of the Twin Lake + Hailo-8L AI development kit using a wall power meter:

  • Power off – 3.4 Watts
  • Idle – 9.6 – 10.3 Watts (fan active at all times) | with SSD: 11.5 – 12.0 Watts
  • Stress test (stress -c 4)
    • First few couple of seconds – 24.3 – 24.4 Watts | with SSD: 25.8 – 26.2 Watts
    • Longer runs – 15.1 – 15.3 Watts | with SSD: 16.5 – 16.7 Watts
  • Object detection – Camera + GPU –  Not test (Hailo-8L not supported in UP AI Toolkit for now)

The board was connected to an HDMI monitor, 2.5GbE, a USB RF dongle for a wireless keyboard/mouse combo, and a USB camera.

Conclusion

The UP Squared Pro TWL AI Dev Kit is potentially a good platform for AI development thanks to the great price/performance ratio. All basic features work well, and compared to the UP TWL AI Dev Kit, it adds extra features like DisplayPort video output and M.2 expansion sockets. The AI performance is almost as good as on the Intel Core Ultra 5 225H-based UP Xtreme ARL (review published soon) for benchmarks that are supported. However, software support is not optimal, with only a few tests working in Nx Meta, and the UP AI Toolkit is not supported yet when combining the Intel N150 SBC with an Hailo-8L AI accelerator. It’s also limited by the 64GB eMMC flash (like on the UP TWL AI Dev Kit), but you can also add an M.2 NVMe SSD for additional storage after removing the heatsink on the Hailo-8L M.2 module. So as things stand now, it’s less than ideal, but once software support improves, it might be an interesting mid-range platform.

I’d like to thank AAEON for sending the UP Squared Pro TWL AI Dev Kit for review with an Hailo-8L AI accelerator. It can currently be purchased for $469.00 with the Hailo-8L AI accelerator, a Full HD USB camera, and a 12V/6A power supply. In the future, the company plans to offer other AI accelerators from DeepX and Axelera for up to 214 TOPS of AI performance.

Continue reading UP Xtreme ARL AI Dev Kit review – Benchmarks and AI workloads on an Intel Core Ultra 5 225H Arrow Lake SBC.

Share this:
FacebookTwitterHacker NewsSlashdotRedditLinkedInPinterestFlipboardMeWeLineEmailShare

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress. We also use affiliate links in articles to earn commissions if you make a purchase after clicking on those links.

Radxa Orion O6 Armv9 mini-ITX motherboard

8 Replies to “UP Squared Pro TWL AI Dev Kit Review – Intel N150 + Hailo-8L accelerator tested on Ubuntu 24.04”

  1. > There’s no thermal throttling here with the CPU temperature measured at 66.0°C after the benchmark

    Speaking of 66°C there may be thermal throttling (maybe only memory controller) since

    In case time permits you could share /var/log/sbc-bench.log for further insights.

    Reply

      1. For reference, the values in LattePanda Mu reviewed here were higher, but not quite as high as the results on GitHub:

        Sadly, the reviewer did not check power limits (Probably PL1=15W).
        https://www.cnx-software.com/2024/07/30/lattepanda-mu-intel-n100-som-and-carrier-board-review-part-2-ubuntu-24-04/#lattepanda-mu-benchmarks-on-ubuntu-24-04

        Reply

      2. Performance degradation is all over the place, the N150 should be able to clock at 3600 MHz but exceeds 3000 MHz only with light tasks. Even single-threaded the older N100 thingy outperforms it (3900 vs. 3700 7-ZIP MIPS) since clocking 150-200 MHz higher during this benchmark.

        But looking at measured temperatures it seems the degraded performance is more likely caused by power than thermal limits.

        Reply

        1. [ “Performance degradation is all over the place”
          Do we recognize a hardware capabilities vs. cost optimization ‘stabilization’ with current situations in economy, international affairs, energy provision and change in goods distribution?
          There’s a big change from DDR3 to DDR4 performance (and smaller increase in cost, 32bit(4GB) to 64bit(up to th. 64GB/ch.) vs. DDR4 to DDR5 (th.512GB/ch., with higher cost for absolute bigger doubling for the memory capabilities, maybe not for the cost/GB, but the systems are more demanding for overall memory supply, ‘DDR6’?)?
          A mobile (Twin Lake) core 3 would be N350/N355.
          Desktop core 3/5/7 is Bartlett Lake. (thx) ]

          Reply

          1. [ There’s no PCH (platform controller hub) with cpus later than 810/860/890 (800 series, additional 8-24 PCIe 4.0), because it gets an integrated chiplet within the cpu case (on-package, new socket LGA 1851 (previous LGA 1700), then LGA 1954)? (thx) ]

          1. Thank you.

            So it looks like it’s only power limits and I start to think that N150 is even a worse SKU than N100 in terms of energy efficiency (N100 being only popular amongst consumers due to Intel’s successful TDP marketing BS)

Leave a Reply

Your email address will not be published. Required fields are marked *

Boardcon MINI1126B-P AI vision system-on-module wit Rockchip RV1126B-P SoC
Boardcon MINI1126B-P AI vision system-on-module wit Rockchip RV1126B-P SoC