Test AI Performance in Android with AI Benchmark App

Many new phones now come with Silicon-on-Chips (SoC) featuring a Neural Processing Unit (NPU) or similar IP block(s) aiming to accelerate A.I workloads such as facial recognition or object detection. Some silicon vendors will report performance in terms of TOPS (Trillion of Operation per Seconds) with for example 2.25 TOPS for the latest Mediatek Helio P90 processor, or 4.20 TOPS for Hisilicon Kirin 980 SoC.

However, those numbers can be deceptive, and nothing beats actual tests. However, last time I checked Android benchmarks such as Antutu do not include tests specific to artificial intelligence just yet. Luckily there’s already an app for that called AI-Benchmark which you’ll find on Google Play.

AI Benchmark App
Click to Enlarge

The benchmark includes 9 tasks:

  1. Object Recognition / Classification with MobileNet – V2
  2. Object Recognition / Classification with Inception – V3
  3. Face Recognition with Inception ResNet V1
  4. Image Deblurring with SRCNN 9-5-5 neural network
  5. Image Super-Resolution with VGG – 19 neural network
  6. Image Super-Resolution with SRGAN neural network(CPU only)
  7. Semantic Image Segmentation with ICNet, 2X parallel  (CPU only)
  8. Photo Enhancement with ResNet – 12
  9. Memory Limits with SRCNN 9-5-5

Unless otherwise noted (CPU only) all tests make use of the CPU, NPU and/or DSP when available. More details on each test can be found on AI Benchmark website.

Xiaomi Mi A2 AI Benchmark
Click to Enlarge

I decided to give it a try on Xiaomi Mi A2 smartphone running Android 9.0. The phone is powered by a Qualcomm Snapdragon 660 mid-range processor which was not advertised with any kind of neural network acceleration at the time of launch. However, the processor features an Hexagon 680 DSP which may have been used, and may explain why the performance may not be as bad as I expected against phones with SoC equipped with an NPU like Google Pixel 3 XL (Qualcomm Snapdragon 845 with Pixel Visual Core) or Huawei P20 Pro (HiSilicon Kirin 970 with NPU).  Xiaomi Mi A2 gets 2,755 points, while the latter achieves 7,833 points.

However, the comparison results in the app do not use the top scores achieved in AI Benchmark, so instead, you may want to go to the website where the gap is much more significant, and top devices currently based on Kirin 980 get over 14,000 points.

Fastest AI ProcessorsIf we filter by processor instead, the upcoming Qualcomm Snapdragon 855 and Mediatek Helio P90 takes the lead with the latter taking the top step with  18,231 points.

Thanks to Jon for the tip.

Support CNX Software - Donate via PayPal or cryptocurrencies, become a Patron on Patreon, or buy review samples
Subscribe
Notify of
guest
4 Comments
oldest
newest most voted
Advertisements