Testing AMD Ryzen 9 7940HS’ AI Engine in Windows 11

Some of the newer AMD Ryzen processors come with an AI Engine (also called NPU or IPU) that works in Windows 11 including the Ryzen 9 7940HS, Ryzen 7 7840HS, Ryzen 5 7640HS, Ryzen 7 7840U, and Ryzen 5 7640U. I’ve just completed the review of the GEEKOM A7 mini PC powered by an AMD Ryzen 9 7940HS CPU with Windows 11 – but need to wait before publishing it – so I decided to try the AI engine on the Ryzen 9 7940HS processor.

The AI Engine relies on AMD XDNA architecture, and AMD provides instructions to get started with examples, demos, and developer resources. So I decided to try some examples, but for some reasons I’ll explain below, I eventually had to settle for some demos.

Ryzen AI software installation (for the demos)

The examples hosted on GitHub require us to install dependencies for the Ryzen AI software stack.

The first step is the installation of the IPU driver which can be done by downloading the “ipu_stack_rel_silicon_prod.zip” after registering an account on the AMD website.

ipu_stack_rel_silicon_prod.zip download

Note that this requires signing a “Beta Software End User License Agreement”, and I initially assumed that was not needed here because the AMD IPU Device driver is already installed in Windows 11 Pro running on the GEEKOM A7 mini PC…

AMD IPU Device

But I was wrong. I had to install the new one (and uninstall the old one) for this part to work.

We are then asked to install a few programs:

  • Visual Studio 2019 (I’d recommend downloading it from a third-party website like TechSpot, because Microsoft requires login to download previous versions of VS).
  • CMake version >= 3.26
  • Python version >= 3.9
  • Anaconda3 or (Miniconda3)

I made sure to select the option add to PATH when installing CMake, Python 3.12, and Anaconda3. Note that Visual Studio 2019 may also be used to install CMake and other tools.

The next step is to download the Ryzen AI Software installation package (ryzen-ai-sw-1.0.1.zip), but this time around, I could not avoid signing the Beta Software End User License Agreement, so I did. The agreement is confidential (although anybody who registers for an AMD account can read it) and also prevents me from reporting any results without a written agreement from AMD. So I’ll just write public information for that part…

We’ll need to run the installation script and accept another EULA for RyzenAI:

If there are any errors (look for CRITICAL string) or the script does not complete, you’ll want to correct that by installing the relevant programs and making sure they are in the PATH.

The next step is to activate the conda environment…

… and run the test:

Once successful, the output should look like that as per the documentation:

It’s a bit longer than that, but you get the idea.

Yolov8 example with Ryzen AI software

Once the installation is complete, we can try some of the examples, and I thought to check out the Yolov8 example.  I spent a few hours on it, but eventually failed, and I did not want to spend more time on it since I couldn’t write about it to the Beta software agreement. But AMD published a video about the similar Yolov8_e2e tutorial from the same GitHub repository about two months ago. Some of the versions have changed, but the procedure is similar and requires quite a few more steps than the example.

YouTube video player

Ryzen AI demos

I then switched to the demos directory in the same GitHub repository, and I was not asked to sign any legal documents… I started with the Ryzen AI “cloud-to-client” demo that showcases searching and sorting images on AMD Ryzen AI-based PCs using Yolov5 and Retinaface AI models. I used this with the older IPU driver that shipped with Windows 11 Pro in the GEEKOM A7 mini PC.

The first step was to run the setup batch script:

It ended with a “critical” file missing, but I was still able to run the one of the programs.

You’ll find the log on pastebin.

I then run the program Qt demo:

AMD Image Categorizer NPU usage

I selected a folder with screenshots (I reckon it may not be the best for this type of test but that is what I had on the test PC), and the AMD Image Categorizer scanned my images, and I could see some NPU usage in HWiNFO64. See full output while scanning for reference.

Each image is now searchable and although many tags are not super relevant because of the source images used, it could find a bird in a screenshot I had done with YouTube.

AMD Ryzen 7940HS AI Engine Image Sorting

The cloud-to-client demo is also available as a web server, but it did not work for me, even after installing some extra Python modules.

AMD Object Detection Web Server

RyzenAI SW Server

I’d kind of expect demos to work out of the box without having to spend hours getting the right module or DLL version although my recent experience with the Orbbec Femto Mega depth camera in Windows 11 seems to imply this is business as usual in Windows…

There’s also another demo called “multi-model-exec” with the following models:

  • MobileNet_v2
  • ResNet50
  • Retinaface
  • Segmentation
  • Yolox

Sadly all failed me with the same error. I had to change some of the commands from the instructions or I would have had to stop earlier…

I first had to create a conda environment:

Full output:

We can now enable activate the environment:

I download the ONNX models and test image/video package (resource_multi_model_demo.zip), and unzip it under demo/multi-model-exec/ipu_modelsx4_demo/. After that, I could run the command to generate the scripts:

AMD Ryzen multi model exec generate_script.py

It looks good so far:

But trying any of the scripts fail with the same error:

Windows programs often have cryptic error messages and don’t know what to do with:

The file is there but the system complain about something for which a web search did not help. I’m suspecting some mismatch between versions… It’s probably a matter of spending a few hours or days to fix the issue, and I’m not going to do that.

So the NPU/IPU in the Ryzen 9 7940HS can be used in Windows 11, but that’s not exactly a straightforward task right now, and I’m not sure any programs for end-users have been released just yet… Let me know in the comments if you know any.

Share this:
FacebookTwitterHacker NewsSlashdotRedditLinkedInPinterestFlipboardMeWeLineEmailShare

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK 5 ITX RK3588 mini-ITX motherboard

13 Replies to “Testing AMD Ryzen 9 7940HS’ AI Engine in Windows 11”

  1. It’s also in the new 8700G and 8600G desktop APUs.

    That’s too bad and it might be considered an AMD failure rather than a Windows failure. They fall flat on their faces repeatedly when it comes to software.

    Best case scenario, XDNA1 is there just to give time for developers to start using it. The real action will happen after XDNA2 lands in the Zen 5 Strix family of APUs, with triple to quintuple the performance (45-50 TOPS instead of 10/16). That may actually outperform the iGPU in some cases, so they can hype up more than laptop battery life.

    1. Surprising they would have an on-package NPU that can’t keep up with the iGPU, unless it’s drastically more power-efficient than using the CPU – bearing in mind that the CPU will still have to feed the NPU, which is still a large part of the compute power budget. With Zen5 that will get even harder (+VNNI), so if it can’t beat the iGPU and CPU significantly on both power and latency then it’s going to be a waste of die space. Esp as the iGPU has PCI bus control and can do some AI without needing the CPU cores awake. I’ll be interesting, and I don’t know whether the NPU has one or more general compute cores with PCI bus control and there will be well documented APIs, perhaps even allowing apps to use it for wake words.

      1. It should be more power efficient and die area efficient for the operations it does, but I don’t have the numbers. It does have some of its own L2 cache. Intel also claims better power efficiency for their NPU in Meteor Lake.

        AMD claims 39 TOPs for Hawk Point, of which 16 TOPS is from the NPU. It was increased from 10 TOPS in Phoenix with clock speed increases alone.

        With Strix Point, the NPU hits 45-50 TOPS, the iGPU grows in size from 12 to 16 CUs, and there are 12 cores (4x Zen 5 + 8x Zen 5C) with new AVX-512 instructions we heard about the other day. They are probably increasing the die area devoted to the NPU. Strix Point is supposed to use the same TSMC N4 node as its predecessors.

        AMD is thinking about mobile first. There is no evidence they will bring XDNA to the non-APU desktop lineup yet. Microsoft is said to be pushing AMD, Intel, and Qualcomm to adopt >40 TOPS NPUs.

        We now know a potential application for using the NPU during gaming. Microsoft announced “Automatic Super Resolution” which may use the NPU for upscaling:

  2. > I’ve just completed the review of the GEEKOM A7 mini PC powered by an AMD Ryzen 9 7940HS CPU with Windows 11 – but need to wait before publishing it

    Why “need to wait”? Geekom requirement?

    1. Yes, GEEKOM asked me to wait until February 20. Sometimes it’s because they don’t have stock. The post is scheduled for that day.

  3. I got the same LoadLibrary Error. But I reran the driver installer: amd_install_kipudrv.bat. I added “cd /d %~dp0” to the top of the bat file. As there seems to be some file Signing with the CAT file. Or start the BAT file from within CMD as Admin. After a reboot the run_modelx4 demo works

  4. When you run a multi-model demo, could you try using CMD, but not powershell, as I have found some errors when using powershell.

    1. Thanks for looking into it. I have some issues with Python in CMD right now. I’ll check again this coming weekend.

  5. Hello
    I use ryzen-ai-sw-1.1 and installation runs without errors. But when I start quicktest I receive the following warning and the test does not bring any result:
    C:\Programs\anaconda3\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider ‘VitisAIExecutionProvider’ is not in available provider names.Available providers: ‘AzureExecutionProvider, CPUExecutionProvider’
    Test Passed

    Any thoughts ? Any help appreciated.

    1. Quicktest works now in a new conda environment and directories without a German character. But I failed to run the Yolo8 example from the tutorial – the build didn’t work.
      Compared to Cuda AMD always causes more problems and difficulties to me which is a bit frustrating.

  6. Thanks @cnxsoft for (re)pointing this to me. Now that I bought a Geekom A7 for myself, this is doubly interesting!

    Given how alpha-testing-like your entire experience with this on Win11 went, I wonder whether it would fare any better on Linux, specially considering that AMD has since then published (and been updating) a XDNA driver for Linux: https://github.com/amd/xdna-driver

    Have you had any experience with this on Linux? Any advice for me, in case I decide to try it? I’m a total newbie re: AMD GPUs, this is actually my first AMD graphics ‘card’ since they bought the ATI business in the 2000s..

    1. I don’t know about the XDNA-driver and I can’t see any reference to 7940HS in it.
      I still think it should be easier in Windows. It’s been almost three months since I tested it on my A7 so it might work better now.

Leave a Reply

Your email address will not be published. Required fields are marked *

Khadas VIM4 SBC
Khadas VIM4 SBC