Testing AMD Ryzen 9 7940HS’ AI Engine in Windows 11

Some of the newer AMD Ryzen processors come with an AI Engine (also called NPU or IPU) that works in Windows 11 including the Ryzen 9 7940HS, Ryzen 7 7840HS, Ryzen 5 7640HS, Ryzen 7 7840U, and Ryzen 5 7640U. I’ve just completed the review of the GEEKOM A7 mini PC powered by an AMD Ryzen 9 7940HS CPU with Windows 11 – but need to wait before publishing it – so I decided to try the AI engine on the Ryzen 9 7940HS processor.

The AI Engine relies on AMD XDNA architecture, and AMD provides instructions to get started with examples, demos, and developer resources. So I decided to try some examples, but for some reasons I’ll explain below, I eventually had to settle for some demos.

Ryzen AI software installation (for the demos)

The examples hosted on GitHub require us to install dependencies for the Ryzen AI software stack.

The first step is the installation of the IPU driver which can be done by downloading the “ipu_stack_rel_silicon_prod.zip” after registering an account on the AMD website.

ipu_stack_rel_silicon_prod.zip download

Note that this requires signing a “Beta Software End User License Agreement”, and I initially assumed that was not needed here because the AMD IPU Device driver is already installed in Windows 11 Pro running on the GEEKOM A7 mini PC…

AMD IPU Device

But I was wrong. I had to install the new one (and uninstall the old one) for this part to work.

We are then asked to install a few programs:

  • Visual Studio 2019 (I’d recommend downloading it from a third-party website like TechSpot, because Microsoft requires login to download previous versions of VS).
  • CMake version >= 3.26
  • Python version >= 3.9
  • Anaconda3 or (Miniconda3)

I made sure to select the option add to PATH when installing CMake, Python 3.12, and Anaconda3. Note that Visual Studio 2019 may also be used to install CMake and other tools.

The next step is to download the Ryzen AI Software installation package (ryzen-ai-sw-1.0.1.zip), but this time around, I could not avoid signing the Beta Software End User License Agreement, so I did. The agreement is confidential (although anybody who registers for an AMD account can read it) and also prevents me from reporting any results without a written agreement from AMD. So I’ll just write public information for that part…

We’ll need to run the installation script and accept another EULA for RyzenAI:


If there are any errors (look for CRITICAL string) or the script does not complete, you’ll want to correct that by installing the relevant programs and making sure they are in the PATH.

The next step is to activate the conda environment…


… and run the test:


Once successful, the output should look like that as per the documentation:


It’s a bit longer than that, but you get the idea.

Yolov8 example with Ryzen AI software

Once the installation is complete, we can try some of the examples, and I thought to check out the Yolov8 example.  I spent a few hours on it, but eventually failed, and I did not want to spend more time on it since I couldn’t write about it to the Beta software agreement. But AMD published a video about the similar Yolov8_e2e tutorial from the same GitHub repository about two months ago. Some of the versions have changed, but the procedure is similar and requires quite a few more steps than the example.

Ryzen AI demos

I then switched to the demos directory in the same GitHub repository, and I was not asked to sign any legal documents… I started with the Ryzen AI “cloud-to-client” demo that showcases searching and sorting images on AMD Ryzen AI-based PCs using Yolov5 and Retinaface AI models. I used this with the older IPU driver that shipped with Windows 11 Pro in the GEEKOM A7 mini PC.

The first step was to run the setup batch script:


It ended with a “critical” file missing, but I was still able to run the one of the programs.


You’ll find the log on pastebin.

I then run the program Qt demo:


AMD Image Categorizer NPU usage

I selected a folder with screenshots (I reckon it may not be the best for this type of test but that is what I had on the test PC), and the AMD Image Categorizer scanned my images, and I could see some NPU usage in HWiNFO64. See full output while scanning for reference.

Each image is now searchable and although many tags are not super relevant because of the source images used, it could find a bird in a screenshot I had done with YouTube.

AMD Ryzen 7940HS AI Engine Image Sorting

The cloud-to-client demo is also available as a web server, but it did not work for me, even after installing some extra Python modules.

AMD Object Detection Web Server

RyzenAI SW Server

I’d kind of expect demos to work out of the box without having to spend hours getting the right module or DLL version although my recent experience with the Orbbec Femto Mega depth camera in Windows 11 seems to imply this is business as usual in Windows…

There’s also another demo called “multi-model-exec” with the following models:

  • MobileNet_v2
  • ResNet50
  • Retinaface
  • Segmentation
  • Yolox

Sadly all failed me with the same error. I had to change some of the commands from the instructions or I would have had to stop earlier…

I first had to create a conda environment:


Full output:


We can now enable activate the environment:


I download the ONNX models and test image/video package (resource_multi_model_demo.zip), and unzip it under demo/multi-model-exec/ipu_modelsx4_demo/. After that, I could run the command to generate the scripts:

AMD Ryzen multi model exec generate_script.py

It looks good so far:


But trying any of the scripts fail with the same error:


Windows programs often have cryptic error messages and don’t know what to do with:


The file is there but the system complain about something for which a web search did not help. I’m suspecting some mismatch between versions… It’s probably a matter of spending a few hours or days to fix the issue, and I’m not going to do that.

So the NPU/IPU in the Ryzen 9 7940HS can be used in Windows 11, but that’s not exactly a straightforward task right now, and I’m not sure any programs for end-users have been released just yet… Let me know in the comments if you know any.

Share this:

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK Pi 4C Plus
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
9 Comments
oldest
newest
Anonymous
Anonymous
19 days ago

It’s also in the new 8700G and 8600G desktop APUs.

That’s too bad and it might be considered an AMD failure rather than a Windows failure. They fall flat on their faces repeatedly when it comes to software.

Best case scenario, XDNA1 is there just to give time for developers to start using it. The real action will happen after XDNA2 lands in the Zen 5 Strix family of APUs, with triple to quintuple the performance (45-50 TOPS instead of 10/16). That may actually outperform the iGPU in some cases, so they can hype up more than laptop battery life.

linuxgeex
19 days ago

Surprising they would have an on-package NPU that can’t keep up with the iGPU, unless it’s drastically more power-efficient than using the CPU – bearing in mind that the CPU will still have to feed the NPU, which is still a large part of the compute power budget. With Zen5 that will get even harder (+VNNI), so if it can’t beat the iGPU and CPU significantly on both power and latency then it’s going to be a waste of die space. Esp as the iGPU has PCI bus control and can do some AI without needing the CPU cores awake.… Read more »

Anonymous
Anonymous
19 days ago

It should be more power efficient and die area efficient for the operations it does, but I don’t have the numbers. It does have some of its own L2 cache. Intel also claims better power efficiency for their NPU in Meteor Lake. AMD claims 39 TOPs for Hawk Point, of which 16 TOPS is from the NPU. It was increased from 10 TOPS in Phoenix with clock speed increases alone. With Strix Point, the NPU hits 45-50 TOPS, the iGPU grows in size from 12 to 16 CUs, and there are 12 cores (4x Zen 5 + 8x Zen 5C)… Read more »

Vall
Vall
19 days ago

> I’ve just completed the review of the GEEKOM A7 mini PC powered by an AMD Ryzen 9 7940HS CPU with Windows 11 – but need to wait before publishing it

Why “need to wait”? Geekom requirement?

Ken Raymond Kristiansen
Ken Raymond Kristiansen
19 days ago

I got the same LoadLibrary Error. But I reran the driver installer: amd_install_kipudrv.bat. I added “cd /d %~dp0” to the top of the bat file. As there seems to be some file Signing with the CAT file. Or start the BAT file from within CMD as Admin. After a reboot the run_modelx4 demo works

Dachang
Dachang
18 days ago

When you run a multi-model demo, could you try using CMD, but not powershell, as I have found some errors when using powershell.

zoobab
11 days ago

Any idea if CUDA would work on there with Zluda? GitHub – vosen/ZLUDA: CUDA on AMD GPUs

Khadas VIM4 SBC