MemryX MX3 edge AI accelerator delivers up to 5 TOPS, is offered in die, package, and M.2 and mPCIe modules

Jean-Luc noted the MemryX MX3 edge AI accelerator module while covering the DeGirum ORCA M.2 and USB Edge AI accelerators last month, so today, we’ll have a look at this AI chip and corresponding modules that run computer vision neural networks using common frameworks such as TensorFlow, TensorFlow Lite, ONNX, PyTorch, and Keras.

MemryX MX3 Specifications

MemryX hasn’t disclosed much performance stats about this chip. All we know is it offers more than 5 TFLOPs. The listed specifications include:

  • Bfloat16 activations
  • Batch = 1
  • Weights: 4, 8, and 16-bit
  • ~10M parameters stored on-die
  • Host interfaces – PCIe Gen 3 I/O and/or USB 2.0/3.x
  • Power consumption – ~1.0W
  • 1-click compilation for the MX-SDK when mapping neural networks that have multiple layers

Under the hood, the MX3 features MemryX Compute Engines (MCE) which are tightly coupled with at-memory computing. This design creates a native, proprietary dataflow architecture that utilizes up to 70% of the chip with just one click compared to 15-30% on traditional CPUs, GPUs, and DSPs that use legacy instruction sets and control-flow architectures after software tuning.

 

MemryX MX3 internal design
MemryX MX3 internal design

Form Factor

Form-factor-wise, this edge AI processor is offered either as a bare die, a single-die package, or in modules (mini PCIe or M.2) with one or more MemryX MX3 chips.

MX3 Form Factors

MemryX MX3 quad chip M2 module
M.2 module with four MemryX MX3 chips – Source: Mark Hachman, PCWorld

MemryX MX3 EVB

The MX3 EVB (Evaluation Board) is a PCBA with four MX3 chips, and you can cascade multiple EVB boards using a single interface to provide the required inferencing power. Each of these four chips has a single-die package.

The MX3 EVB

MX3 SDK

The MX SDK helps in simulating and deploying the trained AI models. MemryX builds its products to:

  • Provide real-world performance per watt
  • Run models trained on any popular framework without requiring software changes or retraining
  • Provide high scalability and granularity
  • Run AI models equally as well on every host processor regardless of the system load
  • Provide the same 1-click SDK (compilation software)

This SDK’s developer hub consists of a compiler (for graph processing, mapping, and assembling), utility tools (a bit-accurate simulator, performance analyzer, profiler, chip helper tools, and template applications), and a runtime environment with APIs, OS drivers, and a dataflow runtime.

MX3 SDK Chart
MX3-SDK architecture

You can use the MX3 EVB with Edge Impulse deployments after installing dependencies like Python 3.8+, MemryX tools and drivers, and Edge Impulse (for Linux). Next, connect the board to Edge Impulse, then verify it is connected by going to your projects and clicking “devices”.

MemryX MX3 demo

While the company hasn’t provided much detail about the chip’s performance, they did upload a video demo using the virtual camera input of AirSim – a software that creates datasets for autonomous driving and flying – comparing a computer fitted to an MX3 M.2 module to one equipped with NVIDIA 4060 GPU.

Latency was very low while running on the MX3 module, but increased drastically when switching over to the NVIDIA 4060 GPU, and the loud noise from the cooling fans was clearly noticeable.

More details may be found on the company’s website.

Share this:

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK 5 ITX Rockchip RK3588 mini-ITX motherboard
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
2 Comments
oldest
newest
persondb
persondb
6 months ago

What is the cost? 5 TOPS isn’t a lot nowasday. There are many in SOC NPUs or accelerators that will be just as fast or faster and consume similar ammount of power.

Mo Haidara
6 months ago

Thanks for asking about the cost. I would be happy to discuss with you directly. Please reach out via our website, or Linked In page.

Thank you.

Boardcon Rockchip RK3588S SBC with 8K, WiFI 6, 4G LTE, NVME SSD, HDMI 2.1...