ASUS UGen300 USB AI accelerator features a 40 TOPS Hailo-10H AI processor and 8GB of LPDDR4 memory to support large language models (LLMs), vision language models (VLMs), and AI vision workloads while consuming less than 2.5 Watts.
In some ways, the hardware is similar to the Raspberry Pi AI HAT+ 2, but in a slim dongle form factor (105 x 50 x 18mm) and a 10 Gbps USB 3.2 Gen1 USB-C Type-C port instead of a 5 or 8 GT/s PCIe interface.
ASUS UGen300 USB AI accelerator (UGen300-USB-8G) specifications:
- AI accelerator – Hailo Hailo-10H delivering up to 40 TOPS (INT4) or 20 TOPS (INT8) inferencing performance
- Memory – 8GB LPDDR4 @ 4266 MT/s
- Host interface – USB 3.1 Gen2 (10 Gbps) Type-C port
- Power Supply – 5V via USB-C port
- Power Consumption – 2.5 Watts (typical)
- Dimensions – 105 x 50 x 18 mm
- Weight – 145 grams
ASUS says the USB AI accelerator works with Windows, Linux, and Android operating systems running on x86 or Arm hosts. It supports both vision and GenAI models, including 150+ pre-trained models, using AI frameworks such as Keras, TensorFlow, TensorFlowLite, PyTorch, and ONNX. ASUS documentation shows how to install the HailoRT SDK on Ubuntu and provides Python tutorials for LLM, VLM, and speech-to-text.
As noted in our review of the Raspberry Pi AI HAT+ 2, the Hailo-10H will not necessarily accelerate LLM workloads, but it will help offload the work to the USB adapter, leaving more memory and CPU/GPU resources on the host, and potentially do so at lower power. So in that respect it should be more of an LLM offloader than an accelerator. However, it will accelerate AI vision workloads and may also help improve performance somewhat with VLMs.
ASUS introduced the UGen300 USB AI accelerator several months ago, but it’s only been publicly available for sale recently, and the company offers the USB adapter for $299.99 on its online store or on Newegg. The latter also lists the ASUS UGen300 M.2 Hailo-10H module with 8GB for $259.99, and a 2GB RAM variant is in the works, but not for sale at this time.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress. We also use affiliate links in articles to earn commissions if you make a purchase after clicking on those links.
