Leveraging GPT-4o and NVIDIA TAO to train TinyML models for microcontrollers using Edge Impulse

Edge Impulse using NVIDIA TAO and GPT-4o LLM to run model on Arduino Nicla Vision

We previously tested Edge Impulse machine learning platform showing how to train and deploy a model with IMU data from the XIAO BLE sense board relatively easily. Since then the company announced support for NVIDIA TAO toolkit in Edge Impulse, and now they’ve added the latest GPT-4o LLM to the ML platform to help users quickly train TinyML models that can run on boards with microcontrollers. What’s interesting is how AI tools from various companies, namely NVIDIA (TAO toolkit) and OpenAI (GPT-4o LLM), are leveraged in Edge Impulse to quickly create some low-end ML model by simply filming a video. Jan Jongboom, CTO and co-founder at Edge Impulse, demonstrated the solution by shooting a video of his kids’ toys and loading it in Edge Impulse to create an “is there a toy?” model that runs on the Arduino Nicla Vision at about 10 FPS. Another way to look at it […]

MemryX MX3 edge AI accelerator delivers up to 5 TOPS, is offered in die, package, and M.2 and mPCIe modules

MemryX MX3 EVB

Jean-Luc noted the MemryX MX3 edge AI accelerator module while covering the DeGirum ORCA M.2 and USB Edge AI accelerators last month, so today, we’ll have a look at this AI chip and corresponding modules that run computer vision neural networks using common frameworks such as TensorFlow, TensorFlow Lite, ONNX, PyTorch, and Keras. MemryX MX3 Specifications MemryX hasn’t disclosed much performance stats about this chip. All we know is it offers more than 5 TFLOPs. The listed specifications include: Bfloat16 activations Batch = 1 Weights: 4, 8, and 16-bit ~10M parameters stored on-die Host interfaces – PCIe Gen 3 I/O and/or USB 2.0/3.x Power consumption – ~1.0W 1-click compilation for the MX-SDK when mapping neural networks that have multiple layers Under the hood, the MX3 features MemryX Compute Engines (MCE) which are tightly coupled with at-memory computing. This design creates a native, proprietary dataflow architecture that utilizes up to 70% […]

Edge Impulse machine learning platform adds support for NVIDIA TAO Toolkit and Omniverse

Edge Impulse NVIDIA TAO Models

Edge Impulse machine learning platform for edge devices has released a new suite of tools developed on NVIDIA TAO Toolkit and Omniverse that brings new AI models to entry-level hardware based on Arm Cortex-A processors, Arm Cortex-M microcontrollers, or Arm Ethos-U NPUs. By combining Edge Impulse and NVIDIA TAO Toolkit, engineers can create computer vision models that can be deployed to edge-optimized hardware such as NXP I.MX RT1170, Alif E3, STMicro STM32H747AI, and Renesas CK-RA8D1. The Edge Impulse platform allows users to provide their own custom data with GPU-trained NVIDIA TAO models such as YOLO and RetinaNet, and optimize them for deployment on edge devices with or without AI accelerators. NVIDIA and Edge Impulse claim this new solution enables the deployment of large-scale NVIDIA models to Arm-based devices, and right now the following object detection and image classification tasks are available: RetinaNet, YOLOv3, YOLOv4, SSD, and image classification. You can […]

nRF7002 Expansion Board adds WiFi 6 to Nordic Thingy:53 devkit

nRF7002 Expansion Board Thingy:53 devkit

Nordic Semi keeps adding more nRF7002 WiFi 6 boards with the launch of the nRF7002 Expansion Board adding WiFi 6 connectivity to the Thingy:53 IoT prototyping platform and transforming it into an all-in-one wireless devkit with Matter, Bluetooth Low Energy, Thread, and WiFi 6 support. The new “nRF7002 EB” board follows the nRF7002 DK development kit combining the nRF7002 WiFi 6 with nRF5340 multiprotocol wireless SoCs, and the nRF7002 EK evaluation kit in Arduino UNO shield form factor adding WiFi 6 to existing Nordic development kits. nRF7002 Expansion Board specifications: Chipset – nRF7002 Wi-Fi Companion IC with support for features such as OFDMA (Orthogonal Frequency Division Multiple Access), Beamforming, Target Wake Time, and SSID-based locationing Antenna – 2.4 GHz / 5 GHz ceramic antenna I/Os Castellations for all pins on nRF7002 Thingy:53 expansion connector with SPI and a 3-wire coexistence interface to allow seamless coexistence with other wireless protocols Misc […]

Particle launches Photon 2 Realtek RTL8721DM dual-band WiFi and BLE IoT board, Particle P2 module

Particle Photon 2

Particle has launched the Photon 2 dual-band WiFi and BLE IoT board powered by a 200 MHz Realtek RTL8721DM Arm Cortex-M33 microcontroller, as well as the corresponding Particle P2 module for integration into commercial products. The original “Spark Photon” WiFi IoT board was launched in 2014 with an STM32 MCU and a BCM43362 wireless module, but the market and company name have changed since then, and Particle has now launched the Photon 2 board and P2 module with a more modern Cortex-M33 WiFi & BLE microcontroller with support for security features such as Arm TrustZone. Particle Photon 2 specifications: Wireless MCU – Realtek RTL8721DM CPU – Arm Cortex-M33 core @ 200 MHz Memory – 4.5MB embedded SRAM of which 3072 KB (3 MB) is available to user applications Connectivity – Dual-band WiFi 4 up to 150Mbps and Bluetooth 5.0 Security Hardware Engine Arm Trustzone-M Secure Boot SWD Protection Wi-Fi WEP, […]

reComputer J4012 mini PC features NVIDIA Jetson Orin NX for AI Edge applications

reComputer J4021 mini PC

reComputer J4012 is a mini PC or “Edge AI computer” based on the new NVIDIA Jetson Orin NX, a cost-down version of the Jetson AGX Orin, delivering up to 100 TOPS modern AI performance. The mini PC is based on the Jetson Orin NX 16GB, comes with a 128GB M.2 SSD preloaded with the NVIDIA JetPack SDK and offers Gigabit Ethernet, four USB 3.2 ports, and HDMI 2.1 output. Wireless connectivity could be added through the system’s M.2 Key E socket. reComputer J4012 / J401 specifications: SoM – NVIDIA Jetson Orin NX 16GB with CPU – 8x Arm Cortex-A78AE core @ up to 2.0 GHz with 2MB L2 + 4MB L3 cache GPU/AI 1024-core NVIDIA Ampere GPU with 32 Tensor Cores @ up to 918 MHz 2x NVDLA v2.0 @ 614 MHz PVA v2 vision accelerator 100 TOPS AI performance (sparse) Video Encoder  (H.265) 1x 4Kp60 | 3x 4Kp30 6x […]

Arducam OCam AI camera adds context to video streams in real-time with the PhysicO platform

Arducam OCam

Arducam OCam, whose name stands for Object Camera, is an AI camera with 3 TOPS of AI performance and designed to work with OStream‘s PhysicO Edge AI media platform that adds context to MP4 video streams in real-time. The camera supports resolutions from QVGA to 2K, takes USB or PoE power, comes with a drag-and-drop AI pipeline for easy programming/configuration, and is also compatible with common AI tools such as TensorFlow, PyTorch, Edge Impulse, and others. Arducam OCam specifications: Resolution – QVGA up to 2K Frame Rate – Up to 60 fps FoV – 80° Audio – Dual beamforming AI processing power – Up to 3 TOPS Power Supply 5V via USB PoE Power Consumption – Up to 5 Watts Dimensions – 10 cm Φ x 3 cm Weight – 400 grams As I understand it, the AI pipeline – named ObjectAgent – runs on the camera itself, and adds […]

Edge Impulse Enables Machine Learning on Cortex-M Embedded Devices

Edge Impulse

Artificial intelligence used to happen almost exclusively in the cloud, but this introduces delays (latency) for the users and higher costs for the provider, so it’s now very common to have on-device AI on mobile phones or other systems powered by application processors. But recently there’s been a push to bring machine learning capabilities to even lower-end embedded systems powered by microcontrollers, as we’ve seen with GAP8 RISC-V IoT processor or Arm Cortex-M55 core and the Ethos-U55 micro NPU for Cortex-M microcontrollers, as well as Tensorflow Lite. Edge Impulse is another solution that aims to ease deployment of machine learning applications on Cortex-M embedded devices (aka Embedded ML or TinyML) by collecting real-world sensor data, training ML models on this data in the cloud, and then deploying the model back to the embedded device. The company collaborated with Arduino and announced support for the Arduino Nano 33 BLE Sense and […]

Exit mobile version