generative ai News - CNX Software - Embedded Systems News

April 26, 2024April 28, 2024 by Jean-Luc Aufranc (CNXSoft) - 1 Comment

Firefly AIBOX-1684X compact AI Box delivers 32 TOPS for large language models, image generation, video analytics, and more

Firefly AIBOX-1684X is a compact AI Box based on SOPHON BM1684X octa-core Arm Cortex-53 processor with a 32 TOPS AI accelerator suitable for large language models (LLM) such as Llama 2, Stable Diffusion image generation solution, and traditional CNN and RNN neural network architectures. Firefly had already released several designs based on the SOPHON BM1684X AI processor with the full-featured Firefly EC-A1684XJD4 FD Edge AI computer and the AIO-1684XQ motherboard, but the AIBOX-1684X AI Box offers the same level of performance, just without as many interfaces, in a compact enclosure measuring just 90.6 x 84.4 x 48.5 mm. AIBOX-1684X AI box specifications: SoC – SOPHGO SOPHON BM1684X CPU – Octa-core Arm Cortex-A53 processor @ up to 2.3 GHz TPU – Up to 32TOPS (INT8), 16 TFLOPS (FP16/BF16), 2 TFLOPS (FP32) VPU Up to 32-channel H.265/H.264 1080p25 video decoding Up to 32-channel 1080p25 HD video processing (decoding + AI analysis) Up […]

April 4, 2024 by Jean-Luc Aufranc (CNXSoft) - 6 Comments

Hailo-10 M.2 Key-M module brings Generative AI to the edge with up to 40 TOPS of performance

Hailo-10 M.2 module generative AI for the edge

Hailo-10 is a new M.2 Key-M module that brings Generative AI capabilities to the edge with up to 40 TOPS of performance at low power. It targets AI PCs supporting only the Windows 11 operating system on x86 or Aarch64 targets at this time. Hailo claims the Hailo-10 is faster and more energy efficient than integrated neural processing unit (NPU) solutions found in Intel SoCs and delivers at least twice the performance at half the power of Intel’s Core Ultra “AI Boost” NPU. Hailo-10 module specifications: AI accelerator – Hailo-10H System Memory – 8GB LPDDR4 on module Host interface – 4-lane PCIe Gen 3 Power consumption – Less than 3.5W (typical) for the chip Form factor – M.2 Key M 2242 / 2280 Supported AI frameworks – TensorFlow, TensorFlow Lite, Keras, PyTorch & ONNX The Hailo-10 can run Llama2-7B with up to 10 tokens per second (TPS) at under 5W […]

January 9, 2024January 10, 2024 by Tomisin Olujinmi - 8 Comments

The EQSP32 is a no-code, no-solder Industrial Internet of Things Controller powered by a generative AI assistant (Crowdfunding)

The EQSP32 controller is a complete, end-to-end solution for IoT applications that recently launched on Kickstarter. It is a compact, wireless Industrial IoT controller based on the ESP32-S3 wireless SoC with a 250MHz dual-core processor, 512KB of RAM, and 8MB of flash memory. The product leverages artificial intelligence and code for automation projects can be generated automatically by the bundled generative AI programming assistant. The EQSP32 controller features 16 terminals that can be configured as analog or digital inputs, or as digital outputs. Switches, pushbuttons, keypads, LED strips, sensors, servos, potentiometers, etc., can be connected to these terminals. It is similar to the EdgeBox-Edge-100 we covered a while back but lacks an Ethernet port and uses less power overall. EQSP32 specifications: SoC: Espressif Systems ESP32-S3 dual-core Tensilica LX7 microcontroller @ 240 MHz, 512KB RAM Memory – 8MB flash Network Connectivity: Bluetooth, WiFi USB – USB-C programming port I/O: 16 multipurpose […]

January 9, 2024January 8, 2024 by Jean-Luc Aufranc (CNXSoft) - 5 Comments

Ambarella N1 SoC brings Generative AI to the edge for video analytics, robotics, industrial applications

Ambarella has been working on adding support for generative AI and multi-modal large language models (LLMs) to its AI Edge processors including the new 50W N1 SoC series with server-grade performance and the 5W CV72S for mid-range applications at a fraction of the power-per-inference of leading GPU solutions. Last year, Generative AI was mostly offered as a cloud solution, but we’ve also seen LLM running on single board computers thanks to open-source projects such as Llama2 and Whispter, and analysts such as Alexander Harrowell, Principal Analyst, at Omdia expect that “virtually every edge application will get enhanced by generative AI in the next 18 months”. The Generative AI and LLM solutions running on Ambarella AI Edge processors will be used for video analytics, robotics, and various industrial applications. Compared to GPUs and other AI accelerators, Ambarella provides AI Edge SoC solutions that are up to 3x more power-efficient per generated […]

November 7, 2023November 7, 2023 by Jean-Luc Aufranc (CNXSoft) - 18 Comments

MediaTek drops efficiency cores in Dimensity 9300 Cortex-X4/A720 mobile SoC

MediaTek Dimensity 9300 is a premium octa-core 5G mobile SoC with two clusters of four Cortex-X4 cores and four Cortex-A720 cores, but doing without any Cortex-A520 efficiency core, plus the latest Arm Mali-G720 GPU, and a MediaTek APU 790 neural processing unit (NPU) capable of support generative AI and large language models (LLM) with up to 33 billion parameters. Arm invented big.LITTLE and then DynamIQ technologies in order to mix cores with different power efficiency and performance characteristics in order to improve power consumption. Their latest launches included the Cortex-X4 premium core, Cortex-A720 performance/big core, and Cortex-A520 efficient/LITTLE core, but MediaTek decided to do without the Cortex-A520 in the Dimensity 9300 which strikes me as odd for a mobile SoC where power efficiency is important for a long battery life. MediaTek Dimensity 9300 specifications: Octa-core CPU with DynamIQ 4x Arm Cortex-X4 at up to 3.25GHz 4x Arm Cortex-A720 up to […]

October 26, 2023October 26, 2023 by Jean-Luc Aufranc (CNXSoft) - 14 Comments

Qualcomm Snapdragon X Elite – A 4.3 GHz 12-core Arm AI processor for next gen PCs and laptops

Qualcomm has now provided Arm chips for mobile PCs (aka laptops) for several years, but apart from a 20-hour battery life, the performance and price of Snapdragon laptops have often been disappointing. The Snapdragon X Elite aims to change that at least on the performance front. The new Qualcomm 12-core 64-bit Arm processor is clocked at up to 3.8 GHz boosting to up to 4.3 GHz, and is said to deliver up to twice the CPU performance against the competition (Intel/AMD/Apple) or provides the same level of performance at a third of the power consumption. The SoC will also be able to run on-device generative AI with over 13B parameters thanks to 75 TOPS of AI performance and support the latest wireless connectivity technologies such as 5G and WiFi 7 through external chips from the company. Qualcomm Snapdragon X Elite specifications: CPU – 12-core 64-bit Armv8 Oryon processor clocked at […]

October 21, 2023February 7, 2024 by Jean-Luc Aufranc (CNXSoft) - 55 Comments

Sophgo SG2380 – A 2.5 GHz 16-core SiFive P670 RISC-V processor with a 20 TOPS AI accelerator

Sophgo SG2380 is an upcoming 2.5 GHz 16-core RISC-V processor based on SiFive Performance P670 cores and also equipped with a 20 TOPS AI accelerator using SiFive Intelligence X280 and Sophgo TPU that will find its way into a $120 desktop-class mini-ITX motherboard in H2 2024. The RISC-V processor also supports up to 64GB RAM, as well as UFS 3.2 and SATA 3.0 storage, comes with an Imagination GPU for 3D graphics and a VPU capable of 4Kp60 H.265, H.264, AV1, and VP9 video decoding, plenty of interfaces, and the system can manage locally deployed larger-scale LLMs like LLaMA-65B without the need for external NVIDIA or AMD accelerator cards. Sophgo SG2380 RISC-V SoC Sophgo SG2380 specifications: CPU 16-core SiFive P670 (RV64GCVH) 64-bit RISC-V processor @ up to 2.5GHz with RISC-V Vector v1.0, Vector Crypto Cluster configuration – 12x 2.5 GHz performance cores, 4x 1.6 GHz efficiency cores Full RISC-V RVA22 […]

October 19, 2023October 19, 2023 by Jean-Luc Aufranc (CNXSoft) - 3 Comments

Generative AI on NVIDIA Jetson Orin, Jetpack 6 SDK to support multiple OSes

NVIDIA has had several announcements at ROSCon 2023 related to robotics & embedded with highlights including generative AI on the NVIDIA Jetson Orin module and the Jetpack 6 SDK will be released next month (November 2023) with supports for Ubuntu as usual, but also other operating systems and platforms such as Debian, Yocto, Wind River, Redhawk RTOS, and Balena. Generative AI on NVIDIA Jetson Orin There’s been a lot of hype in the last year about generative AI thanks to services such as ChatGPT, Google Bard, or Microsoft Bing Chat. But those rely on closed-source software that runs on powerful servers in the cloud. As we noted in our article about the “AI in a box” offline LLM solution there are some open-source projects such as Whisper speech-to-text model and Llama2 language models that could be run on embedded hardware at the edge, but as noted by some readers platforms […]