Archive

Posts Tagged ‘sdk’

NVIDIA DRIVE PX Pegasus Platform is Designed for Fully Autonomous Vehicles

October 11th, 2017 1 comment

Many companies are now involved in the quest to develop self-driving cars, and getting there step by step with 6 levels of autonomous driving defined based on info from  Wikipedia:

  • Level 0 – Automated system issues warnings but has no vehicle control.
  • Level 1 (”hands on”) – Driver and automated system shares control over the vehicle. Examples include Adaptive Cruise Control (ACC), Parking Assistance, and Lane Keeping Assistance (LKA) Type II.
  • Level 2 (”hands off”) – The automated system takes full control of the vehicle (accelerating, braking, and steering), but the driver is still expected to monitor the driving, and be prepared to immediately intervene at any time. You’ll actually have your hands on the steering wheel, just in case…
  • Level 3 (”eyes off”) – The driver can safely turn their attention away from the driving tasks, e.g. the driver can text or watch a movie. The system may ask the driver to take over in some situations specified by the manufacturer such as traffic jams. So no sleeping while driving 🙂 . The Audi A8 Luxury Sedan was the first commercial car to claim to be able to do level 3 self driving.
  • Level 4 (”mind off”) – Similar to level 3, but no driver attention is ever required. You could sleep while the car is driving, or even send the car somewhere without your being in the driver seat. There’s a limitation at this level, as self-driving mode is limited to certain areas, or special circumstances. Outside of these areas or circumstances, the vehicle must be able to safely park the car, if the driver does not retake control.
  • Level 5 (”steering wheel optional”) – Fully autonomous car with no human intervention required, no other limitations

So the goal is obviously to reach level 5, which would allow robotaxis, or safely drive you home whatever your alcohol or THC blood levels. This however requires lots of redundant (for safety) computing power, and current autonomous vehicle prototypes have a trunk full of computing equipments.

NVIDIA has condensed the A.I processing power required  or level 5 autonomous driving into DRIVE PX Pegasus AI computer that’s roughly the size of a license plate, and capable of handling inputs from high-resolution 360-degree surround cameras and lidars, localizing the vehicle within centimeter accuracy, tracking vehicles and people around the car, and planning a safe and comfortable path to the destination.

The computer comes with four A.I processors said to be delivering 320 TOPS (trillion operations per second) of computing power, ten times faster than NVIDIA DRIVE PX 2, or about the performance of a 100-server data center according to Jensen Huang, NVIDIA founder and CEO. Specifically, the board combines two NVIDIA Xavier SoCs and two “next generation” GPUs with hardware accelerated deep learning and computer vision algorithms. Pegasus is designed for ASIL D certification with automotive inputs/outputs, including CAN bus, Flexray, 16 dedicated high-speed sensor inputs for camera, radar, lidar and ultrasonics, plus multiple 10Gbit Ethernet

Machine learning works in two steps with training on the most powerful hardware you can find, and inferencing done on cheaper hardware, and for autonomous driving, data scientists train their deep neural networks NVIDIA DGX-1 AI supercomputer, for example being able to simulate driving 300,000 miles in five hours by harnessing 8 NVIDIA DGX systems. Once trained is completed, the models can be updated over the air to NVIDIA DRIVE PX platforms where inferencing takes place. The process can be repeated regularly so that the system is always up to date.

NVIDIA DRIVE PX Pegasus will be available to NVIDIA automotive partners in H2 2018, together with NVIDIA DRIVE IX (intelligent experience) SDK, meaning level 5 autonomous driving cars, taxis and trucks based on the solution could become available in a few years.

Hologram Unveils Nova 3G USB Dongle and Python SDK; 200 Raspberry Pi Zero W Kits Given Away to Developers

October 6th, 2017 No comments

This summer I discovered Hologram global cellular IoT SIM card, and since they provided free developer samples with 2MB of monthly data includes, I decided to get one to try it out. I received it a few weeks later, and to my surprise it worked, despite my country of residence having some strict requirements with regards to SIM card registration. The SIM card uses roaming, but with a low fixed worldwide pricing, and does not come with a phone number by default, so maybe that’s why I did not have to register.

The company is now back with Nova, an open source hardware cellular modem certified by OSHWA (ID #US000077). It’s basically 2G/3G USB dongle that’s controlled by Hologram Python SDK, specifically suited to Debian systems like Raspberry Pi 3 or BeagleBone Black. Hackster.io is also involved in the launch with a worldwide contest offering 200 free kits comprised of Nova 3G USB dongle and Raspberry Pi Zero W board for the best project ideas leveraging cellular IoT.

Nova will eventually come in three versions

  • 3G (in production now) – Ublox Sara-U201 module;  Global 3G/2G GSM;  GPRS/GSM/UMTS/HSPA: 850, 900, 1800, 1900 MHz;
  • Cat-M1 (November 2017) – Ublox Sara-R404M module; USA LTE Cat M-1; FDD: 13 (Verizon)
  • Cat-M1/NB1 (Q1 2018) – Ublox Sara-R410M-02B module; Global LTE Cat M-1+ NB; FDD: 1,2,3,4,5,8,12,13,17,18,19,20,25,26,28

All should have the same other interfaces and other specs:

 

  • u.FL Antenna Connector
  • Nano SIM card holder
  • UART GPIO Pads
  • USB Serial
  • Network Status LED; Power LED
  • Fully end certified (FCC, PTCRB, CE, and AT&T)
  • Dimensions – 46mm x 19mm x 6mm (Plugged in PCB);  71mm x 23mm x 9mm (w/ case)
  • Weight – 8 grams

The hardware kit includes the dongle, Hologram global IoT SIM card, a transparent enclosure, 2 Quad-band flexible u.FL antennas, and access to Hologram Developer Tools for modem and data management.

 

The dongle can be controlled using Hologram client tool, or Hologram Python SDK requiring ppp and Python 2.7 packages, and will allow you to send SMS, setup data connection, and more. Any SIM card should work, and it’s not tied to Hologram SIM card. While the company claims OSHWA certifications, the number US000077 is not present (empty line) in the OSHWA certification list yet, and so far, they’ve only released the PDF schematics. However, Python SDK is fully open source and released under an MIT license on Github.

More details can be found in the product page, and Nova 3G kit can be purchased now for $49.

But as mentioned in the introduction, if you have a great project idea, you could also get the kit for free, and possibly another “grand prize” (Apple Watch Series 3)once the project is completed. The contest is opened worldwide (except to US sanctioned countries) with the following timeline:

  • Submit your proposal by October 27, 2017
  • Best project ideas will be selected, and be sent their kit within around 14 days
  • Build and submit your project to Hackster.io by January 5, 2018
  • 8 Grand Prize winners will be announced on January 8, 2018 for four categories: gateway, asset tracking, remote controlling, and remote monitoring.

There are already 135 participants. Good luck!

 

STMicro BlueNRG-MESH SDK for Bluetooth Mesh to Include Code for Firmware, Android and iOS Apps

October 2nd, 2017 2 comments

Earlier this summer, the Bluetooth SIG announced Bluetooth Mesh, which supports many-to-many (m:m) device communications for up 32,767 unicast addresses per mesh network (in theory), and is compatible with Bluetooth 4.0 or greater hardware.  Several companies immediately unveiled Bluetooth Mesh SDK at the time including Qualcomm, Nordic Semi, and Silicon Labs.

ST Micro has now unveiled their own BlueNRG-MESH SDK which the company claims is “the market’s only three-part SDK that provides two app developer packages for Android and iOS, and the embedded-development software for building smart objects such as light fittings and sensors”.

Sadly, details about the SDK are near inexistent now, except – as one would expect – BlueNRG-MESH SDK will work with ST BlueNRG Bluetooth low energy wireless network processor based on an ARM Cortex M0 core, and corresponding development kits. [Update: STSW-BNRG-Mesh page has many more details about the SDK including the architecture diagram below.

]

The solution was showcased at Bluetooth Asia in Shenzhen last week, with some selected customers already working with the SDK, before the public release scheduled for Q4 2017.

Thanks to Jon for the tip.

TECHBASE Moduino X Series Industrial IoT Modules / Endpoints are Based on ESP32 WiSoC

September 27th, 2017 3 comments

We’ve previously covered TECHBASE ModBerry industrial IoT gateways leveraging Raspberry Pi 3, FriendlyELEC NanoPi M1 Plus, or AAEON’s UP Linux boards. The company has now launched Moduino X series modules powered by Espressif ESP32 WiFi + Bluetooth SoC to be used as end points together with their ModBerry gateways.

Moduino X1

Two models have been developed so far, namely Moduino X1 and X2, with the following specifications:

  • Wireless Module – ESP32-WROVER with ESP32 dual-core Tensilica LX6 processor @ 240 MHz, 4MB pSRAM (512KB as option), 4MB SPI flash;
  • External Storage – X2 only: micro SD card slot
  • Connectivity
    • 802.11 b/g/n WiFi up to 16 Mbps + Bluetooth 4.2 LE with u.FL antenna connector
    • X2 only: 10/100M Ethernet
    • Options: LoRa (Semtech SX1272); Sigfox (TI CC1125); LTE Cat M1/NB1; Zigbee
  • Serial – 2x RS-232/485
  • Display – Optional 0.96″ OLED display with 128×64 resolution
  • Expansion I/Os
    • 4x Digital I/O (0 ~ 3V)
    • 2x Analog Input:
    • A2 Only: 2x analog output (optional)
    • A2 only: support for Techbase ExCard add-on modules for extr RS-232/485 ports, Ethernet ports, PCIe slots, analog input and output, digital I/Os, relays, M-Bus interface, etc…
  • Battery – Optional battery power support (A1 only); optional UPS function with LiPo battery or Supercapacitor
  • Power Supply -5V DC
  • Dimensions
    • A1 – ABS: 90 x 36 x 32 mm (LxWxH); Aluminum: 95 x 35 x 41 mm (LxWxH)
    • A2 – ABS: 90 x 71 x 32 mm (LxWxH); Aluminum: 95 x 71 x 41 mm (LxWxH)

Moduino A1 consumes less than A2, and can be powered by batteries only, but both models can use battery as UPS. The modules support Espressif ESP-IDF SDK, Zephyr Project, Arduino programming, MicroPython, Mongoose OS, and more, and would typically be used as meters & sensor nodes capable of reporting temperature, humidity, pressure, acceleration, & light with attached sensors. More sensors are being developed by the company.

Moduino X2 (right)

Moduino X1 & X2 appear to be available now, but you’d need to contact the company to get price information. Visit Moduino X series product page for more details.

Checking Out Debian and Linux SDK for VideoStrong VS-RD-RK3399 Board

September 25th, 2017 32 comments

VideoStrong VS-RD-RK3399 (aka VS-RK3399) is a features-packed development board powered by Rockchip RK3399 hexa core core processor which offers an alternative to Firefly-RK3399 board. The company sent me a development kit for evaluation, and I’ve already looked into VS-RD-RK3399 hardware and SDK in the first part of the review. Today, after shortly looking into the pre-installed Android 7.1 OS to make sure the board boots fine, I’ll report my experience with Debian 9, and building it from source.

Click to Enlarge

A Quick Look at Android 7.1

I connected the board to my HDMI TV, added an Ethernet cable, and after powering it, VS-RK3399 promptly booted into Android 7.1 with the following launched.

Click to Enlarge

It just has a few apps pre-installed, and lack Google Play store, but as I’ve seen in the new version of the SDK, a patch for Google Play store is provided, if that’s something you need for your use case.

Click to Enlarge

The version I had pre-installed was built on August 14, 2017 with Android 7.1.2 OS running on top of Linux 4.4.55 kernel.

The storage section shows plenty of space left out of the 29.12GB flash partition, and the NTFS and EXT-4 partitions of my USB drives were mounted properly.

However, Explorer app would keep on reporting all USB partitions were NOT mounted. I had to use my smartphone to transfer the screenshots over Bluetooth. Adb would have been another option. Anyway, the Android version I had on my board was pretty rough, but I can see they’ve released another Android image in VS3399 board’s MEGA folder. I have not tried it, since I’ve spent enough time playing with Android on RK3399 with TV boxes such as Yundoo Y8 and Vorke Z3.

Installing Debian

So instead I went to the aforelinked MEGA folder, and downloaded VS-RD-RK3399-linuxSDK, which not only have the SDK like last month, but also a Debian image.

So I downloaded debia_linux_20170905.rar, and the rar file in linux_update_tools, but the latter actually contains (outdated versions of) Windows based Rockchip’s AndroidTool & DriverAssistant, but since I’m using Ubuntu, I went with upgrade_tool instead just like I did for Rock64 board quick start guide.

We’ll need to enter recovery mode to flash the Debian image. While the board is running keep pressing the recovery button, press the reset button shortly, count to 2 or 3, and release the recovery button. When you connect the USB type C to USB 3.0 cable between the board and your computer, you should see the board in the kernel log (dmesg):

How we can extract the Debian firmware, and flash it with upgrade_tool:

if successful the procedure is successful, the output should like the four lines below:

and the board should have automatically rebooted to Debian 9.

Testing Debian 9 on VS-RK3399 Board

Now that the installation is complete, we can start to play with the Debian on the board. Or can’t we? There’s no menu available at all, and only the Trash icon on the desktop. If I right click I get some more menus…

So I went into Desktop Preferences, enabled some other icons, and change the right click behavior.

Now I can launch some apps such as the terminal emulator and Chromium web browser, but they don’t show on the desktop at all, despite clearly running “somewhere”…

I then noticed I can move the mouse cursor beyond he left side of the HDMI TV, so I’d assumed there must be another display enabled. I tried to connect Dodocool DC30S hub to the USB type C port since it supports DisplayPort, but I get no signal on my other monitor, so the extra display must be connected via eDP or MIPI DSI, and the company sent me neither.

I could see the image is based on Linaro ALIP rootfs, so I connected to the board via SSH, hoping that linaro/linaro username and password would work, and they did:

The image runs Debian 9 with Linux 4.4.55, and we have a 29GB rootfs, and 3877 MB memory in total.

We can get some more details about the CPU with 2 Cortex A72 cores clocked at up to 1.8 GHz, and 4 Cortex A53 cores clocked at up to 1.416 GHz:

The Gigabit Ethernet port and WiFi module are both detected:

I could not find Bluetooth however using “communication”, and the list of buses or bridges only showed USB ports, no PCI(e) interfaces:

lspci did not return anything either.

Build a Debian Image from Source

Since the image is not really usable with LCD display, I won’t perform more tests on Debian firmware, and instead will work with the SDK, trying to build my own image from source. We can extract the Linux SDK we’ve gotten from the MEGA link:

…and follow / adapt the instructions in VS-RK3399 Linux SDK Compile and build (EN).pdf document found in VS-RD software datasheet folder.

The company recommends a build machine running Ubuntu 14.04 / 16.04 64-bit, and I installed some dependencies on my Ubuntu 16.04 computer:

as well as ARM/ARM64 GCC toolchain and other tools and libraries:

We can now build u-boot:

It should only take a few seconds, and we can make sure the build went fine:

Next up is the the Linux kernel:

We have to choose an image to build with the dts file of our choice:

The -edp image is with eDP LCD display, -mipi is for MIPI LCD display, and the -dulelcd must be for both (“dual LCD”). So there’s no option for no LCD at all right now, and I’m not sure why there’s a need for three DTS files for the board, as surely this should be possible to select/configure the LCD outputs are runtime (TBC). But let’s use the MIPI LCD one:

I did not go that well, as the build failed:

I tried with rk3399-videostrong-board-edp.img again, and same error, but I noticed more details by scrolling up:

We can see gsl3673.c file is missing:

So I renamed the c-old file to c, and could complete the build:

Now we can build the complete firmware from source using buildroot:

The build all script will take a long while as it downloads and builds all packages. It took over 2 hours on my machine. I ended with:

It looks OK, except for the script failed to remove one file/directory. Let’s try to create the firmware file:

Sadly that part failed too because of conflicts with a patch:

That failures means the rootfs was not built, and the script is so bad did not stop during mpp build failure, and continued building other libs:

I have not tried to fix the issue, as I don’t know how many other issues are hidden in that big piles of code. VideoStrong should look into and provide an SDK that actually builds.. In case the build works, we should get rootfs.img file that can be flashed with AndroidTool if we follow the instructions in the PDF file.

However, you’ll also be able to flash the files one by one using upgrade_tool in Linux, no need to have a Windows machine for any part of the process.

It’s hard to recommend VS-RD-RK3399 if you are an individual who want a platform for development since software support and documentation are rather poor, but if you are a company that plans to order in quantity, you should have direct support from the company, and you can contact them via Alibaba’s VideoStrong page. If you have a good skills, and are ready to work to solve whatever issues, you can also purchase the board for $168.32 (2GB/16GB) or $213.29 (4GB/32GB) on GearBest where it is sold under the MECOOL brand (Coupon GBCNA will give a 14% discount), or Aliexpress. For reference, Firefly-RK3399 in equivalent memory/storage configuration sells for $179 and $219 on Aliexpress. It’s also listed on Amazon US for $149.99/$199.99.

Google Releases ARCore Augmented Reality Preview SDK for Android

August 30th, 2017 No comments

Augmented reality has been a thing for many years, and Qualcomm released an Augmented Reality SDK working with Android as soon as 2011, but Google has only just released a preview of ARCore augmented reality SDK for Android in order to bring AR capabilities to existing and future Android phones.

ARCore works without any additional hardware (e.g. no 3D depth camera needed such as Tango), relies on Java/OpenGL, Unity or Unreal engines, and focuses on three things:

  • Motion tracking –  ARCore determines the position and orientation of the phone as it moves using both the phone’s camera to observe feature points in the room and IMU sensor data, in order to keep virtual objects accurately placed.
  • Environmental understanding – This handles placement of AR objects on surface like a table or a floor, with the SDK detecting horizontal surfaces using the same feature points it uses for motion tracking.
  • Light estimation – ARCore observes the ambient light in the environment, and applied the same/similar lighting to virtual object to make then more realistic.

While no extra hardware is needed, ARCore won’t not work for all Android smartphones, as the company is working with Samsung, Huawei, LG, ASUS and other manufacturers with the target of supporting 100 million devices – out of the 2 billion Android devices – at the end of the preview.

If you are interested in the project, you can have a look at ARCore SDK on Github. Note however, the company is not accepting pulled requests for now, so it’s a “read-only” project, albeit you can still submit issues. If you want to find more demos / ideas, you may want to check out their AR Experiments showcase.

Intel Introduces Movidius Myriad X Vision Processing Unit with Dedicated Neural Compute Engine

August 29th, 2017 No comments

Intel has just announced the third generation of Movidius Video Processing Units (VPU) with Myriad X VPU, which the company claims is the world’s first SoC shipping with a dedicated Neural Compute Engine for accelerating deep learning inferences at the edge, and giving devices the ability to see, understand and react to their environments in real time.

Movidius Myraid X VPU key features:

  • Neural Compute Engine – Dedicated on-chip accelerator for deep neural networks delivering over 1 trillion operations per second of DNN inferencing performance (based on peak floating-point computational throughput).
  • 16x programmable 128-bit VLIW Vector Processors (SHAVE cores) optimized for computer vision workloads.
  • 16x configurable MIPI Lanes – Connect up to 8 HD resolution RGB cameras for up to 700 million pixels per second of image signal processing throughput.
  • 20x vision hardware accelerators to perform tasks such as optical flow and stereo depth.
  • On-chip Memory – 2.5 MB homogeneous memory with up to 450 GB per second of internal bandwidth
  • Interfaces – PCIe Gen 3, USB 3.1
  • Packages
    • MA2085: No memory in-package; interfaces to external memory
    • MA2485: 4 Gbit LPDDR4 memory in-package

The hardware accelerators allows to offload the neural compute engine, for example, the stereo depth accelerator can simultaneously process 6 camera inputs (3 stereo pairs) each running 720p resolution at 60 Hz frame rate. The slide below also indicates Myriad X to have 10x higher DNN performance compared to Myriad 2 VPU found in Movidius Neural Compute Stick.

Click to Enlarge

The VPU ships with an SDK that contains software development frameworks, tools, drivers and libraries to implement artificial intelligence applications, such as a specialized “FLIC framework with a plug-in approach to developing application pipelines including image processing, computer vision, and deep learning”, and a neural network compiler to port neural networks from Caffe, Tensorflow, and others.

Myriad SDK Architecture

More details can be found on Movidius’ MyriadX product page.

Videostrong VS-RD-RK3399 Development Board Review – Part 1: Unboxing, Kit Assembly, SDK and Documentation

August 18th, 2017 8 comments

Videostrong VS-RD-RK3399 development board is a full-featured development based on Rockchip RK3399 hexa-core processor with up to 4GB RAM, and 32GB eMMC flash, and plenty of I/Os. The company has sent me a sample, and after getting some trouble going through customs with questions like “what is a development board?” and “is it a board for TV boxes or computers?”, I finally got hold of the parcel. Today, I’ll check out the board and its accessories, show how to assemble it, and since the company shared more info about documentation and software, quickly go over what’s available.

Videostrong VS-RD-RK3399 Development Kit Unboxing

The board was in a bland carton box, which is fine since it’s not a consumer product, with a stick showing I got the 4GB LPDDR3 / 32GB eMMC flash version. There’s also a board using 2GB/16GB configuration.


The package includes the board, bottom and top acrylic plates for the “case”, some spacers, WiFi and Bluetooth antenna, USB 3.0 type A to USB type C cable, a user’s manual detailing the board’s specifications and pinout diagram…

Click to Enlarge

… and a 12V/2A power adapter and EU, US, and UK plug adapters.

Click to Enlarge

I also took some closer photos of the development board, and it will only report new information that I have not already written in the announcement post.  First, the eMMC flash is Samsung KLMBG4WEBD-B031, the cheapest 32GB eMMC flash from the company, but still with acceptable performance: 246/46 MB/s R/W speed, and 6K/5K R/W IOPS.

Click to Enlarge

There are many USB ports in the board’s design, and this is done via Genesys Logic GL850G USB 2.0 hub chip, while the audio codec is Realtek ALC5640.

Click to Enlarge

The back of the board includes an mPCIe slot for a 4G LTE card, and a MIPI DSI connector.

VS-RD-RK3399 Kit Assembly

The assembly of the kit is mostly straightforward for may be a little confusing at the beginning. First, we’ll need to remove the protection on the acrylic plates, and use the bottom one with the 6 ventilation lines, and tighten the small and medium spacers around the base, with the small ones facing down. I thought it was a good idea to connect the u.FL to SMA cable for the antenna at this stage, but they come off those easily, it’s better to do it later.

Click to Enlarge

Then we’ll place the main board on top of the medium spacer, and tighten the longer spacers on top. Once it’s done we can remove nuts from the SMA connector, insert the antenna cables on the right and middle hole in the top acrylic, and screw the nuts back to keep the cables in place.

Click to Enlarge

We can now connect the antenna cables the ANT0 (for Bluetooth), and ANT1 (for WiFi) u.FL connectors on the board, place the top cover with the two opening aligned over the MIPI CSI connectors, and tighten it with the four remaining nuts we have, before completing the assembly by installing the two antennas.

Click to Enlarge

VS-RD-RK3399 Board SDK and Documentation

When I asked about Android/Linux software development kit, and documentation last week, the company had nothing to offer, saying the SDK will be provided via a link… Today, they gave me that link on MEGA with most of what is needed for development.

The download is too big for a free MEGA account, unless you are really patient, but you should be able to download everything using megaupload tools in a terminal.

I haven’t completed the download yet, and I’ll look into details during the next part of the review, but we can see 6 main directories:

  • VS-RD-RK3399-linuxSDK – The Linux SDK
  • VS-RD-android7.1-SDK – Android Nougat SDK
  • VS-RD Software image – Android firmware, apparently no Linux OS (yet)
  • VS-RD Software datasheet – Linux, Android, Dual OS documentation
  • VS-RD Hardware – Parts datasheet, RK3399 TRM, LCD datasheet (No schematics apparently)
  • DevelopmentTool – Various tools for development like AndroidTool, DriverAssistant, etc…

If you are interesting in the platform, you can purchase it by contacting Videostrong via Alibaba.

[Update: Part 2 of VS-RD-RK3399 / Mecool VS-RK3399 board: Checking Out Debian and Linux SDK for VideoStrong VS-RD-RK3399 Board]