Archive

Posts Tagged ‘artificial intelligence’

Google Releases Tensorflow Lite Developer Preview for Android & iOS

November 17th, 2017 No comments

Google mentioned TensorFlow Lite at Google I/O 2017 last may, an implementation of TensorFlow open source machine learning library specifically optimized for embedded use cases. The company said support was coming to Android Oreo, but it was not possible to evaluate the solution at the time.

The company has now released a developer preview of TensorFlow Lite for mobile and embedded devices with a lightweight cross-platform runtine that runs on Android and iOS for now.

TensorFlow Lite Architecture – Click to Enlarge

TensorFlow Lite supports the Android Neural Networks API to take advantage of Machine Learning accelerators when available, but falls back to  CPU execution otherwise.

The architecture diagram above shows three components for TensorFlow Lite:

  • TensorFlow Model – A trained TensorFlow model saved on disk.
  • TensorFlow Lite Converter – A program that converts the model to the TensorFlow Lite file format.
  • TensorFlow Lite Model File – A model file format based on FlatBuffers, that has been optimized for maximum speed and minimum size.

The model file is then within a Mobile App using a C++ or Java (Android only) API, and an interpreter optionally using the Neural Networks API.

TensorFlow Lite currently supports three models: MobileNet (A class of vision models to identify across 1000 different object classes),Inception v3 (An image recognition model with higher accuracy, larger size), and Smart Reply (An on-device conversational model for one-touch replies to chat messages).

The preview release is available on Github, where you’ll also find a demo app that can be tried with a pre-build binary, but it’s probably more fun/useful to instead build it from source in Android Studio and try to change the code to experiment and learn. You can also build the complete framework and demo app from source by cloning the repo. TensorFlow Lite may also be coming to Linux soon, as one of the comment in the announcement mentions that “it should be pretty easy to build TensorFlow Lite on Raspberry PI. We plan to make sure this path works seamlessly soon“. While most of the documentation can be found on Github, some more info may be available on TensorFlow Lite page.

Intel Speech Enabling Developer Kit Works with Alexa Voice Service, Raspberry Pi 3 Board

October 28th, 2017 4 comments

We’ve known Intel has been working on Quark S1000 “Sue Creek” processor for voice recognition for several months. S1000 SoC is based on two Tensilica LX6 with HiFi3 DSP, some speech recognition accelerators, and up to 8x microphones interfaces which allows it to perform speech recognition locally. The solution can also be hooked to an application processor via SPI, I2S and USB (optional) when cloud based voice recognition is needed.

Intel has recently introduced their Speech Enabling Developer Kit working with Amazon Alexa Voice Service (AVS) featuring a “dual DSP with inference engine” – which must be Quark S1000 – and an 8-mic array. The kit also includes a 40-pin cable to connect to the Raspberry Pi 3 board.

Click to Enlarge

Intel only provided basic specifications for the kit:

  • Intel’s dual DSP with inference engine
  • Intel 8-mic circular array
  • High-performance algorithms for acoustic echo cancellation, noise reduction, beamforming and custom wake word engine tuned to “Alexa”
  • 6x Washers
  • 3x 6mm screws
  • 3x 40mm female-female standoffs (x3)
  • Raspberry Pi connector cable

I could not find detailed information to get started, except for assembly guide shown in the video below. We do not that the kit will work with Amazon Alexa, and requires a few extra bits, namely a Raspberry Pi 3 board, an Ethernet cable, a HDMI cable and monitor, USB keyboard and mouse, an external speaker, a micro USB power supply (at least 5V/1A), and a micro SD card.

The video also points to Intel’s Smart Home page for more details about software, but again I could not find instructions or guide there,  except links to register to a developer workshop at Amazon Re:Invent in Las Vegas on November 30, 2017.

Intel Speech Enabling Developer Kit can be pre-ordered for $399 directly on Intel website with shipping planned for the end of November. The product is also listed on Amazon Developer page, but again with little specific information about the hardware and how to use it. One can assume the workflow should be similar to other AVS devkits.

Thanks to Mustafa for the tip.

JeVois Smart Machine Vision Camera Review – Part 1: Developer / Robotics Kit Unboxing

October 24th, 2017 No comments

JeVois-A33 computer vision camera was unveiled at the end of last year through a Kickstarter campaign. Powered by an Allwinner A33 quad core Cortex A7 processor, and a 1.3MP camera sensor, the system could detect motion, track faces and eyes, detect & decode ArUco makers & QR codes, follow lines for autonomous cars, etc.. thanks to JeVois framework.

Most rewards from KickStarter shipped in April of this year, so it’s quite possible some of the regular readers of this blog are already familiar the camera. But the developer (Laurent Itti) re-contacted me recently, explaining they add improves the software with Python support, and new features such as the capability of running deep neural networks directly on the processor inside the smart camera. He also wanted to send a review sample, which I received today, but I got a bit more than I expected, so I’ll start the review with an unboxing of they call the “Developer / Robotics Kit”.

I got the kit in a white package, so I’ll skip the photo, and checking out directly the content.

Click to Enlarge

I was really expecting to receive a tiny camera, and not much else. So my first reaction was: “what!?” 🙂

You’ll find 5 mini USB cables inside (from top left to bottom middle):

Power Bank Info

  • USB to micro serial adapter cable, 1m long, to access the serial console in the camera when running in debug mode, or while troubleshooting Arduino code
  • mini USB + micro USB splitter cable, 15cm long, to power both the camera and Arduino board from the power bank
  • mini USB Y cable, 80cm long, to power the board via two USB 2.0 ports or to one USB 3.0 port on your host computer
  • mini USB cable, 23cm long, to power the camera from a USB port or power bank.
  • mini USB cable, 75cm long, to connect the camera to one USB 3.0 port or power bank.

The kit also includes an 8GB micro SD card pre-loaded with JeVois software, an SD adapter, a micro SD card reader, a 5V USB tester compatible with QuickCharge 2.0 to monitor the power consumption of the camera with your chosen algorithm, a 2,600 mAh power bank (large enough to power the camera for several hours), an Arduino compatible Pro mini board based on Microchip Atmel Atmega 32U4 MCU, and a business card providing useful information such as a link to a Quick Start Guide.

Oh… I almost forgot. Can you see the “fan” in the middle of photo above? That’s the actual JeVois-A33 camera. I knew it was small, but once you put it into your hands, you realize how tiny it is. The cable on the left of the camera is a micro serial cable to connect to an MCU board.

Click to Enlarge

The back of the camera features all the ports and connectors with a micro SD slot, a mini USB port, the micro serial port connector (which looks like a battery connector), and a dual color LED on left of the micro serial connector that indicates power and camera status.

Click to Enlarge

The bottom reveals an opening to cool down AXP223 PMIC.

Click to Enlarge

If you’re interested in the exact developer/robotics kit I’ve received, you can purchase it for $99.99 on JeVois, Amazon, or RobotShop (with locations in US, Canada, Japan, and France). But if you just want the camera without all cable and accessories, $49.99 will do.

NVIDIA DRIVE PX Pegasus Platform is Designed for Fully Autonomous Vehicles

October 11th, 2017 1 comment

Many companies are now involved in the quest to develop self-driving cars, and getting there step by step with 6 levels of autonomous driving defined based on info from  Wikipedia:

  • Level 0 – Automated system issues warnings but has no vehicle control.
  • Level 1 (”hands on”) – Driver and automated system shares control over the vehicle. Examples include Adaptive Cruise Control (ACC), Parking Assistance, and Lane Keeping Assistance (LKA) Type II.
  • Level 2 (”hands off”) – The automated system takes full control of the vehicle (accelerating, braking, and steering), but the driver is still expected to monitor the driving, and be prepared to immediately intervene at any time. You’ll actually have your hands on the steering wheel, just in case…
  • Level 3 (”eyes off”) – The driver can safely turn their attention away from the driving tasks, e.g. the driver can text or watch a movie. The system may ask the driver to take over in some situations specified by the manufacturer such as traffic jams. So no sleeping while driving 🙂 . The Audi A8 Luxury Sedan was the first commercial car to claim to be able to do level 3 self driving.
  • Level 4 (”mind off”) – Similar to level 3, but no driver attention is ever required. You could sleep while the car is driving, or even send the car somewhere without your being in the driver seat. There’s a limitation at this level, as self-driving mode is limited to certain areas, or special circumstances. Outside of these areas or circumstances, the vehicle must be able to safely park the car, if the driver does not retake control.
  • Level 5 (”steering wheel optional”) – Fully autonomous car with no human intervention required, no other limitations

So the goal is obviously to reach level 5, which would allow robotaxis, or safely drive you home whatever your alcohol or THC blood levels. This however requires lots of redundant (for safety) computing power, and current autonomous vehicle prototypes have a trunk full of computing equipments.

NVIDIA has condensed the A.I processing power required  or level 5 autonomous driving into DRIVE PX Pegasus AI computer that’s roughly the size of a license plate, and capable of handling inputs from high-resolution 360-degree surround cameras and lidars, localizing the vehicle within centimeter accuracy, tracking vehicles and people around the car, and planning a safe and comfortable path to the destination.

The computer comes with four A.I processors said to be delivering 320 TOPS (trillion operations per second) of computing power, ten times faster than NVIDIA DRIVE PX 2, or about the performance of a 100-server data center according to Jensen Huang, NVIDIA founder and CEO. Specifically, the board combines two NVIDIA Xavier SoCs and two “next generation” GPUs with hardware accelerated deep learning and computer vision algorithms. Pegasus is designed for ASIL D certification with automotive inputs/outputs, including CAN bus, Flexray, 16 dedicated high-speed sensor inputs for camera, radar, lidar and ultrasonics, plus multiple 10Gbit Ethernet

Machine learning works in two steps with training on the most powerful hardware you can find, and inferencing done on cheaper hardware, and for autonomous driving, data scientists train their deep neural networks NVIDIA DGX-1 AI supercomputer, for example being able to simulate driving 300,000 miles in five hours by harnessing 8 NVIDIA DGX systems. Once trained is completed, the models can be updated over the air to NVIDIA DRIVE PX platforms where inferencing takes place. The process can be repeated regularly so that the system is always up to date.

NVIDIA DRIVE PX Pegasus will be available to NVIDIA automotive partners in H2 2018, together with NVIDIA DRIVE IX (intelligent experience) SDK, meaning level 5 autonomous driving cars, taxis and trucks based on the solution could become available in a few years.

Google’s Teachable Machine is a Simple and Fun Way to Understand How Machine Learning Works

October 9th, 2017 4 comments

Artificial intelligence, machine learning, deep learning, neural networks… are all words we hear more and more today, as machines get the ability to recognize objects, answer voice requests / commands, and so on. But many people may not know at all the basics of how machine learning works, and with that in mind, Google launched Teachable Machine website to let people experiment and understand the basics behind machine learning without having to install an SDK or even code.

So I quickly tried it with Google Chrome, as it did not seem to work with Mozilla Firefox. It’s best to have audio on, as a voice explains how to use it.

Basically you connect your webcam, authorize Chrome too use it, and you should see the image in the input section on the left. After you’re being to train the machine in the learning section in the middle with three difference classes. You’ll be asked to wave your hand and keep pressing on the “Train Green” button until you have at least 100 examples. At this stage, the machine will always detect the green class since it’s all that it knows. Then you can train the Purple class by staying still, and again make sure you have at least 100 examples before you release the button. Now the machine should be able to detect when you stay still or move with a varying percentage of confidence. The output section will just show some animated GIFs, or play sound or words depending on what it detects.  It can learn actions (still, wave hands, clap hands) and object detections. My webcam is pretty bad, but if you have a good image, you should be able to also detect feelings like happiness, sadness, anger, anxiousness, etc… Give it a try it’s fun.

The Teacheable Machine has been built with a new open source hardware-accelerated JavaScript library called deeplearn.js,and Google released the source code for the website too.

Google Clips is an A.I. Camera Powered by Movidius Myriad 2 VPU

October 5th, 2017 No comments

Most consumer cameras offers some ways for the photographer to check the framing of the picture, such as a viewfinder or LCD display, before pressing the button. The first time I saw a consumer camera without such features was with MeCam, a tiny snap-on camera that you can wear on your shirt, and just press a button to take a picture. Convenient, but no ideal as subjects were often out of frame with the camera pointing at the wrong angle.

That was in 2013. But today, those cameras can be improved with artificial intelligence, and Google Clips is a camera without viewfinder nor LCD display that can allegedly take good photos – or short clips – automatically, acting in some ways like a human photographer, so that every human in the room / the whole family can be on the shot.

Google Clips specifications:

  • Vision Processing Unit – Movidius Myriad 2 VPU as found in Intel Movidus Neural Compute Stick
  • Storage – 16 GB for photos
  • Camera
    • TBD?? megapixels; 1.55μm pixels;  130° field of view; auto focus; auto low lux/night mode.
    • Motion photos (JPEGs with embedded MP4s) @ 15 fps, MP4, GIF, JPEG. No audio.
  • Connectivity – WiFi direct and Bluetooth LE
  • USB – 1x USB type C port for charging
  • Battery – Good for 3 hours of smart capture
  • Dimensions – 49 x 49 x 20 mm
  • Weight – 42.2 grams without clip, 60.5 grams with clip

The camera works with Google Clips app for “compatible mobile devices” running Android 7.0 Nougat or higher, such as Google Pixel, or Galaxy S7/S8, or iOS devices starting from iPhone 6. Google Clips will ship with a clip stand, a USB-C to USB-A cable, a quick start guide, and a user guide.

Google Clips will sell for $249, and if you’re interested you can join the waiting list on the product page.

Google Introduces Pixel 2 and Pixel 2 XL Smartphones running Android 8.0 on Snapdragon 835 SoC

October 5th, 2017 2 comments

When Google introduced Nexus brand, it aimed to provide affordable yet decently spec’d Android smartphone. The Nexus has now been deprecated, Google left the low/mid range market, leaving other fills the void, and instead launch the Pixel brand for premium devices.

The company announced several new hardware devices yesterday, including two new Pixel smartphones: Pixel 2 and Pixel 2 XL, both powered by a Qualcomm Snapdragon 835 processor, and running the latest Android 8.0 Oreo.

Pixel 2

Both phones share most of the same specifications:

  • SoC – Qualcomm Snapdragon 835 with 4x Kryo 280 “performance” cores @ 2.35GHz, 4x Kryo 280 “efficiency” cores @ 1.90GHz, Adreno 540 GPU, security module
  • System Memory – 4GB LPDDR4x
  • Storage – 64GB or 128GB flash
  • Display
    • Pixel 2 – 5.0″ always-on AMOLED display with 1920×1080 resolution (16:9 aspect ratio); 2.5D Corning Gorilla Glass 5
    • Pixel 2 XL – 6.0″ always-on pOLED display with 2880×1440 resolution (18:9 aspect ratio) ; 3D Corning Gorilla Glass 5
  • Cellular Connectivity
    • Built-in eSIM chip+ single Nano SIM slot
    • GSM/EDGE: Quad-band (850, 900, 1800, 1900 MHz)
    • UMTS/HSPA+/HSDPA: Bands 1/2/4/5/8
    • CDMA EVDO Rev A: BC0/BC1/BC10
    • FDD-LTE : Bands 1*/2*/3*/4*/5/7*/8/12/13/17/20/25/26/28/29/30/32/66*; TD-LTE: Bands 38*/40/41; * = bands that support 4×4 MIMO
    • Supports up to CAT 15 (800Mbps DL / 75Mbps UL), 3x DL CA, 4×4 MIMO, 256-QAM DL and 64-QAM UL depending on carrier support
  • Other Wireless Connectivity – Dual band 802.11 b/g/n/ac 2×2 MIMO WiFi, Bluetooth 5 + LE, NFC, GPS + GLONASS
  • Camera – 12.2 MP rear camera with auto-focus, OIS and EIS, up to 4K @ 30fps / 720p @ 240 fps video recording; 8MP front-facing camera up to 1080p30 video recording
  • USB – 1x USB 3.1 Gen 1 type C port (Google did not bother mentioning about data or DisplayPort support)
  • Sensors – Active Edge, Proximity / Ambient light sensor, Accelerometer / Gyrometer, Magnetometer, Pixel Imprint(Back-mounted fingerprint sensor), Barometer, Hall effect sensor, Android Sensor Hub, advanced x-axis haptics for sharper/defined response
  • Battery
    • Pixel 2 – 2,700 mAh battery with up to 7 hours of go with a 15-minute charge
    • Pixel 2 XL – 3,520 mAh battery with up to 7 hours of go with a 15-minute charge
  • Dimensions / Weight
    • Pixel 2 – 145.7 x 69.7 x 7.8 mm / 143 grams
    • Pixel 2 XL – 157.9 x 76.7 x 7.9 mm / 175 grams
  • Case – Aluminum unibody with hybrid coating, IP67 ingress protection rating

The phone come with a USB-C charger, a USB-C to 3.5mm headphone adapter (since no 3.5mm audio jack), a quick start guide, and a quick switch adapter. Not too many new things here, except for the embedded SIM (eSIM) that was never used in a phone previously and to initially work with Project Fi, Active Edge (squeeze your phone to start Google Assistant, similar to HTC Edge Sense), and the not-so-common pOLED (Plastic OLED) display found in the XL version. The camera must be really good through, as DxOMark rated the Pixel 2 camera with a score of 98, making it the best mobile camera so far. Google further explains that the camera leverages computational photography and machine learning (ML) capabilities.

Pixel 2 XL

Software and support is probably what make the phones stand part, with the latest Android 8.0 OS from Google, including “an exclusive preview of Google Lens”, which will automatically be able to recognize objects, landmarks, books, movies, etc… scan business cards to adds them to contact list, etc… The company also boasts improvement with Google Assistant, which can now start by squeezing the edges of your phone, and has become smarter with more features such as routines like “good night” to turn off light, set alarms. and put your phone to sleep.

Google Pixel 2 will sell for $649 or $749 with respectively 64 GB or 128GB storage, and Pixel 2 XL for $849 or $949. Visit Google Phones page for further details.

NVIDIA Unveils Open Source Hardware NVDLA Deep Learning Accelerator

October 4th, 2017 2 comments

NVIDIA is not exactly known for their commitment to open source projects, but to be fair things have improved since Linus Torvalds gave them the finger a few years ago, although they don’t seem to help much with Nouveau drivers, I’ve usually read positive feedback for Linux for their Nvidia Jetson boards.

So this morning I was quite surprised to read the company had launched NVDLA (NVIDIA Deep Learning Accelerator), “free and open architecture that promotes a standard way to design deep learning inference accelerators”

Comparison of two possible NVDLA systems – Click to Enlarge

The project is based on Xavier hardware architecture designed for automotive products, is scalable from small to large systems, and is said to be a complete solution with Verilog and C-model for the chip, Linux drivers, test suites, kernel- and user-mode software, and software development tools all available on Github’s NVDLA account. The project is not released under a standard open source license like MIT, BSD or GPL, but instead NVIDIA’s own Open NVDLA license.

This an on-going project, and NVIDIA has a roadmap until H1 2018, at which point we should get FPGA support for accelerating software development, as well as support for TensorRT and other supported frameworks.

Via Phoronix