Posts Tagged ‘augmented reality’

$99 Mira Prism Augmented Reality Headset Works with your Phone

July 19th, 2017 7 comments

We’ve seen many virtual reality headsets launched in the last year or so, even through the technology and ecosystem still needs to be improved to offer an acceptable user experience, but so far I have not seen augmented reality headset that combines 3D real and virtual worlds into a single screen. I was pretty enthusiastic about augmented reality technology when I first heard about in 2010,  but fast forward to 2017, I’m not using any such apps myself, with the most popular AR app possibly being Pokemon Go game launched last year, and apps like the upcoming AR Measure look quite useful. Augmented reality normally works by holding your phone into your hands and the camera, but Mira Prism will change that it’s meant to be worn like a virtual reality headset and relies on a phone (iPhone only for now) to mix both worlds right in front of your eyes.

The headset includes a semi-transparent screen allowing you to view the real world, and also reflects the phone screen to mix your view with virtual objects. The Prism is also said to feature 6 degrees of freedom optical tracking and a wireless controller with gyroscope & accelerometer, a touchpad, and buttons. It only supports iPhone 6 / 6s / 7 phones, and I have not really been impressed by the demo in the promo video, although the people seem to have a lot of fun, as the virtual objects look somewhat pale due to the semi-transparent screen…

But that’s all new and an interesting concept, so we’ll have to see how that evolves. The main advantage I see is that when you play in groups, you can still interact with other people. Mira Prism is also not too expensive with pre-orders going for $99 on Mira Reality website. Note that you’d need to wait until the end of the year to receive the headset, and price may go up to $149 after that. It’s also mostly targeted at developers right now, since few AR apps are available.

Via Liliputing and Engadget.

Vapor Cooled ASUS Zenfone AR Smartphone Comes with 8GB RAM, Supports Google DayDream and Tango

January 5th, 2017 1 comment

ASUS Zenfore AR is an interesting beast, powered by a Qualcomm Snapdragon 821 processor, it’s the first processor I’ve heard to come with 8GB RAM, and also the first to support both Google DayDream virtual reality, and Google Tango 3D depth sensing camera. On top of that, it’s allegedly cooled by an “advanced vapor cooling system”.

Zenfone AR (ZS571KL) specifications:

  • SoC – Qualcomm Snapdragon 821 quad core processor up to 2.35 GHz with Adreno 530 GPU
  • System Memory – 6 to 8 GB LPDDR4 RAM
  • Storage – 32, 64, 128 or 256GB UFS 2.0 flash, micro SD/SDCX card slot up to 2TB, 5GB ASUS WebStorage for file, 100GB Google drive for 2 years
  • Display – 5.7″ WQHD (2560×1440) AMOLED display with Gorilla Glass 4, 10-finger capacitive touch
  • Camera
    • Tricam system with 23MP autofocus, motion tracking, and depth sensing cameras (Tango)
    • 8MP front-facing camera with autofocus dual LED flash
  • Video – 4K video recording
  • Audio – Built-in mono speaker, 3.5mm audio jack
  • Cellular Connectivity – Dual SIM card for 2G, 3G, and 4G networks; up to 600Mbps download speed (LTE cat12); up to 75 upload speed (LTE cat13)
  • Connectivity – 802.11 a/b/g/n/ac 2×2 MIMO, Bluetooth 4.2 + A2DP + EDR, GPS/A-GPS/GLONASS/BDS, NFC
  • USB – 1x USB type C port
  • Sensors – Accelerator/E-Compass/Gyroscope/Proximity sensor/Hall sensor/Ambient light sensor/RGB sensor/IR sensor (Laser Focus)/Fingerprint/Barometer
  • Battery – 3,300 mAh (non-removable) with fast charging through PowerDelivery 2.0 and Quick Charge 3.0
  • Power Supply – 9V/2A (18W) power adapter
  • Dimensions – 158.67 x 77.7 x 4.6 to 8.95 mm
  • Weight – 170 grams

The smartphone will run Android 7.0 Nougat with ZenUI 3.0, and ships with a headset and the power adapter.

Zefone AR is expected to be released in Q2 2017, with pricing yet to be announced. You’ll find more details and photos on ZenFone AR product page.

Qualcomm Officially Unveils Snapdragon 835 Octa-core Processor for Smartphones, Mobile PCs, Virtual Reality…

January 4th, 2017 1 comment

Qualcomm first mentioned Snapdragon 835 processor in November, but at the time, they only disclosed it would be manufactured using 10nm process technology in partnership with Samsung, and claimed the obvious “faster and lower power consumption” compared the previous generation. The company has now provided much more info ahead of CES 2017.

snapdragon-835-block-diagramSnapdragon 835 key features and specifications:

  • Processor – 8x Kryo 280 cores used into two clusters:
    • performance cluster with 4x cores @ up to 2.45 GHz with 2MB L2 cache
    • efficient cluster with 4x cores @ up to 1.9 GHz with 1MB L2 cache
  • GPU – Adreno 540 GPU with support for OpenGL ES 3.2, OpenCL 2.0 full, Vulkan, DX12
  • DSP – Hexagon 682 DSP with Hexagon Vector eXtensions and Qualcomm All-Ways Aware technology
  • Memory I/F – dual channel LPDDR4x
  • Storage I/F – UFS2.1 Gear3 2L, SD 3.0 (UHS-I)
  • Display – UltraHD Premium-ready , 4K Ultra HD 60 Hz, 10-bit color depth, DisplayPort, HDMI, and USB Type-C support
  • Video – Up to 4K @ 30 fps capture, up to 4K @ 60 fps playback, H.264, H.265 and VP9 codecs.
  • Audio – Qualcomm Aqstic audio codec and speaker amplifier; Qualcomm aptX audio playback support: aptX Classic, aptX HD
  • Camera – Spectra 180 ISP; dual 14-bit ISPs up to 16MP dual camera, 32MP single camera
  • Connectivity – 802.11ad multi-gigabit, integrated 802.11ac 2×2 WiFi with MU-MIMO (tri-band: 2.4, 5.0 and 60 GHz); Bluetooth 5.0
  • Modem – X16 LTE modem; downlink up to 1 Gbps, uplink up to 150 Mbps
  • Location – GPS, Glonass, BeiDou, Galileo, and QZSS systems content protection
  • Security – Qualcomm SecureMSM technology, Qualcomm Haven security suite, Qualcomm Snapdragon StudioAccess content protection
  • Charging – Quick Charge 4 technology, Quacomm WiPower technology
  • Manufacturing – 10nm FinFET (Samsung)

Snapdragon 835 will use about 25 percent less power than Snapdragon 820, while being 35 percent smaller, and delivering 25 percent faster 3D graphic rendering. The processor is expected to be found in premium consumer devices such as smartphones, VR/AR head-mounted displays, IP cameras, tablets, mobile PCs, and more. The first devices announced with Snapdragon 835 are Osterhout Design Group (ODG) R-8  augmented/virtual reality smartglasses and ODG R-9 smartglasses and devkit for wide field of view (WFOV) experiences

You’ll find more details on Snapdragon 835 product page.

ARM Introduces Bifrost Mali-G51 GPU, and Mali-V61 4K H.265 & VP9 Video Processing Unit

November 1st, 2016 4 comments

Back in May of this year, ARM unveiled Mali-G71 GPU for premium devices, and the first GPU of the company based on Bifrost architecture. The company has now introduced the second Bifrost GPU with Mali-G51 targeting augmented & virtual reality and higher resolution screens to be found in mainstream devices in 2018, as well as Mali-V61 VPU with 4K H.265 & VP9 video decode and encode capabilities, previously unknown under the codename “Egil“.

Mali-G51 GPU

Click to Enlarge

Click to Enlarge

ARM Mali-G51 will be 60% more energy efficiency, and have 60% more performance density compared to Mali-T830 GPU, making the new GPU the most efficient ARM GPU to date. It will also be 30% smaller, and support 1080p to 4K displays.

Under the hood, Mali-G51 include an updated Bifrost’s low level instruction set, a dual-pixel shader core per GPU core to deliver twice the texel and pixel rates, features the latest ARM Frame Buffer Compression (AFBC) 1.2, and supports Vulkan, OpenGL ES 3.2, and OpenCL 2.0 APIs.

More information can be found on the product page, and an ARM community blog post entitled “The Mali-G51 GPU brings premium performance to mainstream mobile“.

Mali-V61 VPU


Mali-V61 can scale from 1 to 8 cores to handle 1080p60 up to 4K @ 120 fps, supports 8-/10-bit HEVC & 8-/10-bit VP9 up to 4K UHD video encoding and decoding, making it ideal for 4K video conference and chat, as well as 32MP multi-shot @ 20 fps.

The company claims H.265 and VP9 video encoding quality is about the same for a given bitrate with Mali-V61 as shown in the diagram below.

Click to Enlarge

VP9 vs HEVC vs H.264 – Click to Enlarge

Beside the capability of selecting 1 to 8 cores, silicon vendors can also decide whether they need encoding or decoding block for their SoC. For example camera SoC may not need video decoding support, while STB SoCs might do without encoding. While Mali-V61 is a premium IP block, ARM is also expecting it in mainstream devices possibly also featuring Cortex A53 processor cores and Mali-G51 GPU.

You’ll find more details on the product page, and ARM community “Mali-V61 – Premium video processing for Generation Z and beyond” blog post.

SoftKinetic Announces DepthSense 544 ToF Camera for Mobile Devices

September 26th, 2016 No comments

SoftKinetic has recently unveiled DepthSense 544 3D depth sensing ToF camera for mobile devices, which they claim is the world’s smallest ToF camera, with the “highest level of efficiency available on the market, perfect for small form factors such as smartphones and wearables”.


That’s all nice, but what does ToF mean? ToF stands for Time of Flight, and ToF cameras calculate the distance by measuring the time the light signal travels between the camera and the object as explained by Wikipedia:

A time-of-flight camera (ToF camera) is a range imaging camera system that resolves distance based on the known speed of light, measuring the time-of-flight of a light signal between the camera and the subject for each point of the image. The time-of-flight camera is a class of scannerless LIDAR, in which the entire scene is captured with each laser or light pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems.

Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes became fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers. The distance resolution is about 1 cm. The lateral resolution of time-of-flight cameras is generally low compared to standard 2D video cameras, with most commercially available devices at 320 × 240 pixels or less as of 2011. TOF cameras operate very quickly, providing up to 160 images per second.

Texas Instruments provides a more detailed technical introduction of Time-of-flight camera, and also compare time-of-flight techniques to the two other 3D imaging technologies, namely stereo vision and structured light.

Click to Enlarge

Click to Enlarge

ToF has the advantage of having lower software complexity, fast response time good low and bright light performance, at the expensive of having a medium cost, low compactness (a problem mostly fixed by SoftKinetic module), and medium depth accuracy. It can be used for most applications include games, 3D scanning, user interface, and augmented reality, but is not suitable to shoot 3D movies.

A 2016 Allied Market Search report shows that among 3D cameras, most are using stereo vision technology, and such 3D cameras are still expect to dominate in 2021, but the market share of ToF cameras should increase at the expense of structure light solutions.


But let’s go back to Softkinetic DepthSense 544 ToF camera, which is said to have the following key features:

  • 10µ pixel, ¼ inch sensor with micro lenses for high depth resolution
  • HDR sensor for both outdoor operation and extra close depth capture
  • MIPI interface for low latency
  • Innovative lens design for small form factor and high efficiency

The module is said to enable fast multi-point auto focus in extreme low light conditions, DSLR quality depth-of-field and background replacement, augmented/virtual reality natural hand interaction in games, indoor navigation and 3D scanning applications for mobile devices.

SoftKinetic DS541 is currently sampling, with mass production planned for Q3 2017. The company will also be showcasing the technology during a two-day DepthSense Workshop, taking place in San Jose, CA on September 26-27, 2016.

Via Image Sensors World Blog

Lenovo Phab 2 Pro Tango-Enabled Smartphone to Sell for $500 in Q3 2016

June 10th, 2016 2 comments

Project Tango uses computer vision to enable mobile devices to detect their position relative to the world using 3D maps of the world created in real-time using 3D depth cameras. Google and Lenovo had already announced a Tango phone would be launched later this year during CES 2016, and the companies have now officially introduced Lenovo Phab 2 Pro, the world’s first Tango-enabled smartphone, scheduled to start selling worldwide in September for $499 and up.

Lenovo_Phab_2_Pro_Tango_SmartphoneLenovo Phab Pro 2 specifications:

  • SoC – Qualcomm Snapdragon 652 Tango Edition processor with 4x ARM Cortex A72 cores @ up to 1.8 GHz, 4x ARM Cortex A53 cores, and Adreno 510 GPU
  • System Memory – 4GB RAM
  • Storage – 64GB flash
  • Display – 6.4″ display with 2560×1440 (QHD) resolution
  • Audio – Dolby Atmos / 5.1 audio capture via 3 microphones with 360 voice noise-cancelling (Note: How can you record 5.1 audio with only 3 microphones?)
  • Cameras
    • 16MP rear camera with fast auto-focus (< 0.3 s)
    • 8 MP front-facing camera
    • Depth camera
    • Motion tracking camera
  • Misc – Fingerprint scanner
  • Battery – 4,050 mAh with 2.4x turbo charging
  • Dimensions – 8.9 mm thick
Lowe App

Lowe’s App

The phone will obviously run Android, and several Tango apps can already be found – or soon will be – in the Google Play Store such as Lowe’s Home Improvement app capable of showing items for sale in your own living before actually buying them, or take measurements, as well as Phantogeist, Raise or Woorld augmented reality games mixing virtual with reality.

Check out Lenovo’s Tango/Phab 2 Pro product page for more details.

Via Liliputing

Amazon Fire Phone Features a Camera Array for Dynamic Perspectives, and Firefly Real-Life Object & Audio Recognition

June 19th, 2014 4 comments

Amazon first phone has been rumored for a few months, and yesterday Jeff Bezos finally unveiled the Fire Phone powered by Qualcomm Snapdragon 800 processor @ 2.2 GHz, with 2 GB of RAM, a 4.7″  720p display. It also supports 3G and LTE cellular connectivity, and beside a 13MP rear camera, and 2MP front camera, it also features a specialized camera array with four sensors to offer “Dynamic Perspective” and show different information depending on the position of the phone. Beside the hardware, the most notable feature is Firefly an augmented reality application that can recognize objects you may want to buy, and automatically link them to the Amazon store.

Firefly Recognized a Magazing (Left), Dynamic Perspectives show Yelp Ratings (Right)

Firefly Recognizes a Magazine (Left), Dynamic Perspective shows Yelp Ratings in Maps (Right)

Let’s go through the hardware specifications first:

  • SoC – Qualcomm Snapdragon 800 quad core CPU @ 2.2 GHz with Adreno 330 GPU
  • System Memory – 2GB RAM
  • Storage – 32 GB or 64 GB
  • Display – 4.7″ HD LCD display, with 1280 x 720 resolution, 590 cd/m2 brightness (typical), 1000:1 contrast ratio (typical)
  • Connectivity – 802.11a/b/g/n/ac Wi-Fi (up to 300 Mbps), Bluetooth 3.0 (BLE coming), NFC, and GPS/AGPS
  • Cellular – UMTS/HSPA+/DC-HSDPA (850, 900, 1700/2100, 1900, 2100 MHz), Quad-band GSM/EDGE (850, 900, 1800, 1900 MHz), 9 bands of LTE (Bands 1, 2, 3, 4, 5, 7, 8, 17, 20), supports carrier aggregation. Nano SIM card slot.
  • Cameras
    • 13 MP rear-facing camera, multi-frame HDR, auto focus, optical image stabilization, f/2.0 5-element wide aperture lens, LED flash
    • 2.1 MP front-facing camera
  • Audio – Dual stereo speakers with Dolby Digital Plus audio processing, 3.5mm headphone jack
  • USB – 1x micro USB OTG port
  • Sensors – Dynamic Perspective sensor system with invisible infrared illumination, gyroscope, accelerometer, magnetometer, barometer, proximity sensor, ambient light sensor
  • Battery – 2400mAh. Talk time: up to 22 hours; standby time: up to 285 hours. Video playback: up to 11 hours; audio playback: up to 65 hours.
  • Dimensions – 139.2mm x 66.5mm x 8.9mm
  • Weight – 160 grams

Fire Phone runs Fire OS 3.5.0 based on Android 4.2.2 with an heavily customized user interface and experience. The device sells with “premium” headphones with remote and mic, a micro USB to USB charging cable, a USB power adapter (5W), and a Quick Start Guide.

Most of the features are pretty standard, but an mentioned in the introduction, Amazon has really innovated with two features

  • Dynamic Perspective – A custom-designed sensor system that responds to how you hold, view, and move your phone. It can be used for:
    • Immersive apps and games – For example: “Peek in maps to show Yelp ratings. In Stubhub stadium view, see 90 degrees to the left and right of your seat, as if you were actually there, looking around. In games, like Lili, take on the character’s viewpoint and move your head to look around corners, obstacles, and other objects. “
    • One-handed short cuts – Access menus, shortcuts, and useful information with tilt, swivel, and peek.
    • One-handed reading – Fire phone can auto-scroll books or web pages, and you don’t need to touch the screen.
  • Firefly Technology – By pressing the Firefly button, the phone will be able to identify the following data from real-life objects:
    • Phone numbers, web, and email addresses – Firefly identifies printed text on posters, magazines, and business cards, and let you take action with the results, e.g. make calls, save new contacts, send emails, and visit websites without typing.
    • Movies & TV –  Firefly recognizes over 240,000 movies and TV episodes, and 160 live TV channels.
    • Music – In a similar fashion to Shazam, Firefly can recognize songs. It is compatible with iHeartRadio and StubHub apps.
    • Over 70 million products – Recognize household items, books, DVDs, CDs, video games, etc…, and links to the Amazon Store, where you can add it to your wish list, or order it directly with your phone.

You can get a better understanding of Dynamic Perspective and Firefly in the video below, and found out why “it’s almost like the world is an hyperlink.” WARNING: Possible superlatives overuse.

Developers can access to Dynamic Perspective and Firefly APIs using Amazon’s Fire Phone SDK.

Amazon Fire Phone is available for pre-order now for $649 on Amazon US. However, if you are ready to enslave yourself with a contract with AT&T you can get it for $199 from the same link. That’s for the 32GB, the 64GB versionwill cost $100 more.

Via Liliputing and XDA Developers.

Intel Unveils Edison Board for Wearables at CES 2014 Keynote

January 7th, 2014 3 comments

After an earlier Intel presentation about Intel RealSense Technology by Mooly Eden, Intel Senior Vice President and General Manager, the newly appointed Intel CEO, Brian Krzanich, officially opened CES 2014 with the pre-show keynote. This time there was nothing about processors for PCs, and the announcement the most interesting and relevant to this blog was Intel Edison, a tiny SD-card sized board powered by a dual core Quark SoC.


  • SoC – Unnamed and new dual core Intel Quark SoC @ 400MHz manufactured with 22nm process technology.
  • MCU – Unnamed Intel MCU (MCS 51??) to manage I/Os and other baseline functions.
  • System Memory – LPDDR2
  • Storage – NAND Flash
  • Connectivity – Wi-Fi and Blutooth 4.0 LE
  • Dimensions – Just like an SD card

The board will support Linux, and common open source tools used by the marker community. It will also support Wolfgram language and Mathematica. There will also be an app store.

They showed off the Edison board with a “Nursery 2.0” demo by placing a modified version of Rest Devices’ Mimo Baby Turtle on a baby doll and showing it transmit heart rate and temperature data to smart illuminated coffee cups, and automatically start a smart bottle warmer if it detects the baby is hungry.

There’s not much more details for now, but further information should eventually pop-up on Intel Edison page, once it becomes available mid 2014.

I’ll also quickly cover the other parts of the keynote, and it started with other wearable reference designs including:

  • Smart earbuds with full stereo audio, and monitor heart rate and pulse.
  • Smart headet called called Jarvis, that’s always listening and can leverage Dragon voice assistant technology we’ve seen earlier this morning.
  • A smart charging bowl, which looks like a wok to me, but you can just drop several devices in there and charge them simultaneously. This appears to be based on rezence technology by AW4P.
  • A smartwatch  that can support geo-fencing to monitor the location of kids or older persons.

In order to get the community more interested in working with its platforms, Intel has announced the ‘Make It Wearable’ Challenge promising a total of 1.3 million US dollars in prizes, with the winner getting $500,000, and the top 10 getting 2-month mentoring to bring their wearable products to market. It will start in summer 2014, check this PDF for a few more details, or sign-up on if you would like to participate.

Intel also announced they would just make McAfee anti-virus for wearables, tablets, smartphones, and other embedded devices.

That was all for wearables at this stage, and the CEO moved on to showcase dual OS computers supporting both Android and Windows, and instantaneous switching between the two by the touch of a button.

The next demo was pretty interesting, as they showed a tablet equipped with RealSense 3D camera scanning a small doll to create 3D model, modified it with gesture to become a keyring, and printed it to a Cube3D printer. This type of tablet will be available at the end of 2014, which could lead, IMHO, to 3D printers becoming mainstream in 2015 or 2016, as it becomes easier to create models, and prices come down.

The rest was a bit less interesting (to me) with Dreamworks coming on stage to show a clip of their latest animation movie (hidden from online stream), another augmented reality demo with the tablet equipped with a 3D depth camera, Valve showed their new game on Gigabyte BRIX 2 running Steam OS, and the CEO announced all Intel processor are now using conflict-free materials, before concluding by bringing winners of previous Intel developer challenges on stage.

You can watch the full keynote below (one hour).