Archive

Posts Tagged ‘augmented reality’

Qualcomm Snapdragon 845 Octa Core Kryo 385 SoC to Power Premium Smartphones, XR Headsets, Windows Laptops

December 7th, 2017 9 comments

Qualcomm Snapdragon 845 processor was expected since May 2017 with four custom Cortex A75 cores, four Cortex A53 cores, Adreno 630 GPU, and X20 LTE modem. with the launch planned for Q1 2018. At least, that what the leaks said.

Qualcomm has now formally launched Snapdragon 845 Mobile Platform and rumors were mostly right, as the the octa-core processor comes with four Kryo 385 Gold cores (custom Cortex A75), four Kryo 385 Silver cores (custom Cortex A55) leveraging DynamIQ technology, an Adreno 630 “Visual Processing System”, and Snapdragon X20 modem supporting LTE Cat18/13.

The processor is said to use more advanced artificial intelligence (AI) allowing what the company calls “extended reality (XR)” applications, and will soon be found in flagship smartphones, XR headsets, mobile PCs, and more.

Qualcomm Snapdragon 845 (SDM845) specifications:

  • Processor
    • 4x Kryo 385 Gold performance cores @ up to 2.80 GHz (custom ARM Cortex A75 cores)
    • 4x Kryo 385 Silver efficiency cores @ up to 1.80 GHz (custom ARM Cortex A55 cores)
    • DynamIQ technology
  • GPU (Visual Processing Subsystem) – Adreno 630 supporting OpenGL ES 3.2, OpenCL 2.0,Vulkan 1.x, DxNext
  • DSP
    • Hexagon 685 with 3rd Gen Vector Extensions, Qualcomm All-Ways Aware Sensor Hub.
    • Supports Snapdragon Neural Processing Engine (NPE) SDK, Caffe, Caffe2, and Tensorflow
  • Memory I/F – LPDDR4x, 4×16 bit up to 1866MHz, 8GB RAM
  • Storage I/F – TBD (Likely UFS 2.1, but maybe UFS 3.0?)
  • Display
    • Up to 4K Ultra HD, 60 FPS, or dual 2400×2400 @ 120 FPS (VR); 10-bit color depth
    • DisplayPort and USB Type-C support
  • Audio
    • Qualcomm Aqstic audio codec and speaker amplifier
    • Qualcomm aptX audio playback with support for aptX Classic and HD
    • Native DSD support, PCM up to 384kHz/32bit
  • Camera
    • Spectra 280 ISP with dual 14-bit ISPs
    • Up to 16 MP dual camera, up to 32 MP single camera
    • Support for 16MP image sensor operating up to 60 frames per second
    • Hybrid Autofocus, Zero Shutter Lag, Multi-frame Noise Reduction (MFNR)
    • Video Capture – Up to 4K @ 60fps HDR (H.265), up to 720p @ 480fps (slow motion)
  • Connectivity
    • Cellular Modem – Snapdragon X20 with peak download speed: 1.2 Gbps (LTE Cat 18), peak upload speed: 150 Mbps (LTE Cat 13)
    • Qualcomm Wi-Fi 802.11ad Multi-gigabit, integrated 802.11ac 2×2 with MU-MIMO, 2.4 GHz, 5 GHz and 60 GHz
    • Qualcomm TrueWireless Bluetooth 5
  • Location – Support for 6 satellite systems: GPS, GLONASS, Beidou, Galileo, QZSS, SBAS; low power geofencing and tracking, sensor-assisted navigation
  • Security – Qualcomm Secure Processing Unit (SPU), Qualcomm Processor Security, Qualcomm Mobile Security, Qualcomm Content Protection
  • Charging – Qualcomm Quick Charge 4/4+ technology
  • Process – 10nm LPP

The company will provide support for Android and Windows operating systems. eXtended Reality (XR) is enabled with features such as room-scale 6DoF with simultaneous localization and mapping (SLAM), advanced visual inertial odometry (VIO), and Adreno Foveation. Maybe I don’t follow the phone market closely enough, but I can’t remember seeing odometry implemented in any other phones, and Adreon Foveation is not quite self-explaining, so the company explains it combines graphics rendering with eye tracking, and directs the highest graphics resources to where you’re physically looking, while using less resources for rendering other areas. This improves the experience, performance, and lower power consumption.

 

Click to Enlarge

Compared to Snapdragon 835, the new processor is said to be around 25 to 30% faster, the Spectra camera and Adreno graphics architectures are claimed to boost power efficiency by up to 30 percent, and the LTE modem is a bit faster (1.2 Gbps/150Mbps vs 1.0 Gbps/150Mbps). Quick Charge 4+ technology should deliver up  to 50 percent charge in 15 minutes. Earlier this year when SD835 was officially launched, there was virtually no mention of artificial intelligence support in mobile APs, but now NNA (Neural Network Accelerator) or NPE (Neural Processing Engine) are part of most high-end mobile processors, which in SD845 appears to be done though the Hexagon 685 DSP. High Dynamic Range (HDR) for video playback and capture is also a novelty in the new Snapdragon processor.

One of the first device powered by Snapdragon 845 will be Xiaomi Mi 7 smartphone, and according to leaks it will come with a 6.1″ display, up to 8GB RAM, dual camera, 3D facial recognition, and more. Further details about the phone are expected for Mobile World Congress 2018. Considering the first Windows 10 laptop based on Snapdragon 835 processor are expected in H1 2018, we may have to wait until the second part of the year for the launch of Snapdragon 845 mobile PCs.

More details may be found on Qualcomm Snapdragon 845 mobile platform product page.

Google Releases ARCore Augmented Reality Preview SDK for Android

August 30th, 2017 No comments

Augmented reality has been a thing for many years, and Qualcomm released an Augmented Reality SDK working with Android as soon as 2011, but Google has only just released a preview of ARCore augmented reality SDK for Android in order to bring AR capabilities to existing and future Android phones.

ARCore works without any additional hardware (e.g. no 3D depth camera needed such as Tango), relies on Java/OpenGL, Unity or Unreal engines, and focuses on three things:

  • Motion tracking –  ARCore determines the position and orientation of the phone as it moves using both the phone’s camera to observe feature points in the room and IMU sensor data, in order to keep virtual objects accurately placed.
  • Environmental understanding – This handles placement of AR objects on surface like a table or a floor, with the SDK detecting horizontal surfaces using the same feature points it uses for motion tracking.
  • Light estimation – ARCore observes the ambient light in the environment, and applied the same/similar lighting to virtual object to make then more realistic.

While no extra hardware is needed, ARCore won’t not work for all Android smartphones, as the company is working with Samsung, Huawei, LG, ASUS and other manufacturers with the target of supporting 100 million devices – out of the 2 billion Android devices – at the end of the preview.

If you are interested in the project, you can have a look at ARCore SDK on Github. Note however, the company is not accepting pulled requests for now, so it’s a “read-only” project, albeit you can still submit issues. If you want to find more demos / ideas, you may want to check out their AR Experiments showcase.

$99 Mira Prism Augmented Reality Headset Works with your Phone

July 19th, 2017 7 comments

We’ve seen many virtual reality headsets launched in the last year or so, even through the technology and ecosystem still needs to be improved to offer an acceptable user experience, but so far I have not seen augmented reality headset that combines 3D real and virtual worlds into a single screen. I was pretty enthusiastic about augmented reality technology when I first heard about in 2010,  but fast forward to 2017, I’m not using any such apps myself, with the most popular AR app possibly being Pokemon Go game launched last year, and apps like the upcoming AR Measure look quite useful. Augmented reality normally works by holding your phone into your hands and the camera, but Mira Prism will change that it’s meant to be worn like a virtual reality headset and relies on a phone (iPhone only for now) to mix both worlds right in front of your eyes.

The headset includes a semi-transparent screen allowing you to view the real world, and also reflects the phone screen to mix your view with virtual objects. The Prism is also said to feature 6 degrees of freedom optical tracking and a wireless controller with gyroscope & accelerometer, a touchpad, and buttons. It only supports iPhone 6 / 6s / 7 phones, and I have not really been impressed by the demo in the promo video, although the people seem to have a lot of fun, as the virtual objects look somewhat pale due to the semi-transparent screen…


But that’s all new and an interesting concept, so we’ll have to see how that evolves. The main advantage I see is that when you play in groups, you can still interact with other people. Mira Prism is also not too expensive with pre-orders going for $99 on Mira Reality website. Note that you’d need to wait until the end of the year to receive the headset, and price may go up to $149 after that. It’s also mostly targeted at developers right now, since few AR apps are available.

Via Liliputing and Engadget.

Vapor Cooled ASUS Zenfone AR Smartphone Comes with 8GB RAM, Supports Google DayDream and Tango

January 5th, 2017 1 comment

ASUS Zenfore AR is an interesting beast, powered by a Qualcomm Snapdragon 821 processor, it’s the first processor I’ve heard to come with 8GB RAM, and also the first to support both Google DayDream virtual reality, and Google Tango 3D depth sensing camera. On top of that, it’s allegedly cooled by an “advanced vapor cooling system”.
asus-zenfone-ar

Zenfone AR (ZS571KL) specifications:

  • SoC – Qualcomm Snapdragon 821 quad core processor up to 2.35 GHz with Adreno 530 GPU
  • System Memory – 6 to 8 GB LPDDR4 RAM
  • Storage – 32, 64, 128 or 256GB UFS 2.0 flash, micro SD/SDCX card slot up to 2TB, 5GB ASUS WebStorage for file, 100GB Google drive for 2 years
  • Display – 5.7″ WQHD (2560×1440) AMOLED display with Gorilla Glass 4, 10-finger capacitive touch
  • Camera
    • Tricam system with 23MP autofocus, motion tracking, and depth sensing cameras (Tango)
    • 8MP front-facing camera with autofocus dual LED flash
  • Video – 4K video recording
  • Audio – Built-in mono speaker, 3.5mm audio jack
  • Cellular Connectivity – Dual SIM card for 2G, 3G, and 4G networks; up to 600Mbps download speed (LTE cat12); up to 75 upload speed (LTE cat13)
  • Connectivity – 802.11 a/b/g/n/ac 2×2 MIMO, Bluetooth 4.2 + A2DP + EDR, GPS/A-GPS/GLONASS/BDS, NFC
  • USB – 1x USB type C port
  • Sensors – Accelerator/E-Compass/Gyroscope/Proximity sensor/Hall sensor/Ambient light sensor/RGB sensor/IR sensor (Laser Focus)/Fingerprint/Barometer
  • Battery – 3,300 mAh (non-removable) with fast charging through PowerDelivery 2.0 and Quick Charge 3.0
  • Power Supply – 9V/2A (18W) power adapter
  • Dimensions – 158.67 x 77.7 x 4.6 to 8.95 mm
  • Weight – 170 grams

The smartphone will run Android 7.0 Nougat with ZenUI 3.0, and ships with a headset and the power adapter.

Zefone AR is expected to be released in Q2 2017, with pricing yet to be announced. You’ll find more details and photos on ZenFone AR product page.

Qualcomm Officially Unveils Snapdragon 835 Octa-core Processor for Smartphones, Mobile PCs, Virtual Reality…

January 4th, 2017 1 comment

Qualcomm first mentioned Snapdragon 835 processor in November, but at the time, they only disclosed it would be manufactured using 10nm process technology in partnership with Samsung, and claimed the obvious “faster and lower power consumption” compared the previous generation. The company has now provided much more info ahead of CES 2017.

snapdragon-835-block-diagramSnapdragon 835 key features and specifications:

  • Processor – 8x Kryo 280 cores used into two clusters:
    • performance cluster with 4x cores @ up to 2.45 GHz with 2MB L2 cache
    • efficient cluster with 4x cores @ up to 1.9 GHz with 1MB L2 cache
  • GPU – Adreno 540 GPU with support for OpenGL ES 3.2, OpenCL 2.0 full, Vulkan, DX12
  • DSP – Hexagon 682 DSP with Hexagon Vector eXtensions and Qualcomm All-Ways Aware technology
  • Memory I/F – dual channel LPDDR4x
  • Storage I/F – UFS2.1 Gear3 2L, SD 3.0 (UHS-I)
  • Display – UltraHD Premium-ready , 4K Ultra HD 60 Hz, 10-bit color depth, DisplayPort, HDMI, and USB Type-C support
  • Video – Up to 4K @ 30 fps capture, up to 4K @ 60 fps playback, H.264, H.265 and VP9 codecs.
  • Audio – Qualcomm Aqstic audio codec and speaker amplifier; Qualcomm aptX audio playback support: aptX Classic, aptX HD
  • Camera – Spectra 180 ISP; dual 14-bit ISPs up to 16MP dual camera, 32MP single camera
  • Connectivity – 802.11ad multi-gigabit, integrated 802.11ac 2×2 WiFi with MU-MIMO (tri-band: 2.4, 5.0 and 60 GHz); Bluetooth 5.0
  • Modem – X16 LTE modem; downlink up to 1 Gbps, uplink up to 150 Mbps
  • Location – GPS, Glonass, BeiDou, Galileo, and QZSS systems content protection
  • Security – Qualcomm SecureMSM technology, Qualcomm Haven security suite, Qualcomm Snapdragon StudioAccess content protection
  • Charging – Quick Charge 4 technology, Quacomm WiPower technology
  • Manufacturing – 10nm FinFET (Samsung)

Snapdragon 835 will use about 25 percent less power than Snapdragon 820, while being 35 percent smaller, and delivering 25 percent faster 3D graphic rendering. The processor is expected to be found in premium consumer devices such as smartphones, VR/AR head-mounted displays, IP cameras, tablets, mobile PCs, and more. The first devices announced with Snapdragon 835 are Osterhout Design Group (ODG) R-8  augmented/virtual reality smartglasses and ODG R-9 smartglasses and devkit for wide field of view (WFOV) experiences

You’ll find more details on Snapdragon 835 product page.

ARM Introduces Bifrost Mali-G51 GPU, and Mali-V61 4K H.265 & VP9 Video Processing Unit

November 1st, 2016 4 comments

Back in May of this year, ARM unveiled Mali-G71 GPU for premium devices, and the first GPU of the company based on Bifrost architecture. The company has now introduced the second Bifrost GPU with Mali-G51 targeting augmented & virtual reality and higher resolution screens to be found in mainstream devices in 2018, as well as Mali-V61 VPU with 4K H.265 & VP9 video decode and encode capabilities, previously unknown under the codename “Egil“.

Mali-G51 GPU

Click to Enlarge

Click to Enlarge

ARM Mali-G51 will be 60% more energy efficiency, and have 60% more performance density compared to Mali-T830 GPU, making the new GPU the most efficient ARM GPU to date. It will also be 30% smaller, and support 1080p to 4K displays.

Under the hood, Mali-G51 include an updated Bifrost’s low level instruction set, a dual-pixel shader core per GPU core to deliver twice the texel and pixel rates, features the latest ARM Frame Buffer Compression (AFBC) 1.2, and supports Vulkan, OpenGL ES 3.2, and OpenCL 2.0 APIs.

More information can be found on the product page, and an ARM community blog post entitled “The Mali-G51 GPU brings premium performance to mainstream mobile“.

Mali-V61 VPU

mali-v61-4k-120hz

Mali-V61 can scale from 1 to 8 cores to handle 1080p60 up to 4K @ 120 fps, supports 8-/10-bit HEVC & 8-/10-bit VP9 up to 4K UHD video encoding and decoding, making it ideal for 4K video conference and chat, as well as 32MP multi-shot @ 20 fps.

The company claims H.265 and VP9 video encoding quality is about the same for a given bitrate with Mali-V61 as shown in the diagram below.

Click to Enlarge

VP9 vs HEVC vs H.264 – Click to Enlarge

Beside the capability of selecting 1 to 8 cores, silicon vendors can also decide whether they need encoding or decoding block for their SoC. For example camera SoC may not need video decoding support, while STB SoCs might do without encoding. While Mali-V61 is a premium IP block, ARM is also expecting it in mainstream devices possibly also featuring Cortex A53 processor cores and Mali-G51 GPU.

You’ll find more details on the product page, and ARM community “Mali-V61 – Premium video processing for Generation Z and beyond” blog post.

SoftKinetic Announces DepthSense 544 ToF Camera for Mobile Devices

September 26th, 2016 No comments

SoftKinetic has recently unveiled DepthSense 544 3D depth sensing ToF camera for mobile devices, which they claim is the world’s smallest ToF camera, with the “highest level of efficiency available on the market, perfect for small form factors such as smartphones and wearables”.

softkimetic-tof-camera

That’s all nice, but what does ToF mean? ToF stands for Time of Flight, and ToF cameras calculate the distance by measuring the time the light signal travels between the camera and the object as explained by Wikipedia:

A time-of-flight camera (ToF camera) is a range imaging camera system that resolves distance based on the known speed of light, measuring the time-of-flight of a light signal between the camera and the subject for each point of the image. The time-of-flight camera is a class of scannerless LIDAR, in which the entire scene is captured with each laser or light pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems.

Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes became fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers. The distance resolution is about 1 cm. The lateral resolution of time-of-flight cameras is generally low compared to standard 2D video cameras, with most commercially available devices at 320 × 240 pixels or less as of 2011. TOF cameras operate very quickly, providing up to 160 images per second.

Texas Instruments provides a more detailed technical introduction of Time-of-flight camera, and also compare time-of-flight techniques to the two other 3D imaging technologies, namely stereo vision and structured light.

Click to Enlarge

Click to Enlarge

ToF has the advantage of having lower software complexity, fast response time good low and bright light performance, at the expensive of having a medium cost, low compactness (a problem mostly fixed by SoftKinetic module), and medium depth accuracy. It can be used for most applications include games, 3D scanning, user interface, and augmented reality, but is not suitable to shoot 3D movies.

A 2016 Allied Market Search report shows that among 3D cameras, most are using stereo vision technology, and such 3D cameras are still expect to dominate in 2021, but the market share of ToF cameras should increase at the expense of structure light solutions.

3d-camera-technology-market-share

But let’s go back to Softkinetic DepthSense 544 ToF camera, which is said to have the following key features:

  • 10µ pixel, ¼ inch sensor with micro lenses for high depth resolution
  • HDR sensor for both outdoor operation and extra close depth capture
  • MIPI interface for low latency
  • Innovative lens design for small form factor and high efficiency

The module is said to enable fast multi-point auto focus in extreme low light conditions, DSLR quality depth-of-field and background replacement, augmented/virtual reality natural hand interaction in games, indoor navigation and 3D scanning applications for mobile devices.

SoftKinetic DS541 is currently sampling, with mass production planned for Q3 2017. The company will also be showcasing the technology during a two-day DepthSense Workshop, taking place in San Jose, CA on September 26-27, 2016.

Via Image Sensors World Blog

Lenovo Phab 2 Pro Tango-Enabled Smartphone to Sell for $500 in Q3 2016

June 10th, 2016 2 comments

Project Tango uses computer vision to enable mobile devices to detect their position relative to the world using 3D maps of the world created in real-time using 3D depth cameras. Google and Lenovo had already announced a Tango phone would be launched later this year during CES 2016, and the companies have now officially introduced Lenovo Phab 2 Pro, the world’s first Tango-enabled smartphone, scheduled to start selling worldwide in September for $499 and up.

Lenovo_Phab_2_Pro_Tango_SmartphoneLenovo Phab Pro 2 specifications:

  • SoC – Qualcomm Snapdragon 652 Tango Edition processor with 4x ARM Cortex A72 cores @ up to 1.8 GHz, 4x ARM Cortex A53 cores, and Adreno 510 GPU
  • System Memory – 4GB RAM
  • Storage – 64GB flash
  • Display – 6.4″ display with 2560×1440 (QHD) resolution
  • Audio – Dolby Atmos / 5.1 audio capture via 3 microphones with 360 voice noise-cancelling (Note: How can you record 5.1 audio with only 3 microphones?)
  • Cameras
    • 16MP rear camera with fast auto-focus (< 0.3 s)
    • 8 MP front-facing camera
    • Depth camera
    • Motion tracking camera
  • Misc – Fingerprint scanner
  • Battery – 4,050 mAh with 2.4x turbo charging
  • Dimensions – 8.9 mm thick
Lowe App

Lowe’s App

The phone will obviously run Android, and several Tango apps can already be found – or soon will be – in the Google Play Store such as Lowe’s Home Improvement app capable of showing items for sale in your own living before actually buying them, or take measurements, as well as Phantogeist, Raise or Woorld augmented reality games mixing virtual with reality.

Check out Lenovo’s Tango/Phab 2 Pro product page for more details.

Via Liliputing